From laurentfdumont at gmail.com Wed Jun 1 01:45:41 2022 From: laurentfdumont at gmail.com (Laurent Dumont) Date: Tue, 31 May 2022 21:45:41 -0400 Subject: error creating image from volume In-Reply-To: References: Message-ID: I am not familiar with the error. 1. What client version are you using? openstack --version? 2. What Openstack version are you running? 3. Any specific properties on the volume? Laurent On Tue, May 31, 2022 at 8:10 AM Russell Stather < Russell.Stather at ignitiongroup.co.za> wrote: > Hi > > Trying to create an image from a volume. > > Getting this very unhelpful error message. > > igadmin at ig-umh-maas:~$ openstack image create --volume > 57bc3efa-6cdc-4e2d-a0e2-262340cc6180 commvaultimage > upload_to_image() got an unexpected keyword argument 'visibility' > > Anyone seen this before? > > Thanks > > Russell > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Wed Jun 1 07:01:07 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 01 Jun 2022 09:01:07 +0200 Subject: [neutron] CI meeting 7.06.2022 cancelled Message-ID: <3190548.KDQWzQIof1@p1> Hi, Next week there is OpenInfra summit in Berlin and some of us will be busy there, so we decided yesterday to cancel next week's Neutron CI meeting. Please ping me on IRC or email if there would be anything urgent related to our CI. See You all on the CI meeting on 14.06.2022 (and in Berlin if You will be there :)) -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From katonalala at gmail.com Wed Jun 1 07:31:19 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 09:31:19 +0200 Subject: [neutron] Drivers meeting - Friday 2.6.2022 & 9. 6. 2022. - cancelled Message-ID: Hi Neutron Drivers! This Friday I will be at a conference and next Friday I will be travelling back from Berlin, so I can't drive these drivers' meetings, let's cancel them. I hope I can meet a few of you in Berlin, and see you online the week after the summit. Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 07:36:58 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 09:36:58 +0200 Subject: [neutron] team meeting 7. 06. 2022 cancelled Message-ID: Hi, Next week I will attend the Openinfra summit in Berlin, so I can't drive the team meeting. See you online on the week after the summit, or I hope we can meet in Berlin. Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Wed Jun 1 08:06:34 2022 From: zigo at debian.org (Thomas Goirand) Date: Wed, 1 Jun 2022 10:06:34 +0200 Subject: Fwd: [Forum] Meet the Projects Session! In-Reply-To: References: Message-ID: <932f56ec-a9f9-eea0-c3d5-69be9e185397@debian.org> On 5/31/22 17:53, Kendall Nelson wrote: > Hello :) > > I wanted to take a moment to invite project maintainers, core > contributors, PTLs, governance officials, etc to a meet the projects > (projects being the OIF top level ones- Kata, Zuul, OpenStack, > StarlingX, and Airship) session during the Forum at the upcoming Summit > in Berlin. It will take place Tue, June 7, 11:20am - 12:30pm local in? A > - A06. > > The idea is to gather in a single place so that newer members of the > community or people unfamiliar with the project can come and meet some > of the active participants and ask questions. It's all really informal > so there is no need to prepare anything ahead of time. > > I realize it is pretty late in everyone's planning their schedules for > the summit, but if you have a few minutes to spare to come hang out and > meet some people and network, please do! We would love to see you there. > > -Kendall Hi, Thanks a lot for this proposal, I very much like the idea of meeting (new) project members in real. I definitively will join and hope to match faces with IRC nicks after the meeting. Cheers, Thomas Goirand (zigo) From skaplons at redhat.com Wed Jun 1 08:39:30 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 01 Jun 2022 10:39:30 +0200 Subject: Fwd: [Forum] Meet the Projects Session! In-Reply-To: <932f56ec-a9f9-eea0-c3d5-69be9e185397@debian.org> References: <932f56ec-a9f9-eea0-c3d5-69be9e185397@debian.org> Message-ID: <4286047.ni3Bx2OuKx@p1> Hi, Dnia ?roda, 1 czerwca 2022 10:06:34 CEST Thomas Goirand pisze: > On 5/31/22 17:53, Kendall Nelson wrote: > > Hello :) > > > > I wanted to take a moment to invite project maintainers, core > > contributors, PTLs, governance officials, etc to a meet the projects > > (projects being the OIF top level ones- Kata, Zuul, OpenStack, > > StarlingX, and Airship) session during the Forum at the upcoming Summit > > in Berlin. It will take place Tue, June 7, 11:20am - 12:30pm local in A > > - A06. > > > > The idea is to gather in a single place so that newer members of the > > community or people unfamiliar with the project can come and meet some > > of the active participants and ask questions. It's all really informal > > so there is no need to prepare anything ahead of time. > > > > I realize it is pretty late in everyone's planning their schedules for > > the summit, but if you have a few minutes to spare to come hang out and > > meet some people and network, please do! We would love to see you there. > > > > -Kendall > > Hi, > > Thanks a lot for this proposal, I very much like the idea of meeting > (new) project members in real. I definitively will join and hope to > match faces with IRC nicks after the meeting. > > Cheers, > > Thomas Goirand (zigo) > > Yeah. Thx for organizing this Kendal. I already added it to my calendar and will be there too :) -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From rdhasman at redhat.com Wed Jun 1 09:36:44 2022 From: rdhasman at redhat.com (Rajat Dhasmana) Date: Wed, 1 Jun 2022 15:06:44 +0530 Subject: error creating image from volume In-Reply-To: References: Message-ID: Hi Russell, Thanks for finding the issue. On Tue, May 31, 2022 at 5:46 PM Russell Stather < Russell.Stather at ignitiongroup.co.za> wrote: > Hi > > Trying to create an image from a volume. > > Getting this very unhelpful error message. > > igadmin at ig-umh-maas:~$ openstack image create --volume > 57bc3efa-6cdc-4e2d-a0e2-262340cc6180 commvaultimage > upload_to_image() got an unexpected keyword argument 'visibility' > > Anyone seen this before? > > This is indeed a valid bug for which I've proposed the fix here[1]. To summarize, this is caused because OSC is calling cinderclient with *visibility* and *protected* arguments[2] without checking the microversion and the support for passing *visibility* and *protected* fields is available only since microversion *3.1* or greater[3][4]. For a quick workaround, pass the* ``--os-volume-api-version 3.1``* parameter along with the command or set the environment variable *``OS_VOLUME_API_VERSION``* to *3.1* or greater value. [1] https://review.opendev.org/c/openstack/python-openstackclient/+/844268 [2] https://opendev.org/openstack/python-openstackclient/src/branch/master/openstackclient/image/v2/image.py#L492-L493 [3] https://opendev.org/openstack/cinder/src/branch/master/cinder/api/openstack/api_version_request.py#L49 [4] https://opendev.org/openstack/python-cinderclient/src/branch/master/cinderclient/v3/volumes.py#L197-L211 Thanks and regards Rajat Dhasmana Thanks > > Russell > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkopec at redhat.com Wed Jun 1 12:06:15 2022 From: mkopec at redhat.com (Martin Kopec) Date: Wed, 1 Jun 2022 14:06:15 +0200 Subject: [qa] Cancelling office hour on June 7th Message-ID: Hello everyone, QA team is cancelling the next office hour Tuesday June 7th, I'm gonna attend the OpenStack Summit. https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_next_Office_hours Thanks, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin -------------- next part -------------- An HTML attachment was scrubbed... URL: From senrique at redhat.com Wed Jun 1 12:12:43 2022 From: senrique at redhat.com (Sofia Enriquez) Date: Wed, 1 Jun 2022 09:12:43 -0300 Subject: [cinder] Bug deputy report for week of 06-01-2022 Message-ID: This is a bug report from 05-25-2022 to 06-01-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Low - https://bugs.launchpad.net/cinder/+bug/1976400 "[Storwize] Fix multiple SVC CLI calls for rc-relationship operations." Fix proposed to master. Wishlist - https://bugs.launchpad.net/cinder/+bug/1975794 "How to configure Cinder to use as much as possible, (post-provision)." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1976331 "Check attachment_specs may lead to poor performance." Unassigned. Cheers, Sofi -- Sof?a Enriquez she/her Software Engineer Red Hat PnT IRC: @enriquetaso @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 12:24:39 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 14:24:39 +0200 Subject: [neutron][networking-odl] Propose to EOL stable/queens, stable/rocky and stable/stein branches of networking-odl Message-ID: Hi, The Queens, Rocky and Stein branches of networking-odl see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/queens, stable/rocky and stable/stein branches of networking-odl End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in networking-odl. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 12:27:57 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 14:27:57 +0200 Subject: [neutron][networking-bagpipe][networking-bgpvpn] Propose to EOL stable/rocky and stable/stein branches of networking-odl Message-ID: Hi, The Rocky and Stein branches of networking-bagpipe and networking-bgpvpn see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/rocky and stable/stein branches of networking-bagpipe and networking-bgpvpn End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in networking-bagpipe and networking-bgpvpn. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 12:30:56 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 14:30:56 +0200 Subject: Fwd: [neutron][networking-bagpipe][networking-bgpvpn] Propose to EOL stable/rocky and stable/stein branches of networking-odl In-Reply-To: References: Message-ID: Sorry: of course the subject is networking-bagpipe & networking-bgpvpn ---------- Forwarded message --------- Felad?: Lajos Katona Date: 2022. j?n. 1., Sze, 14:27 Subject: [neutron][networking-bagpipe][networking-bgpvpn] Propose to EOL stable/rocky and stable/stein branches of networking-bagpipe & bgpvpn To: Openstack Discuss List Hi, The Rocky and Stein branches of networking-bagpipe and networking-bgpvpn see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/rocky and stable/stein branches of networking-bagpipe and networking-bgpvpn End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in networking-bagpipe and networking-bgpvpn. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 12:33:19 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 14:33:19 +0200 Subject: Fwd: [neutron][networking-bagpipe][networking-bgpvpn] Propose to EOL stable/rocky and stable/stein branches of networking-bagpipe & bgpvpn In-Reply-To: References: Message-ID: ---------- Forwarded message --------- Felad?: Lajos Katona Date: 2022. j?n. 1., Sze, 14:27 Subject: [neutron][networking-bagpipe][networking-bgpvpn] Propose to EOL stable/rocky and stable/stein branches of networking-odl To: Openstack Discuss List Hi, The Rocky and Stein branches of networking-bagpipe and networking-bgpvpn see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/rocky and stable/stein branches of networking-bagpipe and networking-bgpvpn End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in networking-bagpipe and networking-bgpvpn. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 12:35:27 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 14:35:27 +0200 Subject: [neutron][networking-sfc] Propose to EOL stable/queens, stable/rocky and stable/stein branches of networking-sfc Message-ID: Hi, The Queens, Rocky and Stein branches of networking-sfc see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/queens, stable/rocky and stable/stein branches of networking-sfc End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in networking-sfc. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 13:17:30 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 15:17:30 +0200 Subject: [neutron][neutron-fwaas] Propose to EOL stable/queens, stable/rocky and stable/stein branches of neutron-fwaas Message-ID: Hi, The Queens, Rocky and Stein branches of neutron-fwaas see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/queens, stable/rocky and stable/stein branches of neutron-fwaas End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in neutron-fwaas. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 13:36:48 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 15:36:48 +0200 Subject: [neutron][tap-as-a-service] Propose to EOL stable/queens, stable/rocky and stable/stein branches of tap-as-a-service Message-ID: Hi, The Queens, Rocky and Stein branches of tap-as-a-service see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/queens, stable/rocky and stable/stein branches of tap-as-a-service End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in tap-as-a-service. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Wed Jun 1 13:40:39 2022 From: katonalala at gmail.com (Lajos Katona) Date: Wed, 1 Jun 2022 15:40:39 +0200 Subject: [neutron][networking-ovn] Propose to EOL stable/queens, stable/rocky and stable/stein branches of networking-ovn Message-ID: Hi, The Queens, Rocky and Stein branches of networking-ovn see little activity and maintaining them and keeping their jobs running is a big effort. We discussed this topic during the team meeting (see [1]), and agreed to propose stable/queens, stable/rocky and stable/stein branches of networking-ovn End Of Life. I will wait until the end of next week for anyone who would like to step up as maintainer of those branches and who would at least try to fix CI of them. If there will be no volunteers for that, I will EOL those branches in networking-ovn. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-05-31-14.00.log.html#l-25 Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim+openstack.org at coote.org Wed Jun 1 13:43:36 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Wed, 1 Jun 2022 14:43:36 +0100 Subject: Novice question In-Reply-To: References: <83EE0BBE-7FE2-4815-912F-A6383A0FFFA8@coote.org> Message-ID: <93811472-C43D-4A13-9B21-A3E53F953B51@coote.org> Thanks, Sean. Unfortunately, this is a small (16GB x86). I did respond with more error messages, but I think they?re probably too obtuse. > On 30 May 2022, at 12:53, Sean Mooney wrote: > > On Mon, 2022-05-30 at 12:06 +0100, tim+openstack.org at coote.org wrote: >> Thanks, Michael. Very reassuring. I?ll have a look and comment back. > > if you have an m1 mac i woudl suggest using utm with the ubuntu 20.04 image > https://mac.getutm.app/gallery/ubuntu-20-04 > to create a qemu vm whihc will use macos's hypervirio api to hardware acclearte > the l1 vm. the l2 vms will still use qemu but by using arm based images for the host os > you can get pretty good perforamce in teh vm and spin up arm based cirrous iamge with nova and get ok performance. > it will be slower then nested virt but the apple silicon macs dont support that a the hardware level. > > > i have mostly for my own use being developing > https://github.com/SeanMooney/ansible_role_devstack > im probably goign to rename that to "ard" by the way in the future if that link does nto work later. > > this repo currently has ansible playbooks that will use the upstream devstack roles we use in ci to deploy multi node devstack > it can create vms using molecule/vagrant and then run the devstack install. > > so on a linux laptop you can do > > bootstrap-repo.sh > molecule create > molecule converge > and that will create a 2 node openstack based on centos 9 stream with master devstack installed and deployed. > > currently the molecule role creates two 8gb vms but i have an example of using it to deploy onto externally provisioned host > as a pr https://github.com/SeanMooney/ansible_role_devstack/pull/4/files > > if you continue to have troble deploying devstack by hand perhaps that will be of use to you. > > it not really ready for primetime/general use but if people find it intersting ye are welcome to fork and use as ye see fit. > the molecule template seamed to set the licence to "BSD" by default. i was plaaing ot make it apache but i guess bsd > is just as good so i shoudl fix that. > anyway i just wanted to point out that UTM is proably beter the virtual box form a performace point of view. > >> >>> On 27 May 2022, at 20:11, Michael Johnson wrote: >>> >>> Hi Tim, >>> >>> This should work fine. You will want a localrc/local.conf file to >>> configure devstack. I didn't see that mentioned in your steps. >>> See this section in the docs: >>> https://docs.openstack.org/devstack/latest/#create-a-local-conf >>> >>> The only caveat I would mention is the VM instances in Nova will run >>> super slow on virtualbox as it lacks the required "nested >>> virtualization" support and will run them all in software emulation. >>> >>> To find the root cause of the issue in nova, I would look through the >>> devstack at n-cpu log file (journal -u devstack at n-cpui) and the >>> devstack at n-sch logs. >>> >>> Also, you might have a look at one of the nova test localrc file as an example: >>> https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_da7/843127/6/check/tempest-integrated-compute-centos-9-stream/da7bebc/controller/logs/local_conf.txt >>> >>> Michael >>> >>> >>> On Fri, May 27, 2022 at 4:16 AM wrote: >>>> >>>> Hullo >>>> >>>> Na?ve question follows. Sorry. >>>> >>>> I?m trying a minimal OS install on a Virtualbox VM on a mac. I?d like to get to the point where I can launch a compute node. I?ve failed with `packstack` and I?m trying `devstack`. >>>> >>>> Should this work out of the box: ie >>>> >>>> Spin up a vm with vagrant: I?m using Centos Stream 9 to ensure that I get a current(ish) version of Python. It has 9GB RAM >>>> Ensure that SELinux and firewalls are turned off >>>> Clone devstack, cd to the directory and run `stack.sh` as user `vagrant` (this fails 1/3 of the time as some repo or other resets a connection. `stack.sh` doesn?t seem to be idempotent as reinvoking it may or may not install and run the OS environment >>>> Upload a ssh keypair through the web interface >>>> Use the web interface to launch the m1.nano flavor with Cirros image (I think that this flavor is quite new as some of the documentation refers to creating such a flavor with 64MB, whereas this one has 128MB. I did try the 64MB route [with `packstack`] and concluded that at least 96MB was needed and the documentation was wrong. I couldn?t log into launchpad.net to report this ? >>>> At this point the launch process fails with the error message: ?Build of instance 157bfa1d-7f8c-4a6c-ba3a-b02fb4f4b6a9 aborted: Failed to allocate the network(s), not rescheduling.? In the web ui >>>> >>>> >>>> Afaict, the vm has enough memory (just: it?s using a bit of swap, but more cache, so it could reclaim that). I?d expected the instance to launch, and I can well believe that I?ve missed something, but the documentation seems to point all over the place for various logs. >>>> >>>> Should this approach work? Is there an alternative that?s better (e.g. use Ubuntu: I?m not keen on apt/dpkg/.deb based distros as I?ve been tripped up in the past over the dependency handling and systemd integration, so I?ve avoided this, but I can see that Canonical is spending money on OS. But so is IBM/Redhat)? Where can I find info on how to trouble shoot the failing process? >>>> >>>> tia >>>> Tim >> >> > From radoslaw.piliszek at gmail.com Wed Jun 1 15:53:14 2022 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Wed, 1 Jun 2022 17:53:14 +0200 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? Message-ID: Hello Fellow OpenStackers, I am wondering if and how OpenStackers are approaching the subject of VDI (Virtual Desktop Infrastructure), also nowadays commonly called DaaS (Desktop as a Service) offering. To be more specific, I am looking forward to a solution covering both Linux and Windows desktops, pre-OS screen access optional, well-integrated (i.e., having automation) with OpenStack Nova, preferably also with Cinder and Glance, optionally with Keystone (apart from self-auth of course!) and Heat, 3D acceleration optional (most of the desktops here don't need it), nice to have read-only sharing of screen. A search brought me to vdi-broker [1] [2] with Guacamole. [3] But the vdi-broker seems dead with no documentation whatsoever. I know Guacamole but normally it offers only static definition of connections to existing machines (VNC or RDP). Looking forward to hearing (reading :D ) your experience with VDI/DaaS on an OpenStack cloud offering. I am also open to collaboration should any of you share the same interest. We can have a relevant SIG created if that is the case. PS: Sadly, I am not able to attend the Berlin summit due to other commitments, so can't ask in person. ;-( [1] https://github.com/cloudbase/vdi-broker [2] https://www.openstack.org/videos/boston-2017/virtual-desktop-infrastructure-vdi-with-openstack [3] https://guacamole.apache.org/ Kind regards, -yoctozepto From haiwu.us at gmail.com Wed Jun 1 16:18:47 2022 From: haiwu.us at gmail.com (hai wu) Date: Wed, 1 Jun 2022 11:18:47 -0500 Subject: [keystone] how to use v3tokenlessauth Message-ID: I could not find any example/usage of v3tokenlessauth, using environment variables. There are no documents to be found. I assume we could need to have this one at least: export OS_AUTH_TYPE="v3tokenlessauth" What other environment variables are required if using this auth type? Would like to use HTTPs x509 auth. This is the only relevant doc found: https://docs.openstack.org/keystone/train/admin/configure_tokenless_x509.html [keystone_authtoken] memcached_servers = localhost:11211 cafile = /etc/keystone/ca.pem project_domain_name = Default project_name = service auth_url = https://192.168.0.10/identity/v3 auth_type = v3tokenlessauth certfile = /etc/glance/certs/glance.pem keyfile = /etc/glance/private/glance_private_key.pem But there are no environment variables mentioned in the above doc. From cboylan at sapwetik.org Wed Jun 1 17:55:08 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Wed, 01 Jun 2022 10:55:08 -0700 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting Message-ID: Hello, Last week the OpenDev team upgraded Zuul which deployed zuul-executors with openstacksdk==0.99.0 installed. After this, test job logs uploaded to the rackspace swift location were no longer viewable in the Zuul dashboard. The reason for this is these objects did not have properly configured CORS headers any longer. The objects uploaded to OVH appear to have been fine (potentially because the headers are not required for OVH). Today we downgraded the version of openstacksdk on the zuul-executors to 0.61.0 and the rackspace uploaded job log objects have functional CORS headers again. For this reason we're fairly confident that something about the 0.99.0 release is impacting the setting of these headers. The code that does these uploads with openstacksdk can be found here [0]. I suspect that the problem is going to be related to this header set on line 227 [1]. In particular that appears to be rackspace specific and this issue is rackspace specific under 0.99.0. However, I wasn't able to find anything in openstacksdk that would have a problem with this. Does anyone else have ideas? I'm somewhat concerned that this represents a class of data loss (metadata is going missing) for swift object uploads. Additionally, we rely on setting the x-delete-after header to timeout, expire, and prune our logs in swift [2]. If there is a general problem with 0.99.0 setting headers on objects it is possible that all of the objects we created using 0.99.0 do not have this metadata set and will leak in their swift containers. [0] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py [1] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L226-L227 [2] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L224 From mtomaska at redhat.com Wed Jun 1 19:21:24 2022 From: mtomaska at redhat.com (Miro Tomaska) Date: Wed, 1 Jun 2022 14:21:24 -0500 Subject: [all][tc] Lets talk about Flake8 E501 In-Reply-To: References: Message-ID: Thank you all for responding and sharing your thoughts. It seems that although the longer limit might be nice it will bring more complications than we want to. For that reason we can consider this discussion as closed. That is, there will be no chances. Thanks again Miro On Tue, May 31, 2022 at 8:22 AM Sylvain Bauza wrote: > > > Le sam. 28 mai 2022 ? 04:36, Miro Tomaska a ?crit : > >> Hello All, >> >> This is probably going to be a hot topic but I was wondering if the >> community ever considered raising the default 79 characters line limit. I >> have seen some places where even a very innocent line of code needs to be >> split into two lines. I have also seen some code where I feel like variable >> names were abbreviated on purpose to squeeze everything into one line. >> >> How does the community feel about raising the E501 limit to 119 >> characters? The 119 character limit is the second most popular limit >> besides the default one. It's long enough to give a developer enough room >> for descriptive variables without being forced to break lines too much. And >> it is short enough for a diff between two files to look OK. >> >> The only downside I can see right now is that it's not easy to convert an >> existing code. So we will end up with files where the new code is 79+ >> characters and the "old" code is <=79. I can also see an argument where >> someone might have trouble reviewing a patch on a laptop screen (assuming >> standard 14" screen) ? >> >> Here is an example of one extreme, a diff of two files maxing out at 119 >> characters >> >> https://review.opendev.org/c/opendev/sandbox/+/843697/1..2 >> >> Thank you for your time and I am looking forward to this conversation :) >> >> > > My personal opinion is while I understand the reasoning behind, my concern > is about the community support. We don't have a lot of contributors at the > moment, and if we would like to modify the default limit, I'm pretty sure > it would take a lot of time for folks at least reviewing some changes that > are not really priorities... > > Also, as said by some folks, changing the default would also create some > problems for some of our contributors and eventually, if some projects > would accept 119 characters while some others not, it would also be a new > community difference between projects... (and yet again some new nit that > new contributors need to understand) > > Sorry, -1 for those reasons. > > > > -- >> Miro Tomaska >> irc: mtomaska >> Red Hat >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Wed Jun 1 20:51:53 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 01 Jun 2022 22:51:53 +0200 Subject: [all][tc] Lets talk about Flake8 E501 In-Reply-To: References: Message-ID: <3430337.j8lITm4QUA@p1> Hi, Dnia wtorek, 31 maja 2022 15:21:50 CEST Sylvain Bauza pisze: > Le sam. 28 mai 2022 ? 04:36, Miro Tomaska a ?crit : > > > Hello All, > > > > This is probably going to be a hot topic but I was wondering if the > > community ever considered raising the default 79 characters line limit. I > > have seen some places where even a very innocent line of code needs to be > > split into two lines. I have also seen some code where I feel like variable > > names were abbreviated on purpose to squeeze everything into one line. > > > > How does the community feel about raising the E501 limit to 119 > > characters? The 119 character limit is the second most popular limit > > besides the default one. It's long enough to give a developer enough room > > for descriptive variables without being forced to break lines too much. And > > it is short enough for a diff between two files to look OK. > > > > The only downside I can see right now is that it's not easy to convert an > > existing code. So we will end up with files where the new code is 79+ > > characters and the "old" code is <=79. I can also see an argument where > > someone might have trouble reviewing a patch on a laptop screen (assuming > > standard 14" screen) ? > > > > Here is an example of one extreme, a diff of two files maxing out at 119 > > characters > > > > https://review.opendev.org/c/opendev/sandbox/+/843697/1..2 > > > > Thank you for your time and I am looking forward to this conversation :) > > > > > > My personal opinion is while I understand the reasoning behind, my concern > is about the community support. We don't have a lot of contributors at the > moment, and if we would like to modify the default limit, I'm pretty sure > it would take a lot of time for folks at least reviewing some changes that > are not really priorities... > > Also, as said by some folks, changing the default would also create some > problems for some of our contributors and eventually, if some projects > would accept 119 characters while some others not, it would also be a new > community difference between projects... (and yet again some new nit that > new contributors need to understand) > > Sorry, -1 for those reasons. I don't have strong opinion about it but I see that e.g. You are saying that this should be changed (or stay like it is now) globally for all projects but earlier fungi said that there wasn't such requirement from e.g. TC to have this configured in same way for all projects. So IIUC every project can actually change it on their own if team wants to do that. Personally I agree with Your point that it should be done for all projects in the same way so maybe TC should have some guidance/rules for that to make sure all projects will follow the same pattern. Wdyt? > > > > -- > > Miro Tomaska > > irc: mtomaska > > Red Hat > > > -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From fungi at yuggoth.org Wed Jun 1 21:41:06 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 1 Jun 2022 21:41:06 +0000 Subject: [all][tc] Lets talk about Flake8 E501 In-Reply-To: <3430337.j8lITm4QUA@p1> References: <3430337.j8lITm4QUA@p1> Message-ID: <20220601214105.ss75dzoidbbfxst3@yuggoth.org> On 2022-06-01 22:51:53 +0200 (+0200), Slawek Kaplonski wrote: [...] > Personally I agree with Your point that it should be done for all > projects in the same way so maybe TC should have some > guidance/rules for that to make sure all projects will follow the > same pattern. Wdyt? Given that the vast majority of the code in OpenStack is consistent on this point even in the absence of any requirement on the part of the TC, I take it as an indication that having the TC make a rule about it would be a redundant time-sink when there's plenty else for them to be working on. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From andy at andybotting.com Wed Jun 1 23:10:22 2022 From: andy at andybotting.com (Andy Botting) Date: Thu, 2 Jun 2022 09:10:22 +1000 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: Hi Rados?aw, > I am wondering if and how OpenStackers are approaching the subject of > VDI (Virtual Desktop Infrastructure), also nowadays commonly called > DaaS (Desktop as a Service) offering. > To be more specific, I am looking forward to a solution covering both > Linux and Windows desktops, pre-OS screen access optional, > well-integrated (i.e., having automation) with OpenStack Nova, > preferably also with Cinder and Glance, optionally with Keystone > (apart from self-auth of course!) and Heat, 3D acceleration optional > (most of the desktops here don't need it), nice to have read-only > sharing of screen. We just launched a service like this on the Nectar Research Cloud. Like you, I looked around but couldn't find any Open Source projects that would be suitable. In the end, we based ours on a project from the University of Melbourne that did initially support Windows, but we removed some of that support to simplify our codebase as we didn't need it. We called it Bumblebee, and the code is here: https://github.com/NeCTAR-RC/bumblebee It's a Django web app with a Redis-based task scheduler that launches Nova instances, booted from a Cinder volume, cloned from a base OS image. The Django app directly modifies Guacamole's database to create the connections once the OpenStack resources have been provisioned and provides the user with a link that takes them to their desktop. We used Keycloak with OIDC for authentication on the web app and Guacamole and it's fairly seamless transitioning between the two. We did initially look at using Heat, but we wanted to support a workflow of being able to destroy the VM and archive the OS image to Swift (and restore again later if needed) so it was easier to just manage the resources directly. There's no 3D acceleration in our solution yet, but we're planning on looking at things like virtual GPUs later on. Sadly, I'm not going to make it to Berlin either, but I was thinking it might make for a good talk at the next summit. I'm in the process of properly documenting it now, so I don't have much to share at this point, but I'm happy to go into more detail about anything. cheers, Andy From juliaashleykreger at gmail.com Wed Jun 1 23:17:12 2022 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 1 Jun 2022 16:17:12 -0700 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: This sounds super interesting. While not a VDI user, I can see a huge appeal. Maybe this might be an interesting topic for OpenInfra Live? https://openinfra.dev/live/ On Wed, Jun 1, 2022 at 4:14 PM Andy Botting wrote: > > Hi Rados?aw, > > > I am wondering if and how OpenStackers are approaching the subject of > > VDI (Virtual Desktop Infrastructure), also nowadays commonly called > > DaaS (Desktop as a Service) offering. > > To be more specific, I am looking forward to a solution covering both > > Linux and Windows desktops, pre-OS screen access optional, > > well-integrated (i.e., having automation) with OpenStack Nova, > > preferably also with Cinder and Glance, optionally with Keystone > > (apart from self-auth of course!) and Heat, 3D acceleration optional > > (most of the desktops here don't need it), nice to have read-only > > sharing of screen. > > We just launched a service like this on the Nectar Research Cloud. > Like you, I looked around but couldn't find any Open Source projects > that would be suitable. > > In the end, we based ours on a project from the University of > Melbourne that did initially support Windows, but we removed some of > that support to simplify our codebase as we didn't need it. > > We called it Bumblebee, and the code is here: > https://github.com/NeCTAR-RC/bumblebee > > It's a Django web app with a Redis-based task scheduler that launches > Nova instances, booted from a Cinder volume, cloned from a base OS > image. > > The Django app directly modifies Guacamole's database to create the > connections once the OpenStack resources have been provisioned and > provides the user with a link that takes them to their desktop. > > We used Keycloak with OIDC for authentication on the web app and > Guacamole and it's fairly seamless transitioning between the two. > > We did initially look at using Heat, but we wanted to support a > workflow of being able to destroy the VM and archive the OS image to > Swift (and restore again later if needed) so it was easier to just > manage the resources directly. > > There's no 3D acceleration in our solution yet, but we're planning on > looking at things like virtual GPUs later on. > > Sadly, I'm not going to make it to Berlin either, but I was thinking > it might make for a good talk at the next summit. > > I'm in the process of properly documenting it now, so I don't have > much to share at this point, but I'm happy to go into more detail > about anything. > > cheers, > Andy > From andy at andybotting.com Thu Jun 2 02:27:41 2022 From: andy at andybotting.com (Andy Botting) Date: Thu, 2 Jun 2022 12:27:41 +1000 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: > This sounds super interesting. While not a VDI user, I can see a huge appeal. > > Maybe this might be an interesting topic for OpenInfra Live? > https://openinfra.dev/live/ Yeah, I could be down for that. Live demos are the best From ueha.ayumu at fujitsu.com Thu Jun 2 08:38:43 2022 From: ueha.ayumu at fujitsu.com (ueha.ayumu at fujitsu.com) Date: Thu, 2 Jun 2022 08:38:43 +0000 Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team In-Reply-To: References: Message-ID: Hi Rico, This proposal is great helpful for us! There seems to be no particular objection, so could you add it to heat-translator-core if there is no problem? Thanks. Best Regards, Ueha -----Original Message----- From: ueha.ayumu at fujitsu.com Sent: Wednesday, May 25, 2022 1:40 PM To: openstack-discuss at lists.openstack.org Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team +1 Best Regards, Ueha -----Original Message----- From: Yoshito Ito Sent: Wednesday, May 25, 2022 12:31 PM To: openstack-discuss at lists.openstack.org Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team Hi heat-translator team! I would like to propose Hiromu Asahina (h-asahina) as a member of the heat-translator core team. I'm now not able to actively contribute to heat-translator, and he can lead the team instead of me. He is mainly contributing to Tacker by fixing issues, refactoring existing codes, and a lot of helpful reviews, with his experience as a research engineer in NTT. Tacker highly depends on heat-translator in its main functionality, so I believe he can manage it with his knowledge of Tacker. Please vote for his nomination by answering this mail till June 1st. Best Regards, Yoshito Ito From uemit.seren at gmail.com Thu Jun 2 09:13:31 2022 From: uemit.seren at gmail.com (=?iso-8859-2?Q?=DCmit_Seren?=) Date: Thu, 2 Jun 2022 09:13:31 +0000 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? Message-ID: We are using xpra (https://xpra.org/) together with VirtualGL (https://www.virtualgl.org/) on our virtualized OpenStack based SLURM cluster to support OpenGL accelerated applications such as MorphoGraphX (https://morphographx.org/) for our users. This works pretty well. There was a bit of work involved because it requires an X11 server to run on the compute nodes but once this was done it worked fairly well. Best ?mit On 02.06.22, 01:10, "Andy Botting" wrote: Hi Rados?aw, > I am wondering if and how OpenStackers are approaching the subject of > VDI (Virtual Desktop Infrastructure), also nowadays commonly called > DaaS (Desktop as a Service) offering. > To be more specific, I am looking forward to a solution covering both > Linux and Windows desktops, pre-OS screen access optional, > well-integrated (i.e., having automation) with OpenStack Nova, > preferably also with Cinder and Glance, optionally with Keystone > (apart from self-auth of course!) and Heat, 3D acceleration optional > (most of the desktops here don't need it), nice to have read-only > sharing of screen. We just launched a service like this on the Nectar Research Cloud. Like you, I looked around but couldn't find any Open Source projects that would be suitable. In the end, we based ours on a project from the University of Melbourne that did initially support Windows, but we removed some of that support to simplify our codebase as we didn't need it. We called it Bumblebee, and the code is here: https://github.com/NeCTAR-RC/bumblebee It's a Django web app with a Redis-based task scheduler that launches Nova instances, booted from a Cinder volume, cloned from a base OS image. The Django app directly modifies Guacamole's database to create the connections once the OpenStack resources have been provisioned and provides the user with a link that takes them to their desktop. We used Keycloak with OIDC for authentication on the web app and Guacamole and it's fairly seamless transitioning between the two. We did initially look at using Heat, but we wanted to support a workflow of being able to destroy the VM and archive the OS image to Swift (and restore again later if needed) so it was easier to just manage the resources directly. There's no 3D acceleration in our solution yet, but we're planning on looking at things like virtual GPUs later on. Sadly, I'm not going to make it to Berlin either, but I was thinking it might make for a good talk at the next summit. I'm in the process of properly documenting it now, so I don't have much to share at this point, but I'm happy to go into more detail about anything. cheers, Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.goncharov at gmail.com Thu Jun 2 10:44:36 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Thu, 2 Jun 2022 12:44:36 +0200 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting In-Reply-To: References: Message-ID: Hey, https://review.opendev.org/c/openstack/openstacksdk/+/844414 should address the issue. For R1 of SDK we want to ensure all interfaces of SDK use same code branch and not 3 different like it was before. So with the latest changes we now rely here on the regular mechanism that actually watches how to handle each known parameter. All of the arbitrary parameters in Swift are managed as container/object metadata and should get X-Container-Meta- or X-Object-Meta- prefixes in order to be considered and returned back by manila Swift. This is sadly not the case for Rax as you rightfully mention. So for now I have added header required by Rax into ?known? exceptions and added explicit unit test for that. Would be cool to find way how I could test that really in Rax to ensure particular Zuul use case will work. Generally we would need to think whether SDK should continue ?tell user he is wrong before reaching out to the server? approach or to switch to the ?believe the user and let server reject? approach generally. This here is precisely one of those cases. Alternatively we would need to widen our concept of vendors and move all of the specific Rax logic over there. We surely have lot of OpenStack based clouds, which do not explicitly implement everything in the compatible way. So in order to make SDK/CLI working with all of them we either drop most of the validation logic in SDK or really start implementing cloud specific overrides as part of ?extensions?. Artem > On 1. Jun 2022, at 19:55, Clark Boylan wrote: > > Hello, > > Last week the OpenDev team upgraded Zuul which deployed zuul-executors with openstacksdk==0.99.0 installed. After this, test job logs uploaded to the rackspace swift location were no longer viewable in the Zuul dashboard. The reason for this is these objects did not have properly configured CORS headers any longer. The objects uploaded to OVH appear to have been fine (potentially because the headers are not required for OVH). > > Today we downgraded the version of openstacksdk on the zuul-executors to 0.61.0 and the rackspace uploaded job log objects have functional CORS headers again. For this reason we're fairly confident that something about the 0.99.0 release is impacting the setting of these headers. > > The code that does these uploads with openstacksdk can be found here [0]. I suspect that the problem is going to be related to this header set on line 227 [1]. In particular that appears to be rackspace specific and this issue is rackspace specific under 0.99.0. However, I wasn't able to find anything in openstacksdk that would have a problem with this. Does anyone else have ideas? I'm somewhat concerned that this represents a class of data loss (metadata is going missing) for swift object uploads. > > Additionally, we rely on setting the x-delete-after header to timeout, expire, and prune our logs in swift [2]. If there is a general problem with 0.99.0 setting headers on objects it is possible that all of the objects we created using 0.99.0 do not have this metadata set and will leak in their swift containers. > > [0] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py > [1] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L226-L227 > [2] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L224 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rg at raghavgururajan.name Thu Jun 2 11:22:41 2022 From: rg at raghavgururajan.name (Raghav Gururajan) Date: Thu, 02 Jun 2022 11:22:41 +0000 Subject: Resources to learn IT Infrastructure Message-ID: Hello All, I am stepping into infrastructure side of the IT world and was wondering if any of you could recommend best books, materials or videos for learning? Regards, Raghav "RG" Gururajan. From gmann at ghanshyammann.com Thu Jun 2 14:22:16 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 02 Jun 2022 09:22:16 -0500 Subject: [all][tc] Technical Committee next weekly meeting on June 2, 2022 at 1500 UTC In-Reply-To: <18116abf909.bcbc5c9a267396.2648997117806913882@ghanshyammann.com> References: <18116abf909.bcbc5c9a267396.2648997117806913882@ghanshyammann.com> Message-ID: <18124cbad36.10270d18c455377.5625384948074577891@ghanshyammann.com> Hello Everyone, Below is the agenda for Today's TC IRC meeting schedule at 1500 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting == Agenda for tomorrow's TC meeting == * Roll call * Follow up on past action items * Gate health check ** Fixing Zuul config error in OpenStack *** https://etherpad.opendev.org/p/zuul-config-error-openstack * New ELK service dashboard: e-r service ** https://opensearch.logs.openstack.org/_dashboards/app/discover?security_tenant=global ** https://review.opendev.org/c/openstack/governance-sigs/+/835838 * RBAC feedback in ops meetup ** https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L53 * TC + Board meeting * Open Reviews ** https://review.opendev.org/q/projects:openstack/governance+is:open -gmann ---- On Mon, 30 May 2022 15:32:58 -0500 Ghanshyam Mann wrote ---- > Hello Everyone, > > The technical Committee's next weekly meeting is scheduled for June 2, 2022, at 1500 UTC. > > If you would like to add topics for discussion, please add them to the below wiki page by > Wednesday, June 1, at 2100 UTC. > > https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting > > -gmann > > > From fungi at yuggoth.org Thu Jun 2 14:22:54 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 2 Jun 2022 14:22:54 +0000 Subject: [dev][docs][security-sig] Retiring security-analysis process and repo Message-ID: <20220602142254.k2j3uctxkoiumwv7@yuggoth.org> In 2016, what was then the Security Project Team embarked on an effort to centrally collect security analyses of OpenStack components. It accumulated a total of two, one for Barbican as of the Newton release, and another for KeystoneMiddleware as of Pike. The latter was finalized in 2017 and took nearly a year to merge due to already waning enthusiasm and reviewer availability: https://docs.openstack.org/security-analysis Given this effort was effectively abandoned years ago, the Security SIG members agree that the repository should be retired in order to reduce confusion. The vulnerability management oversight requirements were amended in February to remove any reference to this process, and we reached a consensus that this sort of documentation is better off inside the projects which are writing it rather than collected centrally with a disconnected (or in this case absent) group of reviewers and maintainers. This message serves as notice to the community that we will be pushing changes to follow the usual OpenStack repository retirement process for openstack/security-analysis in the coming days. As usual, the final state of the documents will be found in the parent commit of the one which wipes all the old files, but for posterity I'll link it here as well: https://opendev.org/openstack/security-analysis/src/commit/ac43025 Many thanks to those who attempted to provide and review these analyses in years past. The idea of maintaining information on the security risks and considerations for the systems we design is still a good one, and something I hope our contributor community might find more time to focus on in the years to come; but the place to document those things is right alongside the rest of the software's documentation, there's nothing inherently special or different about it. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From elod.illes at est.tech Thu Jun 2 14:58:07 2022 From: elod.illes at est.tech (=?UTF-8?B?RWzFkWQgSWxsw6lz?=) Date: Thu, 2 Jun 2022 16:58:07 +0200 Subject: [release] Release countdown for week R-17, Jun 06 - 10 Message-ID: <9192a345-1a05-dcf3-7ca1-f1d4fb0e4908@est.tech> Development Focus ----------------- The Zed-2 milestone will happen in next month, on July 14th, 2022. Zed-related specs should now be finalized so that teams can move to implementation ASAP. Some teams observe specific deadlines on the second milestone (mostly spec freezes): please refer to https://releases.openstack.org/ zed /schedule.html for details. General Information ------------------- Please remember that libraries need to be released at least once per milestone period. At milestone 2, the release team will propose releases for any library that has not been otherwise released since milestone 1. Other non-library deliverables that follow the cycle-with-intermediary release model should have an intermediary release before milestone-2. Those who haven't will be proposed to switch to the cycle-with-rc model, which is more suited to deliverables that are released only once per cycle. At milestone-2 we also freeze the contents of the final release. If you have a new deliverable that should be included in the final release, you should make sure it has a deliverable file in: https://opendev.org/openstack/releases/src/branch/master/deliverables/ zed You should request a beta release (or intermediary release) for those new deliverables by milestone-2. We understand some may not be quite ready for a full release yet, but if you have something minimally viable to get released it would be good to do a 0.x release to exercise the release tooling for your deliverables. See the MembershipFreeze description for more details: https://releases.openstack.org/ zed /schedule.html# z -mf Finally, now may be a good time for teams to check on any stable releases that need to be done for your deliverables. If you have bugfixes that have been backported, but no stable release getting those. If you are unsure what is out there committed but not released, in the openstack/releases repo, running the command "tools/list_stable_unreleased_changes.sh " gives a nice report. Upcoming Deadlines & Dates -------------------------- Zed-2 Milestone: July 14th, 2022 OpenInfra Summit: June 7-9, 2022 (Berlin) El?d Ill?s irc: elodilles -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Thu Jun 2 16:22:25 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Thu, 02 Jun 2022 09:22:25 -0700 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting In-Reply-To: References: Message-ID: <315f833d-2b00-4f01-8ccc-172d7bf7b576@www.fastmail.com> On Thu, Jun 2, 2022, at 3:44 AM, Artem Goncharov wrote: > Hey, > > https://review.opendev.org/c/openstack/openstacksdk/+/844414 should > address the issue. For R1 of SDK we want to ensure all interfaces of > SDK use same code branch and not 3 different like it was before. So > with the latest changes we now rely here on the regular mechanism that > actually watches how to handle each known parameter. All of the > arbitrary parameters in Swift are managed as container/object metadata > and should get X-Container-Meta- or X-Object-Meta- prefixes in order to > be considered and returned back by manila Swift. This is sadly not the > case for Rax as you rightfully mention. Note that the docs [3] still say that headers are passed through, which isn't quite the case in 0.99.0. Filtered headers for what the SDK thinks are valid are passed through. This change should probably update the docs too. > > So for now I have added header required by Rax into ?known? exceptions > and added explicit unit test for that. Would be cool to find way how I > could test that really in Rax to ensure particular Zuul use case will > work. Probably the simplest thing is to have a test that checks the right headers are sent. Then if/when clouds change behavior we can update the tests to follow. > > Generally we would need to think whether SDK should continue ?tell user > he is wrong before reaching out to the server? approach or to switch to > the ?believe the user and let server reject? approach generally. This > here is precisely one of those cases. Alternatively we would need to > widen our concept of vendors and move all of the specific Rax logic > over there. We surely have lot of OpenStack based clouds, which do not > explicitly implement everything in the compatible way. So in order to > make SDK/CLI working with all of them we either drop most of the > validation logic in SDK or really start implementing cloud specific > overrides as part of ?extensions?. The problem with telling the user they are wrong before talking to the server is that it assumes that clouds don't do weird things. History has shown us that cloud definitely do weird things and this isn't related to any single cloud. Off the top of my head I can come up with a few examples and if we look through the git history of nodepool and opendev system-config I'm sure we'll find many many more occurences. This isn't a specific cloud problem either; they all do it one way or another. It is my opinion that tools like openstacksdk should be forgiving in what they accept to enable users to use openstack in its many forms. Shade is a great example of this. Eventually shade was rolled into openstacksdk and doesn't really exist anymore. It feels like the spirit of shade has been lost though and we're regressing back to 2014 when you couldn't talk to two different clouds without 4 different client tools. Similarly support for old versions of APIs have been ripped out of these tools. Unfortunately some clouds are still deploying old versions of clouds and ideally as a user I wouldn't need to think too hard about that. It isn't my fault this is the situation, I just want the software to work. > > Artem > >> On 1. Jun 2022, at 19:55, Clark Boylan wrote: >> >> Hello, >> >> Last week the OpenDev team upgraded Zuul which deployed zuul-executors with openstacksdk==0.99.0 installed. After this, test job logs uploaded to the rackspace swift location were no longer viewable in the Zuul dashboard. The reason for this is these objects did not have properly configured CORS headers any longer. The objects uploaded to OVH appear to have been fine (potentially because the headers are not required for OVH). >> >> Today we downgraded the version of openstacksdk on the zuul-executors to 0.61.0 and the rackspace uploaded job log objects have functional CORS headers again. For this reason we're fairly confident that something about the 0.99.0 release is impacting the setting of these headers. >> >> The code that does these uploads with openstacksdk can be found here [0]. I suspect that the problem is going to be related to this header set on line 227 [1]. In particular that appears to be rackspace specific and this issue is rackspace specific under 0.99.0. However, I wasn't able to find anything in openstacksdk that would have a problem with this. Does anyone else have ideas? I'm somewhat concerned that this represents a class of data loss (metadata is going missing) for swift object uploads. >> >> Additionally, we rely on setting the x-delete-after header to timeout, expire, and prune our logs in swift [2]. If there is a general problem with 0.99.0 setting headers on objects it is possible that all of the objects we created using 0.99.0 do not have this metadata set and will leak in their swift containers. >> >> [0] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py >> [1] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L226-L227 >> [2] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L224 >> [3] https://docs.openstack.org/openstacksdk/latest/user/connection.html#openstack.connection.Connection.create_object From iurygregory at gmail.com Thu Jun 2 16:34:45 2022 From: iurygregory at gmail.com (Iury Gregory) Date: Thu, 2 Jun 2022 13:34:45 -0300 Subject: Fwd: [Forum] Meet the Projects Session! In-Reply-To: <4286047.ni3Bx2OuKx@p1> References: <932f56ec-a9f9-eea0-c3d5-69be9e185397@debian.org> <4286047.ni3Bx2OuKx@p1> Message-ID: Hi Kendall, I will be there, just added to my schedule =) Em qua., 1 de jun. de 2022 ?s 05:40, Slawek Kaplonski escreveu: > Hi, > > Dnia ?roda, 1 czerwca 2022 10:06:34 CEST Thomas Goirand pisze: > > On 5/31/22 17:53, Kendall Nelson wrote: > > > Hello :) > > > > > > I wanted to take a moment to invite project maintainers, core > > > contributors, PTLs, governance officials, etc to a meet the projects > > > (projects being the OIF top level ones- Kata, Zuul, OpenStack, > > > StarlingX, and Airship) session during the Forum at the upcoming > Summit > > > in Berlin. It will take place Tue, June 7, 11:20am - 12:30pm local in > A > > > - A06. > > > > > > The idea is to gather in a single place so that newer members of the > > > community or people unfamiliar with the project can come and meet some > > > of the active participants and ask questions. It's all really informal > > > so there is no need to prepare anything ahead of time. > > > > > > I realize it is pretty late in everyone's planning their schedules for > > > the summit, but if you have a few minutes to spare to come hang out > and > > > meet some people and network, please do! We would love to see you > there. > > > > > > -Kendall > > > > Hi, > > > > Thanks a lot for this proposal, I very much like the idea of meeting > > (new) project members in real. I definitively will join and hope to > > match faces with IRC nicks after the meeting. > > > > Cheers, > > > > Thomas Goirand (zigo) > > > > > > Yeah. Thx for organizing this Kendal. I already added it to my calendar > and will be there too :) > > -- > Slawek Kaplonski > Principal Software Engineer > Red Hat -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the ironic-core and puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From iurygregory at gmail.com Thu Jun 2 17:23:51 2022 From: iurygregory at gmail.com (Iury Gregory) Date: Thu, 2 Jun 2022 14:23:51 -0300 Subject: [ironic] No weekly meeting next week (June 6) Message-ID: Hello ironicers, As you probably already know, next week is the Open Infrastructure Summit in Berlin and some of us will be at the event. We will resume our weekly meeting on June 13. Thanks! -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic PTL* *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.goncharov at gmail.com Thu Jun 2 19:12:33 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Thu, 2 Jun 2022 21:12:33 +0200 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting In-Reply-To: <315f833d-2b00-4f01-8ccc-172d7bf7b576@www.fastmail.com> References: <315f833d-2b00-4f01-8ccc-172d7bf7b576@www.fastmail.com> Message-ID: <27B7F0D2-E382-49EF-B518-1764D6F83344@gmail.com> Hey Clark, Pls find comments below. I need to justify the product that I maintain, but I perfectly understand you. > On 2. Jun 2022, at 18:22, Clark Boylan wrote: > > On Thu, Jun 2, 2022, at 3:44 AM, Artem Goncharov wrote: >> Hey, >> >> https://review.opendev.org/c/openstack/openstacksdk/+/844414 should >> address the issue. For R1 of SDK we want to ensure all interfaces of >> SDK use same code branch and not 3 different like it was before. So >> with the latest changes we now rely here on the regular mechanism that >> actually watches how to handle each known parameter. All of the >> arbitrary parameters in Swift are managed as container/object metadata >> and should get X-Container-Meta- or X-Object-Meta- prefixes in order to >> be considered and returned back by manila Swift. This is sadly not the >> case for Rax as you rightfully mention. > > Note that the docs [3] still say that headers are passed through, which isn't quite the case in 0.99.0. Filtered headers for what the SDK thinks are valid are passed through. This change should probably update the docs too. Partially yes, partially no. Documentation says it will pass them and the SW does that. It passes, however, only what it know would be accepted. I see the point here and will try to address this, but the issue is that we have generally a mixture of header and metadata props (which are implemented as headers). I will try to reimplement current handling to be able to guess better what user means. > >> >> So for now I have added header required by Rax into ?known? exceptions >> and added explicit unit test for that. Would be cool to find way how I >> could test that really in Rax to ensure particular Zuul use case will >> work. > > Probably the simplest thing is to have a test that checks the right headers are sent. Then if/when clouds change behavior we can update the tests to follow. This is how I implemented that, but it is only a unit test. And here I was rather wondering if there would be possibility to have func test for that to save all of us from another analysis sessions on what the heck is going on right now. Point is that we only know that cloud changed behaviour by noticing Zuul fails now (in similar case to the current issue). > >> >> Generally we would need to think whether SDK should continue ?tell user >> he is wrong before reaching out to the server? approach or to switch to >> the ?believe the user and let server reject? approach generally. This >> here is precisely one of those cases. Alternatively we would need to >> widen our concept of vendors and move all of the specific Rax logic >> over there. We surely have lot of OpenStack based clouds, which do not >> explicitly implement everything in the compatible way. So in order to >> make SDK/CLI working with all of them we either drop most of the >> validation logic in SDK or really start implementing cloud specific >> overrides as part of ?extensions?. > > The problem with telling the user they are wrong before talking to the server is that it assumes that clouds don't do weird things. History has shown us that cloud definitely do weird things and this isn't related to any single cloud. Off the top of my head I can come up with a few examples and if we look through the git history of nodepool and opendev system-config I'm sure we'll find many many more occurences. This isn't a specific cloud problem either; they all do it one way or another. > > It is my opinion that tools like openstacksdk should be forgiving in what they accept to enable users to use openstack in its many forms. Shade is a great example of this. Eventually shade was rolled into openstacksdk and doesn't really exist anymore. It feels like the spirit of shade has been lost though and we're regressing back to 2014 when you couldn't talk to two different clouds without 4 different client tools. > > Similarly support for old versions of APIs have been ripped out of these tools. Unfortunately some clouds are still deploying old versions of clouds and ideally as a user I wouldn't need to think too hard about that. It isn't my fault this is the situation, I just want the software to work. I perfectly see your point. To some extend you are right, but let me describe reasoning for all of those changes we have now: - Zuul is using one interface of SDK - Ansible is using other interface (moreover it mixes 3 different interfaces) - OSC is either not using SDK or using one of 3 mentioned interfaces - Those mentioned interfaces try to do same thing, but eventually differently - Vanila services evolve and require we permanently add new features to all of those interfaces simultaneously - Amount of contributions is incredibly low what makes it impossible to maintain multiple interfaces for doing same job - So what I am trying to do now is to have all of the mentioned tools rely on the single branch in code to be able to maintain it easier and reduce amount of questions like ?hmm, it works if I use OSC, but it doesn?t when I use Ansible. What is wrong??. Target is just to exactly reduce amount of those ?different tools for different tasks and or clouds? as you mention and instead to have a single tool that work for every cloud. We do not take care of shade any more. We try to deprecate all of the individual clients exactly to have a single OSC be able to do everything. And it should just know all of those corner cases. We try to get rid of as much ?reimplementation? from Ansible modules as possible. - Yes, maybe idea of trying to help user to know that he will fail once we reach server was wrong, but you can judge only from experience (when clouds behave differently). Idea as such is not bad, is having its right to live and is generally targeting to be more user friendly than the server response that just returns you 409 with ?good luck figuring out what is wrong?. Here I was spending days trying to find issue in my request with only 409 ?Bad Request? back. And when target clouds are different you as a user are left in even more problems. Generally idea of SDK is to save user from even knowing about differences of clouds he want to talk to. Without that the whole burden is on the user and you need to have all of those cloud hacks directly in your code (is it cool to have all of those hacks inside Zuul and reimplementing them through the different use cases? And repeating the same hacks in OSC and Ansible? Same for all of the user written applications?) - I know this work breaks some corner cases. And I perfectly see the pain my work is causing (even for myself). I have invested huge amount of time in rewriting this and joining code branches and tests. Trust me this was not fun. But unless this is done we wither stuck with old code without possibility to add new features (i.e. micro versions) or we continue living in the mess of tons of different ways to do the same thing with no maintainers. =========== So overall, while I understand all of the issues I am causing to projects right now, I kindly ask, be patient, report broken issues to me and help me fix this insanity. Believe me, I am not doing all of that with bad intents, but rather hoping to resolve issues we have and unify things in the light of low contributions. Otherwise SDK, CLI, Ansible, eventually Zuul are not able to properly keep up with OpenStack services themselves. I listen to complains carefully, and that is the reason for R1.0 not to exist as of now. Artem > >> >> Artem >> >>> On 1. Jun 2022, at 19:55, Clark Boylan wrote: >>> >>> Hello, >>> >>> Last week the OpenDev team upgraded Zuul which deployed zuul-executors with openstacksdk==0.99.0 installed. After this, test job logs uploaded to the rackspace swift location were no longer viewable in the Zuul dashboard. The reason for this is these objects did not have properly configured CORS headers any longer. The objects uploaded to OVH appear to have been fine (potentially because the headers are not required for OVH). >>> >>> Today we downgraded the version of openstacksdk on the zuul-executors to 0.61.0 and the rackspace uploaded job log objects have functional CORS headers again. For this reason we're fairly confident that something about the 0.99.0 release is impacting the setting of these headers. >>> >>> The code that does these uploads with openstacksdk can be found here [0]. I suspect that the problem is going to be related to this header set on line 227 [1]. In particular that appears to be rackspace specific and this issue is rackspace specific under 0.99.0. However, I wasn't able to find anything in openstacksdk that would have a problem with this. Does anyone else have ideas? I'm somewhat concerned that this represents a class of data loss (metadata is going missing) for swift object uploads. >>> >>> Additionally, we rely on setting the x-delete-after header to timeout, expire, and prune our logs in swift [2]. If there is a general problem with 0.99.0 setting headers on objects it is possible that all of the objects we created using 0.99.0 do not have this metadata set and will leak in their swift containers. >>> >>> [0] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py >>> [1] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L226-L227 >>> [2] https://opendev.org/zuul/zuul-jobs/src/commit/e69d879caecb454c529a7d757b80ae49c3caa105/roles/upload-logs-base/library/zuul_swift_upload.py#L224 >>> > > [3] https://docs.openstack.org/openstacksdk/latest/user/connection.html#openstack.connection.Connection.create_object -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Thu Jun 2 19:45:12 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 2 Jun 2022 19:45:12 +0000 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting In-Reply-To: <27B7F0D2-E382-49EF-B518-1764D6F83344@gmail.com> References: <315f833d-2b00-4f01-8ccc-172d7bf7b576@www.fastmail.com> <27B7F0D2-E382-49EF-B518-1764D6F83344@gmail.com> Message-ID: <20220602194511.zkkph5zmasr37jkk@yuggoth.org> On 2022-06-02 21:12:33 +0200 (+0200), Artem Goncharov wrote: [...] > I listen to complains carefully, and that is the reason for R1.0 > not to exist as of now. [...] It's probably also worth pointing out, from a user experience perspective, that we anticipated backward-incompatible changes in 1.0.0 and so merged an openstacksdk<1 pin in nodepool's requirements.txt file. Unfortunately, instead the backward-incompatible changes were released as 0.99.0 so when the container images for our launchers updated we started getting another error back from all of the Rackspace regions we're using (traceback attached). I'm happy to file a bug report on that if nobody has beaten me to it, but further diagnostics will probably have to wait until after summit travel since we've rolled back to 0.61.0 for now. -- Jeremy Stanley -------------- next part -------------- Creating server with hostname ubuntu-bionic-rax-dfw-0029866297 in rax-dfw from image ubuntu-bionic Clearing az, flavor and image caches due to 400 error from nova Launch attempt 1/3 failed: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/nodepool/driver/openstack/handler.py", line 247, in launch self._launchNode() File "/usr/local/lib/python3.9/site-packages/nodepool/driver/openstack/handler.py", line 127, in _launchNode server = self.handler.manager.createServer( File "/usr/local/lib/python3.9/site-packages/nodepool/driver/openstack/provider.py", line 290, in createServer return self._client.create_server(wait=False, **create_args) File "/usr/local/lib/python3.9/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), **kw) File "/usr/local/lib/python3.9/site-packages/openstack/cloud/_utils.py", line 365, in func_wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/openstack/cloud/_compute.py", line 862, in create_server server = self.compute.create_server(**kwargs) File "/usr/local/lib/python3.9/site-packages/openstack/compute/v2/_proxy.py", line 629, in create_server return self._create(_server.Server, **attrs) File "/usr/local/lib/python3.9/site-packages/openstack/proxy.py", line 485, in _create return res.create(self, base_path=base_path) File "/usr/local/lib/python3.9/site-packages/openstack/resource.py", line 1388, in create self._translate_response(response, has_body=has_body) File "/usr/local/lib/python3.9/site-packages/openstack/resource.py", line 1200, in _translate_response exceptions.raise_from_response(response, error_message=error_message) File "/usr/local/lib/python3.9/site-packages/openstack/exceptions.py", line 233, in raise_from_response raise cls( openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://dfw.servers.api.rackspacecloud.com/v2/REDACTED/servers, Bad networks format From artem.goncharov at gmail.com Thu Jun 2 19:54:53 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Thu, 2 Jun 2022 21:54:53 +0200 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting In-Reply-To: <20220602194511.zkkph5zmasr37jkk@yuggoth.org> References: <315f833d-2b00-4f01-8ccc-172d7bf7b576@www.fastmail.com> <27B7F0D2-E382-49EF-B518-1764D6F83344@gmail.com> <20220602194511.zkkph5zmasr37jkk@yuggoth.org> Message-ID: Thanks Jeremy, will try to analyze the trace. This is not too extensive, but better then silent pin without possibility to even notice that. I see that there are generally not enough deep tests for Openstack driver in Zuul since mostly issues appear in Prod. I will to try to invest some time there. Artem ---- typed from mobile, auto-correct typos assumed ---- On Thu, Jun 2, 2022, 21:47 Jeremy Stanley wrote: > On 2022-06-02 21:12:33 +0200 (+0200), Artem Goncharov wrote: > [...] > > I listen to complains carefully, and that is the reason for R1.0 > > not to exist as of now. > [...] > > It's probably also worth pointing out, from a user experience > perspective, that we anticipated backward-incompatible changes in > 1.0.0 and so merged an openstacksdk<1 pin in nodepool's > requirements.txt file. Unfortunately, instead the > backward-incompatible changes were released as 0.99.0 so when the > container images for our launchers updated we started getting > another error back from all of the Rackspace regions we're using > (traceback attached). > > I'm happy to file a bug report on that if nobody has beaten me to > it, but further diagnostics will probably have to wait until after > summit travel since we've rolled back to 0.61.0 for now. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Thu Jun 2 20:52:27 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Thu, 02 Jun 2022 13:52:27 -0700 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting In-Reply-To: References: <315f833d-2b00-4f01-8ccc-172d7bf7b576@www.fastmail.com> <27B7F0D2-E382-49EF-B518-1764D6F83344@gmail.com> <20220602194511.zkkph5zmasr37jkk@yuggoth.org> Message-ID: <6d960eb3-78c2-4410-8bfd-ec697e329fc9@www.fastmail.com> On Thu, Jun 2, 2022, at 12:54 PM, Artem Goncharov wrote: > Thanks Jeremy, will try to analyze the trace. > > This is not too extensive, but better then silent pin without > possibility to even notice that. > > I see that there are generally not enough deep tests for Openstack > driver in Zuul since mostly issues appear in Prod. I will to try to > invest some time there. We do functional testing against a devstack cloud. Unfortunately, as previously mentioned every cloud is just sufficiently unique that there are always corner cases that devstack can't catch. In general what we are able to test is that we can upload an image to devstack then boot that image and ssh into it. But then different clouds support different image upload methods and networking and even hypervisors. We do our best but once you get to the variety of clouds in the wild there is a lot more opportunity to find bad assumptions. > > Artem > > ---- > typed from mobile, auto-correct typos assumed > ---- > > On Thu, Jun 2, 2022, 21:47 Jeremy Stanley wrote: >> On 2022-06-02 21:12:33 +0200 (+0200), Artem Goncharov wrote: >> [...] >> > I listen to complains carefully, and that is the reason for R1.0 >> > not to exist as of now. >> [...] >> >> It's probably also worth pointing out, from a user experience >> perspective, that we anticipated backward-incompatible changes in >> 1.0.0 and so merged an openstacksdk<1 pin in nodepool's >> requirements.txt file. Unfortunately, instead the >> backward-incompatible changes were released as 0.99.0 so when the >> container images for our launchers updated we started getting >> another error back from all of the Rackspace regions we're using >> (traceback attached). >> >> I'm happy to file a bug report on that if nobody has beaten me to >> it, but further diagnostics will probably have to wait until after >> summit travel since we've rolled back to 0.61.0 for now. >> -- >> Jeremy Stanley From artem.goncharov at gmail.com Fri Jun 3 07:08:26 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Fri, 3 Jun 2022 09:08:26 +0200 Subject: [sdk] openstacksdk 0.99.0 breaks swift object header setting In-Reply-To: <6d960eb3-78c2-4410-8bfd-ec697e329fc9@www.fastmail.com> References: <315f833d-2b00-4f01-8ccc-172d7bf7b576@www.fastmail.com> <27B7F0D2-E382-49EF-B518-1764D6F83344@gmail.com> <20220602194511.zkkph5zmasr37jkk@yuggoth.org> <6d960eb3-78c2-4410-8bfd-ec697e329fc9@www.fastmail.com> Message-ID: <51744E7E-BCAE-4833-ADBC-40496CAEAAB8@gmail.com> > On 2. Jun 2022, at 22:52, Clark Boylan wrote: > > On Thu, Jun 2, 2022, at 12:54 PM, Artem Goncharov wrote: >> Thanks Jeremy, will try to analyze the trace. >> >> This is not too extensive, but better then silent pin without >> possibility to even notice that. >> >> I see that there are generally not enough deep tests for Openstack >> driver in Zuul since mostly issues appear in Prod. I will to try to >> invest some time there. > > We do functional testing against a devstack cloud. Unfortunately, as previously mentioned every cloud is just sufficiently unique that there are always corner cases that devstack can't catch. In general what we are able to test is that we can upload an image to devstack then boot that image and ssh into it. But then different clouds support different image upload methods and networking and even hypervisors. We do our best but once you get to the variety of clouds in the wild there is a lot more opportunity to find bad assumptions. Right. I didn?t want to say there is no testing. I mean in Zuul you have knowledge of the corner cases you face and they need to be included in tests to guarantee we do not break. But actually now I think it would be a real big benefit to everybody to include knowledge of those cases including corresponding tests into the SDK so that others can also benefit from that. Those who rely on SDK should just work no matter which particular cloud they are working with. This brings me slowly to other point. Our dream of real interoperability has not taken off. What is the sense of any certification if this does not save us even in pretty simple scenarios: provision host with custom image or upload object to object_store with CORS on? As a person who is responsible on our cloud for passing this certification I say openly: at the moment this is as good as useless at all. There are tests, and yes, sometimes it is even a challenge to pass those. But if they are not even able to guarantee us interoperability here this is not worth of effort. With this knowledge we can say that there is not a single OpenStack public cloud out there. They are all ?based on OpenStack?. Artem > >> >> Artem >> >> ---- >> typed from mobile, auto-correct typos assumed >> ---- >> >> On Thu, Jun 2, 2022, 21:47 Jeremy Stanley wrote: >>> On 2022-06-02 21:12:33 +0200 (+0200), Artem Goncharov wrote: >>> [...] >>>> I listen to complains carefully, and that is the reason for R1.0 >>>> not to exist as of now. >>> [...] >>> >>> It's probably also worth pointing out, from a user experience >>> perspective, that we anticipated backward-incompatible changes in >>> 1.0.0 and so merged an openstacksdk<1 pin in nodepool's >>> requirements.txt file. Unfortunately, instead the >>> backward-incompatible changes were released as 0.99.0 so when the >>> container images for our launchers updated we started getting >>> another error back from all of the Rackspace regions we're using >>> (traceback attached). >>> >>> I'm happy to file a bug report on that if nobody has beaten me to >>> it, but further diagnostics will probably have to wait until after >>> summit travel since we've rolled back to 0.61.0 for now. >>> -- >>> Jeremy Stanley > From katonalala at gmail.com Fri Jun 3 08:53:31 2022 From: katonalala at gmail.com (Lajos Katona) Date: Fri, 3 Jun 2022 10:53:31 +0200 Subject: [Forum] Meet the Projects Session! In-Reply-To: References: Message-ID: Hi Kendall, Thanks for this event, I add it to my calendar. Lajos Kendall Nelson ezt ?rta (id?pont: 2022. m?j. 31., K, 18:02): > Hello :) > > I wanted to take a moment to invite project maintainers, core > contributors, PTLs, governance officials, etc to a meet the projects > (projects being the OIF top level ones- Kata, Zuul, OpenStack, StarlingX, > and Airship) session during the Forum at the upcoming Summit in Berlin. It > will take place Tue, June 7, 11:20am - 12:30pm local in A - A06. > > The idea is to gather in a single place so that newer members of the > community or people unfamiliar with the project can come and meet some of > the active participants and ask questions. It's all really informal so > there is no need to prepare anything ahead of time. > > I realize it is pretty late in everyone's planning their schedules for the > summit, but if you have a few minutes to spare to come hang out and meet > some people and network, please do! We would love to see you there. > > -Kendall > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andrew.Norrie at cgg.com Fri Jun 3 12:53:13 2022 From: Andrew.Norrie at cgg.com (Norrie, Andrew) Date: Fri, 3 Jun 2022 12:53:13 +0000 Subject: [kolla-ansible][xena] Cell database HA/redundancy? Message-ID: Hi, We are currently planning some large OpenStack deployments utilizing kolla-ansible and I'm curious what folks are doing with the cell database HA/redundancy. With kolla-ansible (xena) it appears that a loadbalancer setup is only allowed for the default database shard (shard 0)... reference: https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb-guide.html If we are setting up separate cell database shards with Galera, I'm wondering if there is a convenient work-around or configuration for implementation of HA for these cell databases. In the inventory group_vars directory you can specify the group variables (for each cell database) as: nova_cell_database_address: nova_cell_database_group: but these aren't virtual IP hosts/addresses (non default db shard). This works perfectly fine with a single server cell database but not Galera. If that db server goes down the cell instance information is lost. Many thanks ... Andrew ________________________________ "This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately." -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Fri Jun 3 15:12:04 2022 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Fri, 3 Jun 2022 17:12:04 +0200 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: On Thu, 2 Jun 2022 at 01:10, Andy Botting wrote: > > Hi Rados?aw, Hi Andy, > > I am wondering if and how OpenStackers are approaching the subject of > > VDI (Virtual Desktop Infrastructure), also nowadays commonly called > > DaaS (Desktop as a Service) offering. > > To be more specific, I am looking forward to a solution covering both > > Linux and Windows desktops, pre-OS screen access optional, > > well-integrated (i.e., having automation) with OpenStack Nova, > > preferably also with Cinder and Glance, optionally with Keystone > > (apart from self-auth of course!) and Heat, 3D acceleration optional > > (most of the desktops here don't need it), nice to have read-only > > sharing of screen. > > We just launched a service like this on the Nectar Research Cloud. > Like you, I looked around but couldn't find any Open Source projects > that would be suitable. > > In the end, we based ours on a project from the University of > Melbourne that did initially support Windows, but we removed some of > that support to simplify our codebase as we didn't need it. > > We called it Bumblebee, and the code is here: > https://github.com/NeCTAR-RC/bumblebee > > It's a Django web app with a Redis-based task scheduler that launches > Nova instances, booted from a Cinder volume, cloned from a base OS > image. > > The Django app directly modifies Guacamole's database to create the > connections once the OpenStack resources have been provisioned and > provides the user with a link that takes them to their desktop. First of all, wow, that looks very interesting and in fact very much what I'm looking for. As I mentioned in the original message, the things this solution lacks are not something blocking for me. Regarding the approach to Guacamole, I know that it's preferable to have guacamole extension (that provides the dynamic inventory) developed rather than meddle with the internal database but I guess it is a good start. > We used Keycloak with OIDC for authentication on the web app and > Guacamole and it's fairly seamless transitioning between the two. > > We did initially look at using Heat, but we wanted to support a > workflow of being able to destroy the VM and archive the OS image to > Swift (and restore again later if needed) so it was easier to just > manage the resources directly. Yeah, that's not a problem at all for me, the reasoning is sensible. > There's no 3D acceleration in our solution yet, but we're planning on > looking at things like virtual GPUs later on. > > Sadly, I'm not going to make it to Berlin either, but I was thinking > it might make for a good talk at the next summit. Regarding this, I agree with Julia. I think OpenInfra Live would benefit from a VDI/DaaS-oriented session. > I'm in the process of properly documenting it now, so I don't have > much to share at this point, but I'm happy to go into more detail > about anything. Any "quickstart setting up" would be awesome to have at this stage. As this is a Django app, I think I should be able to figure out the bits and bolts to get it up and running in some shape but obviously it will impede wider adoption. On the note of adoption, if I find it usable, I can provide support for it in Kolla [1] and help grow the project's adoption this way. Also, since this is OpenStack-centric, maybe you could consider migrating to OpenDev at some point to collaborate with interested parties using a common system? Just food for thought at the moment. Kind regards, -yoctozepto [1] https://docs.openstack.org/kolla/latest/ > cheers, > Andy From marios at redhat.com Fri Jun 3 15:22:23 2022 From: marios at redhat.com (Marios Andreou) Date: Fri, 3 Jun 2022 18:22:23 +0300 Subject: [TripleO] move stable/victoria tripleo to End Of Life OK? Message-ID: Hello TripleO the tripleo-ci team proposes that we transition the stable/victoria branch across all tripleo repos [1] to End Of Life [2]. Hopefully this is not a suprise to anyone as it has been discussed for a while (most recent discussion I can point to is at Zed PTG [3]). The stable/victoria branch was moved to Extended Maintenance with [4] so we can no longer make any releases for these repos. Victoria is not an active branch for TripleO (not an FFU branch) so we are just merging patches there on the way back to train. Are there any concerns, objections or any other comment? I'll give this a few days and if there are no further comments I'll post the EOL proposal and update here towards the end of next week, regards, marios [1] https://releases.openstack.org/teams/tripleo.html#victoria [2] https://docs.openstack.org/project-team-guide/stable-branches.html#maintenance-phases [3] https://etherpad.opendev.org/p/tripleo-zed-ci-load [4] https://review.opendev.org/c/openstack/releases/+/837982 From gmann at ghanshyammann.com Fri Jun 3 17:46:59 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 03 Jun 2022 12:46:59 -0500 Subject: [all][tc] Change OpenStack release naming policy proposal In-Reply-To: <180e2904a63.ed97d27316807.1879483015684560107@ghanshyammann.com> References: <2175937.irdbgypaU6@p1> <18076dd3a30.116f2ef97411816.4718977843211132330@ghanshyammann.com> <180c03c9614.10ad4d72e148482.5160564778639516267@ghanshyammann.com> <180e2904a63.ed97d27316807.1879483015684560107@ghanshyammann.com> Message-ID: <1812aad73fb.b9dda98c518480.2929777219543202050@ghanshyammann.com> ---- On Fri, 20 May 2022 12:42:28 -0500 Ghanshyam Mann wrote ---- > ---- On Fri, 13 May 2022 20:43:57 -0500 Ghanshyam Mann wrote ---- > > Hello Everyone, > > > > Writing on the top for quick reading as we have a consensus on this. > > > > In today's TC meeting [3], we discussed this topic with Foundation staff and we all > > agreed to give the release name process handling to Foundation. TC will not be involved > > in the process. The release name will be mainly used for marketing purposes and we will > > use the release number as the primary identifier in our release scheduling, automated > > processes, directory structure etc. > > > > I have proposed the patch to document it in the TC release naming process. > > > > - https://review.opendev.org/c/openstack/governance/+/841800 > > As the next step, we will have a voice call to figure out how to use the number/name in our > development cycle/tooling. Meeting details are below: > > * When Tue May 24, 2022 9am ? 10am Pacific Time (16:00 - 17:00 UTC) > * Joining info Join with Google Meet = meet.google.com/gwi-cphb-rya > * Join by phone: (US) +1 563-447-5079 (PIN: 263292866) To update here: As discussed in the meeting, TC passed the resolution[1] and documented what place to use the release number and name, please reach out to TC if you have any queries on this: - https://governance.openstack.org/tc/reference/release-naming.html [1] https://governance.openstack.org/tc/resolutions/20220524-release-identification-process.html -gmann > > > -gmann > > > > > > [1] https://meetings.opendev.org/meetings/tc/2022/tc.2022-05-12-15.00.log.html#l-245 > > > > -gmann > > > > > > ---- On Fri, 29 Apr 2022 14:47:31 -0500 Ghanshyam Mann wrote ---- > > > > > > ---- On Fri, 29 Apr 2022 14:11:12 -0500 Goutham Pacha Ravi wrote ---- > > > > On Fri, Apr 29, 2022 at 8:36 PM Slawek Kaplonski wrote: > > > > > > > > > > Hi, > > > > > > > > > > > > > > > During the last PTG in April 2022 in the TC meeting we were discussing our release naming policy [1]. > > > > > > > > > > It seems that choosing appropriate name for every releases is very hard and time consuming. There is many factors which needs to be taken into consideration there like legal but also meaning of the chosen name in many different languages. > > > > > > > > > > > > > > > Finally we decided that now, after Zed release, when we will go all round through alphabet it is very good time to change this policy and use only numeric version with "year"."release in the year". It is proposed in [2]. > > > > > > > > > > This is also good timing for such change because in the same release we are going to start our "Tick Tock" release cadence which means that every Tick release will be release with .1 (like 2023.1, 2024.1, etc.) and every Tock release will be one with .2 (2023.2, 2024.2, etc.). > > > > > > > > Beloved TC, > > > > > > > > I'm highly disappointed in this 'decision', and would like for you to > > > > reconsider. I see the reasons you cite, but I feel like we're throwing > > > > the baby out with the bathwater here. Disagreements need not be > > > > feared, why not allow them to be aired publicly? That's a tenet of > > > > this open community. Allow names to be downvoted with reason during > > > > the proposal phase, and they'll organically fall-off from favor. > > > > > > > > Release names have always been a bonding factor. I've been happy to > > > > drum up contributor morale with our release names and the > > > > stories/anecdotes behind them. Release naming will not hurt/help the > > > > tick-tock release IMHO. We can append the release number to the name, > > > > and call it a day if you want. > > > > > > I agree with the disagrement ratio and that should not stop us doing the things. > > > But here we need to understand what type of disagrement we have and on what. > > > Most of the disagrement were cutural or historical where people has shown it > > > emotinally. And I personally as well as a TC or communitiy member does not feel > > > goot to ignore them or give them any reasoning not to listen them (because I do not > > > have any reasoning on these cultural/historical disagrement). > > > > > > Zed cycle was one good example of such thing when it was brought up in TC > > > channel about war thing[1] and asked about change the Zed name. I will be happy > > > to know what is best solution for this. > > > > > > 1. Change Zed name: it involve lot of technical work and communciation too. If yes then > > > let's do this now. > > > > > > 2. Do not listen to these emotional request to change name: We did this at the end and I > > > do not feel ok to do that. At least I do not want to ignore such request in future. > > > > > > Those are main reason we in TC decided to remvoe the name as they are culturally, emotionally > > > tied. That is main reason of droping those not any techncial or work wise issue. > > > > > > [1] https://meetings.opendev.org/irclogs/%23openstack-tc/%23openstack-tc.2022-03-08.log.html#t2022-03-08T14:35:26 > > > > > > -gmann > > > > > > > > > > > I do believe our current release naming process is a step out of the > > > > TC's perceived charter. There are many technical challenges that the > > > > TC is tackling, and coordinating a vote/slugfest about names isn't as > > > > important as those. > > > > As Allison suggests, we could seek help from the foundation to run the > > > > community voting and vetting for the release naming process - and > > > > expect the same level of transparency as the 4 opens that the > > > > OpenStack community espouses. > > > > > > Yes we will offcourse open to that but at the same time we will be waiting > > > for the foudnation proposal to sovle such issue irespective of who is doing name > > > selection. So let's wait for that. > > > > > > -gmann > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] https://etherpad.opendev.org/p/tc-zed-ptg#L265 > > > > > > > > > > [2] https://review.opendev.org/c/openstack/governance/+/839897 > > > > > > > > > > > > > > > -- > > > > > > > > > > Slawek Kaplonski > > > > > > > > > > Principal Software Engineer > > > > > > > > > > Red Hat > > > > > > > > > > > > > > > > > > > > From gmann at ghanshyammann.com Fri Jun 3 19:21:53 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 03 Jun 2022 14:21:53 -0500 Subject: [all][tc] What's happening in Technical Committee: summary 03 June 2022: Reading: 5 min Message-ID: <1812b04558e.f62a6fa3520078.8768492805426232760@ghanshyammann.com> Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on 02 June. Most of the topics I am summarizing in this email. This was a video call and recordings are available @ https://www.youtube.com/watch?v=nHG2WJNceMs And logs summary are available @https://meetings.opendev.org/meetings/tc/2022/tc.2022-06-02-15.02.log.html * We are cancelling next week's meeting. The next TC weekly meeting will be on 16 June Thursday at 15:00 UTC, feel free to add the topic to the agenda[1] by 15 June. 2. What we completed this week: ========================= * TC resolution[2] and documentation[3] for OpenStack release identification schema. This clarify what place we need to use the release number and name. * Security SIG chair rotation from Gage to Jeremy[4] * Removed security-analysis repo from Security SIG[5] * Added a repository for the Large Scale SIG[6] * Added project stats check tool[7] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[8], many items are in-progress. Open Reviews ----------------- * Four open reviews for ongoing activities[9]. New release cadence "SLURP" open items -------------------------------------------------- 1. release notes strategy: Brian proposal for ""SLURP release notes approach is up for review[10]. Improve project governance --------------------------------- Slawek has the proposal the framework up and it is under review[11]. New ELK service dashboard: e-r service ----------------------------------------------- dpawlik working on ansible playbooks to deploy e-r and functional tests. TripleO team and dpawlik are working on the e-r repo merge. Consistent and Secure Default RBAC ------------------------------------------- We had a call yesterday and discussed about presenting this to the ops meetup in berlin and getting feedback, especially on the 'scope' concept. knikolla plan to moderate this topic in the ops meetup but unfortunately he will not be to make it to the summit. I have added a few notes and what questions we need to ask the operator in etherpad[12]. Please note that operator feedback is very important for us to proceed on 'scope' in SRBAC. If you are operator or know anyone around you, we request you to provide feedback. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[13]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[14]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[15][16]. Project updates ------------------- * Add a repository for the Large Scale SIG[17] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[18]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [19] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/843214 [3] https://review.opendev.org/c/openstack/governance/+/841800 [4] https://review.opendev.org/c/openstack/governance-sigs/+/844446 [5] https://review.opendev.org/c/openstack/governance/+/844463 [6] https://review.opendev.org/c/openstack/governance/+/843535 [7] https://review.opendev.org/c/openstack/governance/+/810037 [8] https://etherpad.opendev.org/p/tc-zed-tracker [9] https://review.opendev.org/q/projects:openstack/governance+status:open [10] https://review.opendev.org/c/openstack/project-team-guide/+/843457 [11] https://review.opendev.org/c/openstack/governance/+/839880 [12] https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L53 [13] https://review.opendev.org/c/openstack/governance/+/836888 [14] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.html [15] https://etherpad.opendev.org/p/zuul-config-error-openstack [16] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [17] https://review.opendev.org/c/openstack/project-config/+/843534 [18] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [19] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann From haiwu.us at gmail.com Fri Jun 3 20:37:52 2022 From: haiwu.us at gmail.com (hai wu) Date: Fri, 3 Jun 2022 15:37:52 -0500 Subject: [sdk][openstacksdk][keystone] Does openstacksdk support v3tokenlessauth? Message-ID: Does openstacksdk support keystone auth type "v3tokenlessauth"? I got this error message when trying to enable openstacksdk with auth_type="v3tokenlessauth" openstack.exceptions.NotSupported: The identity service for TEST: exists but does not have any supported versions. The following are the possible relevant documents I could find, and there is no place mentioning the support for tokenless auth using openstacksdk: https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html https://docs.openstack.org/keystone/train/admin/configure_tokenless_x509.html From artem.goncharov at gmail.com Sat Jun 4 06:25:28 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Sat, 4 Jun 2022 08:25:28 +0200 Subject: [sdk][openstacksdk][keystone] Does openstacksdk support v3tokenlessauth? In-Reply-To: References: Message-ID: The error you posted suggests you reached keystone, got some answer, but it is not sufficient for SDK to continue. It can be caused by misconfiguration on keystone. But in any way you should provide logs (add `openstack.enable_logging(debug=True)`) and provide it here. Without this it is not possible to give you any other information. Artem ---- typed from mobile, auto-correct typos assumed ---- On Fri, Jun 3, 2022, 22:40 hai wu wrote: > Does openstacksdk support keystone auth type "v3tokenlessauth"? > > I got this error message when trying to enable openstacksdk with > auth_type="v3tokenlessauth" > > openstack.exceptions.NotSupported: The identity service for TEST: > exists but does not have any supported versions. > > The following are the possible relevant documents I could find, and > there is no place mentioning the support for tokenless auth using > openstacksdk: > > https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html > > https://docs.openstack.org/keystone/train/admin/configure_tokenless_x509.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haiwu.us at gmail.com Sat Jun 4 13:37:24 2022 From: haiwu.us at gmail.com (hai wu) Date: Sat, 4 Jun 2022 08:37:24 -0500 Subject: [sdk][openstacksdk][keystone] Does openstacksdk support v3tokenlessauth? In-Reply-To: References: Message-ID: I did add the above debug line in the openstacksdk script before, but I did not get any more information.. With the current keystone setting, the regular 'openstack' command line is working with x509 tokenless (v3tokenlessauth) authentication for this simple test case (with test command: openstack project list), but I could not get openstacksdk script to work the same way.. ~ ? cat /tmp/test.py import openstack from openstack import ( # type: ignore # pylint: disable=import-self connection, exceptions, ) openstack.enable_logging(debug=True) conn = connection.from_config(cloud_name='TEST') for project in sorted(conn.identity.projects(), key=lambda p: p.name): print(project) ~ ? python /tmp/test.py Traceback (most recent call last): File "/tmp/test.py", line 10, in for project in sorted(conn.identity.projects(), key=lambda p: p.name): File "/home/hai/venv/openstack/lib/python3.10/site-packages/openstack/service_description.py", line 87, in __get__ proxy = self._make_proxy(instance) File "/home/hai/venv/openstack/lib/python3.10/site-packages/openstack/service_description.py", line 266, in _make_proxy raise exceptions.NotSupported( openstack.exceptions.NotSupported: The identity service for TEST: exists but does not have any supported versions. So the 'found_version' ends up as 'None' from this line in 'service_description.py' file: found_version = temp_adapter.get_api_major_version() Here is my clouds.yaml for 'TEST': TEST: auth_type: "v3tokenlessauth" auth: project_name: testme auth_url: https://keystonetest:5050/v3 project_domain_name: Default key: /home/hai/hai.key cert: /home/hai/hai.pem cacert: /home/hai/myca.crt region: RegionOne identity_api_version: 3 On Sat, Jun 4, 2022 at 1:25 AM Artem Goncharov wrote: > > The error you posted suggests you reached keystone, got some answer, but it is not sufficient for SDK to continue. It can be caused by misconfiguration on keystone. > > But in any way you should provide logs (add `openstack.enable_logging(debug=True)`) and provide it here. Without this it is not possible to give you any other information. > > Artem > > ---- > typed from mobile, auto-correct typos assumed > ---- > > On Fri, Jun 3, 2022, 22:40 hai wu wrote: >> >> Does openstacksdk support keystone auth type "v3tokenlessauth"? >> >> I got this error message when trying to enable openstacksdk with >> auth_type="v3tokenlessauth" >> >> openstack.exceptions.NotSupported: The identity service for TEST: >> exists but does not have any supported versions. >> >> The following are the possible relevant documents I could find, and >> there is no place mentioning the support for tokenless auth using >> openstacksdk: >> https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html >> https://docs.openstack.org/keystone/train/admin/configure_tokenless_x509.html >> From radoslaw.piliszek at gmail.com Sat Jun 4 18:41:28 2022 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Sat, 4 Jun 2022 20:41:28 +0200 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: Writing to let you know I have also found the following related paper: [1] and reached out to its authors in the hope to enable further collaboration to happen. The paper is not open access so I have only obtained it for myself and am unsure if licensing permits me to share, thus I also asked the authors to share their copy (that they have copyrights to). I have obviously let them know of the existence of this thread. ;-) Let's stay tuned. [1] https://link.springer.com/chapter/10.1007/978-3-030-99191-3_12 Kind regards, -yoctozepto From amonster369 at gmail.com Sun Jun 5 13:15:11 2022 From: amonster369 at gmail.com (A Monster) Date: Sun, 5 Jun 2022 14:15:11 +0100 Subject: Using infiniband for openstack network communications Message-ID: Is it possible to use the infiniband port for openstack networks without having to use the infiniband port as an ethernet port ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Sun Jun 5 13:22:59 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Sun, 5 Jun 2022 15:22:59 +0200 Subject: Using infiniband for openstack network communications In-Reply-To: References: Message-ID: Yes, why not? As long as your infiniband card support sr-iov there should be no problems with that. You can also check one blog which describes the experience https://satishdotpatel.github.io/HPC-on-openstack/ ??, 5 ???. 2022 ?., 15:20 A Monster : > Is it possible to use the infiniband port for openstack networks without > having to use the infiniband port as an ethernet port ? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yasufum.o at gmail.com Sun Jun 5 18:01:43 2022 From: yasufum.o at gmail.com (Yasufumi Ogawa) Date: Mon, 6 Jun 2022 03:01:43 +0900 Subject: [tacker] Weekly IRC meeting on Jun 7th cancelled Message-ID: <19dcb1e5-4fb7-0361-8cf7-14712e5efc94@gmail.com> Hi team, The next meeting on Jun 7th is cancelled due to the Berlin summit. Cheers, Yasufumi From noonedeadpunk at gmail.com Sun Jun 5 18:38:15 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Sun, 5 Jun 2022 20:38:15 +0200 Subject: [openstack-ansible] Meeting on Jun 7th 2022 is cancelled Message-ID: Hi everybody! The next meeting on Jun 7th is cancelled due to the Berlin summit. Hope seeing you in person instead! We will have next meeting Jun 14th. -- Kind regards, Dmitriy Rabotyagov From p.aminian.server at gmail.com Sun Jun 5 19:06:50 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Sun, 5 Jun 2022 23:36:50 +0430 Subject: masakari-hostmonitor error Message-ID: hello on kolla ansible masakari-hostmonitor container on wallaby version I have this error : 2022-06-05 23:30:51.880 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication using 'eth0' is failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2022-06-05 23:30:51.881 7 ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication is failed. 2022-06-05 23:31:22.042 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Exception caught: Unexpected error while running command. Command: crmadmin -S myhostname Exit code: 102 Stdout: '' Stderr: 'error: Could not connect to controller: Connection refused\nerror: Command failed: Connection refused\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Could you please help me ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.aminian.server at gmail.com Sun Jun 5 19:39:17 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Mon, 6 Jun 2022 00:09:17 +0430 Subject: masakari bug Message-ID: on kolla ansible masakari-hostmonitor container on wallaby version I have this error : 2022-06-05 23:30:51.880 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication using 'eth0' is failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2022-06-05 23:30:51.881 7 ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication is failed. 2022-06-05 23:31:22.042 7 WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Exception caught: Unexpected error while running command. Command: crmadmin -S myhostname Exit code: 102 Stdout: '' Stderr: 'error: Could not connect to controller: Connection refused\nerror: Command failed: Connection refused\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tkajinam at redhat.com Mon Jun 6 04:15:50 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Mon, 6 Jun 2022 13:15:50 +0900 Subject: [all][doc] the openstack-tox-docs job is broken with Sphinx >= 5.0.0 Message-ID: Hi, It seems the openstack-tox-docs job is broken in some projects because of the following change made in Sphinx >= 5.0.0 . https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 We currently have Sphinx 4.5.0 in upper-constraints.txt but some projects (eg. Heat, Nova, Neutron) are using only their own docs-constraints.txt and pull the latest version which is 5.0.1 . IMO we need to fix the problem by the following two steps, but I'd appreciate if anybody can share a better idea. 1. Update tox.ini to honor upper-constraints.txt. This is needed in master and stable/branches as well. 2. Update docs/source/conf.py to hard-code the language parameter to 'en'. This is required in master so that we can bump the upper version later. Thank you, Takashi Kajinami -------------- next part -------------- An HTML attachment was scrubbed... URL: From tkajinam at redhat.com Mon Jun 6 04:27:51 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Mon, 6 Jun 2022 13:27:51 +0900 Subject: [all][doc] the openstack-tox-docs job is broken with Sphinx >= 5.0.0 In-Reply-To: References: Message-ID: Sorry it seems I was looking at something wrong and it seems the projects I mentioned as examples(Heat, Nova and Neutron) are already using upper constraints for doc build. However I'd recommend reach project reviews their current tox.ini. Updating conf.py would be still required so it's better to work on it earlier before the upper constraint is updated. On Mon, Jun 6, 2022 at 1:15 PM Takashi Kajinami wrote: > Hi, > > > It seems the openstack-tox-docs job is broken in some projects because of > the following > change made in Sphinx >= 5.0.0 . > > https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 > > We currently have Sphinx 4.5.0 in upper-constraints.txt but some projects > (eg. Heat, Nova, > Neutron) are using only their own docs-constraints.txt and pull the latest > version which is 5.0.1 . > > IMO we need to fix the problem by the following two steps, but I'd > appreciate > if anybody can share a better idea. > > 1. Update tox.ini to honor upper-constraints.txt. This is needed in > master and > stable/branches as well. > > 2. Update docs/source/conf.py to hard-code the language parameter to 'en'. > This is required in master so that we can bump the upper version > later. > > Thank you, > Takashi Kajinami > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Mon Jun 6 08:22:22 2022 From: mark at stackhpc.com (Mark Goddard) Date: Mon, 6 Jun 2022 09:22:22 +0100 Subject: [kolla-ansible][xena] Cell database HA/redundancy? In-Reply-To: References: Message-ID: On Fri, 3 Jun 2022 at 14:10, Norrie, Andrew wrote: > > > Hi, > > > > We are currently planning some large OpenStack deployments utilizing > kolla-ansible and > > I?m curious what folks are doing with the cell database HA/redundancy. > > > > With kolla-ansible (xena) it appears that a loadbalancer setup is only > allowed for the > > default database shard (shard 0)... reference: > https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb-guide.html > > > > If we are setting up separate cell database shards with Galera, I?m > wondering if there is a convenient work-around or configuration for > implementation of HA for these cell databases. > > > > In the inventory group_vars directory you can specify the group variables > (for each cell database) as: > > > > nova_cell_database_address: > > nova_cell_database_group: > > > > but these aren?t virtual IP hosts/addresses (non default db shard). This > works perfectly fine with > > a single server cell database but not Galera. If that db server goes down > the cell instance information is lost. > > > Hi Andrew, You are correct - only the first shard is load balanced. The sharding feature was actually just the first patch in a series intended to support proxysql. I believe this would add the functionality you are looking for. In fact, the proposed test case for proxysql in our CI is a multi-cell deployment. Here is the patch series: https://review.opendev.org/q/topic:proxysql Unfortunately it has been stuck for a while, largely due to core reviewer bandwidth. The author is Michael Arbet, who is often on IRC as kevko. I'd suggest registering your interest in these patches via gerrit, and at one of the weekly kolla IRC meetings (this week is cancelled due to the summit). If you have time available, then testing and providing reviews could help to get the patches moving. In the meantime, you can configure HAProxy to load balance additional services, by placing an HAProxy config snippet in /etc/kolla/config/haproxy/services.d/*.cfg. Regards, Mark Many thanks ... Andrew > ------------------------------ > ?This e-mail and any accompanying attachments are confidential. The > information is intended solely for the use of the individual to whom it is > addressed. Any review, disclosure, copying, distribution, or use of the > email by others is strictly prohibited. If you are not the intended > recipient, you must not review, disclose, copy, distribute or use this > e-mail; please delete it from your system and notify the sender > immediately.? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Mon Jun 6 10:55:42 2022 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Mon, 6 Jun 2022 12:55:42 +0200 Subject: masakari-hostmonitor error In-Reply-To: References: Message-ID: set enable_hacluster to true and redeploy -yoctozepto On Sun, Jun 5, 2022, 21:08 Parsa Aminian wrote: > hello > on kolla ansible masakari-hostmonitor container on wallaby version I have > this error : > > 2022-06-05 23:30:51.880 7 WARNING > masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication using 'eth0' is failed.: > oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while > running command. > 2022-06-05 23:30:51.881 7 ERROR > masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication is failed. > 2022-06-05 23:31:22.042 7 WARNING > masakarimonitors.hostmonitor.host_handler.handle_host [-] Exception caught: > Unexpected error while running command. > Command: crmadmin -S myhostname > Exit code: 102 > Stdout: '' > Stderr: 'error: Could not connect to controller: Connection > refused\nerror: Command failed: Connection refused\n': > oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while > running command. > > Could you please help me ? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnasiadka at gmail.com Mon Jun 6 11:02:24 2022 From: mnasiadka at gmail.com (=?UTF-8?Q?Micha=C5=82_Nasiadka?=) Date: Mon, 6 Jun 2022 13:02:24 +0200 Subject: [kolla] weekly meeting 8th June Message-ID: Hello koalas, This Wednesday (8th June) meeting is cancelled due to OpenInfra Summit in Berlin. See you on a meeting next week. Michal -- Micha? Nasiadka mnasiadka at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Mon Jun 6 13:14:03 2022 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Mon, 6 Jun 2022 09:14:03 -0400 Subject: [all][doc] the openstack-tox-docs job is broken with Sphinx >= 5.0.0 In-Reply-To: References: Message-ID: <4bff2d02-3bd7-48bc-5acc-33233dd63a0b@gmail.com> On 6/6/22 12:27 AM, Takashi Kajinami wrote: [snip] > Updating conf.py would be still required so it's better to work on it > earlier > before the upper constraint is updated. I was just looking into this, and as long as your conf.py does not explicitly set language = None the default language is set to 'en' internally by Sphinx in 5.0.0+ [0]. So as long as you have the language setting either missing or commented out in your conf.py, you should not see this warning (which, when treated as an error, breaks the doc build). [0] https://github.com/sphinx-doc/sphinx/commit/e4e58a4f2791e528cdaa861b96636a1e37a558ba > > On Mon, Jun 6, 2022 at 1:15 PM Takashi Kajinami > wrote: > > Hi, > > > It seems the openstack-tox-docs job is broken in some projects > because of the following > change made in Sphinx >= 5.0.0 . > https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 > > > We currently have Sphinx 4.5.0 in upper-constraints.txt but some > projects (eg. Heat, Nova, > Neutron) are using only their own docs-constraints.txt and pull the > latest version which is 5.0.1 . > > IMO we need to fix the problem by the following two steps, but I'd > appreciate > if anybody can share a better idea. > > ?1. Update tox.ini to honor upper-constraints.txt. This is needed > in master and > ???? stable/branches as well. > > ?2. Update docs/source/conf.py to hard-code the language parameter > to 'en'. > ???? This is required in master so that we can bump the upper > version later. > > Thank you, > Takashi Kajinami > From amonster369 at gmail.com Mon Jun 6 13:15:56 2022 From: amonster369 at gmail.com (A Monster) Date: Mon, 6 Jun 2022 14:15:56 +0100 Subject: Using infiniband for openstack network communications Message-ID: Thank you for your reply, I'll check the link you've sent to me immediately. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amonster369 at gmail.com Mon Jun 6 13:36:38 2022 From: amonster369 at gmail.com (A Monster) Date: Mon, 6 Jun 2022 14:36:38 +0100 Subject: Using infiniband for openstack network communications In-Reply-To: References: Message-ID: What I wanted to actually do is not what is done in this link https://satishdotpatel.github.io/HPC-on-openstack/ , because here vm's are given the possibility to use the infiniband port through Virtual functions, but in my case, I want to use infiniband for openstack management, compute and storage networks, to get better performance than when using ethernet and rg45 cables. so I'm wondering if it's feasible and whether it's a good thing or not. Thank you. On Mon, 6 Jun 2022 at 14:15, A Monster wrote: > Thank you for your reply, I'll check the link you've sent to me > immediately. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Mon Jun 6 13:51:43 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Mon, 6 Jun 2022 15:51:43 +0200 Subject: Using infiniband for openstack network communications In-Reply-To: References: Message-ID: Ah, ok. Well, you can use infiniband for control plane only with IPoIB. There's no RDMA support if that's what you're asking. ??, 6 ???. 2022 ?., 15:36 A Monster : > What I wanted to actually do is not what is done in this link > https://satishdotpatel.github.io/HPC-on-openstack/ , because here vm's > are given the possibility to use the infiniband port through Virtual > functions, but in my case, I want to use infiniband for openstack > management, compute and storage networks, to get better performance than > when using ethernet and rg45 cables. so I'm wondering if it's feasible and > whether it's a good thing or not. > Thank you. > > On Mon, 6 Jun 2022 at 14:15, A Monster wrote: > >> Thank you for your reply, I'll check the link you've sent to me >> immediately. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tkajinam at redhat.com Mon Jun 6 14:15:54 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Mon, 6 Jun 2022 23:15:54 +0900 Subject: [all][doc] the openstack-tox-docs job is broken with Sphinx >= 5.0.0 In-Reply-To: <4bff2d02-3bd7-48bc-5acc-33233dd63a0b@gmail.com> References: <4bff2d02-3bd7-48bc-5acc-33233dd63a0b@gmail.com> Message-ID: Thanks Brian, Actually I hit this problem initially with heat-cfntools which is not using upper-constraints in doc build but has language=None explicitly set, so what you mentioned makes quite clear sense. Quick grep shows a few instances of the hard-coded language=None . These should be commented out before we get the constraint bumped. openstack/python-swiftclient/releasenotes/source/conf.py:language = None openstack/blazar/api-ref/source/conf.py:language = None openstack/heat-agents/doc/source/conf.py:language = None openstack/heat-agents/releasenotes/source/conf.py:language = None openstack/swift/releasenotes/source/conf.py:language = None openstack/storlets/releasenotes/source/conf.py:language = None openstack/placement/releasenotes/source/conf.py:language = None openstack/osc-lib/releasenotes/source/conf.py:language = None openstack/python-heatclient/releasenotes/source/conf.py:language = None openstack/trove-dashboard/releasenotes/source/conf.py:language = None openstack/virtualbmc/releasenotes/source/conf.py:language = None openstack/heat-templates/doc/source/conf.py:language = None Disclaimer: The above list might be missing some projects. On Mon, Jun 6, 2022 at 10:19 PM Brian Rosmaita wrote: > On 6/6/22 12:27 AM, Takashi Kajinami wrote: > [snip] > > Updating conf.py would be still required so it's better to work on it > > earlier > > before the upper constraint is updated. > > I was just looking into this, and as long as your conf.py does not > explicitly set > language = None > the default language is set to 'en' internally by Sphinx in 5.0.0+ [0]. > So as long as you have the language setting either missing or > commented out in your conf.py, you should not see this warning (which, > when treated as an error, breaks the doc build). > > [0] > > https://github.com/sphinx-doc/sphinx/commit/e4e58a4f2791e528cdaa861b96636a1e37a558ba > > > > > On Mon, Jun 6, 2022 at 1:15 PM Takashi Kajinami > > wrote: > > > > Hi, > > > > > > It seems the openstack-tox-docs job is broken in some projects > > because of the following > > change made in Sphinx >= 5.0.0 . > > > https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 > > < > https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 > > > > > > We currently have Sphinx 4.5.0 in upper-constraints.txt but some > > projects (eg. Heat, Nova, > > Neutron) are using only their own docs-constraints.txt and pull the > > latest version which is 5.0.1 . > > > > IMO we need to fix the problem by the following two steps, but I'd > > appreciate > > if anybody can share a better idea. > > > > 1. Update tox.ini to honor upper-constraints.txt. This is needed > > in master and > > stable/branches as well. > > > > 2. Update docs/source/conf.py to hard-code the language parameter > > to 'en'. > > This is required in master so that we can bump the upper > > version later. > > > > Thank you, > > Takashi Kajinami > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amonster369 at gmail.com Mon Jun 6 14:30:37 2022 From: amonster369 at gmail.com (A Monster) Date: Mon, 6 Jun 2022 15:30:37 +0100 Subject: Using infiniband for openstack network communications Message-ID: Alright, thank you again Sir. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Mon Jun 6 15:01:51 2022 From: pierre at stackhpc.com (Pierre Riteau) Date: Mon, 6 Jun 2022 17:01:51 +0200 Subject: [all][doc] the openstack-tox-docs job is broken with Sphinx >= 5.0.0 In-Reply-To: References: <4bff2d02-3bd7-48bc-5acc-33233dd63a0b@gmail.com> Message-ID: Thank you Takashi for raising this. I also found one in the release notes configuration for python-blazarclient. I will fix the blazar repositories. On Mon, 6 Jun 2022 at 16:22, Takashi Kajinami wrote: > Thanks Brian, > > > Actually I hit this problem initially with heat-cfntools which is not > using upper-constraints in doc build > but has language=None explicitly set, so what you mentioned makes quite > clear sense. > > Quick grep shows a few instances of the hard-coded language=None . These > should be commented out > before we get the constraint bumped. > > openstack/python-swiftclient/releasenotes/source/conf.py:language = None > openstack/blazar/api-ref/source/conf.py:language = None > openstack/heat-agents/doc/source/conf.py:language = None > openstack/heat-agents/releasenotes/source/conf.py:language = None > openstack/swift/releasenotes/source/conf.py:language = None > openstack/storlets/releasenotes/source/conf.py:language = None > openstack/placement/releasenotes/source/conf.py:language = None > openstack/osc-lib/releasenotes/source/conf.py:language = None > openstack/python-heatclient/releasenotes/source/conf.py:language = None > openstack/trove-dashboard/releasenotes/source/conf.py:language = None > openstack/virtualbmc/releasenotes/source/conf.py:language = None > openstack/heat-templates/doc/source/conf.py:language = None > > Disclaimer: The above list might be missing some projects. > > On Mon, Jun 6, 2022 at 10:19 PM Brian Rosmaita > wrote: > >> On 6/6/22 12:27 AM, Takashi Kajinami wrote: >> [snip] >> > Updating conf.py would be still required so it's better to work on it >> > earlier >> > before the upper constraint is updated. >> >> I was just looking into this, and as long as your conf.py does not >> explicitly set >> language = None >> the default language is set to 'en' internally by Sphinx in 5.0.0+ [0]. >> So as long as you have the language setting either missing or >> commented out in your conf.py, you should not see this warning (which, >> when treated as an error, breaks the doc build). >> >> [0] >> >> https://github.com/sphinx-doc/sphinx/commit/e4e58a4f2791e528cdaa861b96636a1e37a558ba >> >> > >> > On Mon, Jun 6, 2022 at 1:15 PM Takashi Kajinami > > > wrote: >> > >> > Hi, >> > >> > >> > It seems the openstack-tox-docs job is broken in some projects >> > because of the following >> > change made in Sphinx >= 5.0.0 . >> > >> https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 >> > < >> https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 >> > >> > >> > We currently have Sphinx 4.5.0 in upper-constraints.txt but some >> > projects (eg. Heat, Nova, >> > Neutron) are using only their own docs-constraints.txt and pull the >> > latest version which is 5.0.1 . >> > >> > IMO we need to fix the problem by the following two steps, but I'd >> > appreciate >> > if anybody can share a better idea. >> > >> > 1. Update tox.ini to honor upper-constraints.txt. This is needed >> > in master and >> > stable/branches as well. >> > >> > 2. Update docs/source/conf.py to hard-code the language parameter >> > to 'en'. >> > This is required in master so that we can bump the upper >> > version later. >> > >> > Thank you, >> > Takashi Kajinami >> > >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpodivin at redhat.com Mon Jun 6 15:05:13 2022 From: jpodivin at redhat.com (Jiri Podivin) Date: Mon, 6 Jun 2022 17:05:13 +0200 Subject: [all][doc] the openstack-tox-docs job is broken with Sphinx >= 5.0.0 In-Reply-To: References: <4bff2d02-3bd7-48bc-5acc-33233dd63a0b@gmail.com> Message-ID: Cursory search appears to indicate that these are part of some boilerplate configuration code. So I believe we should be fine if we remove it. Still, I wonder why was the language set to `None` in the first place. On Mon, Jun 6, 2022 at 4:21 PM Takashi Kajinami wrote: > Thanks Brian, > > > Actually I hit this problem initially with heat-cfntools which is not > using upper-constraints in doc build > but has language=None explicitly set, so what you mentioned makes quite > clear sense. > > Quick grep shows a few instances of the hard-coded language=None . These > should be commented out > before we get the constraint bumped. > > openstack/python-swiftclient/releasenotes/source/conf.py:language = None > openstack/blazar/api-ref/source/conf.py:language = None > openstack/heat-agents/doc/source/conf.py:language = None > openstack/heat-agents/releasenotes/source/conf.py:language = None > openstack/swift/releasenotes/source/conf.py:language = None > openstack/storlets/releasenotes/source/conf.py:language = None > openstack/placement/releasenotes/source/conf.py:language = None > openstack/osc-lib/releasenotes/source/conf.py:language = None > openstack/python-heatclient/releasenotes/source/conf.py:language = None > openstack/trove-dashboard/releasenotes/source/conf.py:language = None > openstack/virtualbmc/releasenotes/source/conf.py:language = None > openstack/heat-templates/doc/source/conf.py:language = None > > Disclaimer: The above list might be missing some projects. > > On Mon, Jun 6, 2022 at 10:19 PM Brian Rosmaita > wrote: > >> On 6/6/22 12:27 AM, Takashi Kajinami wrote: >> [snip] >> > Updating conf.py would be still required so it's better to work on it >> > earlier >> > before the upper constraint is updated. >> >> I was just looking into this, and as long as your conf.py does not >> explicitly set >> language = None >> the default language is set to 'en' internally by Sphinx in 5.0.0+ [0]. >> So as long as you have the language setting either missing or >> commented out in your conf.py, you should not see this warning (which, >> when treated as an error, breaks the doc build). >> >> [0] >> >> https://github.com/sphinx-doc/sphinx/commit/e4e58a4f2791e528cdaa861b96636a1e37a558ba >> >> > >> > On Mon, Jun 6, 2022 at 1:15 PM Takashi Kajinami > > > wrote: >> > >> > Hi, >> > >> > >> > It seems the openstack-tox-docs job is broken in some projects >> > because of the following >> > change made in Sphinx >= 5.0.0 . >> > >> https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 >> > < >> https://github.com/sphinx-doc/sphinx/commit/a3d09835522e6487cf93dcc871179db8d69e4180 >> > >> > >> > We currently have Sphinx 4.5.0 in upper-constraints.txt but some >> > projects (eg. Heat, Nova, >> > Neutron) are using only their own docs-constraints.txt and pull the >> > latest version which is 5.0.1 . >> > >> > IMO we need to fix the problem by the following two steps, but I'd >> > appreciate >> > if anybody can share a better idea. >> > >> > 1. Update tox.ini to honor upper-constraints.txt. This is needed >> > in master and >> > stable/branches as well. >> > >> > 2. Update docs/source/conf.py to hard-code the language parameter >> > to 'en'. >> > This is required in master so that we can bump the upper >> > version later. >> > >> > Thank you, >> > Takashi Kajinami >> > >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.harrison at manchester.ac.uk Mon Jun 6 15:11:29 2022 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Mon, 6 Jun 2022 15:11:29 +0000 Subject: [nova] local LVM volume on compute hosts Message-ID: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> I have installed a small test cluster using openstack-ansible (xena) and would like to use local LVM volumes for storing instance volumes - and following https://docs.openstack.org/nova/latest/configuration/config.html and (e.g. https://cloudnull.io/2017/12/nova-lvm-an-iop-love-story/) I have the libvirt section of my /etc/nova/nova.conf as below. -8<--- [libvirt] inject_partition = -2 inject_password = False inject_key = False virt_type = kvm live_migration_with_native_tls = true live_migration_scheme = tls live_migration_inbound_addr = 172.29.236.5 hw_disk_discard = ignore disk_cachemodes = none images_type = lvm images_volume_group = nova volume_clear = zero -8 From kdhall at binghamton.edu Mon Jun 6 16:50:24 2022 From: kdhall at binghamton.edu (Dave Hall) Date: Mon, 6 Jun 2022 12:50:24 -0400 Subject: Openstack-Ansible vs Kolla-Ansible -Xena or later Message-ID: Hello, My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am sensitive to the effort that has been put into both of these projects, what I really need right now is the most fool-proof way to deploy and manage a small production cluster for academic instructional use. I have been working for a couple months with OpenStack-Ansible to deploy a small (3-node) test cluster on VirtualBox VMs in preparation for a larger deployment on real hardware - 6 to 10 nodes. (Details: Debian 11 deployment, Debian 10 infra, compute, and storage nodes) It has been slow going, at least partially due to some issues and limitations with VirtualBox. However, deploying a test cluster on VMs still seems preferable to just diving deployment on real hardware and going through multiple scrubs/reinstalls. Today I saw that Mark asked a question about Kolla-Ansible. So, as a 'beginner', should I shift and look at Kolla-Ansible, or should I stay the course and continue with Openstack-Ansible? What are the pros and cons of each? Thanks. -Dave -- Dave Hall Binghamton University kdhall at binghamton.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.rohmann at inovex.de Mon Jun 6 16:59:41 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Mon, 6 Jun 2022 18:59:41 +0200 Subject: [designate] How to avoid NXDOMAIN or stale data during cold start of a (new) machine In-Reply-To: References: <69ab8e54-f419-4cd1-f289-a0b5efb7f723@inovex.de> Message-ID: Hey Michael, all, sorry for the late reply, just got around to playing with Designate and BIND9 again ... On 10/05/2022 19:04, Michael Johnson wrote: > On startup, BIND9 will start sending SOA serial number queries for all > of the zones it knows about. In the case of Designate, that means > BIND9 will send out requests to the miniDNS instances to check if the > serial number in Designate is newer than the one in BIND9. If the > serial number in Designate is newer, BIND9 will initiate a zone > transfer from the miniDNS in Designate. > > BIND9, by default, will do 20 SOA serial number queries at a time > (less on older versions of BIND). See the serial-query-rate setting in > the rate limiter knowledge base article[1]. > > The tuning knowledge base article[2] also discusses settings that can > be adjusted for secondary servers that may also help speed up a cold > startup. > > Off my head, I don't know of a way to tell BIND9 to not answer queries > via rdnc or such. I usually block network access to a new BIND9 > instance until the "rdnc status" shows the "soa queries in progress" > and "xfers running" drop to 0 or a low number. Thanks for your thoughts and for distinguishing the sync of existing domains via AXFR/IXFR zone-transfers, because I was actually talking about an empty BIND9 in need to actually learn about zones. Maybe I was not clear about that on my initial post. But in the case of BIND9 not having a zone configured it requires input from designate before asking for the latest zone data from mDNS. Think about a freshly restored or a newly added server to the pool that requires to have ALL zones added and until then is not doing any IXFR / AXFR, but wrongfully sending NXDOMAIN. For new zones there is https://github.com/openstack/designate/blob/5d5d83e511acbf5d6f34e9a998ff16d96c5bb162/designate/backend/impl_bind9.py#L76 which creates the zone (via rndc addzone - https://docs.openstack.org/designate/latest/admin/backends/bind9.html#bind9-configuration). and triggers mDNS. But I don't quite get how Designate would recognize that a zone is "missing" on one of the BIND9 servers in the pool. What would be the task that checks or determines that a server of the pool does not have some / all of the zones (anymore)? Is there no form of health check for backends to see if they have all zones? Like a simple count or comparison of the zone list? I'd like to form a readiness check to e.g. block all external queries until all zones exist. There is an open issues about this even: https://gitlab.isc.org/isc-projects/bind9/-/issues/3081 Or is there any way / designate-manage command to trigger this or a re-creating of all zone for a backend or a way to remove and re-add a BIND9 backend to have designate treat it as new and not configured to then do all those "rndc addzone" commands? Regards Christian From gmann at ghanshyammann.com Mon Jun 6 19:56:12 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 06 Jun 2022 14:56:12 -0500 Subject: [all][tc] Canceling this week TC meetings Message-ID: <1813a96d713.b1ad2f74618627.5691800160918441653@ghanshyammann.com> Hello Everyone, We are cancelling this week's TC meeting and will be meeting next on 16 June. -gmann From haleyb.dev at gmail.com Mon Jun 6 21:52:25 2022 From: haleyb.dev at gmail.com (Brian Haley) Date: Mon, 6 Jun 2022 17:52:25 -0400 Subject: [neutron] Bug deputy report for week of May 30th Message-ID: Hi, I was Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * None High bugs --------- * https://bugs.launchpad.net/neutron/+bug/1976357 - [sqlalchemy-20] Missing DB context in subnetpool methods - https://review.opendev.org/c/openstack/neutron/+/844024 * https://bugs.launchpad.net/neutron/+bug/1976360 - [sqlalchemy-20] Missing DB context in networking-sfc flow classifier - Rodolfo assigned to himself as he's working on a number of these Medium bugs ----------- * https://bugs.launchpad.net/neutron/+bug/1976292 - [OVN] AgentCache.agents can change during an iteration operation - https://review.opendev.org/c/openstack/neutron/+/843933 * https://bugs.launchpad.net/neutron/+bug/1976323 - [functional/fullstack][master] move fips periodic job to CentOS 9-Stream - https://review.opendev.org/c/openstack/neutron/+/843252 * https://bugs.launchpad.net/neutron/+bug/1976345 - The sync_routers interface takes too long - Needs owner and further investigation * https://bugs.launchpad.net/neutron/+bug/1976366 - OVN: Metadata is not provisioned for a network with DHCP turned off - Related to https://bugs.launchpad.net/neutron/+bug/1918914 ? * https://bugs.launchpad.net/neutron/+bug/1976439 - The database ml2_port_binding_levels and ml2_distributed_port_bindings tables have a lot of redundant data - Needs assignee * https://bugs.launchpad.net/neutron/+bug/1976355 - remove eager subquery load for PortBindingLevel - Recent fix https://review.opendev.org/c/openstack/neutron/+/841823 - https://review.opendev.org/c/openstack/neutron/+/844699 (potential) - Other suggestions from Oleg as well Low bugs -------- * https://bugs.launchpad.net/neutron/+bug/1977629 - [ovn]agent's heartbeat time is error - Zhou Heng took ownership Misc bugs --------- * https://bugs.launchpad.net/neutron/+bug/1977485 - Neutron deletes port in use and nova errors out when cleaning the VM XML - Possible Nova issue, posted comment * https://bugs.launchpad.net/neutron/+bug/1977470 - [networking-bgpvpn] zed unit test failures - Closed as invalid Wishlist bugs ------------- * https://bugs.launchpad.net/neutron/+bug/1976270 - [RFE] Improve performance of bulk port create/delete with networking-generic-switch From johnsomor at gmail.com Tue Jun 7 00:04:12 2022 From: johnsomor at gmail.com (Michael Johnson) Date: Mon, 6 Jun 2022 17:04:12 -0700 Subject: [designate] How to avoid NXDOMAIN or stale data during cold start of a (new) machine In-Reply-To: References: <69ab8e54-f419-4cd1-f289-a0b5efb7f723@inovex.de> Message-ID: Hi Christian, Sorry I misunderstood the scenario. There are two ways zones can be resynced: 1. Using the "designate-manage pool update" command. This will force an update/recreate of all of the zones. 2. When a zone is in ERROR or PENDING for too long, the WorkerPeriodicRecovery task in producer will attempt to repair the zone. I don't think there is a periodic task that checks the BIND instances for missing content at the moment. Adding one would have to be done very carefully as it would be easy to get into race conditions when new zones are being created and deployed. Michael On Mon, Jun 6, 2022 at 9:59 AM Christian Rohmann wrote: > > Hey Michael, all, > > sorry for the late reply, just got around to playing with Designate and > BIND9 again ... > > > On 10/05/2022 19:04, Michael Johnson wrote: > > On startup, BIND9 will start sending SOA serial number queries for all > > of the zones it knows about. In the case of Designate, that means > > BIND9 will send out requests to the miniDNS instances to check if the > > serial number in Designate is newer than the one in BIND9. If the > > serial number in Designate is newer, BIND9 will initiate a zone > > transfer from the miniDNS in Designate. > > > > BIND9, by default, will do 20 SOA serial number queries at a time > > (less on older versions of BIND). See the serial-query-rate setting in > > the rate limiter knowledge base article[1]. > > > > The tuning knowledge base article[2] also discusses settings that can > > be adjusted for secondary servers that may also help speed up a cold > > startup. > > > > Off my head, I don't know of a way to tell BIND9 to not answer queries > > via rdnc or such. I usually block network access to a new BIND9 > > instance until the "rdnc status" shows the "soa queries in progress" > > and "xfers running" drop to 0 or a low number. > > Thanks for your thoughts and for distinguishing the sync of existing > domains via AXFR/IXFR zone-transfers, because I was actually talking > about an empty BIND9 in need to actually learn about zones. > Maybe I was not clear about that on my initial post. But in the case of > BIND9 not having a zone configured it requires input from designate > before asking for the latest zone data from mDNS. > Think about a freshly restored or a newly added server to the pool that > requires to have ALL zones added and until then is not doing any IXFR / > AXFR, but wrongfully sending NXDOMAIN. > > For new zones there is > https://github.com/openstack/designate/blob/5d5d83e511acbf5d6f34e9a998ff16d96c5bb162/designate/backend/impl_bind9.py#L76 > which creates the zone (via rndc addzone - > https://docs.openstack.org/designate/latest/admin/backends/bind9.html#bind9-configuration). > and triggers mDNS. > > But I don't quite get how Designate would recognize that a zone is > "missing" on one of the BIND9 servers in the pool. > What would be the task that checks or determines that a server of the > pool does not have some / all of the zones (anymore)? > Is there no form of health check for backends to see if they have all > zones? Like a simple count or comparison of the zone list? > I'd like to form a readiness check to e.g. block all external queries > until all zones exist. There is an open issues about this even: > https://gitlab.isc.org/isc-projects/bind9/-/issues/3081 > > Or is there any way / designate-manage command to trigger this or a > re-creating of all zone for a backend or a way to remove and re-add a > BIND9 backend to have designate > treat it as new and not configured to then do all those "rndc addzone" > commands? > > > Regards > > Christian > From sharath.madhava at gmail.com Tue Jun 7 05:09:27 2022 From: sharath.madhava at gmail.com (Sharath Ck) Date: Tue, 7 Jun 2022 10:39:27 +0530 Subject: [kolla-ansible][kolla] merge_configs module replacing colon to equal Message-ID: Hi, In kolla-ansible, I am using the merge_configs module to merge service configs files. I want to merge api-paste.ini config with my custom changes, however while merging if a line consists of one colon, it is getting replaced with equal sign. merge_configs splits the line by considering colon as delimiter and treats two parts as key and value. Later which will be substituted as key = value. Original */v2.1: oscomputeversion_v2* Merged */v2.1= oscomputeversion_v2 * Below, colon will be 1 and equal will be -1, *def _split_key_value(self, line): colon = line.find(':') equal = line.find('=') if colon < 0 and equal < 0: return self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and equal < colon): key, value = line[:equal], line[equal + 1:] else: key, value = line[:colon], line[colon + 1:] value = value.strip() if value and value[0] == value[-1] and value.startswith(("\"", "'")): value = value[1:-1] return key.strip(), [value]* I want to keep the config as it is and merge only my custom changes. Please let me know how to achieve this or colon can be escaped somehow. Regards, Sharath Regards, Sharath -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.rohmann at inovex.de Tue Jun 7 06:54:22 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Tue, 7 Jun 2022 08:54:22 +0200 Subject: [designate] How to avoid NXDOMAIN or stale data during cold start of a (new) machine In-Reply-To: References: <69ab8e54-f419-4cd1-f289-a0b5efb7f723@inovex.de> Message-ID: <21cce5d5-5d21-dc08-4b08-89fdabeb6625@inovex.de> On 07/06/2022 02:04, Michael Johnson wrote: > There are two ways zones can be resynced: > 1. Using the "designate-manage pool update" command. This will force > an update/recreate of all of the zones. > 2. When a zone is in ERROR or PENDING for too long, the > WorkerPeriodicRecovery task in producer will attempt to repair the > zone. Thanks again for these details! I suppose there is no option for a targeted run of this "pool update" command to just hit one backend server? If you don't mind me asking yet another question ... what is the difference between running a regular backend or an agent backend? So https://github.com/openstack/designate/blob/master/designate/backend/impl_bind9.py vs. https://github.com/openstack/designate/blob/master/designate/backend/agent_backend/impl_bind9.py Regards and thanks again, Christian From michal.arbet at ultimum.io Tue Jun 7 07:25:33 2022 From: michal.arbet at ultimum.io (Michal Arbet) Date: Tue, 7 Jun 2022 09:25:33 +0200 Subject: [cinder] Failed to delete volume Message-ID: Hi, I would like to ask if someone saw issue when volume failed to delete, log below : This is happening from time to time. LOG : https://paste.opendev.org/show/b1jRCxuBFnnVzf64i9yV/ Thanks, Michal Arbet Openstack Engineer Ultimum Technologies a.s. Na Po???? 1047/26, 11000 Praha 1 Czech Republic +420 604 228 897 michal.arbet at ultimum.io *https://ultimum.io * LinkedIn | Twitter | Facebook -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbauza at redhat.com Tue Jun 7 09:13:35 2022 From: sbauza at redhat.com (Sylvain Bauza) Date: Tue, 7 Jun 2022 11:13:35 +0200 Subject: [summit][ops][nova] Ops feedback session Message-ID: Hi all, You probably haven't noticed but we'll run a Forum session at Berlin about Nova pain points and features you'd like to have : https://etherpad.opendev.org/p/nova-berlin-meet-and-greet This will be held tomorrow at 14:50 CEST but your feedback is absolutely welcome even if you can't attend our session. Please take a few minutes to look at the etherpad above and provide your thoughts, that'd be more than welcome. The more we have feedback, the better we could have a productive discussion :-) See you hopefully in-person for whose who are lucky to be present at the Summit. -Sylvain and gibi -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Tue Jun 7 12:13:51 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 07 Jun 2022 13:13:51 +0100 Subject: Using infiniband for openstack network communications In-Reply-To: References: Message-ID: On Sun, 2022-06-05 at 15:22 +0200, Dmitriy Rabotyagov wrote: > Yes, why not? As long as your infiniband card support sr-iov there should > be no problems with that. > > You can also check one blog which describes the experience > https://satishdotpatel.github.io/HPC-on-openstack/ well it could work but its not really a tested usecase. security groups for example (which yes i know dont work with normal sriov) more or less assume ethernet. ovs and other backends do assuem ethernet so you cant use an infinaband interface for ovs, ovn or linux bridge. neutrop ports also kind of implictly assume ethernet via the mac adress filed so you cant really use infinaband without ethernet in openstack other then via a direct passthough to a guest in which case you are not using infinaband with neturon netowrks you are using infinaband without integration with neutron netowrking api via flaovr based pci passthoug. that is what https://satishdotpatel.github.io/HPC-on-openstack/ descripbes you will notice that the passthrough_whitelist = { "vendor_id": "10de", "product_id": "1df6" } does not contain a physical_network tag which means this VF is not usable via neutron sriov ports (vnic_type=direct in the case of a vf) they are instead creating a pci alias alias = { "vendor_id":"15b3", "product_id":"101c", "device_type":"type-VF", "name":"mlx5-sriov-ib" } then requesting that in the flavor openstack flavor create --vcpus 8 --ram 16384 --disk 100 --property hw:mem_page_size='large' --property hw:cpu_policy='dedicated' --property "pci_passthrough:alias"="mlx5-sriov-ib" ib.small then when the vm is create i has a neutron network for managment openstack server create --flavor ib.small --image ubuntu_20_04 --nic net-id=private1 ib-vm1 and a passed through infinaband VF. so i i would not cinsider this to be "use the infiniband port for openstack networks" since the infinaband port is not assocated with any neutron network or modeled as a neutron port. as far as i am aware there is no support for using infinaband with a neutron port and vnic_type=direct. so can you use infinaband with openstack yes. can you use infinaband with onpenstack networking no. > > ??, 5 ???. 2022 ?., 15:20 A Monster : > > > Is it possible to use the infiniband port for openstack networks without > > having to use the infiniband port as an ethernet port ? > > From smooney at redhat.com Tue Jun 7 12:25:35 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 07 Jun 2022 13:25:35 +0100 Subject: [nova] local LVM volume on compute hosts In-Reply-To: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> References: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> Message-ID: <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> On Mon, 2022-06-06 at 15:11 +0000, Paul Harrison wrote: > I have installed a small test cluster using openstack-ansible (xena) and would like to use local LVM volumes for storing instance volumes - and following https://docs.openstack.org/nova/latest/configuration/config.html and (e.g. https://cloudnull.io/2017/12/nova-lvm-an-iop-love-story/) I have the libvirt section of my /etc/nova/nova.conf as below. > > -8<--- > [libvirt] > inject_partition = -2 > inject_password = False > inject_key = False > virt_type = kvm > live_migration_with_native_tls = true > live_migration_scheme = tls > live_migration_inbound_addr = 172.29.236.5 > hw_disk_discard = ignore > disk_cachemodes = none > images_type = lvm > images_volume_group = nova > volume_clear = zero > -8 > however, the nova service continues to use cinder volumes from my storage host to store instances. Is there something else that needs to be configured to make this work, or is it obsolete? no there is noting else you need to configure but this option is not what you think it is. the images_type option contols what storage will be used for all non cinder storage. i.e. vms that are booted with out usign a boot volume. by default i belive we woudl use qcow or raw files on disk with images_type=lvm we will instead create a lvm volume for the root disk but only if you do not use the boot form volume workflow. there is no way curretnly to prevent usign boot form volume on a specific host and force only local storage directly via config. you can do this indirectly but with sideffect. effectivly if you have a set of hosts that cant or should not have acess to cinder you can create a spereate avaiablity zone. you would then set [cinder]/cross_az_attch=false. e.g. create local-only az and if the cinder backend is configured to be in a differnt az then the cross_az_attach config will prevent the vm form booting in the local-only az. https://docs.openstack.org/nova/latest/configuration/config.html#cinder.cross_az_attach so your current config will make any non boot from volume nova instance use lvm storage to provision the vm root/swap/epmeral disks but will not prevent end users requesting cinder data voluems or boot volumes via the cli/api. if the opt in to cinder stoage that is what they will recive but if they use teh default storage provided by the flaovr then it will be local. > > > From hjensas at redhat.com Tue Jun 7 12:32:54 2022 From: hjensas at redhat.com (Harald Jensas) Date: Tue, 7 Jun 2022 14:32:54 +0200 Subject: [Tripleo] - Provisioning Baremetal Instances in Wallaby Openstack In-Reply-To: References: Message-ID: <86dca991-09df-afd9-7e84-7fc60bb9efbd@redhat.com> On 6/7/22 11:49, Anirudh Gupta wrote: > Hi Harald/Julia/Openstack Team > > I had successfully?installed Ironic Service in Tripleo Train, Ussuri > Release with the help of openstack discuss team members. > > https://lists.openstack.org/pipermail/openstack-discuss/2022-February/027047.html > > > I had custom network configuration for which I had passed Service net > map as below in network-environment file. > > parameter_defaults: > ? ServiceNetMap: > ? ? IronicApiNetwork: oc_provisioning > ? ? IronicNetwork: oc_provisioning > > But with the Wallaby Release, since the deployment format has changed > There is no network-environment file. > > So I tried passing it in a separate?environment.yaml file along with the > templates. > Command to deploy? in wallaby: > > openstack overcloud deploy --templates \ > ? ? --networks-file /home/stack/templates/custom_network_data.yaml \ > ? ? --vip-file ?/home/stack/templates/custom_vip_data.yaml \ > ? ? --baremetal-deployment > ?/home/stack/templates/overcloud-baremetal-deploy.yaml --deployed-server \ > ? ? --network-config \ > ? ? -e /home/stack/templates/environment.yaml \ > ? ? -e > /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml > \ > ? ? -e > /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml > \ > ? ? -e > /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml > \ > ? ? -e /home/stack/templates/ironic-config.yaml \ > ? ? -e > /usr/share/openstack-tripleo-heat-templates/environments/services/ptp.yaml \ > ? ? -e > /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ > ? ? -e > /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ > ? ? -e /home/stack/containers-prepare-parameter.yaml > > Without Service Net Map, deployment was successful but TFTP of image was > an issue since we are going with custom network approach. > > Passing Service Net map details in a seperate file environment.yaml, is > giving the below error: > > 24d-f43f-6f76-000000006ce9 | TASK | Create containers managed by Podman > for /var/lib/tripleo-config/container-startup-config/step_3 > 2022-06-07 11:16:46.806982 | | WARNING | ERROR: Can't run container > ironic_db_sync > stderr: Error: statfs /var/lib/config-data/puppet-generated/ironic_api: > no such file or directory > 2022-06-07 11:16:46.809252 | 525400fd-f24d-f43f-6f76-000000006c79 | > FATAL | Create containers managed by Podman for > /var/lib/tripleo-config/container-startup-config/step_3 | > overcloud-controller-1 | error={"changed": false, "msg": "Failed > containers: ironic_db_sync"} > 2022-06-07 11:16:46.810197 | 525400fd-f24d-f43f-6f76-000000006c79 | > TIMING | tripleo_container_manage : Create containers managed by Podman > for /var/lib/tripleo-config/container-startup-config/step_3 | > overcloud-controller-1 | 0:11:40.574944 | 10.62s > 2022-06-07 11:16:47.683400 | | WARNING | ERROR: Can't run container > ironic_db_sync > stderr: Error: statfs /var/lib/config-data/puppet-generated/ironic_api: > no such file or directory > 2022-06-07 11:16:47.684455 | 525400fd-f24d-f43f-6f76-000000006ce9 | > FATAL | Create containers managed by Podman for > /var/lib/tripleo-config/container-startup-config/step_3 | > overcloud-controller-2 | error={"changed": false, "msg": "Failed > containers: ironic_db_sync"} > > Can you please suggest the place we can pass the service net map details > in Wallaby Release to resolve this issue? > You should be able to pass it in any of the environment files, like you did. Can you try adding the following to the environment file where you override the ServiceNetMap? parameter_merge_strategies: ServiceNetMap: merge -- Harald From hjensas at redhat.com Tue Jun 7 13:31:47 2022 From: hjensas at redhat.com (Harald Jensas) Date: Tue, 7 Jun 2022 15:31:47 +0200 Subject: [Tripleo] - Provisioning Baremetal Instances in Wallaby Openstack In-Reply-To: <86dca991-09df-afd9-7e84-7fc60bb9efbd@redhat.com> References: <86dca991-09df-afd9-7e84-7fc60bb9efbd@redhat.com> Message-ID: <5ee4d9f4-772d-18c5-bc30-7d5f849f63b3@redhat.com> On 6/7/22 14:32, Harald Jensas wrote: > On 6/7/22 11:49, Anirudh Gupta wrote: >> Hi Harald/Julia/Openstack Team >> >> I had successfully?installed Ironic Service in Tripleo Train, Ussuri >> Release with the help of openstack discuss team members. >> >> https://lists.openstack.org/pipermail/openstack-discuss/2022-February/027047.html >> >> >> >> I had custom network configuration for which I had passed Service net >> map as below in network-environment file. >> >> parameter_defaults: >> ?? ServiceNetMap: >> ?? ? IronicApiNetwork: oc_provisioning >> ?? ? IronicNetwork: oc_provisioning >> >> But with the Wallaby Release, since the deployment format has changed >> There is no network-environment file. >> >> So I tried passing it in a separate?environment.yaml file along with >> the templates. >> Command to deploy? in wallaby: >> >> openstack overcloud deploy --templates \ >> ?? ? --networks-file /home/stack/templates/custom_network_data.yaml \ >> ?? ? --vip-file ?/home/stack/templates/custom_vip_data.yaml \ >> ?? ? --baremetal-deployment >> ??/home/stack/templates/overcloud-baremetal-deploy.yaml >> --deployed-server \ >> ?? ? --network-config \ >> ?? ? -e /home/stack/templates/environment.yaml \ >> ?? ? -e >> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml >> \ >> ?? ? -e >> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml >> \ >> ?? ? -e >> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml >> \ >> ?? ? -e /home/stack/templates/ironic-config.yaml \ >> ?? ? -e >> /usr/share/openstack-tripleo-heat-templates/environments/services/ptp.yaml >> \ >> ?? ? -e >> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ >> ?? ? -e >> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ >> ?? ? -e /home/stack/containers-prepare-parameter.yaml >> >> Without Service Net Map, deployment was successful but TFTP of image >> was an issue since we are going with custom network approach. >> >> Passing Service Net map details in a seperate file environment.yaml, >> is giving the below error: >> >> 24d-f43f-6f76-000000006ce9 | TASK | Create containers managed by >> Podman for /var/lib/tripleo-config/container-startup-config/step_3 >> 2022-06-07 11:16:46.806982 | | WARNING | ERROR: Can't run container >> ironic_db_sync >> stderr: Error: statfs >> /var/lib/config-data/puppet-generated/ironic_api: no such file or >> directory >> 2022-06-07 11:16:46.809252 | 525400fd-f24d-f43f-6f76-000000006c79 | >> FATAL | Create containers managed by Podman for >> /var/lib/tripleo-config/container-startup-config/step_3 | >> overcloud-controller-1 | error={"changed": false, "msg": "Failed >> containers: ironic_db_sync"} >> 2022-06-07 11:16:46.810197 | 525400fd-f24d-f43f-6f76-000000006c79 | >> TIMING | tripleo_container_manage : Create containers managed by >> Podman for /var/lib/tripleo-config/container-startup-config/step_3 | >> overcloud-controller-1 | 0:11:40.574944 | 10.62s >> 2022-06-07 11:16:47.683400 | | WARNING | ERROR: Can't run container >> ironic_db_sync >> stderr: Error: statfs >> /var/lib/config-data/puppet-generated/ironic_api: no such file or >> directory >> 2022-06-07 11:16:47.684455 | 525400fd-f24d-f43f-6f76-000000006ce9 | >> FATAL | Create containers managed by Podman for >> /var/lib/tripleo-config/container-startup-config/step_3 | >> overcloud-controller-2 | error={"changed": false, "msg": "Failed >> containers: ironic_db_sync"} >> >> Can you please suggest the place we can pass the service net map >> details in Wallaby Release to resolve this issue? >> > > You should be able to pass it in any of the environment files, like you > did. Can you try adding the following to the environment file where you > override the ServiceNetMap? > > parameter_merge_strategies: > ? ServiceNetMap: merge > Actually, my bad. (Thanks Rabi for reminding me on IRC.) The merge strategies is after a change in master. With Wallaby we still have ServiceNetMapDefaults which is merged with overrides in ServiceNetMap. AFICT from this error ... > 2022-06-07 11:16:47.683400 | | WARNING | ERROR: Can't run container > ironic_db_sync > stderr: Error: statfs > /var/lib/config-data/puppet-generated/ironic_api: no such file or > directory ... it did not execute $THT/common/container-puppet.sh for the ironic_api service correctly. Could it be because you don't use a custom roles file? The default[1] role file does not have the `oc_provisioning` provisioning network. I wonder if some task is skipped because the node is not on the network the service is mapped to. Can you try using a custom roles file with the `oc_provisioning` network defined on the controller role? I think we probably need more information to understand the problem. Would it be possible to share more data? i.e full deploy log, all custom environment files? Thanks! [1]?https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/wallaby/roles_data.yaml#L18 From jdratlif at globalnoc.iu.edu Tue Jun 7 13:51:05 2022 From: jdratlif at globalnoc.iu.edu (jdratlif at globalnoc.iu.edu) Date: Tue, 7 Jun 2022 09:51:05 -0400 Subject: [sdk] specify mac address when creating ports with nova Message-ID: <007701d87a75$a24d8020$e6e88060$@globalnoc.iu.edu> When using connection.compute.create_server(), what is the structure of the networks parameter? I see in the example is a list of objects with a single key in the list being the uuid of the network. I know there are other parameters that could be in this list though. The os-migrate ansible playbooks add fixed_ip to this. fixed_ip is not a Network or a Port class parameter. Port has fixed_ips, but that would be a list. I want to add mac address to this port, but I'm not sure if that's possible or what key I would need to add. Is the structure of the networks object documented somewhere? Thanks --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6687 bytes Desc: not available URL: From smooney at redhat.com Tue Jun 7 14:26:22 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 07 Jun 2022 15:26:22 +0100 Subject: [sdk] specify mac address when creating ports with nova In-Reply-To: <007701d87a75$a24d8020$e6e88060$@globalnoc.iu.edu> References: <007701d87a75$a24d8020$e6e88060$@globalnoc.iu.edu> Message-ID: <833f128b350476c0ef66339bd9f33f1e7b99524c.camel@redhat.com> On Tue, 2022-06-07 at 09:51 -0400, jdratlif at globalnoc.iu.edu wrote: > When using connection.compute.create_server(), what is the structure of the > networks parameter? > > > > I see in the example is a list of objects with a single key in the list > being the uuid of the network. I know there are other parameters that could > be in this list though. The os-migrate ansible playbooks add fixed_ip to > this. fixed_ip is not a Network or a Port class parameter. Port has > fixed_ips, but that would be a list. I want to add mac address to this port, > but I'm not sure if that's possible or what key I would need to add. this is not something that shoudl be added to the sdk or nova. id you want to create a vm with a set of ports with specific mac or fixed ips you shoudl first precreate the port in neutron with the relevent setting applied and then create teh server with a list of neutron port uuids. nova has legacy proxy apis for create vms with interfaces on spefic neturon networks or subnets but we do not plan to extend them to allow any finer grained contol then is there today https://docs.openstack.org/nova/latest/contributor/project-scope.html#no-more-api-proxies """ No more API Proxies? Nova API current has some APIs that are now (in kilo) mostly just a proxy to other OpenStack services. If it were possible to remove a public API, these are some we might start with. As such, we don?t want to add any more. The first example is the API that is a proxy to the Glance v1 API. As Glance moves to deprecate its v1 API, we need to translate calls from the old v1 API we expose, to Glance?s v2 API. The next API to mention is the networking APIs, in particular the security groups API. Most of these APIs exist from when nova-network existed and the proxies were added during the transition. However, security groups has a much richer Neutron API, and if you use both Nova API and Neutron API, the mismatch can lead to some very unexpected results, in certain cases. Our intention is to avoid adding to the problems we already have in this area. """ the sdk shoudl likel avoid introduceding simiarl proxy apis. os-migrate shoudl simple use the neutron port apis in the sdk to create the ports on the migrate neutron netowrks with the fixed ips and macs that are required and then boot the vms with those ports instead of encodeign this in the sdk. this is they type of logic that shoudl live in the calling application (os-migrate) not in the sdk its self. > > > > Is the structure of the networks object documented somewhere? > > > > Thanks > > > > --- > > John Ratliff > > Systems Automation Engineer > > GlobalNOC @ Indiana University > > > From jdratlif at globalnoc.iu.edu Tue Jun 7 14:50:34 2022 From: jdratlif at globalnoc.iu.edu (jdratlif at globalnoc.iu.edu) Date: Tue, 7 Jun 2022 10:50:34 -0400 Subject: [ansible] [openstack-cloud] how to find port by IP address in ansible Message-ID: <01de01d87a7d$f1491970$d3db4c50$@globalnoc.iu.edu> How can I get a port by it's IP address using the openstack cloud ansible modules? This is what I'm trying, but it never yields any results. I do get results if I use mac_address or device_id until filters, but I need to find the port by it's IP. - name: get port info openstack.cloud.port_info: filters: fixed_ips: - 'ip_address=123.124.125.126' auth: "{{ auth_dict }}" region_name: "{{ region }}" Thanks --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6687 bytes Desc: not available URL: From jdratlif at globalnoc.iu.edu Tue Jun 7 15:12:52 2022 From: jdratlif at globalnoc.iu.edu (jdratlif at globalnoc.iu.edu) Date: Tue, 7 Jun 2022 11:12:52 -0400 Subject: [sdk] specify mac address when creating ports with nova In-Reply-To: <833f128b350476c0ef66339bd9f33f1e7b99524c.camel@redhat.com> References: <007701d87a75$a24d8020$e6e88060$@globalnoc.iu.edu> <833f128b350476c0ef66339bd9f33f1e7b99524c.camel@redhat.com> Message-ID: <021f01d87a81$0eb8edc0$2c2ac940$@globalnoc.iu.edu> I'm looking at these docs here. https://docs.openstack.org/openstacksdk/xena/user/proxies/compute.html There is a create_server() function for the compute object on the connection. It says to pass attrs, which is comprised of properties of the Server class. When I go to the server class, all I see for ports is the networks property, which I can't find any description of. Assuming I already used neutron to create the ports, how would I tell the create_server function to use those ports? Is this the proxy function you said I shouldn't use anymore? I'm not clear on that. Thanks --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University -----Original Message----- From: Sean Mooney Sent: Tuesday, June 7, 2022 10:26 AM To: jdratlif at globalnoc.iu.edu; openstack-discuss at lists.openstack.org Cc: Ratliff, John Subject: Re: [sdk] specify mac address when creating ports with nova On Tue, 2022-06-07 at 09:51 -0400, jdratlif at globalnoc.iu.edu wrote: > When using connection.compute.create_server(), what is the structure > of the networks parameter? > > > > I see in the example is a list of objects with a single key in the > list being the uuid of the network. I know there are other parameters > that could be in this list though. The os-migrate ansible playbooks > add fixed_ip to this. fixed_ip is not a Network or a Port class > parameter. Port has fixed_ips, but that would be a list. I want to add > mac address to this port, but I'm not sure if that's possible or what key I would need to add. this is not something that shoudl be added to the sdk or nova. id you want to create a vm with a set of ports with specific mac or fixed ips you shoudl first precreate the port in neutron with the relevent setting applied and then create teh server with a list of neutron port uuids. nova has legacy proxy apis for create vms with interfaces on spefic neturon networks or subnets but we do not plan to extend them to allow any finer grained contol then is there today https://docs.openstack.org/nova/latest/contributor/project-scope.html#no-more-api-proxies """ No more API Proxies? Nova API current has some APIs that are now (in kilo) mostly just a proxy to other OpenStack services. If it were possible to remove a public API, these are some we might start with. As such, we don?t want to add any more. The first example is the API that is a proxy to the Glance v1 API. As Glance moves to deprecate its v1 API, we need to translate calls from the old v1 API we expose, to Glance?s v2 API. The next API to mention is the networking APIs, in particular the security groups API. Most of these APIs exist from when nova-network existed and the proxies were added during the transition. However, security groups has a much richer Neutron API, and if you use both Nova API and Neutron API, the mismatch can lead to some very unexpected results, in certain cases. Our intention is to avoid adding to the problems we already have in this area. """ the sdk shoudl likel avoid introduceding simiarl proxy apis. os-migrate shoudl simple use the neutron port apis in the sdk to create the ports on the migrate neutron netowrks with the fixed ips and macs that are required and then boot the vms with those ports instead of encodeign this in the sdk. this is they type of logic that shoudl live in the calling application (os-migrate) not in the sdk its self. > > > > Is the structure of the networks object documented somewhere? > > > > Thanks > > > > --- > > John Ratliff > > Systems Automation Engineer > > GlobalNOC @ Indiana University > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6687 bytes Desc: not available URL: From smooney at redhat.com Tue Jun 7 15:32:00 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 07 Jun 2022 16:32:00 +0100 Subject: [sdk] specify mac address when creating ports with nova In-Reply-To: <021f01d87a81$0eb8edc0$2c2ac940$@globalnoc.iu.edu> References: <007701d87a75$a24d8020$e6e88060$@globalnoc.iu.edu> <833f128b350476c0ef66339bd9f33f1e7b99524c.camel@redhat.com> <021f01d87a81$0eb8edc0$2c2ac940$@globalnoc.iu.edu> Message-ID: On Tue, 2022-06-07 at 11:12 -0400, jdratlif at globalnoc.iu.edu wrote: > I'm looking at these docs here. https://docs.openstack.org/openstacksdk/xena/user/proxies/compute.html > > There is a create_server() function for the compute object on the connection. It says to pass attrs, which is comprised of properties of the Server class. When I go to the server class, all I see for ports is the networks property, which I can't find any description of. > > Assuming I already used neutron to create the ports, how would I tell the create_server function to use those ports? Is this the proxy function you said I shouldn't use anymore? I'm not clear on that. > i think you would use the networks object https://docs.openstack.org/openstacksdk/xena/user/resources/compute/v2/server.html#openstack.compute.v2.server.Server.networks based on https://docs.openstack.org/openstacksdk/xena/user/guides/compute.html#create-server and https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server= you woudl change there default example form server = conn.compute.create_server( name=SERVER_NAME, image_id=image.id, flavor_id=flavor.id, networks=[{"uuid": network.id}], key_name=keypair.name) to server = conn.compute.create_server( name=SERVER_NAME, image_id=image.id, flavor_id=flavor.id, networks=[{"port": port.id}], key_name=keypair.name) where port is a https://opendev.org/openstack/openstacksdk/src/branch/master/openstack/network/v2/port.py object that you previously created via https://docs.openstack.org/openstacksdk/xena/user/proxies/network.html#port-operations or https://docs.openstack.org/openstacksdk/xena/user/resources/network/v2/port.html#openstack.network.v2.port.Port depending on if you want to use the service proxies https://docs.openstack.org/openstacksdk/xena/user/index.html#service-proxies or resource interface https://docs.openstack.org/openstacksdk/xena/user/index.html#resource-interface im kind of surprised there isnt an example fo this already. > Thanks > > --- > John Ratliff > Systems Automation Engineer > GlobalNOC @ Indiana University > > -----Original Message----- > From: Sean Mooney > Sent: Tuesday, June 7, 2022 10:26 AM > To: jdratlif at globalnoc.iu.edu; openstack-discuss at lists.openstack.org > Cc: Ratliff, John > Subject: Re: [sdk] specify mac address when creating ports with nova > > On Tue, 2022-06-07 at 09:51 -0400, jdratlif at globalnoc.iu.edu wrote: > > When using connection.compute.create_server(), what is the structure > > of the networks parameter? > > > > > > > > I see in the example is a list of objects with a single key in the > > list being the uuid of the network. I know there are other parameters > > that could be in this list though. The os-migrate ansible playbooks > > add fixed_ip to this. fixed_ip is not a Network or a Port class > > parameter. Port has fixed_ips, but that would be a list. I want to add > > mac address to this port, but I'm not sure if that's possible or what key I would need to add. > > this is not something that shoudl be added to the sdk or nova. > > id you want to create a vm with a set of ports with specific mac or fixed ips you shoudl first precreate the port in neutron with the relevent setting applied and then create teh server with a list of neutron port uuids. > > nova has legacy proxy apis for create vms with interfaces on spefic neturon networks or subnets but we do not plan to extend them to allow any finer grained contol then is there today https://docs.openstack.org/nova/latest/contributor/project-scope.html#no-more-api-proxies > > """ > No more API Proxies? > > Nova API current has some APIs that are now (in kilo) mostly just a proxy to other OpenStack services. If it were possible to remove a public API, these are some we might start with. As such, we don?t want to add any more. > > The first example is the API that is a proxy to the Glance v1 API. As Glance moves to deprecate its v1 API, we need to translate calls from the old v1 API we expose, to Glance?s v2 API. > > The next API to mention is the networking APIs, in particular the security groups API. Most of these APIs exist from when nova-network existed and the proxies were added during the transition. However, security groups has a much richer Neutron API, and if you use both Nova API and Neutron API, the mismatch can lead to some very unexpected results, in certain cases. > > Our intention is to avoid adding to the problems we already have in this area. > """ > > the sdk shoudl likel avoid introduceding simiarl proxy apis. > > os-migrate shoudl simple use the neutron port apis in the sdk to create the ports on the migrate neutron netowrks with the fixed ips and macs that are required and then boot the vms with those ports instead of encodeign this in the sdk. this is they type of logic that shoudl live in the calling application (os-migrate) not in the sdk its self. > > > > > > > > Is the structure of the networks object documented somewhere? > > > > > > > > Thanks > > > > > > > > --- > > > > John Ratliff > > > > Systems Automation Engineer > > > > GlobalNOC @ Indiana University > > > > > > > From johnsomor at gmail.com Tue Jun 7 16:24:19 2022 From: johnsomor at gmail.com (Michael Johnson) Date: Tue, 7 Jun 2022 09:24:19 -0700 Subject: [designate] How to avoid NXDOMAIN or stale data during cold start of a (new) machine In-Reply-To: <21cce5d5-5d21-dc08-4b08-89fdabeb6625@inovex.de> References: <69ab8e54-f419-4cd1-f289-a0b5efb7f723@inovex.de> <21cce5d5-5d21-dc08-4b08-89fdabeb6625@inovex.de> Message-ID: The agent based backends use an agent process to interact with the backend DNS server as opposed to directly talking to them over a management protocol. I wasn't part of the project when the agent based BIND9 backend was added, so I don't have the full context of why there is an agent for BIND9. I will note that all of the agent based backends are in the "untested" or "experimental" category: https://docs.openstack.org/designate/latest/admin/support-matrix.html Since the agent based drivers don't have many contributors now, I would expect to see more of them marked deprecated and removed in a future release. I would highly recommend you use the native backend if possible. Michael On Mon, Jun 6, 2022 at 11:54 PM Christian Rohmann wrote: > > On 07/06/2022 02:04, Michael Johnson wrote: > > There are two ways zones can be resynced: > > 1. Using the "designate-manage pool update" command. This will force > > an update/recreate of all of the zones. > > 2. When a zone is in ERROR or PENDING for too long, the > > WorkerPeriodicRecovery task in producer will attempt to repair the > > zone. > > Thanks again for these details! I suppose there is no option for a > targeted run of this "pool update" command to just hit one backend server? > > If you don't mind me asking yet another question ... what is the > difference between running a regular backend or an agent backend? > So > https://github.com/openstack/designate/blob/master/designate/backend/impl_bind9.py > vs. > https://github.com/openstack/designate/blob/master/designate/backend/agent_backend/impl_bind9.py > > > > Regards and thanks again, > > > Christian > From rlandy at redhat.com Tue Jun 7 17:08:31 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Tue, 7 Jun 2022 13:08:31 -0400 Subject: [tripleo] gate blocker - master/wallaby - "SELinux boolean os_enable_vtpm does not exist." Message-ID: Hello All, We have a consistent failure on multiple jobs running on check/gate/integration lines on master and wallaby changes - which shows up as a deployment error: FATAL | Enable os_enable_vtpm SELinux boolean for vTPM | standalone | error={"changed": false, "msg": "SELinux boolean os_enable_vtpm does not exist."} Details of the failure are in: https://bugs.launchpad.net/tripleo/+bug/1977873 We are investigating a fix/workaround. Please hold rechecks - we will post out again with updates. Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: From senrique at redhat.com Tue Jun 7 17:27:29 2022 From: senrique at redhat.com (Sofia Enriquez) Date: Tue, 7 Jun 2022 14:27:29 -0300 Subject: [cinder] Failed to delete volume In-Reply-To: References: Message-ID: Hi Michal, I do not think we have such a bug on the Launchpad platform.[1] Could you create a new bug report indicating which backend you are using, which Cinder version you have, and if the error occurs after following some steps? Thanks in advance, Sofia [1] https://bugs.launchpad.net/cinder On Tue, Jun 7, 2022 at 4:28 AM Michal Arbet wrote: > Hi, > > I would like to ask if someone saw issue when volume failed to delete, log > below : > This is happening from time to time. > > LOG : https://paste.opendev.org/show/b1jRCxuBFnnVzf64i9yV/ > > Thanks, > > Michal Arbet > Openstack Engineer > > Ultimum Technologies a.s. > Na Po???? 1047/26, 11000 Praha 1 > Czech Republic > > +420 604 228 897 > michal.arbet at ultimum.io > *https://ultimum.io * > > LinkedIn | Twitter > | Facebook > > -- Sof?a Enriquez she/her Software Engineer Red Hat PnT IRC: @enriquetaso @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From gthiemonge at redhat.com Tue Jun 7 17:36:29 2022 From: gthiemonge at redhat.com (Gregory Thiemonge) Date: Tue, 7 Jun 2022 19:36:29 +0200 Subject: [Octavia] Weekly meeting cancelled Message-ID: Hi, This week's upstream meeting is cancelled, because it conflicts with the OpenInfra Summit (and I'm also on PTO), See you next week, Thanks, Gregory -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Tue Jun 7 18:05:40 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Tue, 07 Jun 2022 13:05:40 -0500 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) Message-ID: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Hello Everyone, As you might know, we are redesigning the OpenStack default RBAC. The new design target two things: 1. 'new defaults (reader role)' 2. "Scope" concept It is hard to explain the details in email but the below doc is a good place to start understanding this: - https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html We as a community think 1st target (reader role) is a good thing to do and it will definitely be useful in many cases. But we need feedback on the "Scope" concept. To understand what it is and how it can impact your existing use case/deployment, please ref the documentation mentioned in the etherpad[1] (if there is any question about its design/usage we are planning, feel free to reply here or contact us in #openstack-tc IRC channel). * If you are an operator, we really need your feedback if the 'Scope' concept is a useful thing for your deployment/use-case or not. * If you are attending events have operators also attending (for example, project operator feedback (like nova[2]), forum sessions in berlin summit, ops meetup or any local operator event), please communicate about the required feedback. * Due to various reasons, many of us involved in RBAC work are not travelling to Berlin and we have this topic to be discussed in Berlin ops meetup[3] but we require someone knowing RBAC new design moderate this topic. Please reach out to us if you would like to help. Central Etherpad to collect feedback (this can be used to collect from various forums/places): * https://etherpad.opendev.org/p/rbac-operator-feedback [1] https://etherpad.opendev.org/p/rbac-operator-feedback [2] https://etherpad.opendev.org/p/nova-berlin-meet-and-greet [3]https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L74 -gmann From rajatdhasmana at gmail.com Tue Jun 7 19:14:56 2022 From: rajatdhasmana at gmail.com (Rajat Dhasmana) Date: Wed, 8 Jun 2022 00:44:56 +0530 Subject: [cinder][drivers] Register blueprints for your drivers Message-ID: Hello Driver Maintainers, If you are planning to submit your driver in the Zed development cycle, make sure you have a blueprint registered against your driver. There is a doc available[1] describing how to register a blueprint and also information related to its attributes. Also take a look at the previously filed blueprints here[2] for reference. We had a discussion in the last mid cycle (1st June) and the following is a list of drivers that are currently proposed. If your driver is in the list or not, please file a blueprint and mention the driver patches in the work items section of the blueprint that will help us prioritize the review of that driver for the Zed development cycle. 1) DataCore (reintroduced), FC and iSCSI: https://review.opendev.org/c/openstack/cinder/+/836996 2) YADRO Tatlin.UNIFIED, FC and iSCSI: https://review.opendev.org/c/openstack/cinder/+/825492 3) Dell EMC PowerStore, NFS: https://review.opendev.org/c/openstack/cinder/+/797608 4) Pure Storage, NVMe-RoCE: https://review.opendev.org/c/openstack/cinder/+/799871 5) HPE XP Storage, FC and iSCSI: https://review.opendev.org/c/openstack/cinder/+/815582 [1] https://wiki.openstack.org/wiki/Blueprints [2] https://blueprints.launchpad.net/cinder Thanks and regards Rajat Dhasmana -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlandy at redhat.com Tue Jun 7 19:53:33 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Tue, 7 Jun 2022 15:53:33 -0400 Subject: [tripleo] gate blocker - master/wallaby - "SELinux boolean os_enable_vtpm does not exist." In-Reply-To: References: Message-ID: On Tue, Jun 7, 2022 at 1:08 PM Ronelle Landy wrote: > Hello All, > > We have a consistent failure on multiple jobs running on > check/gate/integration lines on master and wallaby changes - which shows up > as a deployment error: > > FATAL | Enable os_enable_vtpm SELinux boolean for vTPM | standalone > | error={"changed": false, "msg": "SELinux boolean os_enable_vtpm does not > exist."} > > Details of the failure are in: > https://bugs.launchpad.net/tripleo/+bug/1977873 > > We are investigating a fix/workaround. > > Please hold rechecks - we will post out again with updates. > > Thank you! > > A fix is in progress - thank you Rafael and Lon: https://bugs.launchpad.net/tripleo/+bug/1977873/comments/7 https://bugs.launchpad.net/tripleo/+bug/1977873/comments/8 -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy at andybotting.com Tue Jun 7 23:19:38 2022 From: andy at andybotting.com (Andy Botting) Date: Wed, 8 Jun 2022 09:19:38 +1000 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: Hi Rados?aw, > First of all, wow, that looks very interesting and in fact very much > what I'm looking for. As I mentioned in the original message, the > things this solution lacks are not something blocking for me. > Regarding the approach to Guacamole, I know that it's preferable to > have guacamole extension (that provides the dynamic inventory) > developed rather than meddle with the internal database but I guess it > is a good start. An even better approach would be something like the Guacozy project (https://guacozy.readthedocs.io) They were able to use the Guacmole JavaScript libraries directly to embed the HTML5 desktop within a React? app. I think this is a much better approach, and I'd love to be able to do something similar in the future. Would make the integration that much nicer. > > Any "quickstart setting up" would be awesome to have at this stage. As > this is a Django app, I think I should be able to figure out the bits > and bolts to get it up and running in some shape but obviously it will > impede wider adoption. Yeah I agree. I'm in the process of documenting it, so I'll aim to get a quickstart guide together. I have a private repo with code to set up a development environment which uses Heat and Ansible - this might be the quickest way to get started. I'm happy to share this with you privately if you like. > On the note of adoption, if I find it usable, I can provide support > for it in Kolla [1] and help grow the project's adoption this way. Kolla could be useful. We're already using containers for this project now, and I have a helm chart for deploying to k8s. https://github.com/NeCTAR-RC/bumblebee-helm Also, an important part is making sure the images are set up correctly with XRDP, etc. Our images are built using Packer, and the config for them can be found at https://github.com/NeCTAR-RC/bumblebee-images > Also, since this is OpenStack-centric, maybe you could consider > migrating to OpenDev at some point to collaborate with interested > parties using a common system? > Just food for thought at the moment. I think it would be more appropriate to start a new project. I think our codebase has too many assumptions about the underlying cloud. We inherited the code from another project too, so it's got twice the cruft. > Writing to let you know I have also found the following related paper: [1] > and reached out to its authors in the hope to enable further > collaboration to happen. > The paper is not open access so I have only obtained it for myself and > am unsure if licensing permits me to share, thus I also asked the > authors to share their copy (that they have copyrights to). > I have obviously let them know of the existence of this thread. ;-) > Let's stay tuned. > > [1] https://link.springer.com/chapter/10.1007/978-3-030-99191-3_12 This looks interesting. A collaboration would be good if there is enough interest in the community. cheers, Andy From dsneddon at redhat.com Wed Jun 8 02:43:21 2022 From: dsneddon at redhat.com (Dan Sneddon) Date: Tue, 7 Jun 2022 19:43:21 -0700 Subject: Using infiniband for openstack network communications In-Reply-To: References: Message-ID: <005599d1-fcb2-bed5-b63b-0512cd1eb012@redhat.com> On 6/5/22 06:15, A Monster wrote: > Is it possible to use the infiniband port for openstack networks without > having to use the infiniband port as an ethernet port ? In theory you could use Infiniband with IPoIB for API/RPC control plane, and possibly for tenant virtual networks using VXLAN. I think you must use Ethernet mode for provider networks or external provider networks, but I could be mistaken. As of Stein it was not possible to use Infiniband for bare metal nodes using Ironic [0] with DHCP and PXE boot. There may have been additional work done since, however I never saw completion of the dependencies to make this work, but it might be possible using config-drive rather than PXE boot and DHCP. I have worked with Mellanox to use some Infiniband-capable NICs (MLX-ConnectX-5) with ML2/OVS, but only in Ethernet mode. These cards can also be used with DPDK using the "mlx5_core" driver. -- Dan Sneddon | Senior Principal Software Engineer dsneddon at redhat.com | redhat.com/cloud dsneddon:irc | @dxs:twitter From juliaashleykreger at gmail.com Wed Jun 8 05:49:18 2022 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 8 Jun 2022 07:49:18 +0200 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Message-ID: On Tue, Jun 7, 2022 at 8:10 PM Ghanshyam Mann wrote: > Hello Everyone, > > As you might know, we are redesigning the OpenStack default RBAC. The new > design target two things: > > 1. 'new defaults (reader role)' > 2. "Scope" concept > > It is hard to explain the details in email but the below doc is a good > place to start understanding this: > - > https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html > > We as a community think 1st target (reader role) is a good thing to do and > it will definitely be useful > in many cases. > > But we need feedback on the "Scope" concept. To understand what it is and > how it can impact your existing > use case/deployment, please ref the documentation mentioned in the > etherpad[1] (if there is any question > about its design/usage we are planning, feel free to reply here or contact > us in #openstack-tc IRC channel). > > * If you are an operator, we really need your feedback if the 'Scope' > concept is a useful thing for your deployment/use-case > or not. > > * If you are attending events have operators also attending (for example, > project operator feedback (like nova[2]), forum sessions > in berlin summit, ops meetup or any local operator event), please > communicate about the required feedback. > > * Due to various reasons, many of us involved in RBAC work are not > travelling to Berlin and > we have this topic to be discussed in Berlin ops meetup[3] but we > require someone knowing RBAC new design moderate > this topic. Please reach out to us if you would like to help. I previously volunteered to facilitate this at the operators meet up and given others have had to drop out, I discussed it with the ops meetup leaders and will be facilitating a session with the interested operators on Friday. I know from previous discussions I?ve had, there was quite an interest in the system level of scope access to be able to see everything across a system, so I suspect there is tons of value there, but our developer perception is obvious different if we?re questioning it at this point. > > Central Etherpad to collect feedback (this can be used to collect from > various forums/places): > > * https://etherpad.opendev.org/p/rbac-operator-feedback > > > [1] https://etherpad.opendev.org/p/rbac-operator-feedback > [2] https://etherpad.opendev.org/p/nova-berlin-meet-and-greet > [3]https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L74 > > > -gmann > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.harrison at manchester.ac.uk Wed Jun 8 07:26:34 2022 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Wed, 8 Jun 2022 07:26:34 +0000 Subject: [nova] local LVM volume on compute hosts In-Reply-To: <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> References: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> Message-ID: <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> > On 2022-06 -07, at 13:25, Sean Mooney wrote: > > no there is noting else you need to configure but this option is not what you think it is. > the images_type option contols what storage will be used for all non cinder storage. > i.e. vms that are booted with out usign a boot volume. > > by default i belive we woudl use qcow or raw files on disk > with images_type=lvm we will instead create a lvm volume for the root disk but only if you do not use the boot form volume workflow. > > there is no way curretnly to prevent usign boot form volume on a specific host and force only local storage directly via config. > > you can do this indirectly but with sideffect. > > effectivly if you have a set of hosts that cant or should not have acess to cinder you can create a spereate avaiablity zone. > you would then set [cinder]/cross_az_attch=false. e.g. create local-only az and if the cinder backend is configured to be in a differnt az then > the cross_az_attach config will prevent the vm form booting in the local-only az. > https://docs.openstack.org/nova/latest/configuration/config.html#cinder.cross_az_attach > > > so your current config will make any non boot from volume nova instance use lvm storage to provision the vm root/swap/epmeral disks > but will not prevent end users requesting cinder data voluems or boot volumes via the cli/api. if the opt in to cinder stoage that > is what they will recive but if they use teh default storage provided by the flaovr then it will be local. thanks for the explanation - it is a shame that there is not a more direct way in config to force local storage - looks like https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools has never got enough votes for implementation. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2893 bytes Desc: not available URL: From smooney at redhat.com Wed Jun 8 09:35:01 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 08 Jun 2022 10:35:01 +0100 Subject: [nova] local LVM volume on compute hosts In-Reply-To: <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> References: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> Message-ID: On Wed, 2022-06-08 at 07:26 +0000, Paul Harrison wrote: > > > On 2022-06 -07, at 13:25, Sean Mooney wrote: > > > > no there is noting else you need to configure but this option is not what you think it is. > > the images_type option contols what storage will be used for all non cinder storage. > > i.e. vms that are booted with out usign a boot volume. > > > > by default i belive we woudl use qcow or raw files on disk > > with images_type=lvm we will instead create a lvm volume for the root disk but only if you do not use the boot form volume workflow. > > > > there is no way curretnly to prevent usign boot form volume on a specific host and force only local storage directly via config. > > > > you can do this indirectly but with sideffect. > > > > effectivly if you have a set of hosts that cant or should not have acess to cinder you can create a spereate avaiablity zone. > > you would then set [cinder]/cross_az_attch=false. e.g. create local-only az and if the cinder backend is configured to be in a differnt az then > > the cross_az_attach config will prevent the vm form booting in the local-only az. > > https://docs.openstack.org/nova/latest/configuration/config.html#cinder.cross_az_attach > > > > > > so your current config will make any non boot from volume nova instance use lvm storage to provision the vm root/swap/epmeral disks > > but will not prevent end users requesting cinder data voluems or boot volumes via the cli/api. if the opt in to cinder stoage that > > is what they will recive but if they use teh default storage provided by the flaovr then it will be local. > > thanks for the explanation - it is a shame that there is not a more direct way in config to force local storage - looks like https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools has never got enough votes for implementation. > thats not what your looking for really, libvirt has a way to create pools of storage locally but nova has its own way to do it itslef. so we never used libvirts way of managing storage pools becasuse in the end it did not really provide any advantage over what we alreay had. the libvirt driver can use local lvm storage or qcow/raw files but you cannot force people to use local storage without breaking interoperablity. if they request to use cinder storage that request will be fulfilled if posible or the vm will fail to boot. by default nova always uses local stroage unless you config the image backedn to rbd/ceph or the end user request boot form volume its extra work for user to boot form volume so normally they do not do this. if your users are explcitly going out of there way to not use local storage and instead are using the boot form volume flow there might be a reason for that worflow wise. nova will never use or create a cinder volume if the user does not ask for one. if you really want to prevent them more directly and interoperablity is not a concern you can alter novas rbac policy. https://github.com/openstack/nova/blob/master/nova/policies/servers.py#L254-L264= policy.DocumentedRuleDefault( name=SERVERS % 'create:attach_volume', check_str=base.PROJECT_MEMBER, description="Create a server with the requested volume attached to it", operations=[ { 'method': 'POST', 'path': '/servers' } ], scope_types=['project']), unfortunetly i belive that policy applies to both boot form cinder root volume and boot form local storage with a secondary data volume. your other option to acive your goal is to write custom midelware like this https://github.com/openstack/nova/blob/b5029890c1c5b1b5153c9ca2fc9a8ea2437f635d/nova/api/openstack/requestlog.py and then enable it via api-paste.ini so you declare the middelware with the fully qualifed python calsse name https://github.com/openstack/nova/blob/b5029890c1c5b1b5153c9ca2fc9a8ea2437f635d/etc/nova/api-paste.ini#L45-L46= https://github.com/openstack/nova/blob/b5029890c1c5b1b5153c9ca2fc9a8ea2437f635d/etc/nova/api-paste.ini#L62-L63= then you add it to the pipeline https://github.com/openstack/nova/blob/b5029890c1c5b1b5153c9ca2fc9a8ea2437f635d/etc/nova/api-paste.ini#L77-L79= so if you want to block boot form volume but not block boot form local disk with an attached data volume you can do that with middelware. but in general i think it would be good to talk to your uses and understand why they are going through the extra effort to create boot form volume guests instead of just use the local storage which as i said above is the default for server create. > From smooney at redhat.com Wed Jun 8 09:39:46 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 08 Jun 2022 10:39:46 +0100 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Message-ID: On Wed, 2022-06-08 at 07:49 +0200, Julia Kreger wrote: > On Tue, Jun 7, 2022 at 8:10 PM Ghanshyam Mann > wrote: > > > Hello Everyone, > > > > As you might know, we are redesigning the OpenStack default RBAC. The new > > design target two things: > > > > 1. 'new defaults (reader role)' > > 2. "Scope" concept > > > > It is hard to explain the details in email but the below doc is a good > > place to start understanding this: > > - > > https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html > > > > We as a community think 1st target (reader role) is a good thing to do and > > it will definitely be useful > > in many cases. > > > > But we need feedback on the "Scope" concept. To understand what it is and > > how it can impact your existing > > use case/deployment, please ref the documentation mentioned in the > > etherpad[1] (if there is any question > > about its design/usage we are planning, feel free to reply here or contact > > us in #openstack-tc IRC channel). > > > > * If you are an operator, we really need your feedback if the 'Scope' > > concept is a useful thing for your deployment/use-case > > or not. > > > > * If you are attending events have operators also attending (for example, > > project operator feedback (like nova[2]), forum sessions > > in berlin summit, ops meetup or any local operator event), please > > communicate about the required feedback. > > > > * Due to various reasons, many of us involved in RBAC work are not > > travelling to Berlin and > > we have this topic to be discussed in Berlin ops meetup[3] but we > > require someone knowing RBAC new design moderate > > this topic. Please reach out to us if you would like to help. > > > I previously volunteered to facilitate this at the operators meet up and > given others have had to drop out, I discussed it with the ops meetup > leaders and will be facilitating a session with the interested operators on > Friday. > > I know from previous discussions I?ve had, there was quite an interest in > the system level of scope access to be able to see everything across a > system, so I suspect there is tons of value there, but our developer > perception is obvious different if we?re questioning it at this point. the system level of scope does not allow you to see everything across the system it only allows you to see the non project related resouces so you can see the flavors and host aggreates but not the instances as instances are project scoped. and project scoped resouces like ports, instances, images and volumes cannot be accessed with a system scope token if you enabel scope enforcement. that is one of the things we want to get clarity on form operators. is the disticntion between system level resouces and project level resouces useful. > > > > > > Central Etherpad to collect feedback (this can be used to collect from > > various forums/places): > > > > * https://etherpad.opendev.org/p/rbac-operator-feedback > > > > > > [1] https://etherpad.opendev.org/p/rbac-operator-feedback > > [2] https://etherpad.opendev.org/p/nova-berlin-meet-and-greet > > [3]https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L74 > > > > > > -gmann > > > > From senrique at redhat.com Wed Jun 8 13:22:54 2022 From: senrique at redhat.com (Sofia Enriquez) Date: Wed, 8 Jun 2022 10:22:54 -0300 Subject: [cinder] Bug deputy report for week of 06-08-2022 Message-ID: This is a bug report from 06-01-2022 to 06-08-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Low - https://bugs.launchpad.net/cinder/+bug/1976499 "[IBM Storwize] lsfcportsetmember is being called in the wrong SVC code level." Fix proposed to master. Wishlist - https://bugs.launchpad.net/cinder/+bug/1977868 "When create volume from image, image is fetched twice." Unassigned. Cheers, -- Sof?a Enriquez she/her Software Engineer Red Hat PnT IRC: @enriquetaso @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From dms at danplanet.com Wed Jun 8 13:53:24 2022 From: dms at danplanet.com (Dan Smith) Date: Wed, 08 Jun 2022 06:53:24 -0700 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: (Sean Mooney's message of "Wed, 08 Jun 2022 10:39:46 +0100") References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Message-ID: > the system level of scope does not allow you to see everything across the system > it only allows you to see the non project related resouces > > so you can see the flavors and host aggreates but not the instances as instances are project scoped. > and project scoped resouces like ports, instances, images and volumes cannot be accessed with a system scope > token if you enabel scope enforcement. > > that is one of the things we want to get clarity on form operators. > is the disticntion between system level resouces and project level resouces useful. Yep, exactly this. Given the amount of breakage it brings for things like Heat and Tacker, as well as the potential workflow annoyance for human admins, I really want to measure whether any operators see a benefit here. The persona roles, things like a standardized service role, and getting out of this current situation of having two sets of defaults are priorities for me. --Dan From juliaashleykreger at gmail.com Wed Jun 8 14:19:43 2022 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 8 Jun 2022 07:19:43 -0700 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Message-ID: Is that Nova's interpretation, specifically the delineation that non-project owned should only be viewable by system, or was system scope changed at some point? I interpreted it differently, but haven't circled back recently. I guess interpretation and evolution in specific pockets after initial implementation work started ultimately resulted in different perceptions. I'll make sure operators are aware there may be significant nuances and perception differences since they are using their own etherpads and their own communications flow. i.e. we're unlikely to see many find/use our own etherpads as they have their own. And yes, this is a problem, but it is a higher level community/communications feedback issue under active discussion. Granted, I get that the system scope ideas were breaking for some projects in specific use patterns since not everything would be the same nor possible (which is actually a good thing, context of use and all), but it was in theory perfect for a lot of the external audit tooling use cases which arise in so many different ways. Anyway, off to the next $thing with my scattered brain. On Wed, Jun 8, 2022 at 6:53 AM Dan Smith wrote: > > > the system level of scope does not allow you to see everything across the system > > it only allows you to see the non project related resouces > > > > so you can see the flavors and host aggreates but not the instances as instances are project scoped. > > and project scoped resouces like ports, instances, images and volumes cannot be accessed with a system scope > > token if you enabel scope enforcement. > > > > that is one of the things we want to get clarity on form operators. > > is the disticntion between system level resouces and project level resouces useful. > > Yep, exactly this. Given the amount of breakage it brings for things > like Heat and Tacker, as well as the potential workflow annoyance for > human admins, I really want to measure whether any operators see a > benefit here. The persona roles, things like a standardized service > role, and getting out of this current situation of having two sets of > defaults are priorities for me. > > --Dan From michal.arbet at ultimum.io Tue Jun 7 07:23:56 2022 From: michal.arbet at ultimum.io (Michal Arbet) Date: Tue, 7 Jun 2022 09:23:56 +0200 Subject: [cinder] Failed to delete volume Message-ID: Hi, I would like to ask if someone saw issue when volume failed to delete, log below : This is happening from time to time. 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server [req-80b56352-81b5-47b6-841e-94fface5974e fce19e20993647db8ae39d67094a4294 c8d8a385912c4731832bf1f2cf4727fb - - -] Exception during message handling: EOFError 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 359, in _recv_loop 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server self._read_trampoline() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 331, in _read_trampoline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server timeout_exc=socket_timeout('timed out')) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 210, in _trampoline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server mark_as_closed=self._mark_as_closed) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/hubs/__init__.py", line 159, in trampoline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return hub.switch() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/hubs/hub.py", line 298, in switch 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.greenlet.switch() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server eventlet.hubs.IOClosed: [Errno 107] Operation on closed file 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred: 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/osprofiler/profiler.py", line 160, in wrapper 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/osprofiler/profiler.py", line 160, in wrapper 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "", line 2, in delete_volume 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/cinder/coordination.py", line 144, in _synchronized 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server with lock(blocking): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/locking.py", line 29, in __enter__ 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.lock.__enter__(*self.args, **self.kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/locking.py", line 49, in __enter__ 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server acquired = self.acquire(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/drivers/redis.py", line 58, in wrapper 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return func(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/tooz/drivers/redis.py", line 110, in acquire 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server blocking=blocking, blocking_timeout=timeout) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/lock.py", line 182, in acquire 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server if self.do_acquire(token): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/lock.py", line 197, in do_acquire 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server if self.redis.set(self.name, token, nx=True, px=timeout): 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/client.py", line 1519, in set 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.execute_command('SET', *pieces) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/client.py", line 839, in execute_command 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self.parse_response(conn, command_name, **options) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/client.py", line 853, in parse_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server response = connection.read_response() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/sentinel.py", line 55, in read_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return super(SentinelManagedConnection, self).read_response() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 700, in read_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server response = self._parser.read_response() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 310, in read_response 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server response = self._buffer.readline() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 242, in readline 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server self._read_from_socket() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/connection.py", line 184, in _read_from_socket 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server data = recv(self._sock, socket_read_size) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/redis/_compat.py", line 71, in recv 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return sock.recv(*args, **kwargs) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 365, in recv 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server return self._recv_loop(self.fd.recv, b'', bufsize, flags) 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/eventlet/greenio/base.py", line 362, in _recv_loop 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server raise EOFError() 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server EOFError 2022-05-23 10:15:24.414 833 ERROR oslo_messaging.rpc.server Thanks, Michal Arbet Openstack Engineer Ultimum Technologies a.s. Na Po???? 1047/26, 11000 Praha 1 Czech Republic +420 604 228 897 michal.arbet at ultimum.io *https://ultimum.io * LinkedIn | Twitter | Facebook -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andrew.Norrie at cgg.com Mon Jun 6 14:13:22 2022 From: Andrew.Norrie at cgg.com (Norrie, Andrew) Date: Mon, 6 Jun 2022 14:13:22 +0000 Subject: [kolla-ansible][xena] Cell database HA/redundancy? In-Reply-To: References: Message-ID: Hi Mark, Thanks for the reply. I'll look into the HAProxy "custom config" as suggested and using different ports for the cell databases. Maybe that can be worked in. We are expanding our OpenStack/kolla-ansible personnel/knowledge so will look into the possibility for us to contribute as reviewers/testers as a start. Best regards .... Andrew From: Mark Goddard Sent: Monday, June 6, 2022 2:22 AM To: Norrie, Andrew Cc: openstack-discuss at lists.openstack.org Subject: Re: [kolla-ansible][xena] Cell database HA/redundancy? On Fri, 3 Jun 2022 at 14:10, Norrie, Andrew > wrote: Hi, We are currently planning some large OpenStack deployments utilizing kolla-ansible and I'm curious what folks are doing with the cell database HA/redundancy. With kolla-ansible (xena) it appears that a loadbalancer setup is only allowed for the default database shard (shard 0)... reference: https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb-guide.html If we are setting up separate cell database shards with Galera, I'm wondering if there is a convenient work-around or configuration for implementation of HA for these cell databases. In the inventory group_vars directory you can specify the group variables (for each cell database) as: nova_cell_database_address: nova_cell_database_group: but these aren't virtual IP hosts/addresses (non default db shard). This works perfectly fine with a single server cell database but not Galera. If that db server goes down the cell instance information is lost. Hi Andrew, You are correct - only the first shard is load balanced. The sharding feature was actually just the first patch in a series intended to support proxysql. I believe this would add the functionality you are looking for. In fact, the proposed test case for proxysql in our CI is a multi-cell deployment. Here is the patch series: https://review.opendev.org/q/topic:proxysql Unfortunately it has been stuck for a while, largely due to core reviewer bandwidth. The author is Michael Arbet, who is often on IRC as kevko. I'd suggest registering your interest in these patches via gerrit, and at one of the weekly kolla IRC meetings (this week is cancelled due to the summit). If you have time available, then testing and providing reviews could help to get the patches moving. In the meantime, you can configure HAProxy to load balance additional services, by placing an HAProxy config snippet in /etc/kolla/config/haproxy/services.d/*.cfg. Regards, Mark Many thanks ... Andrew ________________________________ "This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately." ________________________________ "This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately." -------------- next part -------------- An HTML attachment was scrubbed... URL: From moshele at nvidia.com Mon Jun 6 20:48:56 2022 From: moshele at nvidia.com (Moshe Levi) Date: Mon, 6 Jun 2022 20:48:56 +0000 Subject: Using infiniband for openstack network communications In-Reply-To: References: Message-ID: If you are using Mellanox (Nvidia) NIC with inbox driver the default mode is ipoib Enhanced mode [1] Which support for acceleration for the IPoIB [1] - https://www.spinics.net/lists/linux-rdma/msg46802.html From: Dmitriy Rabotyagov Sent: Monday, June 6, 2022 4:52 PM Cc: openstack-discuss Subject: Re: Using infiniband for openstack network communications You don't often get email from noonedeadpunk at gmail.com. Learn why this is important External email: Use caution opening links or attachments Ah, ok. Well, you can use infiniband for control plane only with IPoIB. There's no RDMA support if that's what you're asking. ??, 6 ???. 2022 ?., 15:36 A Monster >: What I wanted to actually do is not what is done in this link https://satishdotpatel.github.io/HPC-on-openstack/ , because here vm's are given the possibility to use the infiniband port through Virtual functions, but in my case, I want to use infiniband for openstack management, compute and storage networks, to get better performance than when using ethernet and rg45 cables. so I'm wondering if it's feasible and whether it's a good thing or not. Thank you. On Mon, 6 Jun 2022 at 14:15, A Monster > wrote: Thank you for your reply, I'll check the link you've sent to me immediately. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lokendrarathour at gmail.com Tue Jun 7 07:24:20 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Tue, 7 Jun 2022 12:54:20 +0530 Subject: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version In-Reply-To: References: Message-ID: Hi Swogat, Yes after correcting the j2 template, as suggested by you, it started working. Thanks a lot once again. Now the Overcloud is deployed. Further, as a next requirement, we were also trying to add an *additional Node as a bare-metal Instance* on the overcloud. As it is the composable network approach, we add an entry in parameters default for ServiceNetMap, as like below in environments/network-environments.yaml parameter_defaults: # This section is where deployment-specific configuration is done ServiceNetMap: IronicApiNetwork: oc_provisioning IronicNetwork: oc_provisioning this is now passed as in the environment.yaml as parameter_defaults, but somehow we are not able to deploy the setup. it gives the error for containers. *2022-06-06 19:08:37.560800 | 525400fd-f24d-9703-b0e7-0000000084b2 | FATAL | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | overcloud-controller-0 | error={"changed": false, "msg": "Service ironic_pxe_tftp has not started yet"}* and if we remove this setting as stated above, the deployment is successful, but at the time of node provisioning it is not able to reach the TFTP server to pull files, mainly because of the composable network (oc_provisioning), we faced this issue in Train, so we did the above config changes and it working. But as it looks something more is needed in Wallaby for Baremetal provisioning as an instance. Please share your learnings with the reference to it. Thanks once again for your all-time support. reference link: Bare Metal Instances in Overcloud ? TripleO 3.0.0 documentation (openstack.org) -Lokendra On Wed, Jun 1, 2022 at 12:19 AM Lokendra Rathour wrote: > Ok Swogat, > Thanks once again. > i will try this approach once and will let you know. > > > On Wed, 1 Jun 2022, 00:04 Swogat Pradhan, > wrote: > >> Hi Lokendra, >> Like i said, >> > NOTE: when passing --network-config parameter in node provision step, >> it creates a directory in /etc/os-net-config and in it creates a file >> > config.yaml, do check the indentation of that file. (in my case the >> indentation was wrong when i was using bonding everytime, so i had to >> > manually change the script and run a while loop and then my node >> provision step was successful) >> this parameter --network-config creates a config.yaml(network config) >> file in /etc/os-net-config directory in the overcloud nodes and then >> ansible tries to apply the network config from the generated config file. >> And in wallaby version if you are using bonding then the syntax in the >> generated conf is wrong (atleast was in my case) and then the ansible tries >> to apply the network config (with wrong syntax) so your overcloud nodes >> become unavailable. >> >> Please run node provision separately, and keep on monitoring "metalsmith >> list" command. Once IP is assigned to the overcloud nodes, ssh to the nodes >> and assign a password to heat-admin user so that even if the network >> becomes unavailable you still will be able to access the nodes via console >> access, then you can visit /etc/os-net-config directory and verify the >> config.yaml file. >> >> With regards, >> Swogat pradhan. >> >> On Tue, May 31, 2022 at 11:56 PM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Hi Swogat, >>> Thanks once again for your input it really helped much. >>> >>> instead of running mentioned those three provisioning steps i used >>> alternate method and passed directly in deploy command. now my current >>> deploy command is: >>> >>> openstack overcloud deploy --templates \ >>> --networks-file /home/stack/templates/custom_network_data.yaml \ >>> --vip-file /home/stack/templates/custom_vip_data.yaml \ >>> --baremetal-deployment >>> /home/stack/templates/overcloud-baremetal-deploy.yaml \ >>> --network-config \ >>> -e /home/stack/templates/environment.yaml \ >>> -e >>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml >>> \ >>> -e >>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml >>> \ >>> -e >>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml >>> \ >>> -e /home/stack/templates/ironic-config.yaml \ >>> -e >>> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ >>> -e >>> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ >>> -e /home/stack/containers-prepare-parameter.yaml >>> >>> The files as suggested by you are well created. >>> >>> But once we run the deployment I get this error for all nodes: >>> >>> 0:00:16.064781 | 1.16s >>> 2022-05-31 19:13:00.276954 | 525400ef-b928-9ded-fecc-000000000094 | >>> TASK | Run tripleo_os_net_config_module with network_config >>> 2022-05-31 19:40:30.061582 | 525400ef-b928-9ded-fecc-000000000094 | >>> FATAL | Run tripleo_os_net_config_module with network_config | >>> overcloud-controller-1 | error={"msg": "Data could not be sent to remote >>> host \"30.30.30.117\". Make sure this host can be reached over ssh: ssh: >>> connect to host 30.30.30.117 port 22: No route to host\r\n"} >>> 2022- >>> >>> >>> Baremetal node list are showing in as active. >>> >>> (undercloud) [stack at undercloud ~]$ openstack baremetal node list >>> /usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: >>> DeprecationWarning: The _yaml extension module is now located at yaml._yaml >>> and its location is subject to change. To use the LibYAML-based parser and >>> emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper >>> as Dumper`. >>> DeprecationWarning >>> >>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>> | UUID | Name | Instance UUID >>> | Power State | Provisioning State | Maintenance | >>> >>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>> | 1a4d873c-f9f7-4504-a3af-92c11f954171 | node-a | >>> 901453a1-183f-4de8-aaab-0f38be2be455 | power on | active | >>> False | >>> | d18610fc-9532-410c-918e-8efc326c89f8 | node-b | >>> d059b94a-8357-4f8e-a0d8-15a24b0c1afe | power on | active | >>> False | >>> | b69f2d5a-5b18-4453-8843-15c6af79aca0 | node-c | >>> f196ef3a-7950-47b9-a5ae-751f06b18f75 | power on | active | >>> False | >>> | 8a38c584-f812-4ebc-a0b1-4299f0917637 | node-d | >>> 1636517c-2ab2-43d7-8205-9f02c5290207 | power on | active | >>> False | >>> >>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>> >>> Some config is missing it seems, please check once and advise. >>> >>> >>> On Tue, May 31, 2022 at 11:40 AM Swogat Pradhan < >>> swogatpradhan22 at gmail.com> wrote: >>> >>>> Hi Lokendra, >>>> You need to generate another file also in the following step: openstack >>>> overcloud node provision --stack overcloud --overcloud-ssh-key >>>> /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml also you >>>> need to pass another parameter --network-config. >>>> example: >>>> openstack overcloud node provision --stack overcloud >>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa *--network-config* *--output >>>> overcloud-baremetal-deployed.yaml* overcloud-baremetal-deploy.yaml >>>> >>>> And then all these output files will be passed on to the openstack >>>> overcloud deploy command. >>>> NOTE: when passing --network-config parameter in node provision step, >>>> it creates a directory in /etc/os-net-config and in it creates a file >>>> config.yaml, do check the indentation of that file. (in my case the >>>> indentation was wrong when i was using bondind everytime, so i had to >>>> manually change the script and run a while loop and then my node provision >>>> step was successful) >>>> >>>> On Tue, May 31, 2022 at 8:59 AM Lokendra Rathour < >>>> lokendrarathour at gmail.com> wrote: >>>> >>>>> Hi Swogat, >>>>> I tried generating the scripts as used by you in your deployments >>>>> using the >>>>> >>>>> >>>>> #openstack overcloud network provision --stack overcloud --output >>>>> networks-deployed-environment.yaml custom_network_data.yaml >>>>> # openstack overcloud network vip provision --stack overcloud >>>>> --output vip-deployed-environment.yaml custom_vip_data.yaml >>>>> # openstack overcloud node provision --stack overcloud >>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa >>>>> overcloud-baremetal-deploy.yaml >>>>> >>>>> and used the first two in the final deployment script, but it gives >>>>> the error: >>>>> >>>>> heatclient.exc.HTTPInternalServerError: ERROR: Internal Error >>>>> 2022-05-30 14:14:39.772 479668 ERROR >>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent >>>>> call last):\n', ' File "/usr/lib/python3.6/ted_stack\n >>>>> nested_stack.validate()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in >>>>> wrapper\n result = f(*args, ine 969, in validate\n result = >>>>> res.validate()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/openstack/neutron/port.py", >>>>> line 454site-packages/heat/engine/resources/openstack/neutron/neutron.py", >>>>> line 43, in validate\n res = super(NeutronResource, self).validate()\n', >>>>> ' File "/un return self.validate_template()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 1882, in >>>>> validate_template\n self.t.rpy", line 200, in >>>>> _validate_service_availability\n raise ex\n', >>>>> 'heat.common.exception.ResourceTypeUnavailable: HEAT-E99001 Service neutron >>>>> is not avaieutron network endpoint is not in service catalog.\n', '\nDuring >>>>> handling of the above exception, another exception occurred:\n\n', >>>>> 'Traceback (most recens/stack_resource.py", line 75, in >>>>> validate_nested_stack\n nested_stack.validate()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py"thon3.6/site-packages/heat/engine/stack.py", >>>>> line 969, in validate\n result = res.validate()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/ateResource, >>>>> self).validate()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", >>>>> line 65, in validate\n self.validources/stack_resource.py", line 81, in >>>>> validate_nested_stack\n ex, path=[self.stack.t.RESOURCES, path])\n', >>>>> 'heat.common.exception.StackValidationFaileeploy/overcloud/tripleo-heat-templates/deployed-server/deployed-server.yaml>: >>>>> HEAT-E99001 Service neutron is not available for resource type >>>>> OS::TripleO::vice catalog.\n', '\nDuring handling of the above exception, >>>>> another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' >>>>> File "/usr/lib/pline 320, in validate_nested_stack\n >>>>> nested_stack.validate()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in >>>>> wrappe/heat/engine/stack.py", line 969, in validate\n result = >>>>> res.validate()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/template_relidate()\n', >>>>> ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", >>>>> line 65, in validate\n self.validate_nested_stack()\n'.py", line 81, in >>>>> validate_nested_stack\n ex, path=[self.stack.t.RESOURCES, path])\n', >>>>> 'heat.common.exception.StackValidationFailed: >>>>> ResourceTypeUnavaimplates/puppet/compute-role.yaml>.resources.NovaCompute>>>> reason: neutron network endpoint is not in service catalog.\n', '\nDuring >>>>> handling of the above exception, >>>>> another/lib/python3.6/site-packages/heat/common/context.py", line 416, in >>>>> wrapped\n return func(self, ctx, *args, **kwargs)\n', ' File >>>>> "/usr/lib/python3.6/sirce_name, template_id)\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 756, in >>>>> _parse_template_and_validate_stack\n stack.v line 160, in wrapper\n >>>>> result = f(*args, **kwargs)\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 969, in >>>>> validate\n resesources/stack_resource.py", line 65, in validate\n >>>>> self.validate_nested_stack()\n', ' File >>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/oph=[self.stack.t.RESOURCES, >>>>> path])\n', 'heat.common.exception.StackValidationFailed: >>>>> ResourceTypeUnavailable: >>>>> resources.Compute.resources.0.resources.NovaCompute: >>>>> HEAT-E9900::ControlPlanePort, reason: neutron network endpoint is not in >>>>> service catalog.\n']. >>>>> >>>>> Request you to check once, please. >>>>> >>>>> >>>>> >>>>> >>>>> On Mon, May 30, 2022 at 11:06 AM Lokendra Rathour < >>>>> lokendrarathour at gmail.com> wrote: >>>>> >>>>>> Hi Swogat, >>>>>> Thanks once again. >>>>>> >>>>>> with the files as shown below I am running the overcloud deploy for >>>>>> wallaby using this command: >>>>>> >>>>>> (undercloud) [stack at undercloud ~]$ cat deploy_overcloud_working_1.sh >>>>>> openstack overcloud deploy --templates \ >>>>>> -n /home/stack/templates/network_data.yaml \ >>>>>> -r /home/stack/templates/roles_data.yaml \ >>>>>> -e /home/stack/templates/environment.yaml \ >>>>>> -e /home/stack/templates/environments/network-isolation.yaml \ >>>>>> -e /home/stack/templates/environments/network-environment.yaml \ >>>>>> -e >>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml >>>>>> \ >>>>>> -e >>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml >>>>>> \ >>>>>> -e >>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml >>>>>> \ >>>>>> -e /home/stack/templates/ironic-config.yaml \ >>>>>> -e >>>>>> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ >>>>>> -e >>>>>> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ >>>>>> -e /home/stack/containers-prepare-parameter.yaml >>>>>> (undercloud) [stack at undercloud ~]$ >>>>>> >>>>>> >>>>>> This deployment is on ipv6 using triple0 wallaby, templates, as >>>>>> mentioned below, are generated using rendering steps and the >>>>>> network_data.yaml the roles_data.yaml >>>>>> Steps used to render the templates: >>>>>> cd /usr/share/openstack-tripleo-heat-templates/ >>>>>> ./tools/process-templates.py -o >>>>>> ~/openstack-tripleo-heat-templates-rendered_at_wallaby -n >>>>>> /home/stack/templates/network_data.yaml -r >>>>>> /home/stack/templates/roles_data.yaml >>>>>> >>>>>> *Now if i try adding the related to VIP port I do get the error as:* >>>>>> >>>>>> 2022-05-30 10:37:12.792 979387 WARNING >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template >>>>>> to file: >>>>>> /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/controller-role.yaml >>>>>> 2022-05-30 10:37:12.792 979387 WARNING >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template >>>>>> to file: >>>>>> /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/compute-role.yaml >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured >>>>>> while running the command: ValueError: The environment is not a valid YAML >>>>>> mapping data type. >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent >>>>>> call last): >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, >>>>>> self).run(parsed_args) >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in >>>>>> run >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, >>>>>> self).run(parsed_args) >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = >>>>>> self.take_action(parsed_args) or 0 >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>> line 1189, in take_action >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud stack, parsed_args, >>>>>> new_tht_root, user_tht_root) >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>> line 227, in create_env_files >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud created_env_files, >>>>>> parsed_args, new_tht_root, user_tht_root) >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>> line 204, in build_image_params >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud cleanup=(not >>>>>> parsed_args.no_cleanup)) >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 1929, in >>>>>> process_multiple_environments >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud env_path=env_path, >>>>>> include_env_in_files=include_env_in_files) >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", >>>>>> line 326, in process_environment_and_files >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud env = >>>>>> environment_format.parse(raw_env) >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>> "/usr/lib/python3.6/site-packages/heatclient/common/environment_format.py", >>>>>> line 50, in parse >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud raise >>>>>> ValueError(_('The environment is not a valid ' >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud ValueError: The >>>>>> environment is not a valid YAML mapping data type. >>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud >>>>>> 2022-05-30 10:37:14.457 979387 ERROR openstack [-] The environment is >>>>>> not a valid YAML mapping data type. >>>>>> 2022-05-30 10:37:14.457 979387 INFO osc_lib.shell [-] END return >>>>>> value: 1 >>>>>> (undercloud) [stack at undercloud ~]$ >>>>>> >>>>>> This is more of a syntax error where it is not able to understand the >>>>>> passed VIP data file: >>>>>> >>>>>> undercloud) [stack at undercloud ~]$ cat >>>>>> /home/stack/templates/vip-data-default-network-isolation.yaml >>>>>> - >>>>>> dns_name: overcloud >>>>>> network: internal_api >>>>>> - >>>>>> dns_name: overcloud >>>>>> network: external >>>>>> - >>>>>> dns_name: overcloud >>>>>> network: ctlplane >>>>>> - >>>>>> dns_name: overcloud >>>>>> network: oc_provisioning >>>>>> - >>>>>> dns_name: overcloud >>>>>> network: j3mgmt >>>>>> >>>>>> >>>>>> Please advise, also please note that similar templates generated in >>>>>> prior releases such as train/ussuri works perfectly. >>>>>> >>>>>> >>>>>> >>>>>> Please check the list of *templates *files: >>>>>> >>>>>> drwxr-xr-x. 2 stack stack 68 May 30 09:22 environments >>>>>> -rw-r--r--. 1 stack stack 265 May 27 13:47 environment.yaml >>>>>> -rw-rw-r--. 1 stack stack 297 May 27 13:47 init-repo.yaml >>>>>> -rw-r--r--. 1 stack stack 570 May 27 13:47 ironic-config.yaml >>>>>> drwxrwxr-x. 4 stack stack 4096 May 27 13:53 network >>>>>> -rw-r--r--. 1 stack stack 6370 May 27 14:26 network_data.yaml >>>>>> -rw-r--r--. 1 stack stack 11137 May 27 13:53 roles_data.yaml >>>>>> -rw-r--r--. 1 stack stack 234 May 30 09:23 >>>>>> vip-data-default-network-isolation.yaml >>>>>> >>>>>> >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ cat environment.yaml >>>>>> >>>>>> parameter_defaults: >>>>>> OvercloudControllerFlavor: control >>>>>> OvercloudComputeFlavor: compute >>>>>> ControllerCount: 3 >>>>>> ComputeCount: 1 >>>>>> TimeZone: 'Asia/Kolkata' >>>>>> NtpServer: ['30.30.30.3'] >>>>>> NeutronBridgeMappings: datacentre:br-tenant >>>>>> NeutronFlatNetworks: datacentre >>>>>> (undercloud) [stack at undercloud templates]$ >>>>>> >>>>>> >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ cat ironic-config.yaml >>>>>> >>>>>> parameter_defaults: >>>>>> NovaSchedulerDefaultFilters: >>>>>> - AggregateInstanceExtraSpecsFilter >>>>>> - AvailabilityZoneFilter >>>>>> - ComputeFilter >>>>>> - ComputeCapabilitiesFilter >>>>>> - ImagePropertiesFilter >>>>>> IronicEnabledHardwareTypes: >>>>>> - ipmi >>>>>> - redfish >>>>>> IronicEnabledPowerInterfaces: >>>>>> - ipmitool >>>>>> - redfish >>>>>> IronicEnabledManagementInterfaces: >>>>>> - ipmitool >>>>>> - redfish >>>>>> IronicCleaningDiskErase: metadata >>>>>> IronicIPXEEnabled: true >>>>>> IronicInspectorSubnets: >>>>>> - ip_range: 172.23.3.100,172.23.3.150 >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ cat network_data.yaml >>>>>> >>>>>> - name: J3Mgmt >>>>>> name_lower: j3mgmt >>>>>> vip: true >>>>>> vlan: 400 >>>>>> ipv6: true >>>>>> ipv6_subnet: 'fd80:fd00:fd00:4000::/64' >>>>>> ipv6_allocation_pools: [{'start': 'fd80:fd00:fd00:4000::10', 'end': >>>>>> 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] >>>>>> mtu: 9000 >>>>>> >>>>>> >>>>>> >>>>>> - name: InternalApi >>>>>> name_lower: internal_api >>>>>> vip: true >>>>>> vlan: 418 >>>>>> ipv6: true >>>>>> ipv6_subnet: 'fd00:fd00:fd00:2000::/64' >>>>>> ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:2000::10', 'end': >>>>>> 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] >>>>>> mtu: 9000 >>>>>> >>>>>> >>>>>> - name: External >>>>>> vip: true >>>>>> name_lower: external >>>>>> vlan: 408 >>>>>> ipv6: true >>>>>> gateway_ipv6: 'fd00:fd00:fd00:9900::1' >>>>>> ipv6_subnet: 'fd00:fd00:fd00:9900::/64' >>>>>> ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:9900::10', 'end': >>>>>> 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] >>>>>> mtu: 9000 >>>>>> >>>>>> >>>>>> - name: OCProvisioning >>>>>> vip: true >>>>>> name_lower: oc_provisioning >>>>>> vlan: 412 >>>>>> ip_subnet: '172.23.3.0/24' >>>>>> allocation_pools: [{'start': '172.23.3.10', 'end': '172.23.3.50'}] >>>>>> mtu: 9000 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ cat roles_data.yaml >>>>>> >>>>>> >>>>>> ############################################################################### >>>>>> # File generated by TripleO >>>>>> >>>>>> ############################################################################### >>>>>> >>>>>> ############################################################################### >>>>>> # Role: Controller >>>>>> # >>>>>> >>>>>> ############################################################################### >>>>>> - name: Controller >>>>>> description: | >>>>>> Controller role that has all the controller services loaded and >>>>>> handles >>>>>> Database, Messaging, and Network functions. >>>>>> CountDefault: 1 >>>>>> tags: >>>>>> - primary >>>>>> - controller >>>>>> # Create external Neutron bridge for SNAT (and floating IPs when >>>>>> using >>>>>> # ML2/OVS without DVR) >>>>>> - external_bridge >>>>>> networks: >>>>>> External: >>>>>> subnet: external_subnet >>>>>> InternalApi: >>>>>> subnet: internal_api_subnet >>>>>> OCProvisioning: >>>>>> subnet: oc_provisioning_subnet >>>>>> J3Mgmt: >>>>>> subnet: j3mgmt_subnet >>>>>> >>>>>> >>>>>> # For systems with both IPv4 and IPv6, you may specify a gateway >>>>>> network for >>>>>> # each, such as ['ControlPlane', 'External'] >>>>>> default_route_networks: ['External'] >>>>>> HostnameFormatDefault: '%stackname%-controller-%index%' >>>>>> RoleParametersDefault: >>>>>> OVNCMSOptions: "enable-chassis-as-gw" >>>>>> # Deprecated & backward-compatible values (FIXME: Make parameters >>>>>> consistent) >>>>>> # Set uses_deprecated_params to True if any deprecated params are >>>>>> used. >>>>>> uses_deprecated_params: True >>>>>> deprecated_param_extraconfig: 'controllerExtraConfig' >>>>>> deprecated_param_flavor: 'OvercloudControlFlavor' >>>>>> deprecated_param_image: 'controllerImage' >>>>>> deprecated_nic_config_name: 'controller.yaml' >>>>>> update_serial: 1 >>>>>> ServicesDefault: >>>>>> - OS::TripleO::Services::Aide >>>>>> - OS::TripleO::Services::AodhApi >>>>>> - OS::TripleO::Services::AodhEvaluator >>>>>> >>>>>> .. >>>>>> . >>>>>> >>>>>> >>>>>> ..############################################################################### >>>>>> # Role: Compute >>>>>> # >>>>>> >>>>>> ############################################################################### >>>>>> - name: Compute >>>>>> description: | >>>>>> Basic Compute Node role >>>>>> CountDefault: 1 >>>>>> # Create external Neutron bridge (unset if using ML2/OVS without >>>>>> DVR) >>>>>> tags: >>>>>> - compute >>>>>> - external_bridge >>>>>> networks: >>>>>> InternalApi: >>>>>> subnet: internal_api_subnet >>>>>> J3Mgmt: >>>>>> subnet: j3mgmt_subnet >>>>>> HostnameFormatDefault: '%stackname%-novacompute-%index%' >>>>>> RoleParametersDefault: >>>>>> FsAioMaxNumber: 1048576 >>>>>> TunedProfileName: "virtual-host" >>>>>> # Deprecated & backward-compatible values (FIXME: Make parameters >>>>>> consistent) >>>>>> # Set uses_deprecated_params to True if any deprecated params are >>>>>> used. >>>>>> # These deprecated_params only need to be used for existing roles >>>>>> and not for >>>>>> # composable roles. >>>>>> uses_deprecated_params: True >>>>>> deprecated_param_image: 'NovaImage' >>>>>> deprecated_param_extraconfig: 'NovaComputeExtraConfig' >>>>>> deprecated_param_metadata: 'NovaComputeServerMetadata' >>>>>> deprecated_param_scheduler_hints: 'NovaComputeSchedulerHints' >>>>>> deprecated_param_ips: 'NovaComputeIPs' >>>>>> deprecated_server_resource_name: 'NovaCompute' >>>>>> >>>>>> deprecated_nic_config_name: 'compute.yaml' >>>>>> update_serial: 25 >>>>>> ServicesDefault: >>>>>> - OS::TripleO::Services::Aide >>>>>> - OS::TripleO::Services::AuditD >>>>>> - OS::TripleO::Services::BootParams >>>>>> >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ cat >>>>>> environments/network-environment.yaml >>>>>> >>>>>> #This file is an example of an environment file for defining the >>>>>> isolated >>>>>> #networks and related parameters. >>>>>> resource_registry: >>>>>> # Network Interface templates to use (these files must exist). You >>>>>> can >>>>>> # override these by including one of the net-*.yaml environment >>>>>> files, >>>>>> # such as net-bond-with-vlans.yaml, or modifying the list here. >>>>>> # Port assignments for the Controller >>>>>> OS::TripleO::Controller::Net::SoftwareConfig: OS::Heat::None >>>>>> # Port assignments for the Compute >>>>>> OS::TripleO::Compute::Net::SoftwareConfig: OS::Heat::None >>>>>> >>>>>> >>>>>> parameter_defaults: >>>>>> # This section is where deployment-specific configuration is done >>>>>> # >>>>>> ServiceNetMap: >>>>>> IronicApiNetwork: oc_provisioning >>>>>> IronicNetwork: oc_provisioning >>>>>> >>>>>> >>>>>> >>>>>> # This section is where deployment-specific configuration is done >>>>>> ControllerNetworkConfigTemplate: >>>>>> 'templates/bonds_vlans/bonds_vlans.j2' >>>>>> ComputeNetworkConfigTemplate: 'templates/bonds_vlans/bonds_vlans.j2' >>>>>> >>>>>> >>>>>> >>>>>> # Customize the IP subnet to match the local environment >>>>>> J3MgmtNetCidr: 'fd80:fd00:fd00:4000::/64' >>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>> J3MgmtAllocationPools: [{'start': 'fd80:fd00:fd00:4000::10', 'end': >>>>>> 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] >>>>>> # Customize the VLAN ID to match the local environment >>>>>> J3MgmtNetworkVlanID: 400 >>>>>> >>>>>> >>>>>> >>>>>> # Customize the IP subnet to match the local environment >>>>>> InternalApiNetCidr: 'fd00:fd00:fd00:2000::/64' >>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>> InternalApiAllocationPools: [{'start': 'fd00:fd00:fd00:2000::10', >>>>>> 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] >>>>>> # Customize the VLAN ID to match the local environment >>>>>> InternalApiNetworkVlanID: 418 >>>>>> >>>>>> >>>>>> >>>>>> # Customize the IP subnet to match the local environment >>>>>> ExternalNetCidr: 'fd00:fd00:fd00:9900::/64' >>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>> # Leave room if the external network is also used for floating IPs >>>>>> ExternalAllocationPools: [{'start': 'fd00:fd00:fd00:9900::10', >>>>>> 'end': 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] >>>>>> # Gateway router for routable networks >>>>>> ExternalInterfaceDefaultRoute: 'fd00:fd00:fd00:9900::1' >>>>>> # Customize the VLAN ID to match the local environment >>>>>> ExternalNetworkVlanID: 408 >>>>>> >>>>>> >>>>>> >>>>>> # Customize the IP subnet to match the local environment >>>>>> OCProvisioningNetCidr: '172.23.3.0/24' >>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>> OCProvisioningAllocationPools: [{'start': '172.23.3.10', 'end': >>>>>> '172.23.3.50'}] >>>>>> # Customize the VLAN ID to match the local environment >>>>>> OCProvisioningNetworkVlanID: 412 >>>>>> >>>>>> >>>>>> >>>>>> # List of Neutron network types for tenant networks (will be used >>>>>> in order) >>>>>> NeutronNetworkType: 'geneve,vlan' >>>>>> # Neutron VLAN ranges per network, for example >>>>>> 'datacentre:1:499,tenant:500:1000': >>>>>> NeutronNetworkVLANRanges: 'datacentre:1:1000' >>>>>> # Customize bonding options, e.g. "mode=4 lacp_rate=1 updelay=1000 >>>>>> miimon=100" >>>>>> # for Linux bonds w/LACP, or "bond_mode=active-backup" for OVS >>>>>> active/backup. >>>>>> BondInterfaceOvsOptions: "bond_mode=active-backup" >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ >>>>>> >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ cat >>>>>> environments/network-isolation.yaml >>>>>> >>>>>> # NOTE: This template is now deprecated, and is only included for >>>>>> compatibility >>>>>> # when upgrading a deployment where this template was originally >>>>>> used. For new >>>>>> # deployments, set "ipv6: true" on desired networks in >>>>>> network_data.yaml, and >>>>>> # include network-isolation.yaml. >>>>>> # >>>>>> # Enable the creation of Neutron networks for isolated Overcloud >>>>>> # traffic and configure each role to assign ports (related >>>>>> # to that role) on these networks. >>>>>> resource_registry: >>>>>> # networks as defined in network_data.yaml >>>>>> OS::TripleO::Network::J3Mgmt: ../network/j3mgmt_v6.yaml >>>>>> OS::TripleO::Network::InternalApi: ../network/internal_api_v6.yaml >>>>>> OS::TripleO::Network::External: ../network/external_v6.yaml >>>>>> OS::TripleO::Network::OCProvisioning: >>>>>> ../network/oc_provisioning.yaml >>>>>> >>>>>> >>>>>> # Port assignments for the VIPs >>>>>> OS::TripleO::Network::Ports::J3MgmtVipPort: >>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>> OS::TripleO::Network::Ports::InternalApiVipPort: >>>>>> ../network/ports/internal_api_v6.yaml >>>>>> OS::TripleO::Network::Ports::ExternalVipPort: >>>>>> ../network/ports/external_v6.yaml >>>>>> OS::TripleO::Network::Ports::OCProvisioningVipPort: >>>>>> ../network/ports/oc_provisioning.yaml >>>>>> >>>>>> >>>>>> >>>>>> # Port assignments by role, edit role definition to assign networks >>>>>> to roles. >>>>>> # Port assignments for the Controller >>>>>> OS::TripleO::Controller::Ports::J3MgmtPort: >>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>> OS::TripleO::Controller::Ports::InternalApiPort: >>>>>> ../network/ports/internal_api_v6.yaml >>>>>> OS::TripleO::Controller::Ports::ExternalPort: >>>>>> ../network/ports/external_v6.yaml >>>>>> OS::TripleO::Controller::Ports::OCProvisioningPort: >>>>>> ../network/ports/oc_provisioning.yaml >>>>>> # Port assignments for the Compute >>>>>> OS::TripleO::Compute::Ports::J3MgmtPort: >>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>> OS::TripleO::Compute::Ports::InternalApiPort: >>>>>> ../network/ports/internal_api_v6.yaml >>>>>> >>>>>> >>>>>> >>>>>> parameter_defaults: >>>>>> # Enable IPv6 environment for Manila >>>>>> ManilaIPv6: True >>>>>> >>>>>> (undercloud) [stack at undercloud templates]$ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Tue, May 24, 2022 at 5:04 PM Lokendra Rathour < >>>>>> lokendrarathour at gmail.com> wrote: >>>>>> >>>>>>> Thanks, I'll check them out. >>>>>>> will let you know in case it works out. >>>>>>> >>>>>>> On Tue, May 24, 2022 at 2:37 PM Swogat Pradhan < >>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> Please find the below templates: >>>>>>>> These are for openstack wallaby release: >>>>>>>> >>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>> custom_network_data.yaml >>>>>>>> - name: Storage >>>>>>>> name_lower: storage >>>>>>>> vip: true >>>>>>>> mtu: 1500 >>>>>>>> subnets: >>>>>>>> storage_subnet: >>>>>>>> ip_subnet: 172.25.202.0/26 >>>>>>>> allocation_pools: >>>>>>>> - start: 172.25.202.6 >>>>>>>> end: 172.25.202.20 >>>>>>>> vlan: 1105 >>>>>>>> - name: StorageMgmt >>>>>>>> name_lower: storage_mgmt >>>>>>>> vip: true >>>>>>>> mtu: 1500 >>>>>>>> subnets: >>>>>>>> storage_mgmt_subnet: >>>>>>>> ip_subnet: 172.25.202.64/26 >>>>>>>> allocation_pools: >>>>>>>> - start: 172.25.202.72 >>>>>>>> end: 172.25.202.87 >>>>>>>> vlan: 1106 >>>>>>>> - name: InternalApi >>>>>>>> name_lower: internal_api >>>>>>>> vip: true >>>>>>>> mtu: 1500 >>>>>>>> subnets: >>>>>>>> internal_api_subnet: >>>>>>>> ip_subnet: 172.25.201.192/26 >>>>>>>> allocation_pools: >>>>>>>> - start: 172.25.201.198 >>>>>>>> end: 172.25.201.212 >>>>>>>> vlan: 1104 >>>>>>>> - name: Tenant >>>>>>>> vip: false # Tenant network does not use VIPs >>>>>>>> mtu: 1500 >>>>>>>> name_lower: tenant >>>>>>>> subnets: >>>>>>>> tenant_subnet: >>>>>>>> ip_subnet: 172.25.202.128/26 >>>>>>>> allocation_pools: >>>>>>>> - start: 172.25.202.135 >>>>>>>> end: 172.25.202.150 >>>>>>>> vlan: 1108 >>>>>>>> - name: External >>>>>>>> name_lower: external >>>>>>>> vip: true >>>>>>>> mtu: 1500 >>>>>>>> subnets: >>>>>>>> external_subnet: >>>>>>>> ip_subnet: 172.25.201.128/26 >>>>>>>> allocation_pools: >>>>>>>> - start: 172.25.201.135 >>>>>>>> end: 172.25.201.150 >>>>>>>> gateway_ip: 172.25.201.129 >>>>>>>> vlan: 1103 >>>>>>>> >>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>> custom_vip_data.yaml >>>>>>>> - network: ctlplane >>>>>>>> #dns_name: overcloud >>>>>>>> ip_address: 172.25.201.91 >>>>>>>> subnet: ctlplane-subnet >>>>>>>> - network: external >>>>>>>> #dns_name: overcloud >>>>>>>> ip_address: 172.25.201.150 >>>>>>>> subnet: external_subnet >>>>>>>> - network: internal_api >>>>>>>> #dns_name: overcloud >>>>>>>> ip_address: 172.25.201.250 >>>>>>>> subnet: internal_api_subnet >>>>>>>> - network: storage >>>>>>>> #dns_name: overcloud >>>>>>>> ip_address: 172.25.202.50 >>>>>>>> subnet: storage_subnet >>>>>>>> - network: storage_mgmt >>>>>>>> #dns_name: overcloud >>>>>>>> ip_address: 172.25.202.90 >>>>>>>> subnet: storage_mgmt_subnet >>>>>>>> >>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>> overcloud-baremetal-deploy.yaml >>>>>>>> - name: Controller >>>>>>>> count: 4 >>>>>>>> defaults: >>>>>>>> networks: >>>>>>>> - network: ctlplane >>>>>>>> vif: true >>>>>>>> - network: external >>>>>>>> subnet: external_subnet >>>>>>>> - network: internal_api >>>>>>>> subnet: internal_api_subnet >>>>>>>> - network: storage >>>>>>>> subnet: storage_subnet >>>>>>>> - network: storage_mgmt >>>>>>>> subnet: storage_mgmt_subnet >>>>>>>> - network: tenant >>>>>>>> subnet: tenant_subnet >>>>>>>> network_config: >>>>>>>> template: /home/stack/templates/controller.j2 >>>>>>>> default_route_network: >>>>>>>> - external >>>>>>>> instances: >>>>>>>> - hostname: overcloud-controller-0 >>>>>>>> name: dc1-controller2 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-controller-1 >>>>>>>> name: dc2-controller2 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-controller-2 >>>>>>>> name: dc1-controller1 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-controller-no-ceph-3 >>>>>>>> name: dc2-ceph2 >>>>>>>> #provisioned: false >>>>>>>> #- hostname: overcloud-controller-3 >>>>>>>> #name: dc2-compute3 >>>>>>>> #provisioned: false >>>>>>>> >>>>>>>> - name: Compute >>>>>>>> count: 5 >>>>>>>> defaults: >>>>>>>> networks: >>>>>>>> - network: ctlplane >>>>>>>> vif: true >>>>>>>> - network: internal_api >>>>>>>> subnet: internal_api_subnet >>>>>>>> - network: tenant >>>>>>>> subnet: tenant_subnet >>>>>>>> - network: storage >>>>>>>> subnet: storage_subnet >>>>>>>> network_config: >>>>>>>> template: /home/stack/templates/compute.j2 >>>>>>>> instances: >>>>>>>> - hostname: overcloud-novacompute-0 >>>>>>>> name: dc2-compute1 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-novacompute-1 >>>>>>>> name: dc2-ceph1 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-novacompute-2 >>>>>>>> name: dc1-compute1 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-novacompute-3 >>>>>>>> name: dc1-compute2 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-novacompute-4 >>>>>>>> name: dc2-compute3 >>>>>>>> #provisioned: false >>>>>>>> >>>>>>>> - name: CephStorage >>>>>>>> count: 4 >>>>>>>> defaults: >>>>>>>> networks: >>>>>>>> - network: ctlplane >>>>>>>> vif: true >>>>>>>> - network: internal_api >>>>>>>> subnet: internal_api_subnet >>>>>>>> - network: storage >>>>>>>> subnet: storage_subnet >>>>>>>> - network: storage_mgmt >>>>>>>> subnet: storage_mgmt_subnet >>>>>>>> network_config: >>>>>>>> template: /home/stack/templates/ceph-storage.j2 >>>>>>>> instances: >>>>>>>> - hostname: overcloud-cephstorage-0 >>>>>>>> name: dc2-controller1 >>>>>>>> #provisioned: false >>>>>>>> # - hostname: overcloud-cephstorage-1 >>>>>>>> # name: dc2-ceph2 >>>>>>>> - hostname: overcloud-cephstorage-1 >>>>>>>> name: dc1-ceph1 >>>>>>>> # provisioned: false >>>>>>>> - hostname: overcloud-cephstorage-2 >>>>>>>> name: dc1-ceph2 >>>>>>>> #provisioned: false >>>>>>>> - hostname: overcloud-cephstorage-3 >>>>>>>> name: dc2-compute2 >>>>>>>> #provisioned: false >>>>>>>> >>>>>>>> >>>>>>>> You must use these templates to provision network, vip and nodes. >>>>>>>> You must use the output files generated during the provisioning >>>>>>>> step in openstack overcloud deploy command using -e parameter. >>>>>>>> >>>>>>>> With regards, >>>>>>>> Swogat Pradhan >>>>>>>> >>>>>>>> >>>>>>>> On Mon, May 23, 2022 at 8:33 PM Lokendra Rathour < >>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Swogat, >>>>>>>>> I tried checking your solution and my templates but could not >>>>>>>>> relate much. >>>>>>>>> But issue seems the same >>>>>>>>> >>>>>>>>> http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028401.html >>>>>>>>> >>>>>>>>> I tried somemore ways but looks like some issue with templates. >>>>>>>>> Can you please share the templates used to deploy the overcloud. >>>>>>>>> >>>>>>>>> Mysetup have 3 controller and 1 compute. >>>>>>>>> >>>>>>>>> Thanks once again for reading my mail. >>>>>>>>> >>>>>>>>> Waiting for your reply. >>>>>>>>> >>>>>>>>> -Lokendra >>>>>>>>> >>>>>>>>> On Fri, 20 May 2022, 08:25 Swogat Pradhan, < >>>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> Yes I was able to find the issue and fix it. >>>>>>>>>> The issue was with the overcloud-baremetal-deployed.yaml file i >>>>>>>>>> was trying to provision controller-0, controller-1 and controller-3 and >>>>>>>>>> kept controller-2 aside for later, but the tripleo scripts are built in >>>>>>>>>> such a way that they were taking controller- 0, 1 and 2 inplace of >>>>>>>>>> controller-3, so the network ports and vip were created for controller 0,1 >>>>>>>>>> and 2 but not for 3 , so this error was popping off. Also i would request >>>>>>>>>> you to check the jinja nic templates and once the node provisioning is done >>>>>>>>>> check the /etc/os-net-config/config.json/yaml file for syntax if using >>>>>>>>>> bonded nic template. >>>>>>>>>> If you need any more infor please let me know. >>>>>>>>>> >>>>>>>>>> With regards, >>>>>>>>>> Swogat Pradhan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, May 20, 2022 at 8:01 AM Lokendra Rathour < >>>>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Swogat, >>>>>>>>>>> Thanks for raising this issue. >>>>>>>>>>> Did you find any solution? to this problem ? >>>>>>>>>>> >>>>>>>>>>> Please let me know it might be helpful >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Apr 19, 2022 at 12:43 PM Swogat Pradhan < >>>>>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> I am currently trying to deploy openstack wallaby using tripleo >>>>>>>>>>>> arch. >>>>>>>>>>>> I created the network jinja templates, ran the following >>>>>>>>>>>> commands also: >>>>>>>>>>>> >>>>>>>>>>>> #openstack overcloud network provision --stack overcloud >>>>>>>>>>>> --output networks-deployed-environment.yaml custom_network_data.yaml >>>>>>>>>>>> # openstack overcloud network vip provision --stack overcloud >>>>>>>>>>>> --output vip-deployed-environment.yaml custom_vip_data.yaml >>>>>>>>>>>> # openstack overcloud node provision --stack overcloud >>>>>>>>>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa >>>>>>>>>>>> overcloud-baremetal-deploy.yaml >>>>>>>>>>>> >>>>>>>>>>>> and used the environment files in the openstack overcloud >>>>>>>>>>>> deploy command: >>>>>>>>>>>> >>>>>>>>>>>> (undercloud) [stack at hkg2director ~]$ cat deploy.sh >>>>>>>>>>>> #!/bin/bash >>>>>>>>>>>> THT=/usr/share/openstack-tripleo-heat-templates/ >>>>>>>>>>>> CNF=/home/stack/ >>>>>>>>>>>> openstack overcloud deploy --templates $THT \ >>>>>>>>>>>> -r $CNF/templates/roles_data.yaml \ >>>>>>>>>>>> -n $CNF/workplace/custom_network_data.yaml \ >>>>>>>>>>>> -e ~/containers-prepare-parameter.yaml \ >>>>>>>>>>>> -e $CNF/templates/node-info.yaml \ >>>>>>>>>>>> -e $CNF/templates/scheduler-hints.yaml \ >>>>>>>>>>>> -e $CNF/workplace/networks-deployed-environment.yaml \ >>>>>>>>>>>> -e $CNF/workplace/vip-deployed-environment.yaml \ >>>>>>>>>>>> -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ >>>>>>>>>>>> -e $CNF/workplace/custom-net-bond-with-vlans.yaml >>>>>>>>>>>> >>>>>>>>>>>> Now when i run the ./deploy.sh script i encounter an error >>>>>>>>>>>> stating: >>>>>>>>>>>> >>>>>>>>>>>> ERROR openstack [-] Resource >>>>>>>>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>>>>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>>>>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>>>>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>>>>>>>> included with the deployment command.: >>>>>>>>>>>> tripleoclient.exceptions.InvalidConfiguration: Resource >>>>>>>>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>>>>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>>>>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>>>>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>>>>>>>> included with the deployment command. >>>>>>>>>>>> 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END >>>>>>>>>>>> return value: 1 >>>>>>>>>>>> >>>>>>>>>>>> Can someone tell me where the mistake is? >>>>>>>>>>>> >>>>>>>>>>>> With regards, >>>>>>>>>>>> Swogat Pradhan >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>> >>>>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bragin75 at gmail.com Tue Jun 7 09:40:00 2022 From: bragin75 at gmail.com (Vasiliy Bragin) Date: Tue, 7 Jun 2022 14:40:00 +0500 Subject: trove, error - Exception during message handling: AttributeError: 'TroveContext' object has no attribute 'notification', please help to fix Message-ID: Hello, team trove! trove, error - Exception during message handling: AttributeError: 'TroveContext' object has no attribute 'notification', please help to fix release openstack - yoga, CentOS, CentOS Stream release 8 instance trove - ubuntu, Linux version 5.4.0-113-generic (buildd at lcy02-amd64-067) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 On Yoga Trove guest-agent inside instance not work at all: log 2022-06-01 15:31:13.263 908 DEBUG trove.common.rpc.service [-] Creating RPC server for service guestagent.15e4e2e9-13a4-42f6-bafb-f63fdd074d76 start /opt/guest-agent-venv/lib/python3.8/site-packages/trove/common/rpc/service.py:56 *2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server [-] Exception during message handling: AttributeError: 'TroveContext' object has no attribute 'notification'* 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/osprofiler/profiler.py", line 160, in wrapper 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server result = f(*args, **kwargs) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/trove/guestagent/datastore/manager.py", line 213, in prepare 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server with EndNotification(context, instance_id=CONF.guest_id): 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server File "/opt/guest-agent-venv/lib/python3.8/site-packages/trove/common/notification.py", line 48, in __init__ 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server self.context.notification.payload.update(kwargs) 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server AttributeError: 'TroveContext' object has no attribute 'notification' 2022-06-01 15:31:13.610 908 ERROR oslo_messaging.rpc.server Trove guest-agent on instance execute. But DB don't work and not started. After restart guest-agent on instance next log: 2022-06-06 12:51:09.443 4503 DEBUG trove.common.rpc.service [-] Creating RPC server for service guestagent.15e4e2e9-13a4-42f6-bafb-f63fdd074d76 start /opt/guest-agent-venv/lib/python3.8/site-packages/trove/common/rpc/service.py:56 2022-06-06 12:52:09.541 4503 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.update_status run_periodic_tasks /opt/guest-agent-venv/lib/python3.8/site-packages/oslo_service/periodic_task.py:210 *2022-06-06 12:52:09.546 4503 INFO trove.guestagent.datastore.manager [-] Database service is not installed, skip status check* Please tell me how can I fix this errors? -- Vasily Bragin -------------- next part -------------- An HTML attachment was scrubbed... URL: From anyrude10 at gmail.com Tue Jun 7 09:49:05 2022 From: anyrude10 at gmail.com (Anirudh Gupta) Date: Tue, 7 Jun 2022 15:19:05 +0530 Subject: [Tripleo] - Provisioning Baremetal Instances in Wallaby Openstack Message-ID: Hi Harald/Julia/Openstack Team I had successfully installed Ironic Service in Tripleo Train, Ussuri Release with the help of openstack discuss team members. https://lists.openstack.org/pipermail/openstack-discuss/2022-February/027047.html I had custom network configuration for which I had passed Service net map as below in network-environment file. parameter_defaults: ServiceNetMap: IronicApiNetwork: oc_provisioning IronicNetwork: oc_provisioning But with the Wallaby Release, since the deployment format has changed There is no network-environment file. So I tried passing it in a separate environment.yaml file along with the templates. Command to deploy in wallaby: openstack overcloud deploy --templates \ --networks-file /home/stack/templates/custom_network_data.yaml \ --vip-file /home/stack/templates/custom_vip_data.yaml \ --baremetal-deployment /home/stack/templates/overcloud-baremetal-deploy.yaml --deployed-server \ --network-config \ -e /home/stack/templates/environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml \ -e /home/stack/templates/ironic-config.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/services/ptp.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ -e /home/stack/containers-prepare-parameter.yaml Without Service Net Map, deployment was successful but TFTP of image was an issue since we are going with custom network approach. Passing Service Net map details in a seperate file environment.yaml, is giving the below error: 24d-f43f-6f76-000000006ce9 | TASK | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 2022-06-07 11:16:46.806982 | | WARNING | ERROR: Can't run container ironic_db_sync stderr: Error: statfs /var/lib/config-data/puppet-generated/ironic_api: no such file or directory 2022-06-07 11:16:46.809252 | 525400fd-f24d-f43f-6f76-000000006c79 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-1 | error={"changed": false, "msg": "Failed containers: ironic_db_sync"} 2022-06-07 11:16:46.810197 | 525400fd-f24d-f43f-6f76-000000006c79 | TIMING | tripleo_container_manage : Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-1 | 0:11:40.574944 | 10.62s 2022-06-07 11:16:47.683400 | | WARNING | ERROR: Can't run container ironic_db_sync stderr: Error: statfs /var/lib/config-data/puppet-generated/ironic_api: no such file or directory 2022-06-07 11:16:47.684455 | 525400fd-f24d-f43f-6f76-000000006ce9 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_3 | overcloud-controller-2 | error={"changed": false, "msg": "Failed containers: ironic_db_sync"} Can you please suggest the place we can pass the service net map details in Wallaby Release to resolve this issue? Regards Anirudh Gupta -------------- next part -------------- An HTML attachment was scrubbed... URL: From swogatpradhan22 at gmail.com Wed Jun 8 05:34:22 2022 From: swogatpradhan22 at gmail.com (Swogat Pradhan) Date: Wed, 8 Jun 2022 11:04:22 +0530 Subject: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version In-Reply-To: References: Message-ID: Hi Lokendra, I am not sure why this issue is caused, can you please check the tripleo_dnsmasq service and the tripleo_ironic_pxe service in your director node? IF the services are running fine then please report this here for fast resolvement https://bugzilla.redhat.com/ With regards, Swogat Pradhan On Tue, Jun 7, 2022 at 12:54 PM Lokendra Rathour wrote: > Hi Swogat, > Yes after correcting the j2 template, as suggested by you, it started > working. Thanks a lot once again. > Now the Overcloud is deployed. > > Further, as a next requirement, we were also trying to add an *additional > Node as a bare-metal Instance* on the overcloud. As it is the composable > network approach, we add an entry in parameters default for ServiceNetMap, > as like below in environments/network-environments.yaml > > parameter_defaults: > # This section is where deployment-specific configuration is done > ServiceNetMap: > IronicApiNetwork: oc_provisioning > IronicNetwork: oc_provisioning > > this is now passed as in the environment.yaml as parameter_defaults, but > somehow we are not able to deploy the setup. it gives the error for > containers. > > > *2022-06-06 19:08:37.560800 | 525400fd-f24d-9703-b0e7-0000000084b2 | FATAL > | Manage container systemd services and cleanup old systemd healthchecks > for /var/lib/tripleo-config/container-startup-config/step_4 | > overcloud-controller-0 | error={"changed": false, "msg": "Service > ironic_pxe_tftp has not started yet"}* > > > and if we remove this setting as stated above, the deployment is > successful, but at the time of node provisioning it is not able to reach > the TFTP server to pull files, mainly because of the composable network > (oc_provisioning), we faced this issue in Train, so we did the above config > changes and it working. But as it looks something more is needed in > Wallaby for Baremetal provisioning as an instance. > > Please share your learnings with the reference to it. Thanks once again > for your all-time support. > reference link: Bare Metal Instances in Overcloud ? TripleO 3.0.0 > documentation (openstack.org) > > > -Lokendra > > > On Wed, Jun 1, 2022 at 12:19 AM Lokendra Rathour < > lokendrarathour at gmail.com> wrote: > >> Ok Swogat, >> Thanks once again. >> i will try this approach once and will let you know. >> >> >> On Wed, 1 Jun 2022, 00:04 Swogat Pradhan, >> wrote: >> >>> Hi Lokendra, >>> Like i said, >>> > NOTE: when passing --network-config parameter in node provision step, >>> it creates a directory in /etc/os-net-config and in it creates a file >>> > config.yaml, do check the indentation of that file. (in my case the >>> indentation was wrong when i was using bonding everytime, so i had to >>> > manually change the script and run a while loop and then my node >>> provision step was successful) >>> this parameter --network-config creates a config.yaml(network config) >>> file in /etc/os-net-config directory in the overcloud nodes and then >>> ansible tries to apply the network config from the generated config file. >>> And in wallaby version if you are using bonding then the syntax in the >>> generated conf is wrong (atleast was in my case) and then the ansible tries >>> to apply the network config (with wrong syntax) so your overcloud nodes >>> become unavailable. >>> >>> Please run node provision separately, and keep on monitoring "metalsmith >>> list" command. Once IP is assigned to the overcloud nodes, ssh to the nodes >>> and assign a password to heat-admin user so that even if the network >>> becomes unavailable you still will be able to access the nodes via console >>> access, then you can visit /etc/os-net-config directory and verify the >>> config.yaml file. >>> >>> With regards, >>> Swogat pradhan. >>> >>> On Tue, May 31, 2022 at 11:56 PM Lokendra Rathour < >>> lokendrarathour at gmail.com> wrote: >>> >>>> Hi Swogat, >>>> Thanks once again for your input it really helped much. >>>> >>>> instead of running mentioned those three provisioning steps i used >>>> alternate method and passed directly in deploy command. now my current >>>> deploy command is: >>>> >>>> openstack overcloud deploy --templates \ >>>> --networks-file /home/stack/templates/custom_network_data.yaml \ >>>> --vip-file /home/stack/templates/custom_vip_data.yaml \ >>>> --baremetal-deployment >>>> /home/stack/templates/overcloud-baremetal-deploy.yaml \ >>>> --network-config \ >>>> -e /home/stack/templates/environment.yaml \ >>>> -e >>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml >>>> \ >>>> -e >>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml >>>> \ >>>> -e >>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml >>>> \ >>>> -e /home/stack/templates/ironic-config.yaml \ >>>> -e >>>> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ >>>> -e >>>> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ >>>> -e /home/stack/containers-prepare-parameter.yaml >>>> >>>> The files as suggested by you are well created. >>>> >>>> But once we run the deployment I get this error for all nodes: >>>> >>>> 0:00:16.064781 | 1.16s >>>> 2022-05-31 19:13:00.276954 | 525400ef-b928-9ded-fecc-000000000094 | >>>> TASK | Run tripleo_os_net_config_module with network_config >>>> 2022-05-31 19:40:30.061582 | 525400ef-b928-9ded-fecc-000000000094 | >>>> FATAL | Run tripleo_os_net_config_module with network_config | >>>> overcloud-controller-1 | error={"msg": "Data could not be sent to remote >>>> host \"30.30.30.117\". Make sure this host can be reached over ssh: ssh: >>>> connect to host 30.30.30.117 port 22: No route to host\r\n"} >>>> 2022- >>>> >>>> >>>> Baremetal node list are showing in as active. >>>> >>>> (undercloud) [stack at undercloud ~]$ openstack baremetal node list >>>> /usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: >>>> DeprecationWarning: The _yaml extension module is now located at yaml._yaml >>>> and its location is subject to change. To use the LibYAML-based parser and >>>> emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper >>>> as Dumper`. >>>> DeprecationWarning >>>> >>>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>>> | UUID | Name | Instance UUID >>>> | Power State | Provisioning State | Maintenance | >>>> >>>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>>> | 1a4d873c-f9f7-4504-a3af-92c11f954171 | node-a | >>>> 901453a1-183f-4de8-aaab-0f38be2be455 | power on | active | >>>> False | >>>> | d18610fc-9532-410c-918e-8efc326c89f8 | node-b | >>>> d059b94a-8357-4f8e-a0d8-15a24b0c1afe | power on | active | >>>> False | >>>> | b69f2d5a-5b18-4453-8843-15c6af79aca0 | node-c | >>>> f196ef3a-7950-47b9-a5ae-751f06b18f75 | power on | active | >>>> False | >>>> | 8a38c584-f812-4ebc-a0b1-4299f0917637 | node-d | >>>> 1636517c-2ab2-43d7-8205-9f02c5290207 | power on | active | >>>> False | >>>> >>>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>>> >>>> Some config is missing it seems, please check once and advise. >>>> >>>> >>>> On Tue, May 31, 2022 at 11:40 AM Swogat Pradhan < >>>> swogatpradhan22 at gmail.com> wrote: >>>> >>>>> Hi Lokendra, >>>>> You need to generate another file also in the following step: openstack >>>>> overcloud node provision --stack overcloud --overcloud-ssh-key >>>>> /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml also you >>>>> need to pass another parameter --network-config. >>>>> example: >>>>> openstack overcloud node provision --stack overcloud >>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa *--network-config* *--output >>>>> overcloud-baremetal-deployed.yaml* overcloud-baremetal-deploy.yaml >>>>> >>>>> And then all these output files will be passed on to the openstack >>>>> overcloud deploy command. >>>>> NOTE: when passing --network-config parameter in node provision step, >>>>> it creates a directory in /etc/os-net-config and in it creates a file >>>>> config.yaml, do check the indentation of that file. (in my case the >>>>> indentation was wrong when i was using bondind everytime, so i had to >>>>> manually change the script and run a while loop and then my node provision >>>>> step was successful) >>>>> >>>>> On Tue, May 31, 2022 at 8:59 AM Lokendra Rathour < >>>>> lokendrarathour at gmail.com> wrote: >>>>> >>>>>> Hi Swogat, >>>>>> I tried generating the scripts as used by you in your deployments >>>>>> using the >>>>>> >>>>>> >>>>>> #openstack overcloud network provision --stack overcloud --output >>>>>> networks-deployed-environment.yaml custom_network_data.yaml >>>>>> # openstack overcloud network vip provision --stack overcloud >>>>>> --output vip-deployed-environment.yaml custom_vip_data.yaml >>>>>> # openstack overcloud node provision --stack overcloud >>>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa >>>>>> overcloud-baremetal-deploy.yaml >>>>>> >>>>>> and used the first two in the final deployment script, but it gives >>>>>> the error: >>>>>> >>>>>> heatclient.exc.HTTPInternalServerError: ERROR: Internal Error >>>>>> 2022-05-30 14:14:39.772 479668 ERROR >>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent >>>>>> call last):\n', ' File "/usr/lib/python3.6/ted_stack\n >>>>>> nested_stack.validate()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in >>>>>> wrapper\n result = f(*args, ine 969, in validate\n result = >>>>>> res.validate()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/openstack/neutron/port.py", >>>>>> line 454site-packages/heat/engine/resources/openstack/neutron/neutron.py", >>>>>> line 43, in validate\n res = super(NeutronResource, self).validate()\n', >>>>>> ' File "/un return self.validate_template()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 1882, in >>>>>> validate_template\n self.t.rpy", line 200, in >>>>>> _validate_service_availability\n raise ex\n', >>>>>> 'heat.common.exception.ResourceTypeUnavailable: HEAT-E99001 Service neutron >>>>>> is not avaieutron network endpoint is not in service catalog.\n', '\nDuring >>>>>> handling of the above exception, another exception occurred:\n\n', >>>>>> 'Traceback (most recens/stack_resource.py", line 75, in >>>>>> validate_nested_stack\n nested_stack.validate()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py"thon3.6/site-packages/heat/engine/stack.py", >>>>>> line 969, in validate\n result = res.validate()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/ateResource, >>>>>> self).validate()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", >>>>>> line 65, in validate\n self.validources/stack_resource.py", line 81, in >>>>>> validate_nested_stack\n ex, path=[self.stack.t.RESOURCES, path])\n', >>>>>> 'heat.common.exception.StackValidationFaileeploy/overcloud/tripleo-heat-templates/deployed-server/deployed-server.yaml>: >>>>>> HEAT-E99001 Service neutron is not available for resource type >>>>>> OS::TripleO::vice catalog.\n', '\nDuring handling of the above exception, >>>>>> another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' >>>>>> File "/usr/lib/pline 320, in validate_nested_stack\n >>>>>> nested_stack.validate()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in >>>>>> wrappe/heat/engine/stack.py", line 969, in validate\n result = >>>>>> res.validate()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/template_relidate()\n', >>>>>> ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", >>>>>> line 65, in validate\n self.validate_nested_stack()\n'.py", line 81, in >>>>>> validate_nested_stack\n ex, path=[self.stack.t.RESOURCES, path])\n', >>>>>> 'heat.common.exception.StackValidationFailed: >>>>>> ResourceTypeUnavaimplates/puppet/compute-role.yaml>.resources.NovaCompute>>>>> reason: neutron network endpoint is not in service catalog.\n', '\nDuring >>>>>> handling of the above exception, >>>>>> another/lib/python3.6/site-packages/heat/common/context.py", line 416, in >>>>>> wrapped\n return func(self, ctx, *args, **kwargs)\n', ' File >>>>>> "/usr/lib/python3.6/sirce_name, template_id)\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 756, in >>>>>> _parse_template_and_validate_stack\n stack.v line 160, in wrapper\n >>>>>> result = f(*args, **kwargs)\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 969, in >>>>>> validate\n resesources/stack_resource.py", line 65, in validate\n >>>>>> self.validate_nested_stack()\n', ' File >>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/oph=[self.stack.t.RESOURCES, >>>>>> path])\n', 'heat.common.exception.StackValidationFailed: >>>>>> ResourceTypeUnavailable: >>>>>> resources.Compute.resources.0.resources.NovaCompute: >>>>>> HEAT-E9900::ControlPlanePort, reason: neutron network endpoint is not in >>>>>> service catalog.\n']. >>>>>> >>>>>> Request you to check once, please. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, May 30, 2022 at 11:06 AM Lokendra Rathour < >>>>>> lokendrarathour at gmail.com> wrote: >>>>>> >>>>>>> Hi Swogat, >>>>>>> Thanks once again. >>>>>>> >>>>>>> with the files as shown below I am running the overcloud deploy for >>>>>>> wallaby using this command: >>>>>>> >>>>>>> (undercloud) [stack at undercloud ~]$ cat deploy_overcloud_working_1.sh >>>>>>> openstack overcloud deploy --templates \ >>>>>>> -n /home/stack/templates/network_data.yaml \ >>>>>>> -r /home/stack/templates/roles_data.yaml \ >>>>>>> -e /home/stack/templates/environment.yaml \ >>>>>>> -e /home/stack/templates/environments/network-isolation.yaml \ >>>>>>> -e /home/stack/templates/environments/network-environment.yaml \ >>>>>>> -e >>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml >>>>>>> \ >>>>>>> -e >>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml >>>>>>> \ >>>>>>> -e >>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml >>>>>>> \ >>>>>>> -e /home/stack/templates/ironic-config.yaml \ >>>>>>> -e >>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ >>>>>>> -e >>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ >>>>>>> -e /home/stack/containers-prepare-parameter.yaml >>>>>>> (undercloud) [stack at undercloud ~]$ >>>>>>> >>>>>>> >>>>>>> This deployment is on ipv6 using triple0 wallaby, templates, as >>>>>>> mentioned below, are generated using rendering steps and the >>>>>>> network_data.yaml the roles_data.yaml >>>>>>> Steps used to render the templates: >>>>>>> cd /usr/share/openstack-tripleo-heat-templates/ >>>>>>> ./tools/process-templates.py -o >>>>>>> ~/openstack-tripleo-heat-templates-rendered_at_wallaby -n >>>>>>> /home/stack/templates/network_data.yaml -r >>>>>>> /home/stack/templates/roles_data.yaml >>>>>>> >>>>>>> *Now if i try adding the related to VIP port I do get the error as:* >>>>>>> >>>>>>> 2022-05-30 10:37:12.792 979387 WARNING >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template >>>>>>> to file: >>>>>>> /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/controller-role.yaml >>>>>>> 2022-05-30 10:37:12.792 979387 WARNING >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template >>>>>>> to file: >>>>>>> /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/compute-role.yaml >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured >>>>>>> while running the command: ValueError: The environment is not a valid YAML >>>>>>> mapping data type. >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent >>>>>>> call last): >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, >>>>>>> self).run(parsed_args) >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in >>>>>>> run >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, >>>>>>> self).run(parsed_args) >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = >>>>>>> self.take_action(parsed_args) or 0 >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>>> line 1189, in take_action >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud stack, parsed_args, >>>>>>> new_tht_root, user_tht_root) >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>>> line 227, in create_env_files >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud created_env_files, >>>>>>> parsed_args, new_tht_root, user_tht_root) >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>>> line 204, in build_image_params >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud cleanup=(not >>>>>>> parsed_args.no_cleanup)) >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 1929, in >>>>>>> process_multiple_environments >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud env_path=env_path, >>>>>>> include_env_in_files=include_env_in_files) >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", >>>>>>> line 326, in process_environment_and_files >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud env = >>>>>>> environment_format.parse(raw_env) >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>> "/usr/lib/python3.6/site-packages/heatclient/common/environment_format.py", >>>>>>> line 50, in parse >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud raise >>>>>>> ValueError(_('The environment is not a valid ' >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud ValueError: The >>>>>>> environment is not a valid YAML mapping data type. >>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud >>>>>>> 2022-05-30 10:37:14.457 979387 ERROR openstack [-] The environment >>>>>>> is not a valid YAML mapping data type. >>>>>>> 2022-05-30 10:37:14.457 979387 INFO osc_lib.shell [-] END return >>>>>>> value: 1 >>>>>>> (undercloud) [stack at undercloud ~]$ >>>>>>> >>>>>>> This is more of a syntax error where it is not able to understand >>>>>>> the passed VIP data file: >>>>>>> >>>>>>> undercloud) [stack at undercloud ~]$ cat >>>>>>> /home/stack/templates/vip-data-default-network-isolation.yaml >>>>>>> - >>>>>>> dns_name: overcloud >>>>>>> network: internal_api >>>>>>> - >>>>>>> dns_name: overcloud >>>>>>> network: external >>>>>>> - >>>>>>> dns_name: overcloud >>>>>>> network: ctlplane >>>>>>> - >>>>>>> dns_name: overcloud >>>>>>> network: oc_provisioning >>>>>>> - >>>>>>> dns_name: overcloud >>>>>>> network: j3mgmt >>>>>>> >>>>>>> >>>>>>> Please advise, also please note that similar templates generated in >>>>>>> prior releases such as train/ussuri works perfectly. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Please check the list of *templates *files: >>>>>>> >>>>>>> drwxr-xr-x. 2 stack stack 68 May 30 09:22 environments >>>>>>> -rw-r--r--. 1 stack stack 265 May 27 13:47 environment.yaml >>>>>>> -rw-rw-r--. 1 stack stack 297 May 27 13:47 init-repo.yaml >>>>>>> -rw-r--r--. 1 stack stack 570 May 27 13:47 ironic-config.yaml >>>>>>> drwxrwxr-x. 4 stack stack 4096 May 27 13:53 network >>>>>>> -rw-r--r--. 1 stack stack 6370 May 27 14:26 network_data.yaml >>>>>>> -rw-r--r--. 1 stack stack 11137 May 27 13:53 roles_data.yaml >>>>>>> -rw-r--r--. 1 stack stack 234 May 30 09:23 >>>>>>> vip-data-default-network-isolation.yaml >>>>>>> >>>>>>> >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ cat environment.yaml >>>>>>> >>>>>>> parameter_defaults: >>>>>>> OvercloudControllerFlavor: control >>>>>>> OvercloudComputeFlavor: compute >>>>>>> ControllerCount: 3 >>>>>>> ComputeCount: 1 >>>>>>> TimeZone: 'Asia/Kolkata' >>>>>>> NtpServer: ['30.30.30.3'] >>>>>>> NeutronBridgeMappings: datacentre:br-tenant >>>>>>> NeutronFlatNetworks: datacentre >>>>>>> (undercloud) [stack at undercloud templates]$ >>>>>>> >>>>>>> >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ cat ironic-config.yaml >>>>>>> >>>>>>> parameter_defaults: >>>>>>> NovaSchedulerDefaultFilters: >>>>>>> - AggregateInstanceExtraSpecsFilter >>>>>>> - AvailabilityZoneFilter >>>>>>> - ComputeFilter >>>>>>> - ComputeCapabilitiesFilter >>>>>>> - ImagePropertiesFilter >>>>>>> IronicEnabledHardwareTypes: >>>>>>> - ipmi >>>>>>> - redfish >>>>>>> IronicEnabledPowerInterfaces: >>>>>>> - ipmitool >>>>>>> - redfish >>>>>>> IronicEnabledManagementInterfaces: >>>>>>> - ipmitool >>>>>>> - redfish >>>>>>> IronicCleaningDiskErase: metadata >>>>>>> IronicIPXEEnabled: true >>>>>>> IronicInspectorSubnets: >>>>>>> - ip_range: 172.23.3.100,172.23.3.150 >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ cat network_data.yaml >>>>>>> >>>>>>> - name: J3Mgmt >>>>>>> name_lower: j3mgmt >>>>>>> vip: true >>>>>>> vlan: 400 >>>>>>> ipv6: true >>>>>>> ipv6_subnet: 'fd80:fd00:fd00:4000::/64' >>>>>>> ipv6_allocation_pools: [{'start': 'fd80:fd00:fd00:4000::10', >>>>>>> 'end': 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] >>>>>>> mtu: 9000 >>>>>>> >>>>>>> >>>>>>> >>>>>>> - name: InternalApi >>>>>>> name_lower: internal_api >>>>>>> vip: true >>>>>>> vlan: 418 >>>>>>> ipv6: true >>>>>>> ipv6_subnet: 'fd00:fd00:fd00:2000::/64' >>>>>>> ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:2000::10', >>>>>>> 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] >>>>>>> mtu: 9000 >>>>>>> >>>>>>> >>>>>>> - name: External >>>>>>> vip: true >>>>>>> name_lower: external >>>>>>> vlan: 408 >>>>>>> ipv6: true >>>>>>> gateway_ipv6: 'fd00:fd00:fd00:9900::1' >>>>>>> ipv6_subnet: 'fd00:fd00:fd00:9900::/64' >>>>>>> ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:9900::10', >>>>>>> 'end': 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] >>>>>>> mtu: 9000 >>>>>>> >>>>>>> >>>>>>> - name: OCProvisioning >>>>>>> vip: true >>>>>>> name_lower: oc_provisioning >>>>>>> vlan: 412 >>>>>>> ip_subnet: '172.23.3.0/24' >>>>>>> allocation_pools: [{'start': '172.23.3.10', 'end': '172.23.3.50'}] >>>>>>> mtu: 9000 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ cat roles_data.yaml >>>>>>> >>>>>>> >>>>>>> ############################################################################### >>>>>>> # File generated by TripleO >>>>>>> >>>>>>> ############################################################################### >>>>>>> >>>>>>> ############################################################################### >>>>>>> # Role: Controller >>>>>>> # >>>>>>> >>>>>>> ############################################################################### >>>>>>> - name: Controller >>>>>>> description: | >>>>>>> Controller role that has all the controller services loaded and >>>>>>> handles >>>>>>> Database, Messaging, and Network functions. >>>>>>> CountDefault: 1 >>>>>>> tags: >>>>>>> - primary >>>>>>> - controller >>>>>>> # Create external Neutron bridge for SNAT (and floating IPs when >>>>>>> using >>>>>>> # ML2/OVS without DVR) >>>>>>> - external_bridge >>>>>>> networks: >>>>>>> External: >>>>>>> subnet: external_subnet >>>>>>> InternalApi: >>>>>>> subnet: internal_api_subnet >>>>>>> OCProvisioning: >>>>>>> subnet: oc_provisioning_subnet >>>>>>> J3Mgmt: >>>>>>> subnet: j3mgmt_subnet >>>>>>> >>>>>>> >>>>>>> # For systems with both IPv4 and IPv6, you may specify a gateway >>>>>>> network for >>>>>>> # each, such as ['ControlPlane', 'External'] >>>>>>> default_route_networks: ['External'] >>>>>>> HostnameFormatDefault: '%stackname%-controller-%index%' >>>>>>> RoleParametersDefault: >>>>>>> OVNCMSOptions: "enable-chassis-as-gw" >>>>>>> # Deprecated & backward-compatible values (FIXME: Make parameters >>>>>>> consistent) >>>>>>> # Set uses_deprecated_params to True if any deprecated params are >>>>>>> used. >>>>>>> uses_deprecated_params: True >>>>>>> deprecated_param_extraconfig: 'controllerExtraConfig' >>>>>>> deprecated_param_flavor: 'OvercloudControlFlavor' >>>>>>> deprecated_param_image: 'controllerImage' >>>>>>> deprecated_nic_config_name: 'controller.yaml' >>>>>>> update_serial: 1 >>>>>>> ServicesDefault: >>>>>>> - OS::TripleO::Services::Aide >>>>>>> - OS::TripleO::Services::AodhApi >>>>>>> - OS::TripleO::Services::AodhEvaluator >>>>>>> >>>>>>> .. >>>>>>> . >>>>>>> >>>>>>> >>>>>>> ..############################################################################### >>>>>>> # Role: Compute >>>>>>> # >>>>>>> >>>>>>> ############################################################################### >>>>>>> - name: Compute >>>>>>> description: | >>>>>>> Basic Compute Node role >>>>>>> CountDefault: 1 >>>>>>> # Create external Neutron bridge (unset if using ML2/OVS without >>>>>>> DVR) >>>>>>> tags: >>>>>>> - compute >>>>>>> - external_bridge >>>>>>> networks: >>>>>>> InternalApi: >>>>>>> subnet: internal_api_subnet >>>>>>> J3Mgmt: >>>>>>> subnet: j3mgmt_subnet >>>>>>> HostnameFormatDefault: '%stackname%-novacompute-%index%' >>>>>>> RoleParametersDefault: >>>>>>> FsAioMaxNumber: 1048576 >>>>>>> TunedProfileName: "virtual-host" >>>>>>> # Deprecated & backward-compatible values (FIXME: Make parameters >>>>>>> consistent) >>>>>>> # Set uses_deprecated_params to True if any deprecated params are >>>>>>> used. >>>>>>> # These deprecated_params only need to be used for existing roles >>>>>>> and not for >>>>>>> # composable roles. >>>>>>> uses_deprecated_params: True >>>>>>> deprecated_param_image: 'NovaImage' >>>>>>> deprecated_param_extraconfig: 'NovaComputeExtraConfig' >>>>>>> deprecated_param_metadata: 'NovaComputeServerMetadata' >>>>>>> deprecated_param_scheduler_hints: 'NovaComputeSchedulerHints' >>>>>>> deprecated_param_ips: 'NovaComputeIPs' >>>>>>> deprecated_server_resource_name: 'NovaCompute' >>>>>>> >>>>>>> deprecated_nic_config_name: 'compute.yaml' >>>>>>> update_serial: 25 >>>>>>> ServicesDefault: >>>>>>> - OS::TripleO::Services::Aide >>>>>>> - OS::TripleO::Services::AuditD >>>>>>> - OS::TripleO::Services::BootParams >>>>>>> >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ cat >>>>>>> environments/network-environment.yaml >>>>>>> >>>>>>> #This file is an example of an environment file for defining the >>>>>>> isolated >>>>>>> #networks and related parameters. >>>>>>> resource_registry: >>>>>>> # Network Interface templates to use (these files must exist). You >>>>>>> can >>>>>>> # override these by including one of the net-*.yaml environment >>>>>>> files, >>>>>>> # such as net-bond-with-vlans.yaml, or modifying the list here. >>>>>>> # Port assignments for the Controller >>>>>>> OS::TripleO::Controller::Net::SoftwareConfig: OS::Heat::None >>>>>>> # Port assignments for the Compute >>>>>>> OS::TripleO::Compute::Net::SoftwareConfig: OS::Heat::None >>>>>>> >>>>>>> >>>>>>> parameter_defaults: >>>>>>> # This section is where deployment-specific configuration is done >>>>>>> # >>>>>>> ServiceNetMap: >>>>>>> IronicApiNetwork: oc_provisioning >>>>>>> IronicNetwork: oc_provisioning >>>>>>> >>>>>>> >>>>>>> >>>>>>> # This section is where deployment-specific configuration is done >>>>>>> ControllerNetworkConfigTemplate: >>>>>>> 'templates/bonds_vlans/bonds_vlans.j2' >>>>>>> ComputeNetworkConfigTemplate: >>>>>>> 'templates/bonds_vlans/bonds_vlans.j2' >>>>>>> >>>>>>> >>>>>>> >>>>>>> # Customize the IP subnet to match the local environment >>>>>>> J3MgmtNetCidr: 'fd80:fd00:fd00:4000::/64' >>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>> J3MgmtAllocationPools: [{'start': 'fd80:fd00:fd00:4000::10', >>>>>>> 'end': 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] >>>>>>> # Customize the VLAN ID to match the local environment >>>>>>> J3MgmtNetworkVlanID: 400 >>>>>>> >>>>>>> >>>>>>> >>>>>>> # Customize the IP subnet to match the local environment >>>>>>> InternalApiNetCidr: 'fd00:fd00:fd00:2000::/64' >>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>> InternalApiAllocationPools: [{'start': 'fd00:fd00:fd00:2000::10', >>>>>>> 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] >>>>>>> # Customize the VLAN ID to match the local environment >>>>>>> InternalApiNetworkVlanID: 418 >>>>>>> >>>>>>> >>>>>>> >>>>>>> # Customize the IP subnet to match the local environment >>>>>>> ExternalNetCidr: 'fd00:fd00:fd00:9900::/64' >>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>> # Leave room if the external network is also used for floating IPs >>>>>>> ExternalAllocationPools: [{'start': 'fd00:fd00:fd00:9900::10', >>>>>>> 'end': 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] >>>>>>> # Gateway router for routable networks >>>>>>> ExternalInterfaceDefaultRoute: 'fd00:fd00:fd00:9900::1' >>>>>>> # Customize the VLAN ID to match the local environment >>>>>>> ExternalNetworkVlanID: 408 >>>>>>> >>>>>>> >>>>>>> >>>>>>> # Customize the IP subnet to match the local environment >>>>>>> OCProvisioningNetCidr: '172.23.3.0/24' >>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>> OCProvisioningAllocationPools: [{'start': '172.23.3.10', 'end': >>>>>>> '172.23.3.50'}] >>>>>>> # Customize the VLAN ID to match the local environment >>>>>>> OCProvisioningNetworkVlanID: 412 >>>>>>> >>>>>>> >>>>>>> >>>>>>> # List of Neutron network types for tenant networks (will be used >>>>>>> in order) >>>>>>> NeutronNetworkType: 'geneve,vlan' >>>>>>> # Neutron VLAN ranges per network, for example >>>>>>> 'datacentre:1:499,tenant:500:1000': >>>>>>> NeutronNetworkVLANRanges: 'datacentre:1:1000' >>>>>>> # Customize bonding options, e.g. "mode=4 lacp_rate=1 updelay=1000 >>>>>>> miimon=100" >>>>>>> # for Linux bonds w/LACP, or "bond_mode=active-backup" for OVS >>>>>>> active/backup. >>>>>>> BondInterfaceOvsOptions: "bond_mode=active-backup" >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ >>>>>>> >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ cat >>>>>>> environments/network-isolation.yaml >>>>>>> >>>>>>> # NOTE: This template is now deprecated, and is only included for >>>>>>> compatibility >>>>>>> # when upgrading a deployment where this template was originally >>>>>>> used. For new >>>>>>> # deployments, set "ipv6: true" on desired networks in >>>>>>> network_data.yaml, and >>>>>>> # include network-isolation.yaml. >>>>>>> # >>>>>>> # Enable the creation of Neutron networks for isolated Overcloud >>>>>>> # traffic and configure each role to assign ports (related >>>>>>> # to that role) on these networks. >>>>>>> resource_registry: >>>>>>> # networks as defined in network_data.yaml >>>>>>> OS::TripleO::Network::J3Mgmt: ../network/j3mgmt_v6.yaml >>>>>>> OS::TripleO::Network::InternalApi: ../network/internal_api_v6.yaml >>>>>>> OS::TripleO::Network::External: ../network/external_v6.yaml >>>>>>> OS::TripleO::Network::OCProvisioning: >>>>>>> ../network/oc_provisioning.yaml >>>>>>> >>>>>>> >>>>>>> # Port assignments for the VIPs >>>>>>> OS::TripleO::Network::Ports::J3MgmtVipPort: >>>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>>> OS::TripleO::Network::Ports::InternalApiVipPort: >>>>>>> ../network/ports/internal_api_v6.yaml >>>>>>> OS::TripleO::Network::Ports::ExternalVipPort: >>>>>>> ../network/ports/external_v6.yaml >>>>>>> OS::TripleO::Network::Ports::OCProvisioningVipPort: >>>>>>> ../network/ports/oc_provisioning.yaml >>>>>>> >>>>>>> >>>>>>> >>>>>>> # Port assignments by role, edit role definition to assign >>>>>>> networks to roles. >>>>>>> # Port assignments for the Controller >>>>>>> OS::TripleO::Controller::Ports::J3MgmtPort: >>>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>>> OS::TripleO::Controller::Ports::InternalApiPort: >>>>>>> ../network/ports/internal_api_v6.yaml >>>>>>> OS::TripleO::Controller::Ports::ExternalPort: >>>>>>> ../network/ports/external_v6.yaml >>>>>>> OS::TripleO::Controller::Ports::OCProvisioningPort: >>>>>>> ../network/ports/oc_provisioning.yaml >>>>>>> # Port assignments for the Compute >>>>>>> OS::TripleO::Compute::Ports::J3MgmtPort: >>>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>>> OS::TripleO::Compute::Ports::InternalApiPort: >>>>>>> ../network/ports/internal_api_v6.yaml >>>>>>> >>>>>>> >>>>>>> >>>>>>> parameter_defaults: >>>>>>> # Enable IPv6 environment for Manila >>>>>>> ManilaIPv6: True >>>>>>> >>>>>>> (undercloud) [stack at undercloud templates]$ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, May 24, 2022 at 5:04 PM Lokendra Rathour < >>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>> >>>>>>>> Thanks, I'll check them out. >>>>>>>> will let you know in case it works out. >>>>>>>> >>>>>>>> On Tue, May 24, 2022 at 2:37 PM Swogat Pradhan < >>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> Please find the below templates: >>>>>>>>> These are for openstack wallaby release: >>>>>>>>> >>>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>>> custom_network_data.yaml >>>>>>>>> - name: Storage >>>>>>>>> name_lower: storage >>>>>>>>> vip: true >>>>>>>>> mtu: 1500 >>>>>>>>> subnets: >>>>>>>>> storage_subnet: >>>>>>>>> ip_subnet: 172.25.202.0/26 >>>>>>>>> allocation_pools: >>>>>>>>> - start: 172.25.202.6 >>>>>>>>> end: 172.25.202.20 >>>>>>>>> vlan: 1105 >>>>>>>>> - name: StorageMgmt >>>>>>>>> name_lower: storage_mgmt >>>>>>>>> vip: true >>>>>>>>> mtu: 1500 >>>>>>>>> subnets: >>>>>>>>> storage_mgmt_subnet: >>>>>>>>> ip_subnet: 172.25.202.64/26 >>>>>>>>> allocation_pools: >>>>>>>>> - start: 172.25.202.72 >>>>>>>>> end: 172.25.202.87 >>>>>>>>> vlan: 1106 >>>>>>>>> - name: InternalApi >>>>>>>>> name_lower: internal_api >>>>>>>>> vip: true >>>>>>>>> mtu: 1500 >>>>>>>>> subnets: >>>>>>>>> internal_api_subnet: >>>>>>>>> ip_subnet: 172.25.201.192/26 >>>>>>>>> allocation_pools: >>>>>>>>> - start: 172.25.201.198 >>>>>>>>> end: 172.25.201.212 >>>>>>>>> vlan: 1104 >>>>>>>>> - name: Tenant >>>>>>>>> vip: false # Tenant network does not use VIPs >>>>>>>>> mtu: 1500 >>>>>>>>> name_lower: tenant >>>>>>>>> subnets: >>>>>>>>> tenant_subnet: >>>>>>>>> ip_subnet: 172.25.202.128/26 >>>>>>>>> allocation_pools: >>>>>>>>> - start: 172.25.202.135 >>>>>>>>> end: 172.25.202.150 >>>>>>>>> vlan: 1108 >>>>>>>>> - name: External >>>>>>>>> name_lower: external >>>>>>>>> vip: true >>>>>>>>> mtu: 1500 >>>>>>>>> subnets: >>>>>>>>> external_subnet: >>>>>>>>> ip_subnet: 172.25.201.128/26 >>>>>>>>> allocation_pools: >>>>>>>>> - start: 172.25.201.135 >>>>>>>>> end: 172.25.201.150 >>>>>>>>> gateway_ip: 172.25.201.129 >>>>>>>>> vlan: 1103 >>>>>>>>> >>>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>>> custom_vip_data.yaml >>>>>>>>> - network: ctlplane >>>>>>>>> #dns_name: overcloud >>>>>>>>> ip_address: 172.25.201.91 >>>>>>>>> subnet: ctlplane-subnet >>>>>>>>> - network: external >>>>>>>>> #dns_name: overcloud >>>>>>>>> ip_address: 172.25.201.150 >>>>>>>>> subnet: external_subnet >>>>>>>>> - network: internal_api >>>>>>>>> #dns_name: overcloud >>>>>>>>> ip_address: 172.25.201.250 >>>>>>>>> subnet: internal_api_subnet >>>>>>>>> - network: storage >>>>>>>>> #dns_name: overcloud >>>>>>>>> ip_address: 172.25.202.50 >>>>>>>>> subnet: storage_subnet >>>>>>>>> - network: storage_mgmt >>>>>>>>> #dns_name: overcloud >>>>>>>>> ip_address: 172.25.202.90 >>>>>>>>> subnet: storage_mgmt_subnet >>>>>>>>> >>>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>>> overcloud-baremetal-deploy.yaml >>>>>>>>> - name: Controller >>>>>>>>> count: 4 >>>>>>>>> defaults: >>>>>>>>> networks: >>>>>>>>> - network: ctlplane >>>>>>>>> vif: true >>>>>>>>> - network: external >>>>>>>>> subnet: external_subnet >>>>>>>>> - network: internal_api >>>>>>>>> subnet: internal_api_subnet >>>>>>>>> - network: storage >>>>>>>>> subnet: storage_subnet >>>>>>>>> - network: storage_mgmt >>>>>>>>> subnet: storage_mgmt_subnet >>>>>>>>> - network: tenant >>>>>>>>> subnet: tenant_subnet >>>>>>>>> network_config: >>>>>>>>> template: /home/stack/templates/controller.j2 >>>>>>>>> default_route_network: >>>>>>>>> - external >>>>>>>>> instances: >>>>>>>>> - hostname: overcloud-controller-0 >>>>>>>>> name: dc1-controller2 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-controller-1 >>>>>>>>> name: dc2-controller2 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-controller-2 >>>>>>>>> name: dc1-controller1 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-controller-no-ceph-3 >>>>>>>>> name: dc2-ceph2 >>>>>>>>> #provisioned: false >>>>>>>>> #- hostname: overcloud-controller-3 >>>>>>>>> #name: dc2-compute3 >>>>>>>>> #provisioned: false >>>>>>>>> >>>>>>>>> - name: Compute >>>>>>>>> count: 5 >>>>>>>>> defaults: >>>>>>>>> networks: >>>>>>>>> - network: ctlplane >>>>>>>>> vif: true >>>>>>>>> - network: internal_api >>>>>>>>> subnet: internal_api_subnet >>>>>>>>> - network: tenant >>>>>>>>> subnet: tenant_subnet >>>>>>>>> - network: storage >>>>>>>>> subnet: storage_subnet >>>>>>>>> network_config: >>>>>>>>> template: /home/stack/templates/compute.j2 >>>>>>>>> instances: >>>>>>>>> - hostname: overcloud-novacompute-0 >>>>>>>>> name: dc2-compute1 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-novacompute-1 >>>>>>>>> name: dc2-ceph1 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-novacompute-2 >>>>>>>>> name: dc1-compute1 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-novacompute-3 >>>>>>>>> name: dc1-compute2 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-novacompute-4 >>>>>>>>> name: dc2-compute3 >>>>>>>>> #provisioned: false >>>>>>>>> >>>>>>>>> - name: CephStorage >>>>>>>>> count: 4 >>>>>>>>> defaults: >>>>>>>>> networks: >>>>>>>>> - network: ctlplane >>>>>>>>> vif: true >>>>>>>>> - network: internal_api >>>>>>>>> subnet: internal_api_subnet >>>>>>>>> - network: storage >>>>>>>>> subnet: storage_subnet >>>>>>>>> - network: storage_mgmt >>>>>>>>> subnet: storage_mgmt_subnet >>>>>>>>> network_config: >>>>>>>>> template: /home/stack/templates/ceph-storage.j2 >>>>>>>>> instances: >>>>>>>>> - hostname: overcloud-cephstorage-0 >>>>>>>>> name: dc2-controller1 >>>>>>>>> #provisioned: false >>>>>>>>> # - hostname: overcloud-cephstorage-1 >>>>>>>>> # name: dc2-ceph2 >>>>>>>>> - hostname: overcloud-cephstorage-1 >>>>>>>>> name: dc1-ceph1 >>>>>>>>> # provisioned: false >>>>>>>>> - hostname: overcloud-cephstorage-2 >>>>>>>>> name: dc1-ceph2 >>>>>>>>> #provisioned: false >>>>>>>>> - hostname: overcloud-cephstorage-3 >>>>>>>>> name: dc2-compute2 >>>>>>>>> #provisioned: false >>>>>>>>> >>>>>>>>> >>>>>>>>> You must use these templates to provision network, vip and nodes. >>>>>>>>> You must use the output files generated during the provisioning >>>>>>>>> step in openstack overcloud deploy command using -e parameter. >>>>>>>>> >>>>>>>>> With regards, >>>>>>>>> Swogat Pradhan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, May 23, 2022 at 8:33 PM Lokendra Rathour < >>>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Swogat, >>>>>>>>>> I tried checking your solution and my templates but could not >>>>>>>>>> relate much. >>>>>>>>>> But issue seems the same >>>>>>>>>> >>>>>>>>>> http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028401.html >>>>>>>>>> >>>>>>>>>> I tried somemore ways but looks like some issue with templates. >>>>>>>>>> Can you please share the templates used to deploy the overcloud. >>>>>>>>>> >>>>>>>>>> Mysetup have 3 controller and 1 compute. >>>>>>>>>> >>>>>>>>>> Thanks once again for reading my mail. >>>>>>>>>> >>>>>>>>>> Waiting for your reply. >>>>>>>>>> >>>>>>>>>> -Lokendra >>>>>>>>>> >>>>>>>>>> On Fri, 20 May 2022, 08:25 Swogat Pradhan, < >>>>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> Yes I was able to find the issue and fix it. >>>>>>>>>>> The issue was with the overcloud-baremetal-deployed.yaml file i >>>>>>>>>>> was trying to provision controller-0, controller-1 and controller-3 and >>>>>>>>>>> kept controller-2 aside for later, but the tripleo scripts are built in >>>>>>>>>>> such a way that they were taking controller- 0, 1 and 2 inplace of >>>>>>>>>>> controller-3, so the network ports and vip were created for controller 0,1 >>>>>>>>>>> and 2 but not for 3 , so this error was popping off. Also i would request >>>>>>>>>>> you to check the jinja nic templates and once the node provisioning is done >>>>>>>>>>> check the /etc/os-net-config/config.json/yaml file for syntax if using >>>>>>>>>>> bonded nic template. >>>>>>>>>>> If you need any more infor please let me know. >>>>>>>>>>> >>>>>>>>>>> With regards, >>>>>>>>>>> Swogat Pradhan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, May 20, 2022 at 8:01 AM Lokendra Rathour < >>>>>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Swogat, >>>>>>>>>>>> Thanks for raising this issue. >>>>>>>>>>>> Did you find any solution? to this problem ? >>>>>>>>>>>> >>>>>>>>>>>> Please let me know it might be helpful >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Apr 19, 2022 at 12:43 PM Swogat Pradhan < >>>>>>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> I am currently trying to deploy openstack wallaby using >>>>>>>>>>>>> tripleo arch. >>>>>>>>>>>>> I created the network jinja templates, ran the following >>>>>>>>>>>>> commands also: >>>>>>>>>>>>> >>>>>>>>>>>>> #openstack overcloud network provision --stack overcloud >>>>>>>>>>>>> --output networks-deployed-environment.yaml custom_network_data.yaml >>>>>>>>>>>>> # openstack overcloud network vip provision --stack overcloud >>>>>>>>>>>>> --output vip-deployed-environment.yaml custom_vip_data.yaml >>>>>>>>>>>>> # openstack overcloud node provision --stack overcloud >>>>>>>>>>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa >>>>>>>>>>>>> overcloud-baremetal-deploy.yaml >>>>>>>>>>>>> >>>>>>>>>>>>> and used the environment files in the openstack overcloud >>>>>>>>>>>>> deploy command: >>>>>>>>>>>>> >>>>>>>>>>>>> (undercloud) [stack at hkg2director ~]$ cat deploy.sh >>>>>>>>>>>>> #!/bin/bash >>>>>>>>>>>>> THT=/usr/share/openstack-tripleo-heat-templates/ >>>>>>>>>>>>> CNF=/home/stack/ >>>>>>>>>>>>> openstack overcloud deploy --templates $THT \ >>>>>>>>>>>>> -r $CNF/templates/roles_data.yaml \ >>>>>>>>>>>>> -n $CNF/workplace/custom_network_data.yaml \ >>>>>>>>>>>>> -e ~/containers-prepare-parameter.yaml \ >>>>>>>>>>>>> -e $CNF/templates/node-info.yaml \ >>>>>>>>>>>>> -e $CNF/templates/scheduler-hints.yaml \ >>>>>>>>>>>>> -e $CNF/workplace/networks-deployed-environment.yaml \ >>>>>>>>>>>>> -e $CNF/workplace/vip-deployed-environment.yaml \ >>>>>>>>>>>>> -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ >>>>>>>>>>>>> -e $CNF/workplace/custom-net-bond-with-vlans.yaml >>>>>>>>>>>>> >>>>>>>>>>>>> Now when i run the ./deploy.sh script i encounter an error >>>>>>>>>>>>> stating: >>>>>>>>>>>>> >>>>>>>>>>>>> ERROR openstack [-] Resource >>>>>>>>>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>>>>>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>>>>>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>>>>>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>>>>>>>>> included with the deployment command.: >>>>>>>>>>>>> tripleoclient.exceptions.InvalidConfiguration: Resource >>>>>>>>>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>>>>>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>>>>>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>>>>>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>>>>>>>>> included with the deployment command. >>>>>>>>>>>>> 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END >>>>>>>>>>>>> return value: 1 >>>>>>>>>>>>> >>>>>>>>>>>>> Can someone tell me where the mistake is? >>>>>>>>>>>>> >>>>>>>>>>>>> With regards, >>>>>>>>>>>>> Swogat Pradhan >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>> >>>>>> >>>> >>>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anyrude10 at gmail.com Wed Jun 8 13:08:33 2022 From: anyrude10 at gmail.com (Anirudh Gupta) Date: Wed, 8 Jun 2022 18:38:33 +0530 Subject: [Tripleo] - Provisioning Baremetal Instances in Wallaby Openstack In-Reply-To: <5ee4d9f4-772d-18c5-bc30-7d5f849f63b3@redhat.com> References: <86dca991-09df-afd9-7e84-7fc60bb9efbd@redhat.com> <5ee4d9f4-772d-18c5-bc30-7d5f849f63b3@redhat.com> Message-ID: Hi Harald Thanks for your response. After passing the role data with relevant details we were able to successfully deploy Overcloud and pass Service NetMap as well. Thanks a lot for your help Regards Anirudh Gupta On Tue, Jun 7, 2022 at 7:01 PM Harald Jensas wrote: > On 6/7/22 14:32, Harald Jensas wrote: > > On 6/7/22 11:49, Anirudh Gupta wrote: > >> Hi Harald/Julia/Openstack Team > >> > >> I had successfully installed Ironic Service in Tripleo Train, Ussuri > >> Release with the help of openstack discuss team members. > >> > >> > https://lists.openstack.org/pipermail/openstack-discuss/2022-February/027047.html > >> < > https://lists.openstack.org/pipermail/openstack-discuss/2022-February/027047.html> > > >> > >> > >> I had custom network configuration for which I had passed Service net > >> map as below in network-environment file. > >> > >> parameter_defaults: > >> ServiceNetMap: > >> IronicApiNetwork: oc_provisioning > >> IronicNetwork: oc_provisioning > >> > >> But with the Wallaby Release, since the deployment format has changed > >> There is no network-environment file. > >> > >> So I tried passing it in a separate environment.yaml file along with > >> the templates. > >> Command to deploy in wallaby: > >> > >> openstack overcloud deploy --templates \ > >> --networks-file /home/stack/templates/custom_network_data.yaml \ > >> --vip-file /home/stack/templates/custom_vip_data.yaml \ > >> --baremetal-deployment > >> /home/stack/templates/overcloud-baremetal-deploy.yaml > >> --deployed-server \ > >> --network-config \ > >> -e /home/stack/templates/environment.yaml \ > >> -e > >> > /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml > > >> \ > >> -e > >> > /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml > > >> \ > >> -e > >> > /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml > > >> \ > >> -e /home/stack/templates/ironic-config.yaml \ > >> -e > >> > /usr/share/openstack-tripleo-heat-templates/environments/services/ptp.yaml > >> \ > >> -e > >> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml > \ > >> -e > >> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ > >> -e /home/stack/containers-prepare-parameter.yaml > >> > >> Without Service Net Map, deployment was successful but TFTP of image > >> was an issue since we are going with custom network approach. > >> > >> Passing Service Net map details in a seperate file environment.yaml, > >> is giving the below error: > >> > >> 24d-f43f-6f76-000000006ce9 | TASK | Create containers managed by > >> Podman for /var/lib/tripleo-config/container-startup-config/step_3 > >> 2022-06-07 11:16:46.806982 | | WARNING | ERROR: Can't run container > >> ironic_db_sync > >> stderr: Error: statfs > >> /var/lib/config-data/puppet-generated/ironic_api: no such file or > >> directory > >> 2022-06-07 11:16:46.809252 | 525400fd-f24d-f43f-6f76-000000006c79 | > >> FATAL | Create containers managed by Podman for > >> /var/lib/tripleo-config/container-startup-config/step_3 | > >> overcloud-controller-1 | error={"changed": false, "msg": "Failed > >> containers: ironic_db_sync"} > >> 2022-06-07 11:16:46.810197 | 525400fd-f24d-f43f-6f76-000000006c79 | > >> TIMING | tripleo_container_manage : Create containers managed by > >> Podman for /var/lib/tripleo-config/container-startup-config/step_3 | > >> overcloud-controller-1 | 0:11:40.574944 | 10.62s > >> 2022-06-07 11:16:47.683400 | | WARNING | ERROR: Can't run container > >> ironic_db_sync > >> stderr: Error: statfs > >> /var/lib/config-data/puppet-generated/ironic_api: no such file or > >> directory > >> 2022-06-07 11:16:47.684455 | 525400fd-f24d-f43f-6f76-000000006ce9 | > >> FATAL | Create containers managed by Podman for > >> /var/lib/tripleo-config/container-startup-config/step_3 | > >> overcloud-controller-2 | error={"changed": false, "msg": "Failed > >> containers: ironic_db_sync"} > >> > >> Can you please suggest the place we can pass the service net map > >> details in Wallaby Release to resolve this issue? > >> > > > > You should be able to pass it in any of the environment files, like you > > did. Can you try adding the following to the environment file where you > > override the ServiceNetMap? > > > > parameter_merge_strategies: > > ServiceNetMap: merge > > > > Actually, my bad. (Thanks Rabi for reminding me on IRC.) The merge > strategies is after a change in master. With Wallaby we still have > ServiceNetMapDefaults which is merged with overrides in ServiceNetMap. > > AFICT from this error ... > > > 2022-06-07 11:16:47.683400 | | WARNING | ERROR: Can't run container > > ironic_db_sync > > stderr: Error: statfs > > /var/lib/config-data/puppet-generated/ironic_api: no such file or > > directory > > ... it did not execute $THT/common/container-puppet.sh for the > ironic_api service correctly. > > Could it be because you don't use a custom roles file? > The default[1] role file does not have the `oc_provisioning` > provisioning network. I wonder if some task is skipped because the node > is not on the network the service is mapped to. Can you try using a > custom roles file with the `oc_provisioning` network defined on the > controller role? > > I think we probably need more information to understand the problem. > Would it be possible to share more data? i.e full deploy log, all custom > environment files? > > Thanks! > > [1] > https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/wallaby/roles_data.yaml#L18 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafaelweingartner at gmail.com Wed Jun 8 13:59:11 2022 From: rafaelweingartner at gmail.com (=?UTF-8?Q?Rafael_Weing=C3=A4rtner?=) Date: Wed, 8 Jun 2022 10:59:11 -0300 Subject: [CLOUDKITTY] presentation used in the forum panels Message-ID: Hello CloudKitters! It was a pleasure to meet you guys during the OpenStack summit. Also, it was a good surprise to see that so many people already use CloudKitty. I really hope that we can all collaborate together to keep improving CloudKitty, and making it more and more solid. Attached is the presentation we used during the Forum, so you all can consult this material. Also, as always, if you need some guidance/help/or anything else, let us know. Link for the presentation online: https://docs.google.com/presentation/d/15e37xGKCJoTMwbarrGcSlLpECUkcJ2exazW_TztCehk/edit?usp=sharing -- Rafael Weing?rtner -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: CloudKitty onboarding and operators feedback forum panels.pdf Type: application/pdf Size: 1091998 bytes Desc: not available URL: From tkajinam at redhat.com Wed Jun 8 14:29:40 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Wed, 8 Jun 2022 23:29:40 +0900 Subject: [neutron][networking-l2gw][puppet] Status of networking-l2gw project Message-ID: Hello team, # I understand networking-l2gw is not under neutron stadium # but adding [neutron] tag hoping it would attract more attention. We found a recent change in neutron[1] broke networking-l2gw[2], and some jobs in puppet repos and rdo, with l2gw enabled, are failing. I proposed a potential fix for the bug[3], but I noticed the project looks unmaintained according to the following points. - No change has been merged for more than 1 year - No release was created since Wallaby - The zuul setting still points the victoria job template (openstack-python3-victoria-jobs-neutron) I'm wondering if anybody is actually interested in maintaining the project. If not, I'll propose retiring the plugin support from puppet (and TripleO because it requires puppet implementation). I appreciate any feedback about this. [1] https://review.opendev.org/c/openstack/neutron/+/837392 [2] https://bugs.launchpad.net/networking-l2gw/+bug/1977980 [3] https://review.opendev.org/c/x/networking-l2gw/+/845141 Thank you, Takashi -------------- next part -------------- An HTML attachment was scrubbed... URL: From dms at danplanet.com Wed Jun 8 14:43:59 2022 From: dms at danplanet.com (Dan Smith) Date: Wed, 08 Jun 2022 07:43:59 -0700 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: (Julia Kreger's message of "Wed, 8 Jun 2022 07:19:43 -0700") References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Message-ID: Julia Kreger writes: > Is that Nova's interpretation, specifically the delineation that > non-project owned should only be viewable by system, or was system > scope changed at some point? I interpreted it differently, but haven't > circled back recently. I guess interpretation and evolution in > specific pockets after initial implementation work started ultimately > resulted in different perceptions. Nope, not a Nova thing. Here's the relevant course correction from two PTGs ago: https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change Mohammed is going to be there and primed to discuss this as well. I think he's pretty well caught up on the current state of things. Having your experience with what it means in Ironic, as well as his context from the sticky implementation issues in the other projects should mean we have pretty good coverage. Thanks! --Dan From smooney at redhat.com Wed Jun 8 14:48:04 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 08 Jun 2022 15:48:04 +0100 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Message-ID: <913612422bb6844e05dfee31dc3caab60f71200a.camel@redhat.com> On Wed, 2022-06-08 at 07:19 -0700, Julia Kreger wrote: > Is that Nova's interpretation, specifically the delineation that > non-project owned should only be viewable by system, or was system > scope changed at some point? > no its not just the nova interpreation. there was some differnt view in differnt porjects but we did an openstack wide reset on this in yoga and on the defintions of the personas when it became apprent we coudl not proceed with the previous approch. > I interpreted it differently, but haven't > circled back recently. I guess interpretation and evolution in > specific pockets after initial implementation work started ultimately > resulted in different perceptions. > > I'll make sure operators are aware there may be significant nuances > and perception differences since they are using their own etherpads > and their own communications flow. i.e. we're unlikely to see many > find/use our own etherpads as they have their own. And yes, this is a > problem, but it is a higher level community/communications feedback > issue under active discussion. well it was under active discussion for a few cycle both at the ptgs and mailing lists, and common understaind was codeified as part of a comunity wide goal https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html with pahses agree to be implemnted across all project based on the updated common understanding of the personas and definitons to ensure that seperate interpreation and perspective nolonger differed. the direction change and rational are documend in the goal document https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change with the high level roadmap cpature in teh pahse 1,2 and 3 defintions. but in general we woudl like to get feedback from operatros if what is being propsoed in that document is useful. As dan noted the scope enforcment aspect is proving a challange for higher level porject like heat and tacker to support. the is more agreement on the usfullnes of a standard reader role, service role and perhaps manager role as thos standarised role dont really break exising usage but scope enforcement is a big change. so the dicusion of RBAC is not finalised but a direction has been chosen and there is a certin amount of moment behind it now. parts of the design can still change an evlonve but other parts are more concrate like the reader role. > > Granted, I get that the system scope ideas were breaking for some > projects in specific use patterns since not everything would be the > same nor possible (which is actually a good thing, context of use and > all), but it was in theory perfect for a lot of the external audit > tooling use cases which arise in so many different ways. > > Anyway, off to the next $thing with my scattered brain. > > On Wed, Jun 8, 2022 at 6:53 AM Dan Smith wrote: > > > > > the system level of scope does not allow you to see everything across the system > > > it only allows you to see the non project related resouces > > > > > > so you can see the flavors and host aggreates but not the instances as instances are project scoped. > > > and project scoped resouces like ports, instances, images and volumes cannot be accessed with a system scope > > > token if you enabel scope enforcement. > > > > > > that is one of the things we want to get clarity on form operators. > > > is the disticntion between system level resouces and project level resouces useful. > > > > Yep, exactly this. Given the amount of breakage it brings for things > > like Heat and Tacker, as well as the potential workflow annoyance for > > human admins, I really want to measure whether any operators see a > > benefit here. The persona roles, things like a standardized service > > role, and getting out of this current situation of having two sets of > > defaults are priorities for me. > > > > --Dan > From rlandy at redhat.com Wed Jun 8 15:13:33 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Wed, 8 Jun 2022 11:13:33 -0400 Subject: [tripleo] gate blocker - master/wallaby - "SELinux boolean os_enable_vtpm does not exist." In-Reply-To: References: Message-ID: On Tue, Jun 7, 2022 at 3:53 PM Ronelle Landy wrote: > > > On Tue, Jun 7, 2022 at 1:08 PM Ronelle Landy wrote: > >> Hello All, >> >> We have a consistent failure on multiple jobs running on >> check/gate/integration lines on master and wallaby changes - which shows up >> as a deployment error: >> >> FATAL | Enable os_enable_vtpm SELinux boolean for vTPM | standalone >> | error={"changed": false, "msg": "SELinux boolean os_enable_vtpm does not >> exist."} >> >> Details of the failure are in: >> https://bugs.launchpad.net/tripleo/+bug/1977873 >> >> We are investigating a fix/workaround. >> >> Please hold rechecks - we will post out again with updates. >> >> Thank you! >> >> > A fix is in progress - thank you Rafael and Lon: > > https://bugs.launchpad.net/tripleo/+bug/1977873/comments/7 > https://bugs.launchpad.net/tripleo/+bug/1977873/comments/8 > > Updated linselinux versions are available in centos-9 production composes: libselinux-3.4-2.el9.i686.rpm 2022-06-07 12:12 93K libselinux-3.4-2.el9.x86_64.rpm 2022-06-07 12:12 86K https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/BaseOS/x86_64/os/Packages/?C=M;O=A Waiting on mirrors to get this update before closing this out. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Wed Jun 8 16:49:57 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 08 Jun 2022 11:49:57 -0500 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> Message-ID: <18144390aeb.f7aa2f0856684.396006809241741590@ghanshyammann.com> ---- On Wed, 08 Jun 2022 09:43:59 -0500 Dan Smith wrote ---- > Julia Kreger writes: > > > Is that Nova's interpretation, specifically the delineation that > > non-project owned should only be viewable by system, or was system > > scope changed at some point? I interpreted it differently, but haven't > > circled back recently. I guess interpretation and evolution in > > specific pockets after initial implementation work started ultimately > > resulted in different perceptions. > > Nope, not a Nova thing. Here's the relevant course correction from two > PTGs ago: > > https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change > > Mohammed is going to be there and primed to discuss this as well. I > think he's pretty well caught up on the current state of things. Having > your experience with what it means in Ironic, as well as his context > from the sticky implementation issues in the other projects should mean > we have pretty good coverage. Yes. and it is more than just a single service use case especially when heat discussion[1] came up and the scope complexity for heat/NVF users is brought up. We want to make sure by introducing scope at the service level which is all good for us does not break others users/tooling like heat, tacker, and deployment projects. We discussed one solution for heat[2] which is sent on ML for feedback not still now response and that is why operators' feedback is critical before we try to implement something that can break them. [1] https://etherpad.opendev.org/p/rbac-zed-ptg#L104 [2] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028490.html -gmann > > Thanks! > > --Dan > > From gmann at ghanshyammann.com Wed Jun 8 16:53:03 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 08 Jun 2022 11:53:03 -0500 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: <913612422bb6844e05dfee31dc3caab60f71200a.camel@redhat.com> References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> <913612422bb6844e05dfee31dc3caab60f71200a.camel@redhat.com> Message-ID: <181443bdf1f.12767362256831.7510285921554510482@ghanshyammann.com> ---- On Wed, 08 Jun 2022 09:48:04 -0500 Sean Mooney wrote ---- > On Wed, 2022-06-08 at 07:19 -0700, Julia Kreger wrote: > > Is that Nova's interpretation, specifically the delineation that > > non-project owned should only be viewable by system, or was system > > scope changed at some point? > > > no its not just the nova interpreation. there was some differnt view in differnt porjects but > we did an openstack wide reset on this in yoga and on the defintions of the personas > when it became apprent we coudl not proceed with the previous approch. > > > I interpreted it differently, but haven't > > circled back recently. I guess interpretation and evolution in > > specific pockets after initial implementation work started ultimately > > resulted in different perceptions. > > > > I'll make sure operators are aware there may be significant nuances > > and perception differences since they are using their own etherpads > > and their own communications flow. i.e. we're unlikely to see many > > find/use our own etherpads as they have their own. And yes, this is a > > problem, but it is a higher level community/communications feedback > > issue under active discussion. > well it was under active discussion for a few cycle both at the ptgs and mailing lists, and common understaind was codeified as > part of a comunity wide goal https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html > with pahses agree to be implemnted across all project based on the updated common understanding of the personas and definitons > to ensure that seperate interpreation and perspective nolonger differed. > > the direction change and rational are documend in the goal document > https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change > > with the high level roadmap cpature in teh pahse 1,2 and 3 defintions. > > but in general we woudl like to get feedback from operatros if what is being propsoed in that document is useful. > As dan noted the scope enforcment aspect is proving a challange for higher level porject like heat and tacker to support. > the is more agreement on the usfullnes of a standard reader role, service role and perhaps manager role as thos standarised > role dont really break exising usage but scope enforcement is a big change. > > so the dicusion of RBAC is not finalised but a direction has been chosen and there is a certin amount of moment behind it now. > parts of the design can still change an evlonve but other parts are more concrate like the reader role. True, our overall goal is to ship improvement without breaking any use case. For example, 'scope' can be good for just nova admin/users but from heat, tacker users these might be breaking change more than improvement so we want to make sure what operators think by considering all use cases. -gmann > > > > Granted, I get that the system scope ideas were breaking for some > > projects in specific use patterns since not everything would be the > > same nor possible (which is actually a good thing, context of use and > > all), but it was in theory perfect for a lot of the external audit > > tooling use cases which arise in so many different ways. > > > > Anyway, off to the next $thing with my scattered brain. > > > > On Wed, Jun 8, 2022 at 6:53 AM Dan Smith wrote: > > > > > > > the system level of scope does not allow you to see everything across the system > > > > it only allows you to see the non project related resouces > > > > > > > > so you can see the flavors and host aggreates but not the instances as instances are project scoped. > > > > and project scoped resouces like ports, instances, images and volumes cannot be accessed with a system scope > > > > token if you enabel scope enforcement. > > > > > > > > that is one of the things we want to get clarity on form operators. > > > > is the disticntion between system level resouces and project level resouces useful. > > > > > > Yep, exactly this. Given the amount of breakage it brings for things > > > like Heat and Tacker, as well as the potential workflow annoyance for > > > human admins, I really want to measure whether any operators see a > > > benefit here. The persona roles, things like a standardized service > > > role, and getting out of this current situation of having two sets of > > > defaults are priorities for me. > > > > > > --Dan > > > > > From james.slagle at gmail.com Wed Jun 8 19:35:55 2022 From: james.slagle at gmail.com (James Slagle) Date: Wed, 8 Jun 2022 15:35:55 -0400 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami Message-ID: I'm formally proposing that we add Brendan Shephard and Takashi Kajinami to the core reviewers team for TripleO. Takashi is already considered core on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and likewise for Brendan. They both have been contributing and reviewing for a little while now, and I feel it's time to add them to the team. Please +1 or let me know any concerns before Friday June 15th. Thanks! -- -- James Slagle -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From hjensas at redhat.com Wed Jun 8 19:47:20 2022 From: hjensas at redhat.com (Harald Jensas) Date: Wed, 8 Jun 2022 21:47:20 +0200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: On Wed, 8 Jun 2022, 21:36 James Slagle, wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- > +1 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abishop at redhat.com Wed Jun 8 19:56:19 2022 From: abishop at redhat.com (Alan Bishop) Date: Wed, 8 Jun 2022 12:56:19 -0700 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: On Wed, Jun 8, 2022 at 12:38 PM James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > +1 for both of them! > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbaker at redhat.com Wed Jun 8 21:09:37 2022 From: sbaker at redhat.com (Steve Baker) Date: Thu, 9 Jun 2022 09:09:37 +1200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: +1! On 9/06/22 07:35, James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi > Kajinami to the core reviewers team for TripleO. Takashi is already > considered core on puppet-tripleo, and I'm proposing we expand that to > all of TripleO, and likewise for Brendan. > > They both have been contributing and reviewing for a little while now, > and I feel it's time to add them to the team. Please +1 or let me know > any concerns? before Friday June 15th. Thanks! > > -- > -- James Slagle > -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From johfulto at redhat.com Wed Jun 8 22:29:49 2022 From: johfulto at redhat.com (John Fulton) Date: Wed, 8 Jun 2022 18:29:49 -0400 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: +1 On Wed, Jun 8, 2022, 5:11 PM Steve Baker wrote: > +1! > On 9/06/22 07:35, James Slagle wrote: > > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ramishra at redhat.com Thu Jun 9 02:27:49 2022 From: ramishra at redhat.com (Rabi Mishra) Date: Thu, 9 Jun 2022 07:57:49 +0530 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: On Thu, Jun 9, 2022 at 1:08 AM James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > +1! > -- > -- James Slagle > -- > -- Regards, Rabi Mishra -------------- next part -------------- An HTML attachment was scrubbed... URL: From ueha.ayumu at fujitsu.com Thu Jun 9 05:14:15 2022 From: ueha.ayumu at fujitsu.com (ueha.ayumu at fujitsu.com) Date: Thu, 9 Jun 2022 05:14:15 +0000 Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team In-Reply-To: References: Message-ID: Hi Rico, This is just a friendly reminder that we are waiting for your action. Or please let us know if there are any other necessary processes. Thanks. Best Regards, Ueha -----Original Message----- From: ueha.ayumu at fujitsu.com Sent: Thursday, June 2, 2022 5:39 PM To: openstack-discuss at lists.openstack.org; 'Rico Lin' Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team Hi Rico, This proposal is great helpful for us! There seems to be no particular objection, so could you add it to heat-translator-core if there is no problem? Thanks. Best Regards, Ueha -----Original Message----- From: ueha.ayumu at fujitsu.com Sent: Wednesday, May 25, 2022 1:40 PM To: openstack-discuss at lists.openstack.org Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team +1 Best Regards, Ueha -----Original Message----- From: Yoshito Ito Sent: Wednesday, May 25, 2022 12:31 PM To: openstack-discuss at lists.openstack.org Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team Hi heat-translator team! I would like to propose Hiromu Asahina (h-asahina) as a member of the heat-translator core team. I'm now not able to actively contribute to heat-translator, and he can lead the team instead of me. He is mainly contributing to Tacker by fixing issues, refactoring existing codes, and a lot of helpful reviews, with his experience as a research engineer in NTT. Tacker highly depends on heat-translator in its main functionality, so I believe he can manage it with his knowledge of Tacker. Please vote for his nomination by answering this mail till June 1st. Best Regards, Yoshito Ito From marios at redhat.com Thu Jun 9 05:15:26 2022 From: marios at redhat.com (Marios Andreou) Date: Thu, 9 Jun 2022 08:15:26 +0300 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: On Wednesday, June 8, 2022, James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > +1 and +1 easily ;) > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- > -- _sent from my mobile - sorry for spacing spelling etc_ -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregory.orange at pawsey.org.au Thu Jun 9 08:10:49 2022 From: gregory.orange at pawsey.org.au (Gregory Orange) Date: Thu, 9 Jun 2022 16:10:49 +0800 Subject: [kolla] restart mariadb container on host 1 Message-ID: Hi all, I wanted to temporarily change a mariadb config in /etc/kolla/mariadb/galera.cnf so I intended to stop mariadb on host 1, make the change, start it again, then repeat in turn for hosts 2 and 3. Upon starting it on host 1, it doesn't rejoin, but creates its own separate cluster. Dutiful really, since logs show: WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster How do I avoid that? General advice on handling this kind of activity would also be welcome. Thank you, Greg. From juliaashleykreger at gmail.com Thu Jun 9 08:41:09 2022 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Thu, 9 Jun 2022 01:41:09 -0700 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: <18144390aeb.f7aa2f0856684.396006809241741590@ghanshyammann.com> References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> <18144390aeb.f7aa2f0856684.396006809241741590@ghanshyammann.com> Message-ID: On Wed, Jun 8, 2022 at 9:50 AM Ghanshyam Mann wrote: > > ---- On Wed, 08 Jun 2022 09:43:59 -0500 Dan Smith wrote ---- > > Julia Kreger writes: > > > > > Is that Nova's interpretation, specifically the delineation that > > > non-project owned should only be viewable by system, or was system > > > scope changed at some point? I interpreted it differently, but haven't > > > circled back recently. I guess interpretation and evolution in > > > specific pockets after initial implementation work started ultimately > > > resulted in different perceptions. > > > > Nope, not a Nova thing. Here's the relevant course correction from two > > PTGs ago: > > > > https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change Thanks Sean! Now I'm remembering! This is what happens when we implement things and then change directions and already have some variety of models. I guess I mentally dropped some of the context due to this after reading it since it was already "done" and committed from my perspective. Thanks for the refresher! > > > > Mohammed is going to be there and primed to discuss this as well. I > > think he's pretty well caught up on the current state of things. Having > > your experience with what it means in Ironic, as well as his context > > from the sticky implementation issues in the other projects should mean > > we have pretty good coverage. > > Yes. and it is more than just a single service use case especially when heat discussion[1] > came up and the scope complexity for heat/NVF users is brought up. We want to make > sure by introducing scope at the service level which is all good for us does not break > others users/tooling like heat, tacker, and deployment projects. > > We discussed one solution for heat[2] which is sent on ML for feedback not still now response and that > is why operators' feedback is critical before we try to implement something that can break them. > Thanks! I think we will try to remember to bring this up, but we also need to let the operators drive based upon their wants and needs. Operators tend to want to focus on the nuts and bolts of older software, so hopefully we will be able to move the discussion forward. There is entirely the possibility they just won't want to discuss it because it is well over the horizon for them. Semi-on topic, but also on topic to the base different topic: How many people would be interested in actually trying to meet up with operators on some semi-regular basis? Having a past operations background would be a major bonus point because the same words would be able to be coalesced far faster. Additionally, travel? How far? etc? I'm feeling like we need to build a "bridging" group of some sort. There are a ton of threads I'm on on trying to help bridge some of these communication gaps right now. > [1] https://etherpad.opendev.org/p/rbac-zed-ptg#L104 > [2] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028490.html > > -gmann > > > > > Thanks! > > > > --Dan > > > > From mark at stackhpc.com Thu Jun 9 08:50:22 2022 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 9 Jun 2022 09:50:22 +0100 Subject: [kolla-ansible][kolla] merge_configs module replacing colon to equal In-Reply-To: References: Message-ID: Hi Sharath, I had not realised this. Wikipedia suggests colons are sometimes used instead of equals. Perhaps the function should return which delimiter was found, and use it to generate the merged file? Mark On Tue, 7 Jun 2022 at 06:19, Sharath Ck wrote: > Hi, > > In kolla-ansible, I am using the merge_configs module to merge service > configs files. I want to merge api-paste.ini config with my custom changes, > however while merging if a line consists of one colon, it is getting > replaced with equal sign. merge_configs splits the line by considering > colon as delimiter and treats two parts as key and value. Later which will > be substituted as key = value. > > Original > */v2.1: oscomputeversion_v2* > Merged > */v2.1= oscomputeversion_v2 * > > Below, colon will be 1 and equal will be -1, > > > > > > > > > > > > > > > > *def _split_key_value(self, line): colon = line.find(':') > equal = line.find('=') if colon < 0 and equal < 0: return > self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and > equal < colon): key, value = line[:equal], line[equal + 1:] > else: key, value = line[:colon], line[colon + 1:] value > = value.strip() if value and value[0] == value[-1] and > value.startswith(("\"", "'")): value = value[1:-1] return > key.strip(), [value]* > > I want to keep the config as it is and merge only my custom changes. > Please let me know how to achieve this or colon can be escaped somehow. > > Regards, > Sharath > Regards, > Sharath > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Thu Jun 9 08:53:52 2022 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 9 Jun 2022 09:53:52 +0100 Subject: [kolla] restart mariadb container on host 1 In-Reply-To: References: Message-ID: Hi Greg, It should only have that option set when bootstrapping the cluster. How are you stopping and starting the container? Could you provide logs? Mark On Thu, 9 Jun 2022 at 09:17, Gregory Orange wrote: > Hi all, > > I wanted to temporarily change a mariadb config in > /etc/kolla/mariadb/galera.cnf so I intended to stop mariadb on host 1, > make the change, start it again, then repeat in turn for hosts 2 and 3. > > Upon starting it on host 1, it doesn't rejoin, but creates its own > separate cluster. Dutiful really, since logs show: > > WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster > > How do I avoid that? General advice on handling this kind of activity > would also be welcome. > > Thank you, > Greg. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Thu Jun 9 08:55:27 2022 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Thu, 9 Jun 2022 01:55:27 -0700 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: <181443bdf1f.12767362256831.7510285921554510482@ghanshyammann.com> References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> <913612422bb6844e05dfee31dc3caab60f71200a.camel@redhat.com> <181443bdf1f.12767362256831.7510285921554510482@ghanshyammann.com> Message-ID: On Wed, Jun 8, 2022 at 9:53 AM Ghanshyam Mann wrote: > > ---- On Wed, 08 Jun 2022 09:48:04 -0500 Sean Mooney wrote ---- [trim] > > so the dicusion of RBAC is not finalised but a direction has been chosen and there is a certin amount of moment behind it now. > > parts of the design can still change an evlonve but other parts are more concrate like the reader role. > > True, our overall goal is to ship improvement without breaking any use case. For example, 'scope' can be good > for just nova admin/users but from heat, tacker users these might be breaking change more than improvement > so we want to make sure what operators think by considering all use cases. > > -gmann So, one thought. Ironic views system scope as *critical* for our usage based upon the consensus we built before the direction change, because the system fundamentally is the owner/manager of $things. We can and likely should extend that out to project admin (granted, I suspect at least one ironic admin will reply with a strong -1 to such a change... :\. ) given the direction change. We also have had some operators jump on it, but... again, entirely different models of usage/interaction given the base state. If system scope were to suddenly disappear or be completely redefined, it would be a hard break for us at this point. > > > > > > > Granted, I get that the system scope ideas were breaking for some > > > projects in specific use patterns since not everything would be the > > > same nor possible (which is actually a good thing, context of use and > > > all), but it was in theory perfect for a lot of the external audit > > > tooling use cases which arise in so many different ways. > > > > > > Anyway, off to the next $thing with my scattered brain. > > > > > > On Wed, Jun 8, 2022 at 6:53 AM Dan Smith wrote: > > > > > > > > > the system level of scope does not allow you to see everything across the system > > > > > it only allows you to see the non project related resouces > > > > > > > > > > so you can see the flavors and host aggreates but not the instances as instances are project scoped. > > > > > and project scoped resouces like ports, instances, images and volumes cannot be accessed with a system scope > > > > > token if you enabel scope enforcement. > > > > > > > > > > that is one of the things we want to get clarity on form operators. > > > > > is the disticntion between system level resouces and project level resouces useful. > > > > > > > > Yep, exactly this. Given the amount of breakage it brings for things > > > > like Heat and Tacker, as well as the potential workflow annoyance for > > > > human admins, I really want to measure whether any operators see a > > > > benefit here. The persona roles, things like a standardized service > > > > role, and getting out of this current situation of having two sets of > > > > defaults are priorities for me. > > > > > > > > --Dan > > > > > > > > > From sharath.madhava at gmail.com Thu Jun 9 08:57:03 2022 From: sharath.madhava at gmail.com (Sharath Ck) Date: Thu, 9 Jun 2022 14:27:03 +0530 Subject: [kolla-ansible][kolla] merge_configs module replacing colon to equal In-Reply-To: References: Message-ID: Hi Mark, Yes, that would be correct. If the function returns the delimiter and uses the same while mapping key value, it would keep the config content intact. If colon can be replaced by equal and vice versa in openstack configs, especially api-paste.ini. I can continue with the merge now and update the merge_configs module when a fix is available. Regards, Sharath On Thu, Jun 9, 2022 at 2:20 PM Mark Goddard wrote: > Hi Sharath, > > I had not realised this. Wikipedia suggests colons are sometimes used > instead of equals. Perhaps the function should return which delimiter was > found, and use it to generate the merged file? > > Mark > > On Tue, 7 Jun 2022 at 06:19, Sharath Ck wrote: > >> Hi, >> >> In kolla-ansible, I am using the merge_configs module to merge service >> configs files. I want to merge api-paste.ini config with my custom changes, >> however while merging if a line consists of one colon, it is getting >> replaced with equal sign. merge_configs splits the line by considering >> colon as delimiter and treats two parts as key and value. Later which will >> be substituted as key = value. >> >> Original >> */v2.1: oscomputeversion_v2* >> Merged >> */v2.1= oscomputeversion_v2 * >> >> Below, colon will be 1 and equal will be -1, >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> *def _split_key_value(self, line): colon = line.find(':') >> equal = line.find('=') if colon < 0 and equal < 0: return >> self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and >> equal < colon): key, value = line[:equal], line[equal + 1:] >> else: key, value = line[:colon], line[colon + 1:] value >> = value.strip() if value and value[0] == value[-1] and >> value.startswith(("\"", "'")): value = value[1:-1] return >> key.strip(), [value]* >> >> I want to keep the config as it is and merge only my custom changes. >> Please let me know how to achieve this or colon can be escaped somehow. >> >> Regards, >> Sharath >> Regards, >> Sharath >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjeanner at redhat.com Thu Jun 9 09:02:37 2022 From: cjeanner at redhat.com (=?UTF-8?Q?C=c3=a9dric_Jeanneret?=) Date: Thu, 9 Jun 2022 11:02:37 +0200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: <6c26f05b-1814-fb7d-7a85-8070dd9b7b24@redhat.com> Of course +42 !! On 6/8/22 21:35, James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered > core on puppet-tripleo, and I'm proposing we expand that to all of > TripleO, and likewise for Brendan. > > They both have been contributing and reviewing for a little while now, > and I feel it's time to add them to the team. Please +1 or let me know > any concerns? before Friday June 15th. Thanks! > > -- > -- James Slagle > -- -- C?dric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: From christian.rohmann at inovex.de Thu Jun 9 09:11:46 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Thu, 9 Jun 2022 11:11:46 +0200 Subject: [rbac][nova][cinder] Canned roles for service users / inter-service communication (e.g. event submission) Message-ID: Hey openstack-discuss, I posted to the ML quite a while ago about an issue of resized (Cinder) volumes not being propagated to the (Nova) instance. See http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020476.html. The issue there was Cinder being not allowed to send the "volume-extended" event (or any event for that matter via the Nova API just using the user token. For this a configurable additional "privileged user" was added to the config quite a while back with https://opendev.org/openstack/cinder/commit/04003d7c513ed4dd5129cbd5ad1af14a5b200677. While the functionality then works I suppose there should be canned and maintained RBAC roles for such kind of inter service to service communications, e.g. to emit events to others. Otherwise deployments likely will use admin privileged users ignoring the least privilege principle and creating an large attack surface. And there are quite few of those relations even with the most commonly used services. Cinder -> nova, nova-> cincer, nova->ironic, .... nova-> neutron, .... Are such canned RBAC rules for "special" inter service users on the backlog somewhere? Or am I totally misconceiving the issue here? I know there is https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#isolate-system-specific-api-policies and also the question for feedback at https://etherpad.opendev.org/p/rbac-operator-feedback, but that all seems to focus on the impact of roles used by humans / users and not about service roles at all. Regards Christian From christian.rohmann at inovex.de Thu Jun 9 10:39:44 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Thu, 9 Jun 2022 12:39:44 +0200 Subject: [rbac][nova][cinder] Canned roles for service users / inter-service communication (e.g. event submission) In-Reply-To: References: Message-ID: <26833ce2-e486-d953-b7ea-999d94862631@inovex.de> On 09/06/2022 11:11, Christian Rohmann wrote: > And there are quite few of those relations even with the most commonly > used services. > Cinder -> nova, nova-> cincer, nova->ironic, .... nova-> neutron, .... > > Are such canned RBAC rules for "special" inter service users on the > backlog somewhere? Or am I totally misconceiving the issue here? > > I know there is > https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#isolate-system-specific-api-policies > and also the question for feedback at > https://etherpad.opendev.org/p/rbac-operator-feedback, but that all > seems to focus on the impact of roles used by humans / users and not > about service roles at all. I just noticed that Christian Berendt does a forum talk on "Deprivilization of the internal service accounts" TODAY at 2:40pm - 3:10pm at A05 on apparently that exact question :-) Regards Christian From Russell.Stather at ignitiongroup.co.za Thu Jun 9 10:41:10 2022 From: Russell.Stather at ignitiongroup.co.za (Russell Stather) Date: Thu, 9 Jun 2022 10:41:10 +0000 Subject: Error creating octavia amphora Message-ID: Hi I am seeing the below error when creating a load balancer. The security group does exist, and it is the correct group (tagged with charm-octavia) What am I missing here to resolve this? 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server octavia.common.exceptions.ComputeBuildException: Failed to build compute instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', 'message': 'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in build_instances\n scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.page at canonical.com Thu Jun 9 10:53:30 2022 From: james.page at canonical.com (James Page) Date: Thu, 9 Jun 2022 12:53:30 +0200 Subject: [charms][octavia] Re: Error creating octavia amphora In-Reply-To: References: Message-ID: Hi Russell On Thu, Jun 9, 2022 at 12:44 PM Russell Stather < Russell.Stather at ignitiongroup.co.za> wrote: > Hi > > I am seeing the below error when creating a load balancer. The security > group does exist, and it is the correct group (tagged with charm-octavia) > > What am I missing here to resolve this? > I'm wondering whether the configured security group is not owned by the right project - how did you create the required network and security configuration for Octavia? > > 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server > octavia.common.exceptions.ComputeBuildException: Failed to build compute > instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', > 'message': 'Exceeded maximum number of retries. Exceeded max scheduling > attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last > exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', > 'details': 'Traceback (most recent call last):\n File > "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in > build_instances\n scheduler_utils.populate_retry(\n File > "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in > populate_retry\n raise > exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: > Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for > instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security > group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} > 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bshephar at redhat.com Thu Jun 9 11:16:30 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Thu, 9 Jun 2022 21:16:30 +1000 Subject: Error creating octavia amphora In-Reply-To: References: Message-ID: Hey Russell, Are you able to share the outputs from: $ openstack server show 524ae27b-1542-4c2d-9118-138d9e7f3770 -c id -c project_id -c security_groups -c status -f yaml And: $ openstack security group show0b683c75-d900-4e45-acb2-8bc321580666 -c id -c project_id -f yaml I agree with James, my assumption would be that we will find they aren't in the same project, so Nova can't use that security group for the Amphorae. I'm not familiar with charmed openstack though, but it does look like James is from Canonical and might be able to advise on the specifics. All the best, Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Thu, Jun 9, 2022 at 8:48 PM Russell Stather < Russell.Stather at ignitiongroup.co.za> wrote: > Hi > > I am seeing the below error when creating a load balancer. The security > group does exist, and it is the correct group (tagged with charm-octavia) > > What am I missing here to resolve this? > > 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server > octavia.common.exceptions.ComputeBuildException: Failed to build compute > instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', > 'message': 'Exceeded maximum number of retries. Exceeded max scheduling > attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last > exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', > 'details': 'Traceback (most recent call last):\n File > "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in > build_instances\n scheduler_utils.populate_retry(\n File > "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in > populate_retry\n raise > exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: > Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for > instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security > group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} > 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Jun 9 11:24:10 2022 From: smooney at redhat.com (Sean Mooney) Date: Thu, 09 Jun 2022 12:24:10 +0100 Subject: [kolla-ansible][kolla] merge_configs module replacing colon to equal In-Reply-To: References: Message-ID: <042a070e63901559e99fddf3a9a45724f88d82d0.camel@redhat.com> On Thu, 2022-06-09 at 14:27 +0530, Sharath Ck wrote: > Hi Mark, > > Yes, that would be correct. If the function returns the delimiter and uses > the same while mapping key value, it would keep the config content intact. > > If colon can be replaced by equal and vice versa in openstack configs, > especially api-paste.ini. I can continue with the merge now and update the > merge_configs module when a fix is available. just not that openstack for most file does not use ini format. it juse conf format or strictly speaking the oslo config format but the openstack service dont actully use ini unless we are using lib that have config that we need to generate. so not evenything that is vaild in conf format is vlaid in ini format and visversa for example ini formant defient that if repeate a key the last value is used with oslo config if the key is defiend as a multiopt type repeated values are formed into a list. so? key=val1 key=val2 in ini format result in key=val2 when parsed but in conforma that can be intpertated as k=[val1,val2] the merge_configs module is based on pythons config parser which is inteded for ini files but we use them for conf files. api-paste.ini is not form openstack it s a config for a lib we use so in principal it might support : instead of = but that replacemnt is not intended to be support in nova.conf for example which is not an ini file its a .conf or oslo conf file. kolla can extend merge_config to alow : for ini file if they want too but its not a bug if you try to use : in nova.conf and it does not work. we dont support that for openstack's config files intentinally. it may work because of the the code oslo.config is built on but that woudl not be intentinal as far as im aware and it might break. in this case you are modifying nova's api-paste.ini https://github.com/openstack/nova/blob/master/etc/nova/api-paste.ini specificaly https://github.com/openstack/nova/blob/master/etc/nova/api-paste.ini#L21-L22= im not entirely sure why we use : [composite:osapi_compute] use = call:nova.api.openstack.urlmap:urlmap_factory /: oscomputeversions /v2: oscomputeversion_legacy_v2 /v2.1: oscomputeversion_v2 # v21 is an exactly feature match for v2, except it has more stringent # input validation on the wsgi surface (prevents fuzzing early on the # API). It also provides new features via API microversions which are # opt into for clients. Unaware clients will receive the same frozen # v2 API feature set, but with some relaxed validation /v2/+: openstack_compute_api_v21_legacy_v2_compatible /v2.1/+: openstack_compute_api_v21 looking a the pastedeploy docs breifly https://docs.pylonsproject.org/projects/pastedeploy/en/latest/index.html#paste-composite-factory it seams to be using = and none of the test code used : so i dont think they inted to suport that either. it might be better for nova and other poject to not use : in the api-paste.ini as we seem to be in undefiend behavior land on this one. https://github.com/openstack/nova/blob/master/nova/api/openstack/urlmap.py implementes the factory in nova which delegets the path parsign to paste https://github.com/cdent/paste/blob/225c7ebf34a9c90435932b690967538493a943ce/paste/urlmap.py#L35-L74= [composite:osapi_compute] use = call:nova.api.openstack.urlmap:urlmap_factory / = oscomputeversions /v2 = oscomputeversion_legacy_v2 /v2.1 = oscomputeversion_v2 # v21 is an exactly feature match for v2, except it has more stringent # input validation on the wsgi surface (prevents fuzzing early on the # API). It also provides new features via API microversions which are # opt into for clients. Unaware clients will receive the same frozen # v2 API feature set, but with some relaxed validation /v2/+ = openstack_compute_api_v21_legacy_v2_compatible /v2.1/+ = openstack_compute_api_v21 i assume ^ does not work today? i have not tried it. by all means add supprot to kolla for this use case if you would like too but longter we proably shoudl remove the use of paste since it undeploy or at least consider a syntax that is tested by them. if this syntax is somethign we added we do not appear to have any testing for it lookign at nova quickly. > > Regards, > Sharath > > > On Thu, Jun 9, 2022 at 2:20 PM Mark Goddard wrote: > > > Hi Sharath, > > > > I had not realised this. Wikipedia suggests colons are sometimes used > > instead of equals. Perhaps the function should return which delimiter was > > found, and use it to generate the merged file? > > > > Mark > > > > On Tue, 7 Jun 2022 at 06:19, Sharath Ck wrote: > > > > > Hi, > > > > > > In kolla-ansible, I am using the merge_configs module to merge service > > > configs files. I want to merge api-paste.ini config with my custom changes, > > > however while merging if a line consists of one colon, it is getting > > > replaced with equal sign. merge_configs splits the line by considering > > > colon as delimiter and treats two parts as key and value. Later which will > > > be substituted as key = value. > > > > > > Original > > > */v2.1: oscomputeversion_v2* > > > Merged > > > */v2.1= oscomputeversion_v2 * > > > > > > Below, colon will be 1 and equal will be -1, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *def _split_key_value(self, line): colon = line.find(':') > > > equal = line.find('=') if colon < 0 and equal < 0: return > > > self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and > > > equal < colon): key, value = line[:equal], line[equal + 1:] > > > else: key, value = line[:colon], line[colon + 1:] value > > > = value.strip() if value and value[0] == value[-1] and > > > value.startswith(("\"", "'")): value = value[1:-1] return > > > key.strip(), [value]* > > > > > > I want to keep the config as it is and merge only my custom changes. > > > Please let me know how to achieve this or colon can be escaped somehow. > > > > > > Regards, > > > Sharath > > > Regards, > > > Sharath > > > > > From smooney at redhat.com Thu Jun 9 11:28:54 2022 From: smooney at redhat.com (Sean Mooney) Date: Thu, 09 Jun 2022 12:28:54 +0100 Subject: [rbac][nova][cinder] Canned roles for service users / inter-service communication (e.g. event submission) In-Reply-To: References: Message-ID: On Thu, 2022-06-09 at 11:11 +0200, Christian Rohmann wrote: > Hey openstack-discuss, > > I posted to the ML quite a while ago about an issue of resized (Cinder) > volumes not being propagated to the (Nova) instance. > See > http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020476.html. > > The issue there was Cinder being not allowed to send the > "volume-extended" event (or any event for that matter via the Nova API > just using the user token. > For this a configurable additional "privileged user" was added to the > config quite a while back with > https://opendev.org/openstack/cinder/commit/04003d7c513ed4dd5129cbd5ad1af14a5b200677. > > While the functionality then works I suppose there should be canned and > maintained RBAC roles for such kind of inter service to service > communications, e.g. to emit events to others. Otherwise deployments > likely will use admin privileged users ignoring the least privilege > principle and creating an large attack surface. > > And there are quite few of those relations even with the most commonly > used services. > Cinder -> nova, nova-> cincer, nova->ironic, .... nova-> neutron, .... > > Are such canned RBAC rules for "special" inter service users on the > backlog somewhere? Or am I totally misconceiving the issue here? yes its called the service role its been discussed many times over the laste 3-4 virutal ptgs. its part of phase 2 https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#phase-2 """Isolate service-to-service APIs to the service role""" so the target is to start implementing the service role in AA and compelte it in all service by BB https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#aa-release-timeline > > I know there is > https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#isolate-system-specific-api-policies > and also the question for feedback at > https://etherpad.opendev.org/p/rbac-operator-feedback, but that all > seems to focus on the impact of roles used by humans / users and not > about service roles at all. > > > Regards > > Christian > > From tim+openstack.org at coote.org Thu Jun 9 11:43:39 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Thu, 9 Jun 2022 12:43:39 +0100 Subject: simple build fails to allocate network interface In-Reply-To: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> References: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> Message-ID: <0044A33F-0E87-433C-8130-77C66D79926A@coote.org> On 9 Jun 2022, at 09:58, tim+openstack.org at coote.org wrote: Hullo I?m trying to spin up a simple OS environment in a single vm and think that I?m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I?ve tried this on a couple of laptops, with the same result, so I suspect that it?s something to do with how I?ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I?ve got to get it working to log any bugs). I?m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try: `openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble` The network id is identified from this: ??" [vagrant at localhost devstack]$ openstack network list +--------------------------------------+---------+----------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+----------------------------------------------------------------------------+ | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 | +--------------------------------------+---------+??????????????????????????????????????+ ??? Using a public network fails with an error that this isn?t allowed. With the private network above, the failure throws out the following errors. ??" [vagrant at localhost devstack]$ sudo journalctl -f |grep ERROR Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): [snip] ??" The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): ??" # -*- mode: ruby -*- # vi: set ft=ruby : [snip] Vagrant.configure(2) do |config| config.vm.box = "eurolinux-vagrant/centos-stream-9" #config.vm.box = "hashicorp/bionic64" config.hostmanager.enabled = true config.hostmanager.manage_host = true config.hostmanager.manage_guest = true config.hostmanager.ignore_private_ip = false config.hostmanager.include_offline = true config.ssh.pty = true config.vm.provision "shell", inline: $script config.vm.network "forwarded_port", guest: 80, host: 8080 config.vm.provider "virtualbox" do | v | v.memory = "9216" v.cpus = "2" end end ??? I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files. Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details? Tim From gregory.orange at pawsey.org.au Thu Jun 9 11:45:20 2022 From: gregory.orange at pawsey.org.au (Gregory Orange) Date: Thu, 9 Jun 2022 19:45:20 +0800 Subject: [kolla] restart mariadb container on host 1 In-Reply-To: References: Message-ID: <55d9fd44-aaca-2bd7-adce-e3cda10140e0@pawsey.org.au> Right, thank you - I thought and hoped that was the intent. I'm just doing: docker stop mariadb docker start mariadb I've just done the same with host 3, and while stopped grastate.dat changed to `safe_to_bootstrap: 1` on host 2, and seqno: appeared in host 3's grastate.dat. Starting it again worked fine. docker inspect mariadb differs on the hosts, 1 has: "Env": [ ... "BOOTSTRAP_ARGS=--wsrep-new-cluster", while the other two don't. /var/log/kolla/mariadb/mariadb.log https://termbin.com/px15 with safe_to_bootstrap: 0, container stops again https://termbin.com/meli after removing /var/lib/docker/volumes/mariadb/_data/, separate cluster results I've just tested a dev cluster with the same steps, and it doesn't have the same problem - no Env entry, no problems restarting. I'm now wondering if it relates to the database migration and restore we did a while back, coming from another cluster with rsync replication and switching over to mariabackup. Next, I deleted the container in the dev cluster, and ran mariadb_recovery play - worked fine. I was hesitant to do this in the test cluster I'm working on because of other settings we have handcrafted for now, but after a few trial runs, I'm more confident. ... Yep, that fixed it - docker stop/start mariadb now works. BOOTSTRAP_ARGS is gone from the Env in the container too. Thank you for giving me the confidence to prod it further. Greg. On 9/6/22 16:53, Mark Goddard wrote: > Hi Greg, > > It should only have that option set when bootstrapping the cluster. How > are you stopping and starting the container? Could you provide logs? > > Mark > > On Thu, 9 Jun 2022 at 09:17, Gregory Orange > > wrote: > > Hi all, > > I wanted to temporarily change a mariadb config in > /etc/kolla/mariadb/galera.cnf so I intended to stop mariadb on host 1, > make the change, start it again, then repeat in turn for hosts 2 and 3. > > Upon starting it on host 1, it doesn't rejoin, but creates its own > separate cluster. Dutiful really, since logs show: > > WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster > > How do I avoid that? General advice on handling this kind of activity > would also be welcome. > > Thank you, > Greg. > -- Gregory Orange Cloud System Administrator Scientific Platforms Team building representative Pawsey Supercomputing Centre, CSIRO From ralonsoh at redhat.com Thu Jun 9 12:20:22 2022 From: ralonsoh at redhat.com (Rodolfo Alonso Hernandez) Date: Thu, 9 Jun 2022 14:20:22 +0200 Subject: simple build fails to allocate network interface In-Reply-To: <0044A33F-0E87-433C-8130-77C66D79926A@coote.org> References: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> <0044A33F-0E87-433C-8130-77C66D79926A@coote.org> Message-ID: Hello: This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too. Regards. On Thu, Jun 9, 2022 at 1:53 PM wrote: > > > On 9 Jun 2022, at 09:58, tim+openstack.org at coote.org wrote: > > Hullo > > I?m trying to spin up a simple OS environment in a single vm and think > that I?m making some silly mistake, but cannot spot it. Any hints would be > greatly expected. I?ve tried this on a couple of laptops, with the same > result, so I suspect that it?s something to do with how I?ve set up my vm, > but I cannot identify it. (I think that there may be a few other devstack > issues, but I?ve got to get it working to log any bugs). > > I?m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the > devstack install/build process. Everything seems to install, but when I try: > > `openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk > --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default > wibble` > > The network id is identified from this: > ??" > [vagrant at localhost devstack]$ openstack network list > > +--------------------------------------+---------+----------------------------------------------------------------------------+ > | ID | Name | Subnets > | > > +--------------------------------------+---------+----------------------------------------------------------------------------+ > | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | > 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | > | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | > e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | > | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | > 97e6d8e6-0f57-48bc-8f1d-e30587086153 | > > +--------------------------------------+---------+??????????????????????????????????????+ > ??? > > Using a public network fails with an error that this isn?t allowed. > > With the private network above, the failure throws out the following > errors. > ??" > [vagrant at localhost devstack]$ sudo journalctl -f |grep ERROR > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR > nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo > demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to > spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface > creation failed > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR > nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] > Traceback (most recent call last): > > [snip] > ??" > > The Vagrantfile (commented out lines relate to trying this with Ubunutu, > which failed the same way, and attempts to use multiple vms): > ??" > # -*- mode: ruby -*- > # vi: set ft=ruby : > > [snip] > Vagrant.configure(2) do |config| > > config.vm.box = "eurolinux-vagrant/centos-stream-9" > #config.vm.box = "hashicorp/bionic64" > > > config.hostmanager.enabled = true > config.hostmanager.manage_host = true > config.hostmanager.manage_guest = true > config.hostmanager.ignore_private_ip = false > config.hostmanager.include_offline = true > > config.ssh.pty = true > > config.vm.provision "shell", inline: $script > > config.vm.network "forwarded_port", guest: 80, host: 8080 > > config.vm.provider "virtualbox" do | v | > v.memory = "9216" > v.cpus = "2" > end > > end > ??? > > I think that the vm has enough RAM, although there is minimal swap being > used, but I think that this is not significant as there is much more RAM > used for caching files. > > Any obvious hints as to why the spin up fails to create the NIC - or > somewhere to look for further and better details? > > Tim > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlandy at redhat.com Thu Jun 9 12:51:12 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Thu, 9 Jun 2022 08:51:12 -0400 Subject: [tripleo] gate blocker - master/wallaby - "SELinux boolean os_enable_vtpm does not exist." In-Reply-To: References: Message-ID: On Wed, Jun 8, 2022 at 11:13 AM Ronelle Landy wrote: > > On Tue, Jun 7, 2022 at 3:53 PM Ronelle Landy wrote: > >> >> >> On Tue, Jun 7, 2022 at 1:08 PM Ronelle Landy wrote: >> >>> Hello All, >>> >>> We have a consistent failure on multiple jobs running on >>> check/gate/integration lines on master and wallaby changes - which shows up >>> as a deployment error: >>> >>> FATAL | Enable os_enable_vtpm SELinux boolean for vTPM | >>> standalone | error={"changed": false, "msg": "SELinux boolean >>> os_enable_vtpm does not exist."} >>> >>> Details of the failure are in: >>> https://bugs.launchpad.net/tripleo/+bug/1977873 >>> >>> We are investigating a fix/workaround. >>> >>> Please hold rechecks - we will post out again with updates. >>> >>> Thank you! >>> >>> >> A fix is in progress - thank you Rafael and Lon: >> >> https://bugs.launchpad.net/tripleo/+bug/1977873/comments/7 >> https://bugs.launchpad.net/tripleo/+bug/1977873/comments/8 >> >> > Updated linselinux versions are available in centos-9 production composes: > > libselinux-3.4-2.el9.i686.rpm 2022-06-07 12:12 93K libselinux-3.4-2.el9.x86_64.rpm 2022-06-07 12:12 86K > > > > https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/BaseOS/x86_64/os/Packages/?C=M;O=A > > Waiting on mirrors to get this update before closing this out. > https://review.opendev.org/c/openstack/tripleo-quickstart/+/845229 merged as a workaround to unblcok most check/gate jobs on Master and Wallaby c9. Overcloud build image jobs are still failing at the moment - patches to address this are in progress -------------- next part -------------- An HTML attachment was scrubbed... URL: From dms at danplanet.com Thu Jun 9 13:30:08 2022 From: dms at danplanet.com (Dan Smith) Date: Thu, 09 Jun 2022 06:30:08 -0700 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: (Julia Kreger's message of "Thu, 9 Jun 2022 01:55:27 -0700") References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> <913612422bb6844e05dfee31dc3caab60f71200a.camel@redhat.com> <181443bdf1f.12767362256831.7510285921554510482@ghanshyammann.com> Message-ID: > So, one thought. Ironic views system scope as *critical* for our usage > based upon the consensus we built before the direction change, because > the system fundamentally is the owner/manager of $things. We can and > likely should extend that out to project admin (granted, I suspect at > least one ironic admin will reply with a strong -1 to such a change... > :\. ) given the direction change. We also have had some operators jump > on it, but... again, entirely different models of usage/interaction > given the base state. If system scope were to suddenly disappear or be > completely redefined, it would be a hard break for us at this point. I don't think system scope would (or could) disappear at this point, so I don't think there's much to worry about. I think it's totally reasonable to say that there are certain things that a user would never interact with directly, which are entirely system-scoped. This might be things like ironic and maybe even placement. You could also make the argument that a good chunk of keystone's API is system-only. If people are already using ironic with scopes turned on, it proves the point that it's isolated enough that it doesn't suffer from all the other problems that caused the direction change. --Dan From bdobreli at redhat.com Thu Jun 9 13:53:54 2022 From: bdobreli at redhat.com (Bogdan Dobrelya) Date: Thu, 9 Jun 2022 15:53:54 +0200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: <29949013-7b00-daa3-f10a-be25a596c5b2@redhat.com> +1 On 6/8/22 21:35, James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered > core on puppet-tripleo, and I'm proposing we expand that to all of > TripleO, and likewise for Brendan. > > They both have been contributing and reviewing for a little while now, > and I feel it's time to add them to the team. Please +1 or let me know > any concerns? before Friday June 15th. Thanks! > > -- > -- James Slagle > -- -- Best regards, Bogdan Dobrelya, Irc #bogdando From Russell.Stather at ignitiongroup.co.za Thu Jun 9 14:24:01 2022 From: Russell.Stather at ignitiongroup.co.za (Russell Stather) Date: Thu, 9 Jun 2022 14:24:01 +0000 Subject: Error creating octavia amphora In-Reply-To: References: Message-ID: Hi I've built new security groups to make sure they are in the correct project. I can start a new server manually using these security groups. The error is coming from the nova compute node. Same error can't find security group. I am creating the load balancer and specifying the project on the command line even. openstack loadbalancer create --name lb25 --vip-subnet-id admin_subnet1 --project 5e168b652a374a02aff855a5e250b7f8 Kind of running out of ideas. ________________________________ From: Brendan Shephard Sent: 09 June 2022 13:16 To: Russell Stather Cc: openstack-discuss at lists.openstack.org Subject: Re: Error creating octavia amphora Hey Russell, Are you able to share the outputs from: $ openstack server show 524ae27b-1542-4c2d-9118-138d9e7f3770 -c id -c project_id -c security_groups -c status -f yaml And: $ openstack security group show0b683c75-d900-4e45-acb2-8bc321580666 -c id -c project_id -f yaml I agree with James, my assumption would be that we will find they aren't in the same project, so Nova can't use that security group for the Amphorae. I'm not familiar with charmed openstack though, but it does look like James is from Canonical and might be able to advise on the specifics. All the best, Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat [https://static.redhat.com/libs/redhat/brand-assets/latest/corp/logo.png] [https://static.redhat.com/libs/redhat/brand-assets/latest/events/red-hat-summit.png] On Thu, Jun 9, 2022 at 8:48 PM Russell Stather > wrote: Hi I am seeing the below error when creating a load balancer. The security group does exist, and it is the correct group (tagged with charm-octavia) What am I missing here to resolve this? 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server octavia.common.exceptions.ComputeBuildException: Failed to build compute instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', 'message': 'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in build_instances\n scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server -------------- next part -------------- An HTML attachment was scrubbed... URL: From kdhall at binghamton.edu Thu Jun 9 14:58:41 2022 From: kdhall at binghamton.edu (Dave Hall) Date: Thu, 9 Jun 2022 10:58:41 -0400 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: Message-ID: Hello, My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am sensitive to the effort that has been put into both of these projects, what I really need right now is the most fool-proof way to deploy and manage a small production cluster for academic instructional use. (BTW, I do understand that there are other differences between Kolla and regular openstack.) I have been working for a couple months with OpenStack-Ansible to deploy a small (3-node) Xena test cluster on VirtualBox VMs in preparation for a larger deployment on real hardware - 6 to 10 nodes. My VirtualBox deployment has been: Debian 11 deployment, Debian 10 infra, compute, and storage nodes It has been slow going, at least partially due to some issues and limitations with VirtualBox (and Vagrant early on). However, deploying a test cluster on VMs still seems preferable to just diving into deployment on real hardware and going through multiple scrubs/reinstalls. I've recently seen more posts in the list about Kolla-Ansible. So, as a 'beginner', should I shift and look at Kolla-Ansible, or should I stay the course and continue with Openstack-Ansible? What are the pros and cons of each? For that matter, is there some other deployment mechanism that would be more fool-proof for a first-timer? Although I'm more familiar with Ansible than the other tools (Chef, Puppet, etc.) I'm most interested in how to get a cluster up and running regardless of the underlying tools. Thanks. -Dave -- Dave Hall Binghamton University kdhall at binghamton.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Thu Jun 9 15:00:57 2022 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Thu, 9 Jun 2022 08:00:57 -0700 Subject: [all][operator][policy] Operator feedback on 'Consistent and Secure RBAC" (new design for RBAC) In-Reply-To: References: <1813f5800a2.dea83620696760.7937991544769180537@ghanshyammann.com> <913612422bb6844e05dfee31dc3caab60f71200a.camel@redhat.com> <181443bdf1f.12767362256831.7510285921554510482@ghanshyammann.com> Message-ID: On Thu, Jun 9, 2022 at 6:30 AM Dan Smith wrote: > > > So, one thought. Ironic views system scope as *critical* for our usage > > based upon the consensus we built before the direction change, because > > the system fundamentally is the owner/manager of $things. We can and > > likely should extend that out to project admin (granted, I suspect at > > least one ironic admin will reply with a strong -1 to such a change... > > :\. ) given the direction change. We also have had some operators jump > > on it, but... again, entirely different models of usage/interaction > > given the base state. If system scope were to suddenly disappear or be > > completely redefined, it would be a hard break for us at this point. > > I don't think system scope would (or could) disappear at this point, so > I don't think there's much to worry about. Whew! That is a weight off my shoulders! > I think it's totally > reasonable to say that there are certain things that a user would never > interact with directly, which are entirely system-scoped. This might be > things like ironic and maybe even placement. You could also make the > argument that a good chunk of keystone's API is system-only. Definitely. > If people > are already using ironic with scopes turned on, it proves the point that > it's isolated enough that it doesn't suffer from all the other problems > that caused the direction change. > > --Dan Really, starting out as an admin-only service and then later adding multi-tenancy support which only got turned on by default with SRBAC meant we never had to deal with the can of worms that were initially encountered with system scope. From jonathan.rosser at rd.bbc.co.uk Thu Jun 9 15:34:40 2022 From: jonathan.rosser at rd.bbc.co.uk (Jonathan Rosser) Date: Thu, 9 Jun 2022 16:34:40 +0100 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: Message-ID: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> Hi Dave, I have been hesitating to reply to your mailing list post because it doesn't feel right to pitch two community tools against each other here on the mailing list - so i won't do that here. I would say that the deployment tool is a means to an end. So you should look at the technology choices, upgrade paths, support for "day 2 operations", how bugs get addressed, documentation, operator experience etc etc. Only you can decide which is appropriate for the constraints and requirements of your deployment. My reply will obviously be biassed, as I am a big contributor to openstack-ansible. My observation is that the operators that gain the most out of any of these tools are the ones who engage with the community around those tools, primarily in the case of openstack-ansible that would be through our IRC channel, weekly meetings and bug reports on Launchpad. You will gain insight and be able to leverage the knowledge of other operators who in some cases have literally written the book on various aspects of OpenStack. Trying to fight though every decision or problem on your own is the worst way to approach using any of these community driven tools. If you instead want a more "shrink wrap" approach to an installer, or more formal support, then it would be wise to look at the product oriented offerings from the large vendors. Both openstack-ansible and kolla-ansible will expect you to make a good number of decisions about the specifics of your deployment, for example storage, networking and security concerns. Both would also expect you to gain sufficient knowledge about how OpenStack itself works to be able to make good use of the customization opportunities that both provide. This is really the unique selling point of the community tooling, you get a very high degree of customization potential but that can come at the cost of some complexity. As you are already using openstack-ansible I would suggest that you continue, but as I've already said I have an existing interest here and I really don't want to start a tooling debate. Join us in IRC in #openstack-ansible and discuss any pain points you have. This way we can hopefully help you out, or address specific issues in the code - you may have discovered legitimate bugs or a use case that is not straightforward to fulfill. This is how all of the community tools get improved and evolved over time. On one specific point I would recommend that you move entirely to Debian 11 as Xena will be the last release that openstack-ansible supports Buster. I'm not sure there is a fool-proof installer really. Installing the code is one thing, being effective though upgrades and applying bugfixes to a production environment is a different and a more important concern in the long term. Both openstack-ansible and kolla-ansible offer "all-in-one" deployments which are intended as "should-just-work" demonstrators of how things fit together and for lightweight testing. Scaling those out to larger deployments is where the real work is, and neither tool sets out to be particularly prescriptive about some parts of how you build your environment. Hopefully this is helpful, Jonathan. On 09/06/2022 15:58, Dave Hall wrote: > Hello, > > My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am > sensitive to the effort that has been put into both of these projects, > what I really need right now is the most fool-proof way to deploy and > manage a small production cluster for academic instructional use. > > (BTW, I do understand that there are other differences between Kolla > and regular openstack.) > > I have been working for a couple months with OpenStack-Ansible?to > deploy a small (3-node) Xena test cluster on VirtualBox VMs in > preparation for a larger deployment on real hardware - 6 to 10 nodes.? > My VirtualBox deployment has been: > > Debian 11 deployment, Debian 10 infra, compute, and storage nodes > > > It has been slow going, at least partially due to some issues and > limitations with VirtualBox (and Vagrant early on).? However, > deploying a test cluster on VMs still seems preferable to just > diving?into deployment on real hardware and going through multiple > scrubs/reinstalls. > > I've?recently seen?more posts in the list about Kolla-Ansible. So, as > a 'beginner', should I shift and look at Kolla-Ansible, or should I > stay the course and continue with Openstack-Ansible? What are the pros > and cons of each? > > For that matter, is there some other deployment mechanism that would > be more fool-proof for a first-timer?? Although I'm more familiar with > Ansible than the other tools (Chef, Puppet, etc.) I'm most interested > in how to get a cluster up and running regardless of the underlying tools. > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.rohmann at inovex.de Thu Jun 9 16:18:00 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Thu, 9 Jun 2022 18:18:00 +0200 Subject: [rbac][nova][cinder] Canned roles for service users / inter-service communication (e.g. event submission) In-Reply-To: <26833ce2-e486-d953-b7ea-999d94862631@inovex.de> References: <26833ce2-e486-d953-b7ea-999d94862631@inovex.de> Message-ID: <6c7210b0-fad6-e31e-ecf5-3a977079e33a@inovex.de> On 09/06/2022 12:39, Christian Rohmann wrote: > I just noticed that Christian Berendt does a forum talk on > "Deprivilization of the internal service accounts" TODAY at 2:40pm - > 3:10pm at A05 on apparently that exact question :-) The notes on the discussion can be found here: https://etherpad.opendev.org/p/deprivilization-of-service-accounts Regards Christian From kdhall at binghamton.edu Thu Jun 9 18:03:47 2022 From: kdhall at binghamton.edu (Dave Hall) Date: Thu, 9 Jun 2022 14:03:47 -0400 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> References: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> Message-ID: Jonathan, Thank you for replying. I want to stress that I also understand that both Openstack-Ansible and Kolla-Ansible are serious projects and that I would not try to play one against the other. I do perceive that although both use Ansible, both also take different approaches in their organization and configuration. My query was whether one might be easier for beginners to grok whereas the other might be the tool of choice for the enlightened ones. Or perhaps it's just that Kolla uses Docker instead of LXC. It's just a lot to learn either way - this is big software. Regarding Openstack-Ansible, I will check out the IRC, although last I knew the University was blocking all IRC. I will also ask more questions here. Thanks. -Dave -- Dave Hall Binghamton University kdhall at binghamton.edu On Thu, Jun 9, 2022 at 11:35 AM Jonathan Rosser < jonathan.rosser at rd.bbc.co.uk> wrote: > Hi Dave, > > I have been hesitating to reply to your mailing list post because it > doesn't feel right to pitch two community tools against each other here on > the mailing list - so i won't do that here. > > I would say that the deployment tool is a means to an end. So you should > look at the technology choices, upgrade paths, support for "day 2 > operations", how bugs get addressed, documentation, operator experience etc > etc. Only you can decide which is appropriate for the constraints and > requirements of your deployment. > My reply will obviously be biassed, as I am a big contributor to > openstack-ansible. My observation is that the operators that gain the most > out of any of these tools are the ones who engage with the community around > those tools, primarily in the case of openstack-ansible that would be > through our IRC channel, weekly meetings and bug reports on Launchpad. You > will gain insight and be able to leverage the knowledge of other operators > who in some cases have literally written the book on various aspects of > OpenStack. Trying to fight though every decision or problem on your own is > the worst way to approach using any of these community driven tools. > > If you instead want a more "shrink wrap" approach to an installer, or more > formal support, then it would be wise to look at the product oriented > offerings from the large vendors. > > Both openstack-ansible and kolla-ansible will expect you to make a good > number of decisions about the specifics of your deployment, for example > storage, networking and security concerns. Both would also expect you to > gain sufficient knowledge about how OpenStack itself works to be able to > make good use of the customization opportunities that both provide. This is > really the unique selling point of the community tooling, you get a very > high degree of customization potential but that can come at the cost of > some complexity. > > As you are already using openstack-ansible I would suggest that you > continue, but as I've already said I have an existing interest here and I > really don't want to start a tooling debate. Join us in IRC in > #openstack-ansible and discuss any pain points you have. This way we can > hopefully help you out, or address specific issues in the code - you may > have discovered legitimate bugs or a use case that is not straightforward > to fulfill. This is how all of the community tools get improved and evolved > over time. > > On one specific point I would recommend that you move entirely to Debian > 11 as Xena will be the last release that openstack-ansible supports Buster. > > I'm not sure there is a fool-proof installer really. Installing the code > is one thing, being effective though upgrades and applying bugfixes to a > production environment is a different and a more important concern in the > long term. Both openstack-ansible and kolla-ansible offer "all-in-one" > deployments which are intended as "should-just-work" demonstrators of how > things fit together and for lightweight testing. Scaling those out to > larger deployments is where the real work is, and neither tool sets out to > be particularly prescriptive about some parts of how you build your > environment. > > Hopefully this is helpful, > Jonathan. > > On 09/06/2022 15:58, Dave Hall wrote: > > Hello, > > My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am > sensitive to the effort that has been put into both of these projects, what > I really need right now is the most fool-proof way to deploy and manage a > small production cluster for academic instructional use. > > (BTW, I do understand that there are other differences between Kolla and > regular openstack.) > > I have been working for a couple months with OpenStack-Ansible to deploy > a small (3-node) Xena test cluster on VirtualBox VMs in preparation for a > larger deployment on real hardware - 6 to 10 nodes. My VirtualBox > deployment has been: > > Debian 11 deployment, Debian 10 infra, compute, and storage nodes > > > It has been slow going, at least partially due to some issues and > limitations with VirtualBox (and Vagrant early on). However, deploying a > test cluster on VMs still seems preferable to just diving into deployment > on real hardware and going through multiple scrubs/reinstalls. > > I've recently seen more posts in the list about Kolla-Ansible. So, as a > 'beginner', should I shift and look at Kolla-Ansible, or should I stay > the course and continue with Openstack-Ansible? What are the pros and > cons of each? > > For that matter, is there some other deployment mechanism that would be > more fool-proof for a first-timer? Although I'm more familiar with Ansible > than the other tools (Chef, Puppet, etc.) I'm most interested in how to get > a cluster up and running regardless of the underlying tools. > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kdhall at binghamton.edu Thu Jun 9 18:19:11 2022 From: kdhall at binghamton.edu (Dave Hall) Date: Thu, 9 Jun 2022 14:19:11 -0400 Subject: [nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox Message-ID: Hello, I'm trying to run a test/learning cluster on VirtualBox VMs deployed with Openstack-Ansible prior to deploying on real hardware. The playbooks succeed, and the basic functional checks all look good. However, I am unable to deploy an instance due to 'No Valid Host Found'. I have seen that this might be because VirtualBox does not support KVM. But I am also perplexed - I am able to deploy instances on an AIO running in VirtualBox on the same physical host. If there is an easy fix for this please point me in the right direction. If I need to destroy and re-clone my VMs I can try that as well. In an effort to learn, though, I'd really like to learn how to adjust the current deployment. Thanks. -Dave -- Dave Hall Binghamton University kdhall at binghamton.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Thu Jun 9 19:18:01 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Thu, 9 Jun 2022 21:18:01 +0200 Subject: [nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox In-Reply-To: References: Message-ID: Hey, One question - have you set nova_virt_type to qemu in overrides? As otherwise you indeed need to have nested virtualization enabled. ??, 9 ???. 2022 ?., 20:21 Dave Hall : > Hello, > > I'm trying to run a test/learning cluster on VirtualBox VMs deployed with > Openstack-Ansible prior to deploying on real hardware. > > The playbooks succeed, and the basic functional checks all look good. > However, I am unable to deploy an instance due to 'No Valid Host Found'. I > have seen that this might be because VirtualBox does not support KVM. But > I am also perplexed - I am able to deploy instances on an AIO running in > VirtualBox on the same physical host. > > If there is an easy fix for this please point me in the right direction. > If I need to destroy and re-clone my VMs I can try that as well. In an > effort to learn, though, I'd really like to learn how to adjust the current > deployment. > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnsomor at gmail.com Thu Jun 9 20:49:43 2022 From: johnsomor at gmail.com (Michael Johnson) Date: Thu, 9 Jun 2022 13:49:43 -0700 Subject: Error creating octavia amphora In-Reply-To: References: Message-ID: This isn't an issue with how you are creating the load balancer. It's a nova error booting a VM, and most likely a configuration issue in the deployment. It sounds like it is a charms bug. At boot time, we only plug the lb-mgmt-network in the amphora instance. All of the required settings for this are in the octavia.conf. First thing to check is the security group Octavia is configured to used: [controller_worker] amp_secgroup_list For each security group ID in that list, do an "openstack security group show " and note the project ID that owns the group. Then, also in the octavia.conf, check the project ID used to create the amphora instances in nova: [service_auth] project_id If project_id is not specified, then look up the project_name with "openstack project show ". All of these project IDs must match. You can also lookup the project IDs of the amphora VM "openstack server show" while it's attempting to boot and compare that to the security group project ID, just to make sure the current running configuration is also correct (as mentioned in a previous email). I haven't seen this error before, but I also don't use charms to deploy OpenStack. Michael On Thu, Jun 9, 2022 at 7:37 AM Russell Stather < Russell.Stather at ignitiongroup.co.za> wrote: > Hi > > I've built new security groups to make sure they are in the correct > project. I can start a new server manually using these security groups. > > The error is coming from the nova compute node. Same error can't find > security group. > > I am creating the load balancer and specifying the project on the command > line even. > > openstack loadbalancer create --name lb25 --vip-subnet-id admin_subnet1 > --project 5e168b652a374a02aff855a5e250b7f8 > > Kind of running out of ideas. > > > ------------------------------ > *From:* Brendan Shephard > *Sent:* 09 June 2022 13:16 > *To:* Russell Stather > *Cc:* openstack-discuss at lists.openstack.org < > openstack-discuss at lists.openstack.org> > *Subject:* Re: Error creating octavia amphora > > Hey Russell, > > Are you able to share the outputs from: > $ openstack server show 524ae27b-1542-4c2d-9118-138d9e7f3770 -c id -c > project_id -c security_groups -c status -f yaml > > And: > $ openstack security group show0b683c75-d900-4e45-acb2-8bc321580666 -c id > -c project_id -f yaml > > I agree with James, my assumption would be that we will find they aren't > in the same project, so Nova can't use that security group for the > Amphorae. I'm not familiar with charmed openstack though, but it does look > like James is from Canonical and might be able to advise on the specifics. > > All the best, > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Thu, Jun 9, 2022 at 8:48 PM Russell Stather < > Russell.Stather at ignitiongroup.co.za> wrote: > > Hi > > I am seeing the below error when creating a load balancer. The security > group does exist, and it is the correct group (tagged with charm-octavia) > > What am I missing here to resolve this? > > 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server > octavia.common.exceptions.ComputeBuildException: Failed to build compute > instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', > 'message': 'Exceeded maximum number of retries. Exceeded max scheduling > attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last > exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', > 'details': 'Traceback (most recent call last):\n File > "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in > build_instances\n scheduler_utils.populate_retry(\n File > "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in > populate_retry\n raise > exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: > Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for > instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security > group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} > 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihalis68 at gmail.com Thu Jun 9 21:32:24 2022 From: mihalis68 at gmail.com (Chris Morgan) Date: Thu, 9 Jun 2022 23:32:24 +0200 Subject: [ops] details for Ops Meetup, Berlin, Tomorrow, Friday 2022-6-10 Message-ID: A brief twitter thread here https://twitter.com/osopsmeetup/status/1535004027032940544?s=20&t=AgLzV6QvgMUFiOm0lG6Agg has some reminders about tomorrow's event. Looking forward to seeing everyone. Chris -- Chris Morgan -------------- next part -------------- An HTML attachment was scrubbed... URL: From laurentfdumont at gmail.com Fri Jun 10 00:02:26 2022 From: laurentfdumont at gmail.com (Laurent Dumont) Date: Thu, 9 Jun 2022 20:02:26 -0400 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> Message-ID: For the sake of figuring out your options, I would also look at what features do you actually require at the Openstack level. I would start with the following questions to identify gaps or advantages between the two tools. - What Openstack projects do you require beyond the basic services? - Manilla, Trove, Skydive, Octavia, Ceilometer? - What network topologies are you looking to support? - OVS, OVN, SRIOV? - What support model are you looking for? - Using community deployers means you wear many different hats. Don't get me wrong, support through IRC and mailing lists is great but it's a community effort. If this breaks at 2AM, you will be mostly on your own. - Canonical can offer a Juju based deployment for Openstack with commercial support. - RedHat offers RHOSP that is based around tripleo with commercial support. - Are you expecting to configure CEPH through your deployer? - Are there any specific tech you are already familiar with in either tool? I am actually working on some deep dive into kolla-ansible and ansible-openstack for $work but I am a bit early in that process. In the end, like Jonathan mentioned, they both are a means to an end. That said, it is fair to say that having two different tools that essentially serve the same purpose is a strange position to be in. But I guess that comes with the flexibility of opensource! On Thu, Jun 9, 2022 at 2:07 PM Dave Hall wrote: > Jonathan, > > Thank you for replying. I want to stress that I also understand that both > Openstack-Ansible and Kolla-Ansible are serious projects and that I > would not try to play one against the other. I do perceive that although > both use Ansible, both also take different approaches in their organization > and configuration. > > My query was whether one might be easier for beginners to grok whereas the > other might be the tool of choice for the enlightened ones. Or perhaps > it's just that Kolla uses Docker instead of LXC. > > It's just a lot to learn either way - this is big software. > > Regarding Openstack-Ansible, I will check out the IRC, although last I > knew the University was blocking all IRC. > > I will also ask more questions here. > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > On Thu, Jun 9, 2022 at 11:35 AM Jonathan Rosser < > jonathan.rosser at rd.bbc.co.uk> wrote: > >> Hi Dave, >> >> I have been hesitating to reply to your mailing list post because it >> doesn't feel right to pitch two community tools against each other here on >> the mailing list - so i won't do that here. >> >> I would say that the deployment tool is a means to an end. So you should >> look at the technology choices, upgrade paths, support for "day 2 >> operations", how bugs get addressed, documentation, operator experience etc >> etc. Only you can decide which is appropriate for the constraints and >> requirements of your deployment. >> My reply will obviously be biassed, as I am a big contributor to >> openstack-ansible. My observation is that the operators that gain the most >> out of any of these tools are the ones who engage with the community around >> those tools, primarily in the case of openstack-ansible that would be >> through our IRC channel, weekly meetings and bug reports on Launchpad. You >> will gain insight and be able to leverage the knowledge of other operators >> who in some cases have literally written the book on various aspects of >> OpenStack. Trying to fight though every decision or problem on your own is >> the worst way to approach using any of these community driven tools. >> >> If you instead want a more "shrink wrap" approach to an installer, or >> more formal support, then it would be wise to look at the product oriented >> offerings from the large vendors. >> >> Both openstack-ansible and kolla-ansible will expect you to make a good >> number of decisions about the specifics of your deployment, for example >> storage, networking and security concerns. Both would also expect you to >> gain sufficient knowledge about how OpenStack itself works to be able to >> make good use of the customization opportunities that both provide. This is >> really the unique selling point of the community tooling, you get a very >> high degree of customization potential but that can come at the cost of >> some complexity. >> >> As you are already using openstack-ansible I would suggest that you >> continue, but as I've already said I have an existing interest here and I >> really don't want to start a tooling debate. Join us in IRC in >> #openstack-ansible and discuss any pain points you have. This way we can >> hopefully help you out, or address specific issues in the code - you may >> have discovered legitimate bugs or a use case that is not straightforward >> to fulfill. This is how all of the community tools get improved and evolved >> over time. >> >> On one specific point I would recommend that you move entirely to Debian >> 11 as Xena will be the last release that openstack-ansible supports Buster. >> >> I'm not sure there is a fool-proof installer really. Installing the code >> is one thing, being effective though upgrades and applying bugfixes to a >> production environment is a different and a more important concern in the >> long term. Both openstack-ansible and kolla-ansible offer "all-in-one" >> deployments which are intended as "should-just-work" demonstrators of how >> things fit together and for lightweight testing. Scaling those out to >> larger deployments is where the real work is, and neither tool sets out to >> be particularly prescriptive about some parts of how you build your >> environment. >> >> Hopefully this is helpful, >> Jonathan. >> >> On 09/06/2022 15:58, Dave Hall wrote: >> >> Hello, >> >> My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am >> sensitive to the effort that has been put into both of these projects, what >> I really need right now is the most fool-proof way to deploy and manage a >> small production cluster for academic instructional use. >> >> (BTW, I do understand that there are other differences between Kolla and >> regular openstack.) >> >> I have been working for a couple months with OpenStack-Ansible to deploy >> a small (3-node) Xena test cluster on VirtualBox VMs in preparation for a >> larger deployment on real hardware - 6 to 10 nodes. My VirtualBox >> deployment has been: >> >> Debian 11 deployment, Debian 10 infra, compute, and storage nodes >> >> >> It has been slow going, at least partially due to some issues and >> limitations with VirtualBox (and Vagrant early on). However, deploying a >> test cluster on VMs still seems preferable to just diving into deployment >> on real hardware and going through multiple scrubs/reinstalls. >> >> I've recently seen more posts in the list about Kolla-Ansible. So, as a >> 'beginner', should I shift and look at Kolla-Ansible, or should I stay >> the course and continue with Openstack-Ansible? What are the pros and >> cons of each? >> >> For that matter, is there some other deployment mechanism that would be >> more fool-proof for a first-timer? Although I'm more familiar with Ansible >> than the other tools (Chef, Puppet, etc.) I'm most interested in how to get >> a cluster up and running regardless of the underlying tools. >> >> Thanks. >> >> -Dave >> >> -- >> Dave Hall >> Binghamton University >> kdhall at binghamton.edu >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Russell.Stather at ignitiongroup.co.za Fri Jun 10 06:01:26 2022 From: Russell.Stather at ignitiongroup.co.za (Russell Stather) Date: Fri, 10 Jun 2022 06:01:26 +0000 Subject: Error creating octavia amphora In-Reply-To: References: Message-ID: Hi all Thank you for the assistance. checking octavia.conf I found that this: [service_auth] auth_section = keystone_authtoken [keystone_authtoken] auth_uri = https://10.0.143.28:5000 auth_url = https://10.0.143.28:35357 auth_type = password project_domain_name = service_domain user_domain_name = service_domain project_name = services username = octavia password = FN9BztN8tYcWrtxbHYPR2myrpz5HgSJ4T7479ynkCNFyWPhbcfgdbfc5xSCxkPH6 memcached_servers = inet6:[::1]:11211 So I rebult the security groups and put them in the services project. All working fine now. Thanks so much! Russell ________________________________ From: Michael Johnson Sent: 09 June 2022 22:49 To: Russell Stather Cc: Brendan Shephard ; openstack-discuss at lists.openstack.org Subject: Re: Error creating octavia amphora This isn't an issue with how you are creating the load balancer. It's a nova error booting a VM, and most likely a configuration issue in the deployment. It sounds like it is a charms bug. At boot time, we only plug the lb-mgmt-network in the amphora instance. All of the required settings for this are in the octavia.conf. First thing to check is the security group Octavia is configured to used: [controller_worker] amp_secgroup_list For each security group ID in that list, do an "openstack security group show " and note the project ID that owns the group. Then, also in the octavia.conf, check the project ID used to create the amphora instances in nova: [service_auth] project_id If project_id is not specified, then look up the project_name with "openstack project show ". All of these project IDs must match. You can also lookup the project IDs of the amphora VM "openstack server show" while it's attempting to boot and compare that to the security group project ID, just to make sure the current running configuration is also correct (as mentioned in a previous email). I haven't seen this error before, but I also don't use charms to deploy OpenStack. Michael On Thu, Jun 9, 2022 at 7:37 AM Russell Stather > wrote: Hi I've built new security groups to make sure they are in the correct project. I can start a new server manually using these security groups. The error is coming from the nova compute node. Same error can't find security group. I am creating the load balancer and specifying the project on the command line even. openstack loadbalancer create --name lb25 --vip-subnet-id admin_subnet1 --project 5e168b652a374a02aff855a5e250b7f8 Kind of running out of ideas. ________________________________ From: Brendan Shephard > Sent: 09 June 2022 13:16 To: Russell Stather > Cc: openstack-discuss at lists.openstack.org > Subject: Re: Error creating octavia amphora Hey Russell, Are you able to share the outputs from: $ openstack server show 524ae27b-1542-4c2d-9118-138d9e7f3770 -c id -c project_id -c security_groups -c status -f yaml And: $ openstack security group show0b683c75-d900-4e45-acb2-8bc321580666 -c id -c project_id -f yaml I agree with James, my assumption would be that we will find they aren't in the same project, so Nova can't use that security group for the Amphorae. I'm not familiar with charmed openstack though, but it does look like James is from Canonical and might be able to advise on the specifics. All the best, Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat [X] [X] On Thu, Jun 9, 2022 at 8:48 PM Russell Stather > wrote: Hi I am seeing the below error when creating a load balancer. The security group does exist, and it is the correct group (tagged with charm-octavia) What am I missing here to resolve this? 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server octavia.common.exceptions.ComputeBuildException: Failed to build compute instance due to: {'code': 500, 'created': '2022-06-09T10:09:59Z', 'message': 'Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 654, in build_instances\n scheduler_utils.populate_retry(\n File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 989, in populate_retry\n raise exception.MaxRetriesExceeded(reason=msg)\nnova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 524ae27b-1542-4c2d-9118-138d9e7f3770. Last exception: Security group 0b683c75-d900-4e45-acb2-8bc321580666 not found.\n'} 2022-06-09 10:10:03.789 678859 ERROR oslo_messaging.rpc.server -------------- next part -------------- An HTML attachment was scrubbed... URL: From fpantano at redhat.com Fri Jun 10 06:57:28 2022 From: fpantano at redhat.com (Francesco Pantano) Date: Fri, 10 Jun 2022 08:57:28 +0200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: +1 On Wed, Jun 8, 2022 at 9:45 PM James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- > -- Francesco Pantano GPG KEY: F41BD75C -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Fri Jun 10 07:02:29 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 10 Jun 2022 07:02:29 +0000 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> Message-ID: <20220610070009.3syvxwzeuhos2odn@yuggoth.org> On 2022-06-09 14:03:47 -0400 (-0400), Dave Hall wrote: [...] > I will check out the IRC, although last I knew the University was > blocking all IRC. [...] If you're able to reach Matrix, you can participate quite effectively in OFTC's IRC channels through their Matrix bridge. A concise writeup can be found here: https://blog.christophersmart.com/2022/03/21/joining-a-bridged-irc-network-on-element-matrix/ -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Fri Jun 10 07:06:35 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 10 Jun 2022 07:06:35 +0000 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> Message-ID: <20220610070635.cahozvgv6klxuoib@yuggoth.org> On 2022-06-09 20:02:26 -0400 (-0400), Laurent Dumont wrote: [...] > That said, it is fair to say that having two different tools that > essentially serve the same purpose is a strange position to be in. > But I guess that comes with the flexibility of opensource! [...] As I'm sure you can imagine, that's not how things started out, it's just where they ended up. In the long-ago, one was a container-focused deployment project while the other was an Ansible-focused deployment project. Over time, they each made convergent tooling choices, leading to the present situation. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From jonathan.rosser at rd.bbc.co.uk Fri Jun 10 07:09:06 2022 From: jonathan.rosser at rd.bbc.co.uk (Jonathan Rosser) Date: Fri, 10 Jun 2022 08:09:06 +0100 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> Message-ID: On 09/06/2022 19:03, Dave Hall wrote: > > > Regarding Openstack-Ansible, I will check out the IRC, although last I > knew the?University was blocking all IRC. > I see you managed to join the IRC channel yesterday. A lot of people will have been at the OpenInfra summit so things are quiet this week. Your best bet it to also try to overlap with the EU timezone. The weekly meeting is held at Tuesday at 15:00 UTC. Web clients such as IRCCloud can help overcome direct blocking of IRC and also can provide a persistent connection, plus there is a bridge to Matrix chat documented here https://docs.openstack.org/contributors/common/irc.html#irc-chatting-with-matrix, although unfortunately that doc does not give a specific example for joining an openstack related channel. Jonathan. From mark at stackhpc.com Fri Jun 10 08:53:30 2022 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 10 Jun 2022 09:53:30 +0100 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: Message-ID: Hi Dave, I would suggest getting a minimal deployment up and running with each environment, before making your choice. For Kolla Ansible, you can follow the quickstart documentation to get a single node deployment up and running: https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html If you also need bare metal provisioning capabilities for your control & compute nodes, as well as more advanced host configuration, you might consider Kayobe, which builds on top of Kolla Ansible. Here is a guide to building up the configuration for a single node deployment: https://docs.openstack.org/kayobe/latest/configuration/scenarios/all-in-one/index.html Regards, Mark On Thu, 9 Jun 2022 at 16:05, Dave Hall wrote: > Hello, > > My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am > sensitive to the effort that has been put into both of these projects, what > I really need right now is the most fool-proof way to deploy and manage a > small production cluster for academic instructional use. > > (BTW, I do understand that there are other differences between Kolla and > regular openstack.) > > I have been working for a couple months with OpenStack-Ansible to deploy > a small (3-node) Xena test cluster on VirtualBox VMs in preparation for a > larger deployment on real hardware - 6 to 10 nodes. My VirtualBox > deployment has been: > > Debian 11 deployment, Debian 10 infra, compute, and storage nodes > > > It has been slow going, at least partially due to some issues and > limitations with VirtualBox (and Vagrant early on). However, deploying a > test cluster on VMs still seems preferable to just diving into deployment > on real hardware and going through multiple scrubs/reinstalls. > > I've recently seen more posts in the list about Kolla-Ansible. So, as a > 'beginner', should I shift and look at Kolla-Ansible, or should I stay > the course and continue with Openstack-Ansible? What are the pros and > cons of each? > > For that matter, is there some other deployment mechanism that would be > more fool-proof for a first-timer? Although I'm more familiar with Ansible > than the other tools (Chef, Puppet, etc.) I'm most interested in how to get > a cluster up and running regardless of the underlying tools. > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Fri Jun 10 08:54:30 2022 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 10 Jun 2022 09:54:30 +0100 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> References: <8b0c70a6-53d4-f314-c6d9-8de65a11826d@rd.bbc.co.uk> Message-ID: This has been discussed before, but it would be nice if we as a community had an easier way to answer this question. I know many of us feel conflicted about aggressively promoting our own projects over alternatives, but also want to give them a fair chance. Maybe each project needs a "why pick me" page that could be linked to in a standard response. Mark On Thu, 9 Jun 2022 at 16:40, Jonathan Rosser wrote: > Hi Dave, > > I have been hesitating to reply to your mailing list post because it > doesn't feel right to pitch two community tools against each other here on > the mailing list - so i won't do that here. > > I would say that the deployment tool is a means to an end. So you should > look at the technology choices, upgrade paths, support for "day 2 > operations", how bugs get addressed, documentation, operator experience etc > etc. Only you can decide which is appropriate for the constraints and > requirements of your deployment. > My reply will obviously be biassed, as I am a big contributor to > openstack-ansible. My observation is that the operators that gain the most > out of any of these tools are the ones who engage with the community around > those tools, primarily in the case of openstack-ansible that would be > through our IRC channel, weekly meetings and bug reports on Launchpad. You > will gain insight and be able to leverage the knowledge of other operators > who in some cases have literally written the book on various aspects of > OpenStack. Trying to fight though every decision or problem on your own is > the worst way to approach using any of these community driven tools. > > If you instead want a more "shrink wrap" approach to an installer, or more > formal support, then it would be wise to look at the product oriented > offerings from the large vendors. > > Both openstack-ansible and kolla-ansible will expect you to make a good > number of decisions about the specifics of your deployment, for example > storage, networking and security concerns. Both would also expect you to > gain sufficient knowledge about how OpenStack itself works to be able to > make good use of the customization opportunities that both provide. This is > really the unique selling point of the community tooling, you get a very > high degree of customization potential but that can come at the cost of > some complexity. > > As you are already using openstack-ansible I would suggest that you > continue, but as I've already said I have an existing interest here and I > really don't want to start a tooling debate. Join us in IRC in > #openstack-ansible and discuss any pain points you have. This way we can > hopefully help you out, or address specific issues in the code - you may > have discovered legitimate bugs or a use case that is not straightforward > to fulfill. This is how all of the community tools get improved and evolved > over time. > > On one specific point I would recommend that you move entirely to Debian > 11 as Xena will be the last release that openstack-ansible supports Buster. > > I'm not sure there is a fool-proof installer really. Installing the code > is one thing, being effective though upgrades and applying bugfixes to a > production environment is a different and a more important concern in the > long term. Both openstack-ansible and kolla-ansible offer "all-in-one" > deployments which are intended as "should-just-work" demonstrators of how > things fit together and for lightweight testing. Scaling those out to > larger deployments is where the real work is, and neither tool sets out to > be particularly prescriptive about some parts of how you build your > environment. > > Hopefully this is helpful, > Jonathan. > > On 09/06/2022 15:58, Dave Hall wrote: > > Hello, > > My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am > sensitive to the effort that has been put into both of these projects, what > I really need right now is the most fool-proof way to deploy and manage a > small production cluster for academic instructional use. > > (BTW, I do understand that there are other differences between Kolla and > regular openstack.) > > I have been working for a couple months with OpenStack-Ansible to deploy > a small (3-node) Xena test cluster on VirtualBox VMs in preparation for a > larger deployment on real hardware - 6 to 10 nodes. My VirtualBox > deployment has been: > > Debian 11 deployment, Debian 10 infra, compute, and storage nodes > > > It has been slow going, at least partially due to some issues and > limitations with VirtualBox (and Vagrant early on). However, deploying a > test cluster on VMs still seems preferable to just diving into deployment > on real hardware and going through multiple scrubs/reinstalls. > > I've recently seen more posts in the list about Kolla-Ansible. So, as a > 'beginner', should I shift and look at Kolla-Ansible, or should I stay > the course and continue with Openstack-Ansible? What are the pros and > cons of each? > > For that matter, is there some other deployment mechanism that would be > more fool-proof for a first-timer? Although I'm more familiar with Ansible > than the other tools (Chef, Puppet, etc.) I'm most interested in how to get > a cluster up and running regardless of the underlying tools. > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From james.page at canonical.com Fri Jun 10 09:15:02 2022 From: james.page at canonical.com (James Page) Date: Fri, 10 Jun 2022 10:15:02 +0100 Subject: Error creating octavia amphora In-Reply-To: References: Message-ID: On Fri, Jun 10, 2022 at 7:06 AM Russell Stather < Russell.Stather at ignitiongroup.co.za> wrote: > Hi all > > Thank you for the assistance. > > checking octavia.conf I found that this: > > [service_auth] > > auth_section = keystone_authtoken > > > [keystone_authtoken] > > auth_uri = https://10.0.143.28:5000 > > auth_url = https://10.0.143.28:35357 > > auth_type = password > > project_domain_name = service_domain > > user_domain_name = service_domain > > project_name = services > > username = octavia > > password = FN9BztN8tYcWrtxbHYPR2myrpz5HgSJ4T7479ynkCNFyWPhbcfgdbfc5xSCxkPH6 > > memcached_servers = inet6:[::1]:11211 > > > So I rebult the security groups and put them in the services project. > > All working fine now. Thanks so much! > Glad to hear you have it working now. For future followers the octavia charm has a 'configure-resources' action that will either create all of the required resources (networking and security groups) in OpenStack for Octavia operation or discover them if they have been created by the user of the charm by hand - see the deployment guide doc on this topic: https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/app-octavia.html#resource-configuration Cheers James -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim+openstack.org at coote.org Fri Jun 10 10:52:19 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Fri, 10 Jun 2022 11:52:19 +0100 Subject: simple build fails to allocate network interface In-Reply-To: References: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> <0044A33F-0E87-433C-8130-77C66D79926A@coote.org> Message-ID: > On 9 Jun 2022, at 13:20, Rodolfo Alonso Hernandez wrote: > > Hello: > > This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too. > > Regards. > Hullo thanks for the response. I don?t know what to look for wrt ml2 driver - I?m using the default `stack.sh` installation script from the devstack repo. The neutron log has helped, I think. I found this useful as a starting point:https://bit.ly/3MG6jnR as it enables log extraction of events relating to the creation of a specific server (although that?s still quite large for a mailing list). I think that this may help with your question (but as I said, I don?t really understand the question). There may be a dhcp issue: I cannot ping either ipv4 or ipv6 dhcp addresses mentioned in the log from the vm. I?ve no idea where they come from, however. Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updated VIF entry in instance network info cache for port 5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc. {{(pid=20725) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3420}} Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f be4f] Updating instance_info_cache with network_info: [{"id": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "address": "fa:16:3e:02:7a:6a", "network": {"id": "2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63", "bridge": "br-int", "label": "private", "subnets": [{"cidr": "10.0.0.0/26", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}, {"cidr": "fd5b:4cf4:f7c1::/64", "dns": [], "gateway": {"address": "fd5b:4cf4:f7c1::1", "type": "gateway", "version": 6, "meta": {}}, "ips": [{"address": "fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a", "type": "fixed", "version": 6, "meta": {}, "floating_ips": []}], "routes": [], "version": 6, "meta": {"ipv6_address_mode": "slaac", "dhcp_server": "fd5b:4cf4:f7c1::1"}}], "meta": {"injected": false, "tenant_id": "6c0b876005f44086bd9bde5fef672dfe", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5ae4c0d6-2a", "ovs_interfaceid": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=20725) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jun 10 10:14:11 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:15:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:16:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, resource={'id': '5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc', 'name': '', 'network_id': '2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'tenant_id': '6c0b876005f44086bd9bde5fef672dfe', 'mac_address': 'fa:16:3e:02:7a:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'device_owner': 'compute:nova', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd', 'ip_address': '10.0.0.51'}, {'subnet_id': 'f915b88c-9988-4e1f-9060-a6295465699a', 'ip_address': 'fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': ['ed75ab0b-d9a4-4023-9204-93038729f6d3'], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'localhost.localdomain', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'tags': [], 'created_at': '2022-06-10T10:13:44Z', 'updated_at': '2022-06-10T10:13:46Z', 'revision_number': 3, 'project_id': '6c0b876005f44086bd9bde5fef672dfe'}, resource_type=ports, if_exists=True) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLSwitchPortCommand(lport=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, columns={'external_ids': {'neutron:port_name': '', 'neutron:device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'neutron:project_id': '6c0b876005f44086bd9bde5fef672dfe', 'neutron:cidrs': '10.0.0.51/26 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a/64', 'neutron:device_owner': 'compute:nova', 'neutron:network_name': 'neutron-2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'neutron:security_group_ids': 'ed75ab0b-d9a4-4023-9204-93038729f6d3', 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], 'options': {'requested-chassis': 'localhost.localdomain', 'mcast_flood_reports': 'true'}, 'enabled': True, 'port_security': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'dhcpv4_options': [UUID('01cae053-612b-4c91-81e8-39884ef035dd')], 'dhcpv6_options': [], 'type': '', 'addresses': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'ha_chassis_group': []}, if_exists=False) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} Jun 10 10:18:13 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} Jun 10 10:18:49 localhost.localdomain nova-compute[20725]: WARNING nova.compute.manager [None req-4a03d9e1-6053-4167-ab43-fba30072a79c demo admin] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Timeout waiting for ['network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc'] for instance with vm_state building and task_state spawning. Event states are: network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc: timed out after 300.00 seconds: eventlet.timeout.Timeout: 300 seconds Jun 08 15:24:48 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b> -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.arbet at ultimum.io Fri Jun 10 12:05:34 2022 From: michal.arbet at ultimum.io (Michal Arbet) Date: Fri, 10 Jun 2022 14:05:34 +0200 Subject: [cinder] Failed to delete volume In-Reply-To: References: Message-ID: Hi, Cinder is from victoria, backend is ceph, steps to reproduce ? Just delete volume. I think that it could be an error not in cinder but somehow a rabbitmq issue. :/ Michal Arbet Openstack Engineer Ultimum Technologies a.s. Na Po???? 1047/26, 11000 Praha 1 Czech Republic +420 604 228 897 michal.arbet at ultimum.io *https://ultimum.io * LinkedIn | Twitter | Facebook ?t 7. 6. 2022 v 19:27 odes?latel Sofia Enriquez napsal: > Hi Michal, > > I do not think we have such a bug on the Launchpad platform.[1] > Could you create a new bug report indicating which backend you are using, > which Cinder version you have, and if the error occurs after following some > steps? > > Thanks in advance, > Sofia > > [1] https://bugs.launchpad.net/cinder > > On Tue, Jun 7, 2022 at 4:28 AM Michal Arbet > wrote: > >> Hi, >> >> I would like to ask if someone saw issue when volume failed to delete, >> log below : >> This is happening from time to time. >> >> LOG : https://paste.opendev.org/show/b1jRCxuBFnnVzf64i9yV/ >> >> Thanks, >> >> Michal Arbet >> Openstack Engineer >> >> Ultimum Technologies a.s. >> Na Po???? 1047/26, 11000 Praha 1 >> Czech Republic >> >> +420 604 228 897 >> michal.arbet at ultimum.io >> *https://ultimum.io * >> >> LinkedIn | >> Twitter | Facebook >> >> > > > -- > > Sof?a Enriquez > > she/her > > Software Engineer > > Red Hat PnT > > IRC: @enriquetaso > @RedHat Red Hat > Red Hat > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Greg.Waines at windriver.com Fri Jun 10 12:30:23 2022 From: Greg.Waines at windriver.com (Waines, Greg) Date: Fri, 10 Jun 2022 12:30:23 +0000 Subject: [doc] question/enhancement-request on Sphinx theme used by openstack docs Message-ID: Hey ... I am on the technical steering committee for the openstack starlingx team and also provide primary technical review for our docs at https://docs.starlingx.io/ . docs.starlingx.io basically uses the openstack docs team's Sphinx theme for building our html docs ... see https://opendev.org/starlingx/docs . docs.starlingx.io is a very large doc suite. We are interested in improvements to the Table-of-Contents (TOC) navigation and presentation ... but we have limited Sphinx expertise. Specifically we are looking for the TOC support to have - more subsection header levels in TOC // such that user has full view of where he currently is in the doc hierarchy - highlighting to indicate currently selected section - separate scroll bar for TOC // ... so you don't lose sight of selected TOC selection when you scroll down on the main page which has large content. Have you ever considered such doc improvements? Did you have Sphinx expertise to do such improvements ? let us know, Greg. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.arbet at ultimum.io Fri Jun 10 12:40:25 2022 From: michal.arbet at ultimum.io (Michal Arbet) Date: Fri, 10 Jun 2022 14:40:25 +0200 Subject: [kolla-ansible][xena] Cell database HA/redundancy? In-Reply-To: References: Message-ID: Hi, I think my patchsets are in good shape and ready for merge, just need some reviewers finally :(. I am glad that there is this type of question, who knows, maybe this will speed up the process of merge :) You can still backport it to your downstream branches, we are using it in this way for several kolla-ansible versions and it works. Thanks, Michal Michal Arbet Openstack Engineer Ultimum Technologies a.s. Na Po???? 1047/26, 11000 Praha 1 Czech Republic +420 604 228 897 michal.arbet at ultimum.io *https://ultimum.io * LinkedIn | Twitter | Facebook st 8. 6. 2022 v 16:37 odes?latel Norrie, Andrew napsal: > Hi Mark, > > > > Thanks for the reply. > > > > I?ll look into the HAProxy ?custom config? as suggested and using > different ports for the cell databases. > > Maybe that can be worked in. > > > > We are expanding our OpenStack/kolla-ansible personnel/knowledge so will > look into the possibility for > > us to contribute as reviewers/testers as a start. > > > > Best regards .... Andrew > > > > *From:* Mark Goddard > *Sent:* Monday, June 6, 2022 2:22 AM > *To:* Norrie, Andrew > *Cc:* openstack-discuss at lists.openstack.org > *Subject:* Re: [kolla-ansible][xena] Cell database HA/redundancy? > > > > > > > > On Fri, 3 Jun 2022 at 14:10, Norrie, Andrew wrote: > > > > Hi, > > > > We are currently planning some large OpenStack deployments utilizing > kolla-ansible and > > I?m curious what folks are doing with the cell database HA/redundancy. > > > > With kolla-ansible (xena) it appears that a loadbalancer setup is only > allowed for the > > default database shard (shard 0)... reference: > https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb-guide.html > > > > > If we are setting up separate cell database shards with Galera, I?m > wondering if there is a convenient work-around or configuration for > implementation of HA for these cell databases. > > > > In the inventory group_vars directory you can specify the group variables > (for each cell database) as: > > > > nova_cell_database_address: > > nova_cell_database_group: > > > > but these aren?t virtual IP hosts/addresses (non default db shard). This > works perfectly fine with > > a single server cell database but not Galera. If that db server goes down > the cell instance information is lost. > > > > > > Hi Andrew, > > > > You are correct - only the first shard is load balanced. The sharding > feature was actually just the first patch in a series intended to support > proxysql. I believe this would add the functionality you are looking for. > In fact, the proposed test case for proxysql in our CI is a multi-cell > deployment. > > > > Here is the patch series: > > > > https://review.opendev.org/q/topic:proxysql > > > > > Unfortunately it has been stuck for a while, largely due to core reviewer > bandwidth. The author is Michael Arbet, who is often on IRC as kevko. I'd > suggest registering your interest in these patches via gerrit, and at one > of the weekly kolla IRC meetings (this week is cancelled due to the > summit). If you have time available, then testing and providing reviews > could help to get the patches moving. > > > > In the meantime, you can configure HAProxy to load balance additional > services, by placing an HAProxy config snippet in > /etc/kolla/config/haproxy/services.d/*.cfg. > > > > Regards, > > Mark > > > > Many thanks ... Andrew > ------------------------------ > > ?This e-mail and any accompanying attachments are confidential. The > information is intended solely for the use of the individual to whom it is > addressed. Any review, disclosure, copying, distribution, or use of the > email by others is strictly prohibited. If you are not the intended > recipient, you must not review, disclose, copy, distribute or use this > e-mail; please delete it from your system and notify the sender > immediately.? > > ------------------------------ > ?This e-mail and any accompanying attachments are confidential. The > information is intended solely for the use of the individual to whom it is > addressed. Any review, disclosure, copying, distribution, or use of the > email by others is strictly prohibited. If you are not the intended > recipient, you must not review, disclose, copy, distribute or use this > e-mail; please delete it from your system and notify the sender > immediately.? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.harrison at manchester.ac.uk Fri Jun 10 13:13:05 2022 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Fri, 10 Jun 2022 13:13:05 +0000 Subject: [nova] local LVM volume on compute hosts In-Reply-To: <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> References: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> Message-ID: <92824CD0-05FB-4941-B9C0-C27296B958F0@manchester.ac.uk> > On 2022-06 -08, at 08:26, Paul Harrison wrote: > > > >> On 2022-06 -07, at 13:25, Sean Mooney > wrote: >> >> no there is noting else you need to configure but this option is not what you think it is. >> the images_type option contols what storage will be used for all non cinder storage. >> i.e. vms that are booted with out usign a boot volume. >> >> >> >> so your current config will make any non boot from volume nova instance use lvm storage to provision the vm root/swap/epmeral disks >> but will not prevent end users requesting cinder data voluems or boot volumes via the cli/api. if the opt in to cinder stoage that >> is what they will recive but if they use teh default storage provided by the flaovr then it will be local. > > thanks for the explanation - it is a shame that there is not a more direct way in config to force local storage - looks like https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools has never got enough votes for implementation. > > I have discovered the reason why my original setup was not working (it would fail if I tried to force in the GUI, by specifying ?no? to "create new volume?) - I think it was failing here https://opendev.org/openstack/nova/src/commit/d86916360858daa06164ebc0d012b78d19ae6497/nova/virt/libvirt/imagebackend.py#L722 as the volume group device does not appear when there are no logical volumes in the group (in Centos 8 stream at least). So I forced the creation of that device by adding a dummy logical volume. Anyway, the situation is now that I can create instances that will use compute node local LVM storage, if I start them from the GUI, but not from the command line. I had a look at what the GUI sends to the /api/nova/servers/ endpoint {"availability_zone":"nova","config_drive":false,"user_data":"","disk_config":"AUTO","instance_count":1,"name":"fromgui","scheduler_hints":{},"security_groups":["48648c10-ce91-4916-9347-d88fbdba9ce6"],"create_volume_default":true,"hide_create_volume":false,"source_id":"b2a3ca46-8b0b-4748-863f-6f3e11301872","flavor_id":"da28d141-3d05-4f0f-a188-229352ccf0a3","nics":[{"net-id":"5e7a171a-ceda-4051-abe6-0496e2e8e154","v4-fixed-ip":""}],"key_name":"cloud?} However, I have not been able to find the set of command line switches for "openstack server create" that achieve the same effect - if someone knows, I would be grateful. Thanks, Paul. p.s. I was not really able to match up what the GUI sends with the API spec either. https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2893 bytes Desc: not available URL: From vince.mulhollon at springcitysolutions.com Fri Jun 10 13:29:15 2022 From: vince.mulhollon at springcitysolutions.com (Vince Mulhollon) Date: Fri, 10 Jun 2022 08:29:15 -0500 Subject: [magnum] Docker Swarm errors on Yoga Ubuntu using Install Guide cut-n-paste Message-ID: I'm trying to use Magnum on a fresh Yoga install on Ubuntu to run some non-load balanced docker swarm workloads. Its a long story but the parts of the swarm workload are heterogeneous so a cluster would just get in the way. Everything else about the cluster is working fine, all is well, etc. I'm a small time VMware and OS admin with years of experience but absolute zero experience with Magnum. Also I have "some" experience with Docker Swarm clusters mostly created and assembled with Ansible. So I know quite well how to look at log files or configs in a general sense, but I have no idea what to expect from Magnum specifically as a "glue" technology. I've followed every detail of the "Magnum Installation Guide" for Yoga as seen at: https://docs.openstack.org/magnum/yoga/install/index.html When provisioning a docker swarm, I very literally cut and paste from: https://docs.openstack.org/magnum/yoga/install/launch-instance.html#provision-a-docker-swarm-cluster-and-create-a-container cluster creation fails every time with "Property allowed_cidrs not assigned" exactly as seen on this mailing list post from 2021-August-20 about ten months ago on a Wallaby install, they were also using Ubuntu. http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024273.html Advice at that point ranged from, try installing Octavia, which I was trying to avoid and I definitely do NOT want a load balancer in front of that particular workload, to Docker swarm is no longer tested for Magnum and it's in practice a K8S exclusive product. I played around for an afternoon and got nowhere WRT configuring the allowed cidrs for a k8s load balancer on a config that does not use a load balancer and does not use k8s. Heat works fine, its super fun, and one alternative is I could give up on Magnum and use some Heat templates, which I know how to do and seems simpler than getting Magnum to work at this point. Or I could brave the scary looking mostly-undocumented world of kuryr and zun, which I was attempting to avoid by using Magnum which is at least simple and easy to install. Or just keep spinning up manual images and letting ansible turn them into docker swarms for me. But, ideally, I'd like to use Magnum to spin up some Docker Swarms for personal curiosity reasons, if nothing else, it would be very convenient if it worked. So the point of this long ramble is, can anyone successfully running Docker Swarm on Yoga on Ubuntu using a very typical "Install Guide" installation provide a pointer or link or github or something that Google searches is not finding to get it to work? Also I hate to piggyback multiple questions but this paragraph also interests me, does anyone have any information and experience with this: https://docs.openstack.org/magnum/yoga/user/index.html#heat-stack-templates "Sample cluster driver To help developers in creating new COE drivers, a minimal cluster driver is provided as an example. The ?docker? cluster driver will simply deploy a single VM running Ubuntu with the latest Docker version installed. It is not a true cluster, but the simplicity will help to illustrate the key concepts. To be filled in" That "To be filled in" part would also be very interesting to me as worst case I could shove the Docker Swarm's workload onto a single Docker host, although then I'm back to orchestrating with Heat and presumably Ansible but via a different path. -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Fri Jun 10 13:43:55 2022 From: smooney at redhat.com (Sean Mooney) Date: Fri, 10 Jun 2022 14:43:55 +0100 Subject: [nova] local LVM volume on compute hosts In-Reply-To: <92824CD0-05FB-4941-B9C0-C27296B958F0@manchester.ac.uk> References: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> <92824CD0-05FB-4941-B9C0-C27296B958F0@manchester.ac.uk> Message-ID: On Fri, 2022-06-10 at 13:13 +0000, Paul Harrison wrote: > > > On 2022-06 -08, at 08:26, Paul Harrison wrote: > > > > > > > > > On 2022-06 -07, at 13:25, Sean Mooney > wrote: > > > > > > no there is noting else you need to configure but this option is not what you think it is. > > > the images_type option contols what storage will be used for all non cinder storage. > > > i.e. vms that are booted with out usign a boot volume. > > > > > > > > > > > > > so your current config will make any non boot from volume nova instance use lvm storage to provision the vm root/swap/epmeral disks > > > but will not prevent end users requesting cinder data voluems or boot volumes via the cli/api. if the opt in to cinder stoage that > > > is what they will recive but if they use teh default storage provided by the flaovr then it will be local. > > > > thanks for the explanation - it is a shame that there is not a more direct way in config to force local storage - looks like https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools has never got enough votes for implementation. > > > > > > I have discovered the reason why my original setup was not working (it would fail if I tried to force in the GUI, by specifying ?no? to "create new volume?) - I think it was failing here > > https://opendev.org/openstack/nova/src/commit/d86916360858daa06164ebc0d012b78d19ae6497/nova/virt/libvirt/imagebackend.py#L722 > > as the volume group device does not appear when there are no logical volumes in the group (in Centos 8 stream at least). So I forced the creation of that device by adding a dummy logical volume. > > Anyway, the situation is now that I can create instances that will use compute node local LVM storage, if I start them from the GUI, but not from the command line. so on the commandlien to use local storage assume you have a flavor with root disk not equal to 0 is as follows openstack server create --flavor --image --network you can try somethign like this export IMAGE_URL=https://github.com/cirros-dev/cirros/releases/download/0.5.1/ ARCH=$(uname -m) export IMAGE=cirros-0.5.1-${ARCH}-disk.img curl --fail -L -o cirros.qcow ${IMAGE_URL}/${IMAGE} image create --disk-format qcow2 --container-format bare --public --file cirros.qcow cirros openstack flavor create --public local-storage-tiny --id auto --ram 256 --disk 1 --vcpus 1 openstack network create lvm-test openstack subnet create --subnet-range 10.0.0.0/24 --network lvm-test \ --gateway 10.0.0.1 --dns-nameserver 8.8.8.8 demo-subnet openstack router create lvm-router openstack router add subnet lvm-router lvm-subnet # assuming your public external network is called public openstack router set --external-gateway public lvm-router openstack server create --flavor local-storage-tiny --image cirros --network lvm-test local-storage-vm this is losely based on https://github.com/openstack/kolla-ansible/blob/master/tools/init-runonce > > I had a look at what the GUI sends to the /api/nova/servers/ endpoint > > {"availability_zone":"nova","config_drive":false,"user_data":"","disk_config":"AUTO","instance_count":1,"name":"fromgui","scheduler_hints":{},"security_groups":["48648c10-ce91-4916-9347-d88fbdba9ce6"],"create_volume_default":true,"hide_create_volume":false,"source_id":"b2a3ca46-8b0b-4748-863f-6f3e11301872","flavor_id":"da28d141-3d05-4f0f-a188-229352ccf0a3","nics":[{"net-id":"5e7a171a-ceda-4051-abe6-0496e2e8e154","v4-fixed-ip":""}],"key_name":"cloud?} > > However, I have not been able to find the set of command line switches for "openstack server create" that achieve the same effect - if someone knows, I would be grateful. > > Thanks, > Paul. > > p.s. I was not really able to match up what the GUI sends with the API spec either. https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server > > From haleyb.dev at gmail.com Fri Jun 10 13:56:52 2022 From: haleyb.dev at gmail.com (Brian Haley) Date: Fri, 10 Jun 2022 09:56:52 -0400 Subject: simple build fails to allocate network interface In-Reply-To: References: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> <0044A33F-0E87-433C-8130-77C66D79926A@coote.org> Message-ID: <54ad0900-9524-013c-587b-50b0f00bf8f1@gmail.com> Hi, From your log below you have this: nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b That comes from nova/network/neutron.py: if net.get('router:external') and not net.get('shared'): raise exception.ExternalNetworkAttachForbidden( network_uuid=net['id']) And from your original email 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f is the "public" network, not the "shared" network you created. My guess if you don't have the "shared" attribute set on that network, or you're using the wrong network. What does 'openstack network show 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f' output look like? -Brian On 6/10/22 06:52, tim+openstack.org at coote.org wrote: > > >> On 9 Jun 2022, at 13:20, Rodolfo Alonso Hernandez > > wrote: >> >> Hello: >> >> This seems to be a problem in Neutron. Please provide what backend you >> are using, and the Neutron server logs. If the ML2 driver has agents >> (DHCP, ML2 agent), provide the agents logs too. >> >> Regards. >> > Hullo > thanks for the response. I don?t know what to look for wrt ml2 driver - > I?m using the default `stack.sh` installation script from the devstack repo. > > The neutron log has helped, I think. I found this useful as a starting > point:https://bit.ly/3MG6jnR as it enables log > extraction of events relating to the creation of a specific server > (although that?s still quite large for a mailing list). > > I think that this may help with your question (but as I said, I don?t > really understand the question). > > There may be a dhcp issue: I cannot ping either ipv4 or ipv6 dhcp > addresses mentioned in the log from the vm. I?ve no idea where they come > from, however. > Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG > nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 > req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: > fc8c3558-e9f5-488f-96b6-8a2eeb1f > be4f] Updated VIF entry in instance network info cache for port > 5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc. {{(pid=20725) > _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3420}} > Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG > nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 > req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: > fc8c3558-e9f5-488f-96b6-8a2eeb1f > be4f] Updating instance_info_cache with network_info: [{"id": > "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "address": "fa:16:3e:02:7a:6a", > "network": {"id": "2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63", "bridge": > "br-int", "label": "private", "subnets": [{"cidr": "10.0.0.0/26", "dns": > [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, > "meta": {}}, "ips": [{"address": "10.0.0.51", "type": "fixed", > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": > 4, "meta": {"dhcp_server": "10.0.0.1"}}, {"cidr": "fd5b:4cf4:f7c1::/64", > "dns": [], "gateway": {"address": "fd5b:4cf4:f7c1::1", "type": > "gateway", "version": 6, "meta": {}}, "ips": [{"address": > "fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a", "type": "fixed", "version": 6, > "meta": {}, "floating_ips": []}], "routes": [], "version": 6, "meta": > {"ipv6_address_mode": "slaac", "dhcp_server": "fd5b:4cf4:f7c1::1"}}], > "meta": {"injected": false, "tenant_id": > "6c0b876005f44086bd9bde5fef672dfe", "mtu": 1442, "physical_network": > null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": > true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": > "tap5ae4c0d6-2a", "ovs_interfaceid": > "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "qbh_params": null, > "qbg_params": null, "active": false, "vnic_type": "normal", "profile": > {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] > {{(pid=20725) update_instance_cache_with_nw_info > /opt/stack/nova/nova/network/neutron.py:117}} > Jun 10 10:14:11 localhost.localdomain nova-compute[20725]: DEBUG > nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None > None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network > cache update for instance because it is Building. {{(pid=20725) > _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} > Jun 10 10:15:12 localhost.localdomain nova-compute[20725]: DEBUG > nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None > None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network > cache update for instance because it is Building. {{(pid=20725) > _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} > Jun 10 10:16:12 localhost.localdomain nova-compute[20725]: DEBUG > nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None > None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network > cache update for instance because it is Building. {{(pid=20725) > _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} > Jun 10 10:17:12 localhost.localdomain nova-compute[20725]: DEBUG > nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None > None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network > cache update for instance because it is Building. {{(pid=20725) > _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} > Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG > ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): > CheckRevisionNumberCommand(name=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, > resource={'id': '5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc', 'name': '', > 'network_id': '2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'tenant_id': > '6c0b876005f44086bd9bde5fef672dfe', 'mac_address': 'fa:16:3e:02:7a:6a', > 'admin_state_up': True, 'status': 'DOWN', 'device_id': > 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'device_owner': 'compute:nova', > 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': > 'e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd', 'ip_address': '10.0.0.51'}, > {'subnet_id': 'f915b88c-9988-4e1f-9060-a6295465699a', 'ip_address': > 'fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'}], 'allowed_address_pairs': [], > 'extra_dhcp_opts': [], 'security_groups': > ['ed75ab0b-d9a4-4023-9204-93038729f6d3'], 'description': '', > 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': > 'localhost.localdomain', 'binding:vif_type': 'ovs', > 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', > 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'tags': > [], 'created_at': '2022-06-10T10:13:44Z', 'updated_at': > '2022-06-10T10:13:46Z', 'revision_number': 3, 'project_id': > '6c0b876005f44086bd9bde5fef672dfe'}, resource_type=ports, > if_exists=True) {{(pid=20918) do_commit > /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} > Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG > ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): > SetLSwitchPortCommand(lport=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, > columns={'external_ids': {'neutron:port_name': '', 'neutron:device_id': > 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'neutron:project_id': > '6c0b876005f44086bd9bde5fef672dfe', 'neutron:cidrs': '10.0.0.51/26 > fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a/64', 'neutron:device_owner': > 'compute:nova', 'neutron:network_name': > 'neutron-2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', > 'neutron:security_group_ids': 'ed75ab0b-d9a4-4023-9204-93038729f6d3', > 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], > 'options': {'requested-chassis': 'localhost.localdomain', > 'mcast_flood_reports': 'true'}, 'enabled': True, 'port_security': > ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], > 'dhcpv4_options': [UUID('01cae053-612b-4c91-81e8-39884ef035dd')], > 'dhcpv6_options': [], 'type': '', 'addresses': ['fa:16:3e:02:7a:6a > 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'ha_chassis_group': > []}, if_exists=False) {{(pid=20918) do_commit > /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} > Jun 10 10:18:13 localhost.localdomain nova-compute[20725]: DEBUG > nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None > None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network > cache update for instance because it is Building. {{(pid=20725) > _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} > Jun 10 10:18:49 localhost.localdomain nova-compute[20725]: WARNING > nova.compute.manager [None req-4a03d9e1-6053-4167-ab43-fba30072a79c demo > admin] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Timeout waiting > for ['network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc'] for > instance with vm_state building and task_state spawning. Event states > are: network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc: timed out > after 300.00 seconds: eventlet.timeout.Timeout: 300 seconds > Jun 08 15:24:48 localhost.localdomain nova-compute[87555]: ERROR > nova.compute.manager nova.exception.ExternalNetworkAttachForbidden: It > is not allowed to create an interface on external network > 205fa7d7-7067-4f63-b3d9-6a9b37b> > > From paul.harrison at manchester.ac.uk Fri Jun 10 14:38:15 2022 From: paul.harrison at manchester.ac.uk (Paul Harrison) Date: Fri, 10 Jun 2022 14:38:15 +0000 Subject: [nova] local LVM volume on compute hosts In-Reply-To: <92824CD0-05FB-4941-B9C0-C27296B958F0@manchester.ac.uk> References: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> <92824CD0-05FB-4941-B9C0-C27296B958F0@manchester.ac.uk> Message-ID: <7EE30BD6-929F-435D-8FB2-7456F852FECF@manchester.ac.uk> > On 2022-06 -10, at 14:13, Paul Harrison wrote: > > > >> On 2022-06 -08, at 08:26, Paul Harrison > wrote: >> >> >> >>> On 2022-06 -07, at 13:25, Sean Mooney > wrote: >>> >>> no there is noting else you need to configure but this option is not what you think it is. >>> the images_type option contols what storage will be used for all non cinder storage. >>> i.e. vms that are booted with out usign a boot volume. >>> >>> > >>> >>> so your current config will make any non boot from volume nova instance use lvm storage to provision the vm root/swap/epmeral disks >>> but will not prevent end users requesting cinder data voluems or boot volumes via the cli/api. if the opt in to cinder stoage that >>> is what they will recive but if they use teh default storage provided by the flaovr then it will be local. >> >> thanks for the explanation - it is a shame that there is not a more direct way in config to force local storage - looks like https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools has never got enough votes for implementation. >> >> > > I have discovered the reason why my original setup was not working (it would fail if I tried to force in the GUI, by specifying ?no? to "create new volume?) - I think it was failing here > > https://opendev.org/openstack/nova/src/commit/d86916360858daa06164ebc0d012b78d19ae6497/nova/virt/libvirt/imagebackend.py#L722 > > as the volume group device does not appear when there are no logical volumes in the group (in Centos 8 stream at least). So I forced the creation of that device by adding a dummy logical volume. > > Anyway, the situation is now that I can create instances that will use compute node local LVM storage, if I start them from the GUI, but not from the command line. > > I had a look at what the GUI sends to the /api/nova/servers/ endpoint > > {"availability_zone":"nova","config_drive":false,"user_data":"","disk_config":"AUTO","instance_count":1,"name":"fromgui","scheduler_hints":{},"security_groups":["48648c10-ce91-4916-9347-d88fbdba9ce6"],"create_volume_default":true,"hide_create_volume":false,"source_id":"b2a3ca46-8b0b-4748-863f-6f3e11301872","flavor_id":"da28d141-3d05-4f0f-a188-229352ccf0a3","nics":[{"net-id":"5e7a171a-ceda-4051-abe6-0496e2e8e154","v4-fixed-ip":""}],"key_name":"cloud?} > > However, I have not been able to find the set of command line switches for "openstack server create" that achieve the same effect - if someone knows, I would be grateful. > > Thanks, > Paul. > > p.s. I was not really able to match up what the GUI sends with the API spec either. https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server > > Have just realised, taking the hint from the GUI - if I do not try to force anything disk/volume related on the CLI, it does what I want! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2893 bytes Desc: not available URL: From smooney at redhat.com Fri Jun 10 14:52:25 2022 From: smooney at redhat.com (Sean Mooney) Date: Fri, 10 Jun 2022 15:52:25 +0100 Subject: [nova] local LVM volume on compute hosts In-Reply-To: <7EE30BD6-929F-435D-8FB2-7456F852FECF@manchester.ac.uk> References: <11D720A5-672B-4636-84C2-BFB4BED450C0@manchester.ac.uk> <215f5ae11529ce856ef6daaad7bc21d3bba3e525.camel@redhat.com> <52687CFB-C0D3-4E5B-A6F3-A35DA838C7B6@manchester.ac.uk> <92824CD0-05FB-4941-B9C0-C27296B958F0@manchester.ac.uk> <7EE30BD6-929F-435D-8FB2-7456F852FECF@manchester.ac.uk> Message-ID: <827b54689e61a89a53bfc2df45b14d327c40806c.camel@redhat.com> On Fri, 2022-06-10 at 14:38 +0000, Paul Harrison wrote: > > > On 2022-06 -10, at 14:13, Paul Harrison wrote: > > > > > > > > > On 2022-06 -08, at 08:26, Paul Harrison > wrote: > > > > > > > > > > > > > On 2022-06 -07, at 13:25, Sean Mooney > wrote: > > > > > > > > no there is noting else you need to configure but this option is not what you think it is. > > > > the images_type option contols what storage will be used for all non cinder storage. > > > > i.e. vms that are booted with out usign a boot volume. > > > > > > > > > > > > > > > > > > so your current config will make any non boot from volume nova instance use lvm storage to provision the vm root/swap/epmeral disks > > > > but will not prevent end users requesting cinder data voluems or boot volumes via the cli/api. if the opt in to cinder stoage that > > > > is what they will recive but if they use teh default storage provided by the flaovr then it will be local. > > > > > > thanks for the explanation - it is a shame that there is not a more direct way in config to force local storage - looks like https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools has never got enough votes for implementation. > > > > > > > > > > I have discovered the reason why my original setup was not working (it would fail if I tried to force in the GUI, by specifying ?no? to "create new volume?) - I think it was failing here > > > > https://opendev.org/openstack/nova/src/commit/d86916360858daa06164ebc0d012b78d19ae6497/nova/virt/libvirt/imagebackend.py#L722 > > > > as the volume group device does not appear when there are no logical volumes in the group (in Centos 8 stream at least). So I forced the creation of that device by adding a dummy logical volume. > > > > Anyway, the situation is now that I can create instances that will use compute node local LVM storage, if I start them from the GUI, but not from the command line. > > > > I had a look at what the GUI sends to the /api/nova/servers/ endpoint > > > > {"availability_zone":"nova","config_drive":false,"user_data":"","disk_config":"AUTO","instance_count":1,"name":"fromgui","scheduler_hints":{},"security_groups":["48648c10-ce91-4916-9347-d88fbdba9ce6"],"create_volume_default":true,"hide_create_volume":false,"source_id":"b2a3ca46-8b0b-4748-863f-6f3e11301872","flavor_id":"da28d141-3d05-4f0f-a188-229352ccf0a3","nics":[{"net-id":"5e7a171a-ceda-4051-abe6-0496e2e8e154","v4-fixed-ip":""}],"key_name":"cloud?} > > > > However, I have not been able to find the set of command line switches for "openstack server create" that achieve the same effect - if someone knows, I would be grateful. > > > > Thanks, > > Paul. > > > > p.s. I was not really able to match up what the GUI sends with the API spec either. https://docs.openstack.org/api-ref/compute/?expanded=create-server-detail#create-server > > > > > > Have just realised, taking the hint from the GUI - if I do not try to force anything disk/volume related on the CLI, it does what I want! yep the default behavior is what you were after. unfortunetly horizons default is differnt form nova. im not sure if you can tweak that in the horizon config so the gui also default to local sotrage but for cli users if they dont do anything fancy they will use the local lvm storage not cinder. > From tim+openstack.org at coote.org Fri Jun 10 15:06:00 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Fri, 10 Jun 2022 16:06:00 +0100 Subject: simple build fails to allocate network interface In-Reply-To: <54ad0900-9524-013c-587b-50b0f00bf8f1@gmail.com> References: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> <0044A33F-0E87-433C-8130-77C66D79926A@coote.org> <54ad0900-9524-013c-587b-50b0f00bf8f1@gmail.com> Message-ID: Hi Brian You?re correct about that. The Attachment to public networks means, I think, that it needs to be a private network. In my follow up, I managed to get much more detail about a specific server creation, and showed, I think, that the vm had been trying to use a couple of dhcp servers (v4 and v6), that afaict didn?t exist - I couldn?t ping them and didn?t recognise the network segments. I couldn?t work out where they came from. I got that detail by reviewing this: https://bit.ly/3MG6jnR atm, I?m trying to see if I can get this to work: https://bit.ly/3mD60PU However, I immediately ran into some VirtuaBox changes that invalidated the Vagrantfile. I think that I can fix these. The experience highlights to me the need for a decent test environment. Even then, I suspect that it?s very easy to have some of the latent bugs that you get in distributed systems, which undergrad courses note (https://bit.ly/2OgyWiI: I?m particularly fond of the example that lurks in Cassandra from that lecture set), and which AWS had to resort to formal proofs with TLA+ to even spot. Tim > On 10 Jun 2022, at 14:56, Brian Haley wrote: > > Hi, > > From your log below you have this: > > nova.exception.ExternalNetworkAttachForbidden: It > is not allowed to create an interface on external network > 205fa7d7-7067-4f63-b3d9-6a9b37b > > That comes from nova/network/neutron.py: > > if net.get('router:external') and not net.get('shared'): > raise exception.ExternalNetworkAttachForbidden( > network_uuid=net['id']) > > And from your original email 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f is the "public" network, not the "shared" network you created. > > My guess if you don't have the "shared" attribute set on that network, or you're using the wrong network. What does 'openstack network show 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f' output look like? > > -Brian > > On 6/10/22 06:52, tim+openstack.org at coote.org wrote: >>> On 9 Jun 2022, at 13:20, Rodolfo Alonso Hernandez > wrote: >>> >>> Hello: >>> >>> This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too. >>> >>> Regards. >>> >> Hullo >> thanks for the response. I don?t know what to look for wrt ml2 driver - I?m using the default `stack.sh` installation script from the devstack repo. >> The neutron log has helped, I think. I found this useful as a starting point:https://bit.ly/3MG6jnR as it enables log extraction of events relating to the creation of a specific server (although that?s still quite large for a mailing list). >> I think that this may help with your question (but as I said, I don?t really understand the question). >> There may be a dhcp issue: I cannot ping either ipv4 or ipv6 dhcp addresses mentioned in the log from the vm. I?ve no idea where they come from, however. >> Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f >> be4f] Updated VIF entry in instance network info cache for port 5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc. {{(pid=20725) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3420}} >> Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f >> be4f] Updating instance_info_cache with network_info: [{"id": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "address": "fa:16:3e:02:7a:6a", "network": {"id": "2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63", "bridge": "br-int", "label": "private", "subnets": [{"cidr": "10.0.0.0/26", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}, {"cidr": "fd5b:4cf4:f7c1::/64", "dns": [], "gateway": {"address": "fd5b:4cf4:f7c1::1", "type": "gateway", "version": 6, "meta": {}}, "ips": [{"address": "fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a", "type": "fixed", "version": 6, "meta": {}, "floating_ips": []}], "routes": [], "version": 6, "meta": {"ipv6_address_mode": "slaac", "dhcp_server": "fd5b:4cf4:f7c1::1"}}], "meta": {"injected": false, "tenant_id": "6c0b876005f44086bd9bde5fef672dfe", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5ae4c0d6-2a", "ovs_interfaceid": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=20725) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} >> Jun 10 10:14:11 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} >> Jun 10 10:15:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} >> Jun 10 10:16:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} >> Jun 10 10:17:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} >> Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, resource={'id': '5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc', 'name': '', 'network_id': '2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'tenant_id': '6c0b876005f44086bd9bde5fef672dfe', 'mac_address': 'fa:16:3e:02:7a:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'device_owner': 'compute:nova', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd', 'ip_address': '10.0.0.51'}, {'subnet_id': 'f915b88c-9988-4e1f-9060-a6295465699a', 'ip_address': 'fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': ['ed75ab0b-d9a4-4023-9204-93038729f6d3'], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'localhost.localdomain', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'tags': [], 'created_at': '2022-06-10T10:13:44Z', 'updated_at': '2022-06-10T10:13:46Z', 'revision_number': 3, 'project_id': '6c0b876005f44086bd9bde5fef672dfe'}, resource_type=ports, if_exists=True) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} >> Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLSwitchPortCommand(lport=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, columns={'external_ids': {'neutron:port_name': '', 'neutron:device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'neutron:project_id': '6c0b876005f44086bd9bde5fef672dfe', 'neutron:cidrs': '10.0.0.51/26 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a/64', 'neutron:device_owner': 'compute:nova', 'neutron:network_name': 'neutron-2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'neutron:security_group_ids': 'ed75ab0b-d9a4-4023-9204-93038729f6d3', 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], 'options': {'requested-chassis': 'localhost.localdomain', 'mcast_flood_reports': 'true'}, 'enabled': True, 'port_security': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'dhcpv4_options': [UUID('01cae053-612b-4c91-81e8-39884ef035dd')], 'dhcpv6_options': [], 'type': '', 'addresses': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'ha_chassis_group': []}, if_exists=False) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}} >> Jun 10 10:18:13 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}} >> Jun 10 10:18:49 localhost.localdomain nova-compute[20725]: WARNING nova.compute.manager [None req-4a03d9e1-6053-4167-ab43-fba30072a79c demo admin] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Timeout waiting for ['network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc'] for instance with vm_state building and task_state spawning. Event states are: network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc: timed out after 300.00 seconds: eventlet.timeout.Timeout: 300 seconds >> Jun 08 15:24:48 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b> From marios at redhat.com Fri Jun 10 15:31:02 2022 From: marios at redhat.com (Marios Andreou) Date: Fri, 10 Jun 2022 18:31:02 +0300 Subject: [TripleO] move stable/victoria tripleo to End Of Life OK? In-Reply-To: References: Message-ID: As promised last week and since there were no further comments the proposal to move tripleo stable/victoria to EOL is at [1]. Before we can move ahead with that, we first need to get folks to stop posting patches against stable/victoria. So we should give some time to get any final patches merged (links to all the repos/gerrit open reviews at [2] for reference) and then we can re-spin [1] to pick up those final commits . I'll send a separate request to stop posting to stable/victoria next week for better visibility. thanks, marios [1] https://review.opendev.org/c/openstack/releases/+/845148 [2] https://gist.github.com/marios/97c7ac7dcdcdabb744a823b1a59ffd74?permalink_comment_id=4196484 On Fri, Jun 3, 2022 at 6:22 PM Marios Andreou wrote: > > Hello TripleO > > the tripleo-ci team proposes that we transition the stable/victoria > branch across all tripleo repos [1] to End Of Life [2]. > Hopefully this is not a suprise to anyone as it has been discussed for > a while (most recent discussion I can point to is at Zed PTG [3]). > > The stable/victoria branch was moved to Extended Maintenance with [4] > so we can no longer make any releases for these repos. Victoria is not > an active branch for TripleO (not an FFU branch) so we are just > merging patches there on the way back to train. > > Are there any concerns, objections or any other comment? I'll give > this a few days and if there are no further comments I'll post the EOL > proposal and update here towards the end of next week, > > regards, marios > > [1] https://releases.openstack.org/teams/tripleo.html#victoria > [2] https://docs.openstack.org/project-team-guide/stable-branches.html#maintenance-phases > [3] https://etherpad.opendev.org/p/tripleo-zed-ci-load > [4] https://review.opendev.org/c/openstack/releases/+/837982 From artem.goncharov at gmail.com Fri Jun 10 15:31:30 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Fri, 10 Jun 2022 17:31:30 +0200 Subject: [doc] question/enhancement-request on Sphinx theme used by openstack docs In-Reply-To: References: Message-ID: <2C29D595-A175-4D81-ACE9-E7CD56A18746@gmail.com> Hi, Not in any way an advertisement, but we quite heavily reworked openstackdocstheme for our cloud (https://github.com/opentelekomcloud/otcdocstheme => https://docs-beta.otc.t-systems.com/otcdocstheme/ ). We have updated to the latest bootstrap, fixed some of the layout issues, added right sidebar with local top and some other tunings. Our there is surely not to be reused by anybody else, but it may give you some hints what can be improved. Regards, Artem > On 10. Jun 2022, at 14:30, Waines, Greg wrote: > > Hey ? I am on the technical steering committee for the openstack starlingx team and also provide primary technical review for our docs at https://docs.starlingx.io/ . > > docs.starlingx.io basically uses the openstack docs team?s Sphinx theme for building our html docs ? seehttps://opendev.org/starlingx/docs . > > docs.starlingx.io is a very large doc suite. > > We are interested in improvements to the Table-of-Contents (TOC) navigation and presentation ? but we have limited Sphinx expertise. > > Specifically we are looking for the TOC support to have > - more subsection header levels in TOC > // such that user has full view of where he currently is in the doc hierarchy > - highlighting to indicate currently selected section > - separate scroll bar for TOC > // ... so you don't lose sight of selected TOC selection when you scroll down on the main page which has large content. > > Have you ever considered such doc improvements? > Did you have Sphinx expertise to do such improvements ? > > let us know, > Greg. -------------- next part -------------- An HTML attachment was scrubbed... URL: From marios at redhat.com Fri Jun 10 16:06:41 2022 From: marios at redhat.com (Marios Andreou) Date: Fri, 10 Jun 2022 19:06:41 +0300 Subject: [TripleO] move stable/victoria tripleo to End Of Life OK? In-Reply-To: References: Message-ID: On Fri, Jun 10, 2022 at 6:31 PM Marios Andreou wrote: > > As promised last week and since there were no further comments the > proposal to move tripleo stable/victoria to EOL is at [1]. > > Before we can move ahead with that, we first need to get folks to stop > posting patches against stable/victoria. > > So we should give some time to get any final patches merged (links to > all the repos/gerrit open reviews at [2] for reference) and then we > can re-spin [1] to pick up those final commits . > > I'll send a separate request to stop posting to stable/victoria next > week for better visibility. > > thanks, marios > > [1] https://review.opendev.org/c/openstack/releases/+/845148 > [2] https://gist.github.com/marios/97c7ac7dcdcdabb744a823b1a59ffd74?permalink_comment_id=4196484 ^^^ that one is more like https://gist.github.com/marios/97c7ac7dcdcdabb744a823b1a59ffd74?permalink_comment_id=4196484#gistcomment-4196484 > > > On Fri, Jun 3, 2022 at 6:22 PM Marios Andreou wrote: > > > > Hello TripleO > > > > the tripleo-ci team proposes that we transition the stable/victoria > > branch across all tripleo repos [1] to End Of Life [2]. > > Hopefully this is not a suprise to anyone as it has been discussed for > > a while (most recent discussion I can point to is at Zed PTG [3]). > > > > The stable/victoria branch was moved to Extended Maintenance with [4] > > so we can no longer make any releases for these repos. Victoria is not > > an active branch for TripleO (not an FFU branch) so we are just > > merging patches there on the way back to train. > > > > Are there any concerns, objections or any other comment? I'll give > > this a few days and if there are no further comments I'll post the EOL > > proposal and update here towards the end of next week, > > > > regards, marios > > > > [1] https://releases.openstack.org/teams/tripleo.html#victoria > > [2] https://docs.openstack.org/project-team-guide/stable-branches.html#maintenance-phases > > [3] https://etherpad.opendev.org/p/tripleo-zed-ci-load > > [4] https://review.opendev.org/c/openstack/releases/+/837982 From kdhall at binghamton.edu Fri Jun 10 17:01:47 2022 From: kdhall at binghamton.edu (Dave Hall) Date: Fri, 10 Jun 2022 13:01:47 -0400 Subject: [nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox In-Reply-To: References: Message-ID: Dmitry, I do have 'nested VT-x/AMD-V' turned on on all of my VirtualBox VMs, but not 'PAE/NX'. Also, I have installed the VBox Guest Additions, but I disabled vboxadd-service.service due to clock synchronization issues. Lastly, I added nova_virt_type: qemu to /etc/openstack_deploy/user_variables.yml and re-ran the playbooks. If re-running the playbooks won't flip this setting, is there some other approach, like destroying the LXC containers for nova and then re-run the playbooks? Add another compute node? Side note: I also added # rsyslog server log_hosts: log1: ip: 172.29.236.14 to openstack_user_config.yml. After doing so, 'openstack-ansible setup-hosts' hangs during a task related to rsyslog, but the other two playbooks complete successfully. Thanks. -Dave -- Dave Hall Binghamton University kdhall at binghamton.edu On Thu, Jun 9, 2022 at 3:18 PM Dmitriy Rabotyagov wrote: > Hey, > > One question - have you set nova_virt_type to qemu in overrides? As > otherwise you indeed need to have nested virtualization enabled. > > ??, 9 ???. 2022 ?., 20:21 Dave Hall : > >> Hello, >> >> I'm trying to run a test/learning cluster on VirtualBox VMs deployed with >> Openstack-Ansible prior to deploying on real hardware. >> >> The playbooks succeed, and the basic functional checks all look good. >> However, I am unable to deploy an instance due to 'No Valid Host Found'. I >> have seen that this might be because VirtualBox does not support KVM. But >> I am also perplexed - I am able to deploy instances on an AIO running in >> VirtualBox on the same physical host. >> >> If there is an easy fix for this please point me in the right direction. >> If I need to destroy and re-clone my VMs I can try that as well. In an >> effort to learn, though, I'd really like to learn how to adjust the current >> deployment. >> >> Thanks. >> >> -Dave >> >> -- >> Dave Hall >> Binghamton University >> kdhall at binghamton.edu >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Fri Jun 10 17:27:40 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Fri, 10 Jun 2022 19:27:40 +0200 Subject: [nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox In-Reply-To: References: Message-ID: The only playbook needs rerunning is os-nova-install.yml. And it should target "compute" VM where nova-compute service runs. There's no reason to destroy containers to apply setting, but yes, we have playbooks for destroying/recreating containers . We actually don't use rsyslog as of today. We should likely fix documentation to reflect that. All loges are stored in journald. What does `openstack compute service list` return as a result? Is there any nova-compute in the list and it's alive? if yes, you should check logs for nova-scheduler in nova_api container and placement in it's container. Eventually logs are exposed to host also but can't recall from top of my head the command to check from host efficiently. ??, 10 ???. 2022 ?., 19:04 Dave Hall : > Dmitry, > > I do have 'nested VT-x/AMD-V' turned on on all of my VirtualBox VMs, but > not 'PAE/NX'. Also, I have installed the VBox Guest Additions, but I > disabled vboxadd-service.service due to clock synchronization issues. > > Lastly, I added > > nova_virt_type: qemu > > to /etc/openstack_deploy/user_variables.yml and re-ran the playbooks. If > re-running the playbooks won't flip this setting, is there some other > approach, like destroying the LXC containers for nova and then re-run the > playbooks? Add another compute node? > > Side note: I also added > > # rsyslog server > log_hosts: > log1: > ip: 172.29.236.14 > > to openstack_user_config.yml. After doing so, 'openstack-ansible > setup-hosts' hangs during a task related to rsyslog, but the other two > playbooks complete successfully. > > Thanks. > > -Dave > > > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > > On Thu, Jun 9, 2022 at 3:18 PM Dmitriy Rabotyagov > wrote: > >> Hey, >> >> One question - have you set nova_virt_type to qemu in overrides? As >> otherwise you indeed need to have nested virtualization enabled. >> >> ??, 9 ???. 2022 ?., 20:21 Dave Hall : >> >>> Hello, >>> >>> I'm trying to run a test/learning cluster on VirtualBox VMs deployed >>> with Openstack-Ansible prior to deploying on real hardware. >>> >>> The playbooks succeed, and the basic functional checks all look good. >>> However, I am unable to deploy an instance due to 'No Valid Host Found'. I >>> have seen that this might be because VirtualBox does not support KVM. But >>> I am also perplexed - I am able to deploy instances on an AIO running in >>> VirtualBox on the same physical host. >>> >>> If there is an easy fix for this please point me in the right >>> direction. If I need to destroy and re-clone my VMs I can try that as >>> well. In an effort to learn, though, I'd really like to learn how to >>> adjust the current deployment. >>> >>> Thanks. >>> >>> -Dave >>> >>> -- >>> Dave Hall >>> Binghamton University >>> kdhall at binghamton.edu >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Fri Jun 10 19:10:08 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Sat, 11 Jun 2022 00:40:08 +0530 Subject: Regarding Policy.json entries for glance image update not working for a user Message-ID: Hi Team, I have a use case where I have to give a user restriction on updating the image properties as a member. I have created a policy Json file and give the modify_image rule to the particular role, but still it is not working "modify_image": "role:user", This role is created in OpenStack. but still it is failing while updating properties with a particular user assigned to a role as "access denied" and unauthorized access Regards Adivya Singh -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Fri Jun 10 19:36:06 2022 From: satish.txt at gmail.com (Satish Patel) Date: Fri, 10 Jun 2022 21:36:06 +0200 Subject: [openstack-ansible] [kolla-ansible] Which deployment tool for first timer? In-Reply-To: References: Message-ID: Let me share my experience because I?m end user and cloud operator. I?m running 5 large production deployment on openstack-ansible (each cloud has more than 250+ computes nodes so you know scale). Openstack-ansible community is very supportive and helped me a lot to bring up massive deployment with scale. Few month back one of my customer ask me to use kolla-ansible so I use kolla-ansible to reply their cloud. For me both are great but I have more command on openstack-ansible and after deploying 5 large cloud it?s in my blood. Both ansible based so I don?t think you will have any difficulty to play. You can find me or others on IRC if you have more questions. Sent from my iPhone > On Jun 10, 2022, at 10:56 AM, Mark Goddard wrote: > > ? > This has been discussed before, but it would be nice if we as a community had an easier way to answer this question. I know many of us feel conflicted about aggressively promoting our own projects over alternatives, but also want to give them a fair chance. Maybe each project needs a "why pick me" page that could be linked to in a standard response. > > Mark > >> On Thu, 9 Jun 2022 at 16:40, Jonathan Rosser wrote: >> Hi Dave, >> >> I have been hesitating to reply to your mailing list post because it doesn't feel right to pitch two community tools against each other here on the mailing list - so i won't do that here. >> >> I would say that the deployment tool is a means to an end. So you should look at the technology choices, upgrade paths, support for "day 2 operations", how bugs get addressed, documentation, operator experience etc etc. Only you can decide which is appropriate for the constraints and requirements of your deployment. >> >> My reply will obviously be biassed, as I am a big contributor to openstack-ansible. My observation is that the operators that gain the most out of any of these tools are the ones who engage with the community around those tools, primarily in the case of openstack-ansible that would be through our IRC channel, weekly meetings and bug reports on Launchpad. You will gain insight and be able to leverage the knowledge of other operators who in some cases have literally written the book on various aspects of OpenStack. Trying to fight though every decision or problem on your own is the worst way to approach using any of these community driven tools. >> >> If you instead want a more "shrink wrap" approach to an installer, or more formal support, then it would be wise to look at the product oriented offerings from the large vendors. >> >> Both openstack-ansible and kolla-ansible will expect you to make a good number of decisions about the specifics of your deployment, for example storage, networking and security concerns. Both would also expect you to gain sufficient knowledge about how OpenStack itself works to be able to make good use of the customization opportunities that both provide. This is really the unique selling point of the community tooling, you get a very high degree of customization potential but that can come at the cost of some complexity. >> >> As you are already using openstack-ansible I would suggest that you continue, but as I've already said I have an existing interest here and I really don't want to start a tooling debate. Join us in IRC in #openstack-ansible and discuss any pain points you have. This way we can hopefully help you out, or address specific issues in the code - you may have discovered legitimate bugs or a use case that is not straightforward to fulfill. This is how all of the community tools get improved and evolved over time. >> >> On one specific point I would recommend that you move entirely to Debian 11 as Xena will be the last release that openstack-ansible supports Buster. >> >> I'm not sure there is a fool-proof installer really. Installing the code is one thing, being effective though upgrades and applying bugfixes to a production environment is a different and a more important concern in the long term. Both openstack-ansible and kolla-ansible offer "all-in-one" deployments which are intended as "should-just-work" demonstrators of how things fit together and for lightweight testing. Scaling those out to larger deployments is where the real work is, and neither tool sets out to be particularly prescriptive about some parts of how you build your environment. >> >> Hopefully this is helpful, >> Jonathan. >> >>> On 09/06/2022 15:58, Dave Hall wrote: >>> Hello, >>> >>> My question is about OpenStack-Ansible vs. Kolla-Ansible. While I am sensitive to the effort that has been put into both of these projects, what I really need right now is the most fool-proof way to deploy and manage a small production cluster for academic instructional use. >>> >>> (BTW, I do understand that there are other differences between Kolla and regular openstack.) >>> >>> I have been working for a couple months with OpenStack-Ansible to deploy a small (3-node) Xena test cluster on VirtualBox VMs in preparation for a larger deployment on real hardware - 6 to 10 nodes. My VirtualBox deployment has been: >>> >>> Debian 11 deployment, Debian 10 infra, compute, and storage nodes >>> >>> It has been slow going, at least partially due to some issues and limitations with VirtualBox (and Vagrant early on). However, deploying a test cluster on VMs still seems preferable to just diving into deployment on real hardware and going through multiple scrubs/reinstalls. >>> >>> I've recently seen more posts in the list about Kolla-Ansible. So, as a 'beginner', should I shift and look at Kolla-Ansible, or should I stay the course and continue with Openstack-Ansible? What are the pros and cons of each? >>> >>> For that matter, is there some other deployment mechanism that would be more fool-proof for a first-timer? Although I'm more familiar with Ansible than the other tools (Chef, Puppet, etc.) I'm most interested in how to get a cluster up and running regardless of the underlying tools. >>> >>> Thanks. >>> >>> -Dave >>> >>> -- >>> Dave Hall >>> Binghamton University >>> kdhall at binghamton.edu >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Sat Jun 11 01:51:01 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 10 Jun 2022 20:51:01 -0500 Subject: [all][tc] What's happening in Technical Committee: summary 10 June 2022: Reading: 5 min Message-ID: <18150751f5f.11e69cf6d82575.1368494779385694947@ghanshyammann.com> Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We cancelled this week's meeting. The next TC weekly meeting will be on 16 June Thursday at 15:00 UTC, feel free to add the topic to the agenda[1] by 15 June. 2. What we completed this week: ========================= * Added community infrastructure contributor help in upstream investment[2] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[3], many items are in-progress. Open Reviews ----------------- * Five open reviews for ongoing activities[4]. New release cadence "SLURP" open items -------------------------------------------------- 1. release notes strategy: Brian proposal for ""SLURP release notes approach is up for review[5]. Improve project governance --------------------------------- Slawek has the framework proposal up and is under review[6]. New ELK service dashboard: e-r service ----------------------------------------------- No update this week. Consistent and Secure Default RBAC ------------------------------------------- We called for the operator's feedback on RBAC new directions, I sent an email about it[7] and also created etherpad to collect feedback in a central place[8]. My humble request is to use this central etherpad for feedback/discussion otherwise it might be difficult for me to collect/know feedback conclusions from a lot of 100s places. We have feedback from the berlin summit but we still encourage operators/local events or so keep adding the feedback and your view on RBAC. Please note that operator feedback is very important for us to proceed on 'scope' in SRBAC. If you are operator or know anyone around you, we request you to provide feedback. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[9]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[10]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[11][12]. Project updates ------------------- * None 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[13]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [14] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/843749 [3] https://etherpad.opendev.org/p/tc-zed-tracker [4] https://review.opendev.org/q/projects:openstack/governance+status:open [5] https://review.opendev.org/c/openstack/project-team-guide/+/843457 [6] https://review.opendev.org/c/openstack/governance/+/839880 [7] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028878.html [8] https://etherpad.opendev.org/p/rbac-operator-feedback [9] https://review.opendev.org/c/openstack/governance/+/836888 [10] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.html [11] https://etherpad.opendev.org/p/zuul-config-error-openstack [12] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [13] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [14] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann From kdhall at binghamton.edu Sat Jun 11 22:28:01 2022 From: kdhall at binghamton.edu (Dave Hall) Date: Sat, 11 Jun 2022 18:28:01 -0400 Subject: [nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox In-Reply-To: References: Message-ID: Dmitry, I reran os-nova-install.yml. I observed from the dashboard that my compute node is listed as running QEMU. `openstack compute service list` shows my compute node as up and enabled. I think I did find a way to get past 'No Valid Host'. But then the instance failed with a volume error. I think maybe I don't have all of the needed pieces in place - networks, ports, security groups, etc. Is there a tutorial that you could point me to? Thanks. -Dave -- Dave Hall Binghamton University kdhall at binghamton.edu On Fri, Jun 10, 2022 at 1:28 PM Dmitriy Rabotyagov wrote: > The only playbook needs rerunning is os-nova-install.yml. And it should > target "compute" VM where nova-compute service runs. There's no reason to > destroy containers to apply setting, but yes, we have playbooks for > destroying/recreating containers . > > We actually don't use rsyslog as of today. We should likely fix > documentation to reflect that. > All loges are stored in journald. > > What does `openstack compute service list` return as a result? Is there > any nova-compute in the list and it's alive? > > if yes, you should check logs for nova-scheduler in nova_api container and > placement in it's container. > Eventually logs are exposed to host also but can't recall from top of my > head the command to check from host efficiently. > > > > ??, 10 ???. 2022 ?., 19:04 Dave Hall : > >> Dmitry, >> >> I do have 'nested VT-x/AMD-V' turned on on all of my VirtualBox VMs, but >> not 'PAE/NX'. Also, I have installed the VBox Guest Additions, but I >> disabled vboxadd-service.service due to clock synchronization issues. >> >> Lastly, I added >> >> nova_virt_type: qemu >> >> to /etc/openstack_deploy/user_variables.yml and re-ran the playbooks. >> If re-running the playbooks won't flip this setting, is there some other >> approach, like destroying the LXC containers for nova and then re-run the >> playbooks? Add another compute node? >> >> Side note: I also added >> >> # rsyslog server >> log_hosts: >> log1: >> ip: 172.29.236.14 >> >> to openstack_user_config.yml. After doing so, 'openstack-ansible >> setup-hosts' hangs during a task related to rsyslog, but the other two >> playbooks complete successfully. >> >> Thanks. >> >> -Dave >> >> >> >> -- >> Dave Hall >> Binghamton University >> kdhall at binghamton.edu >> >> >> On Thu, Jun 9, 2022 at 3:18 PM Dmitriy Rabotyagov < >> noonedeadpunk at gmail.com> wrote: >> >>> Hey, >>> >>> One question - have you set nova_virt_type to qemu in overrides? As >>> otherwise you indeed need to have nested virtualization enabled. >>> >>> ??, 9 ???. 2022 ?., 20:21 Dave Hall : >>> >>>> Hello, >>>> >>>> I'm trying to run a test/learning cluster on VirtualBox VMs deployed >>>> with Openstack-Ansible prior to deploying on real hardware. >>>> >>>> The playbooks succeed, and the basic functional checks all look good. >>>> However, I am unable to deploy an instance due to 'No Valid Host Found'. I >>>> have seen that this might be because VirtualBox does not support KVM. But >>>> I am also perplexed - I am able to deploy instances on an AIO running in >>>> VirtualBox on the same physical host. >>>> >>>> If there is an easy fix for this please point me in the right >>>> direction. If I need to destroy and re-clone my VMs I can try that as >>>> well. In an effort to learn, though, I'd really like to learn how to >>>> adjust the current deployment. >>>> >>>> Thanks. >>>> >>>> -Dave >>>> >>>> -- >>>> Dave Hall >>>> Binghamton University >>>> kdhall at binghamton.edu >>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.aminian.server at gmail.com Sun Jun 12 06:37:06 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Sun, 12 Jun 2022 11:07:06 +0430 Subject: snapshot problem Message-ID: When I snapshot from the instance , server will gone away and its not reachable until the snapshot is complete here is the logs : 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting 2022-06-12 09:42:34.755 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) 2022-06-12 09:42:34.995 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance has a pending task (image_pending_upload). Skip. 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver [req-e79e4177-4712-4795-91da-853bc524fac0 93fb420b3c604d4fae408b81135b58e9 ef940663426b4c87a751afaf13b758e0 - default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning image upload 2022-06-12 09:43:08.778 7 INFO nova.compute.manager [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Sun Jun 12 08:10:38 2022 From: eblock at nde.ag (Eugen Block) Date: Sun, 12 Jun 2022 08:10:38 +0000 Subject: snapshot problem In-Reply-To: Message-ID: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> Hi, can you share more details about your environment? Which openstack version is it? What is the storage backend? In earlier releases there was an option: #disable_libvirt_livesnapshot = false but this option has been deprecated. But if you're on an older version that could explain it. Zitat von Parsa Aminian : > When I snapshot from the instance , server will gone away and its not > reachable until the snapshot is complete here is the logs : > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance > has a pending task (image_pending_upload). Skip. > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver > [req-e79e4177-4712-4795-91da-853bc524fac0 93fb420b3c604d4fae408b81135b58e9 > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning image > upload > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) From noonedeadpunk at gmail.com Sun Jun 12 08:59:35 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Sun, 12 Jun 2022 10:59:35 +0200 Subject: [nova][openstack-ansible] "No Valid Host Found" when running on VirtualBox In-Reply-To: References: Message-ID: What's the volume error? What does 'openstack volume show ' show for status? For networking good set of useful commands you can find here https://docs.openstack.org/neutron/yoga/admin/archives/use.html But all really depends on setup, as openstack is very flexible so it all depends. ??, 12 ???. 2022 ?., 00:28 Dave Hall : > Dmitry, > > I reran os-nova-install.yml. I observed from the dashboard that my > compute node is listed as running QEMU. > > `openstack compute service list` shows my compute node as up and enabled. > > I think I did find a way to get past 'No Valid Host'. But then the > instance failed with a volume error. > > I think maybe I don't have all of the needed pieces in place - networks, > ports, security groups, etc. Is there a tutorial that you could point me > to? > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > > > On Fri, Jun 10, 2022 at 1:28 PM Dmitriy Rabotyagov < > noonedeadpunk at gmail.com> wrote: > >> The only playbook needs rerunning is os-nova-install.yml. And it should >> target "compute" VM where nova-compute service runs. There's no reason to >> destroy containers to apply setting, but yes, we have playbooks for >> destroying/recreating containers . >> >> We actually don't use rsyslog as of today. We should likely fix >> documentation to reflect that. >> All loges are stored in journald. >> >> What does `openstack compute service list` return as a result? Is there >> any nova-compute in the list and it's alive? >> >> if yes, you should check logs for nova-scheduler in nova_api container >> and placement in it's container. >> Eventually logs are exposed to host also but can't recall from top of my >> head the command to check from host efficiently. >> >> >> >> ??, 10 ???. 2022 ?., 19:04 Dave Hall : >> >>> Dmitry, >>> >>> I do have 'nested VT-x/AMD-V' turned on on all of my VirtualBox VMs, but >>> not 'PAE/NX'. Also, I have installed the VBox Guest Additions, but I >>> disabled vboxadd-service.service due to clock synchronization issues. >>> >>> Lastly, I added >>> >>> nova_virt_type: qemu >>> >>> to /etc/openstack_deploy/user_variables.yml and re-ran the playbooks. >>> If re-running the playbooks won't flip this setting, is there some other >>> approach, like destroying the LXC containers for nova and then re-run the >>> playbooks? Add another compute node? >>> >>> Side note: I also added >>> >>> # rsyslog server >>> log_hosts: >>> log1: >>> ip: 172.29.236.14 >>> >>> to openstack_user_config.yml. After doing so, 'openstack-ansible >>> setup-hosts' hangs during a task related to rsyslog, but the other two >>> playbooks complete successfully. >>> >>> Thanks. >>> >>> -Dave >>> >>> >>> >>> -- >>> Dave Hall >>> Binghamton University >>> kdhall at binghamton.edu >>> >>> >>> On Thu, Jun 9, 2022 at 3:18 PM Dmitriy Rabotyagov < >>> noonedeadpunk at gmail.com> wrote: >>> >>>> Hey, >>>> >>>> One question - have you set nova_virt_type to qemu in overrides? As >>>> otherwise you indeed need to have nested virtualization enabled. >>>> >>>> ??, 9 ???. 2022 ?., 20:21 Dave Hall : >>>> >>>>> Hello, >>>>> >>>>> I'm trying to run a test/learning cluster on VirtualBox VMs deployed >>>>> with Openstack-Ansible prior to deploying on real hardware. >>>>> >>>>> The playbooks succeed, and the basic functional checks all look good. >>>>> However, I am unable to deploy an instance due to 'No Valid Host Found'. I >>>>> have seen that this might be because VirtualBox does not support KVM. But >>>>> I am also perplexed - I am able to deploy instances on an AIO running in >>>>> VirtualBox on the same physical host. >>>>> >>>>> If there is an easy fix for this please point me in the right >>>>> direction. If I need to destroy and re-clone my VMs I can try that as >>>>> well. In an effort to learn, though, I'd really like to learn how to >>>>> adjust the current deployment. >>>>> >>>>> Thanks. >>>>> >>>>> -Dave >>>>> >>>>> -- >>>>> Dave Hall >>>>> Binghamton University >>>>> kdhall at binghamton.edu >>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Sun Jun 12 11:02:08 2022 From: eblock at nde.ag (Eugen Block) Date: Sun, 12 Jun 2022 11:02:08 +0000 Subject: snapshot problem In-Reply-To: References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> Message-ID: <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> Have you tried with debug logs? Has it worked with live snapshots before for other instances or did it never work and all snapshots were "cold"? Zitat von Parsa Aminian : > Hi > kolla-ansible victoria version with ceph backend without volume > > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block wrote: > >> Hi, >> >> can you share more details about your environment? Which openstack >> version is it? What is the storage backend? In earlier releases there >> was an option: >> >> #disable_libvirt_livesnapshot = false >> >> but this option has been deprecated. But if you're on an older version >> that could explain it. >> >> Zitat von Parsa Aminian : >> >> > When I snapshot from the instance , server will gone away and its not >> > reachable until the snapshot is complete here is the logs : >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> instance >> > has a pending task (image_pending_upload). Skip. >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> 93fb420b3c604d4fae408b81135b58e9 >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning image >> > upload >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> >> >> >> From eblock at nde.ag Sun Jun 12 13:14:36 2022 From: eblock at nde.ag (Eugen Block) Date: Sun, 12 Jun 2022 13:14:36 +0000 Subject: snapshot problem In-Reply-To: References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> Message-ID: <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> You should respond to the list so other users can try to support you. So nova is trying to live snapshot the instance: > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > instance snapshotting > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning > live snapshot process But I don't see any 'rbd snap create' command. Either the rbd image doesn't support it or there is some setting to keep all rbd images "flat". Can you check any relevant configs you might have in nova? Also can you show the output of 'rbd info /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if the underlying rbd functions work as expected you could try to create a live snapshot manually: rbd snap create /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap And paste any relevant output here. Zitat von Parsa Aminian : > Its not working for any instances and all of them are paused . I enable > debug logs please check the logs : > > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > :: waited 0.000s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute host > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > 63426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot process > default default] Lazy-loading 'pci_devices' on Instance uuid > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 > obj_load_attr > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event > => Paused> emit_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state > after lifecycle event "Paused"; current vm_state: active, current > task_state: image_pending_upload, current DB power_state: 1, VM > power_state: 3 handle_lifecycle_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance > has a pending task (image_pending_upload). Skip. > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd > (subprocess): qemu-img convert -t none -O raw -f raw > rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad > execute > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event > => Stopped> emit_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > INFO nova.compute.manager [-] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - - > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - - > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing > instance power state after lifecycle event "Stopped"; current vm_state: > active, current task_state: image_pending_upload, current DB power_state: > 1, VM power_state: 4 handle_lifecycle_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > INFO nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - - - > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state > the instance has a pending task (image_pending_upload). Skip. > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock > "25c8d676-e20a-4238 > -a45c-d51daa62b941" acquired by > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > :: waited 0.000s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the instance > has a pending task (image_pending_upload). Skip. > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock > "25c8d676-e20a-4238-a45c-d51daa62b941" released by > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > :: held 0.001s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img convert > -t none -O raw -f raw > rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" > returned: 0 in 23.730s execute > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot > extracted, beginning image upload > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event > => Started> emit_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > _get_power_state > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state > after lifecycle event "Started"; current vm_state: active, current > task_state: image_pending_upload, current DB power_state: 1, VM > power_state: 1 handle_lifecycle_event > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network info > cache for instance _get_instance_nw_info > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache with > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "address": > "fa:16:3e:ca:00:d9", "network": {"id": > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", "label": > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": {}}], > "gateway": {"address": "141.11.42.1", "type": "gateway", "version": 4, > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, > "meta": {}}], "meta": {"injected": true, "tenant_id": > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, "physical_network": > "physnet1", "tunneled": false}}, "type": "ovs", "details": {"connectivity": > "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "qbh_params": > null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": > {}, "preserve_on_delete": false, "meta": {}}] > update_instance_cache_with_nw_info > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > compute host and has allocations in placement: {'resources': {'VCPU': 1, > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to snapshot the > instance on the hypervisor. > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > :: waited 0.000s inner > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute host > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager > [req-e79e4177-4712-4795-91da-853bc524fac0 93fb420b3c604d4fae408b81135b58e9 > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: > >> Have you tried with debug logs? Has it worked with live snapshots >> before for other instances or did it never work and all snapshots were >> "cold"? >> >> Zitat von Parsa Aminian : >> >> > Hi >> > kolla-ansible victoria version with ceph backend without volume >> > >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block wrote: >> > >> >> Hi, >> >> >> >> can you share more details about your environment? Which openstack >> >> version is it? What is the storage backend? In earlier releases there >> >> was an option: >> >> >> >> #disable_libvirt_livesnapshot = false >> >> >> >> but this option has been deprecated. But if you're on an older version >> >> that could explain it. >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> > When I snapshot from the instance , server will gone away and its not >> >> > reachable until the snapshot is complete here is the logs : >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> >> instance >> >> > has a pending task (image_pending_upload). Skip. >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> >> 93fb420b3c604d4fae408b81135b58e9 >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning >> image >> >> > upload >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From p.aminian.server at gmail.com Sun Jun 12 19:33:16 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Mon, 13 Jun 2022 00:03:16 +0430 Subject: snapshot problem In-Reply-To: <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> Message-ID: Hi kolla-ansible victoria version with ceph backend . rbd info output : rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': size 20 GiB in 5120 objects order 22 (4 MiB objects) snapshot_count: 0 id: b69aaf907284da block_name_prefix: rbd_data.b69aaf907284da format: 2 features: layering, exclusive-lock, object-map, fast-diff, deep-flatten op_features: flags: create_timestamp: Fri May 20 00:04:47 2022 access_timestamp: Sun Jun 12 16:26:02 2022 --------------- also live snapshot seems to work correctly without any error or any downtime : docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk SNAPID NAME SIZE PROTECTED TIMESTAMP 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 also on compute nova.conf, images_type is set on rbd . On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: > You should respond to the list so other users can try to support you. > > So nova is trying to live snapshot the instance: > > > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - > > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > > instance snapshotting > > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning > > live snapshot process > > But I don't see any 'rbd snap create' command. Either the rbd image > doesn't support it or there is some setting to keep all rbd images > "flat". Can you check any relevant configs you might have in nova? > Also can you show the output of 'rbd info > /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if > the underlying rbd functions work as expected you could try to create > a live snapshot manually: > > rbd snap create /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap > > And paste any relevant output here. > > Zitat von Parsa Aminian : > > > Its not working for any instances and all of them are paused . I enable > > debug logs please check the logs : > > > > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager > > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock > > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > > > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > > :: waited 0.000s inner > > > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker > > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute > host > > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': > > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager > > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b > > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b > > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > > 63426b4c87a751afaf13b758e0 - default default] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot process > > default default] Lazy-loading 'pci_devices' on Instance uuid > > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance > > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b > > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading > > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 > > obj_load_attr > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event > > > => Paused> emit_event > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager > > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager > > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state > > after lifecycle event "Paused"; current vm_state: active, current > > task_state: image_pending_upload, current DB power_state: 1, VM > > power_state: 3 handle_lifecycle_event > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager > > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > instance > > has a pending task (image_pending_upload). Skip. > > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils > > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b > > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd > > (subprocess): qemu-img convert -t none -O raw -f raw > > > rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > > > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad > > execute > > > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 > > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event > > > => Stopped> emit_event > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > > INFO nova.compute.manager [-] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - > - > > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > > _get_power_state > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - > - > > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing > > instance power state after lifecycle event "Stopped"; current vm_state: > > active, current task_state: image_pending_upload, current DB power_state: > > 1, VM power_state: 4 handle_lifecycle_event > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > > INFO nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - > - - > > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During > sync_power_state > > the instance has a pending task (image_pending_upload). Skip. > > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager > > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock > > "25c8d676-e20a-4238 > > -a45c-d51daa62b941" acquired by > > > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > > :: waited 0.000s inner > > > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > instance > > has a pending task (image_pending_upload). Skip. > > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock > > "25c8d676-e20a-4238-a45c-d51daa62b941" released by > > > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > > :: held 0.001s inner > > > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 > > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils > > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b > > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img convert > > -t none -O raw -f raw > > > rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > > > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" > > returned: 0 in 23.730s execute > > > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 > > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > Snapshot > > extracted, beginning image upload > > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver > > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event > > > => Started> emit_event > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager > > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) > > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > > _get_power_state > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state > > after lifecycle event "Started"; current vm_state: active, current > > task_state: image_pending_upload, current DB power_state: 1, VM > > power_state: 1 handle_lifecycle_event > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager > > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event > > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils > > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock > > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock > > > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 > > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network info > > cache for instance _get_instance_nw_info > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 > > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron > > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache with > > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "address": > > "fa:16:3e:ca:00:d9", "network": {"id": > > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", "label": > > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": > > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, > > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": {}}], > > "gateway": {"address": "141.11.42.1", "type": "gateway", "version": 4, > > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", > > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": > 4, > > "meta": {}}], "meta": {"injected": true, "tenant_id": > > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, "physical_network": > > "physnet1", "tunneled": false}}, "type": "ovs", "details": > {"connectivity": > > "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": > > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", > > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "qbh_params": > > null, "qbg_params": null, "active": true, "vnic_type": "normal", > "profile": > > {}, "preserve_on_delete": false, "meta": {}}] > > update_instance_cache_with_nw_info > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 > > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > > compute host and has allocations in placement: {'resources': {'VCPU': 1, > > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager > > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > 4dbffaa9c14e401c8c210e23ebe0ab7b > > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to snapshot the > > instance on the hypervisor. > > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager > > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for > > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > > > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > > :: waited 0.000s inner > > > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker > > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute > host > > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': > > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager > > [req-e79e4177-4712-4795-91da-853bc524fac0 > 93fb420b3c604d4fae408b81135b58e9 > > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > > > > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: > > > >> Have you tried with debug logs? Has it worked with live snapshots > >> before for other instances or did it never work and all snapshots were > >> "cold"? > >> > >> Zitat von Parsa Aminian : > >> > >> > Hi > >> > kolla-ansible victoria version with ceph backend without volume > >> > > >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block wrote: > >> > > >> >> Hi, > >> >> > >> >> can you share more details about your environment? Which openstack > >> >> version is it? What is the storage backend? In earlier releases there > >> >> was an option: > >> >> > >> >> #disable_libvirt_livesnapshot = false > >> >> > >> >> but this option has been deprecated. But if you're on an older > version > >> >> that could explain it. > >> >> > >> >> Zitat von Parsa Aminian : > >> >> > >> >> > When I snapshot from the instance , server will gone away and its > not > >> >> > reachable until the snapshot is complete here is the logs : > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager > >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager > >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > >> >> instance > >> >> > has a pending task (image_pending_upload). Skip. > >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver > >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> >> 93fb420b3c604d4fae408b81135b58e9 > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning > >> image > >> >> > upload > >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager > >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) > >> >> > >> >> > >> >> > >> >> > >> >> > >> > >> > >> > >> > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Sun Jun 12 22:10:19 2022 From: zigo at debian.org (Thomas Goirand) Date: Mon, 13 Jun 2022 00:10:19 +0200 Subject: Upgrading to a more recent version of jsonschema Message-ID: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> Hi, A few DDs are pushing me to upgrade jsonschema in Debian Unstable. However, OpenStack global requirements are still stuck at 3.2.0. Is there any reason for it, or should we attempt to upgrade to 4.6.0? I'd really appreciate if someone (else than me) was driving this... Cheers, Thomas Goirand (zigo) From eblock at nde.ag Mon Jun 13 07:33:00 2022 From: eblock at nde.ag (Eugen Block) Date: Mon, 13 Jun 2022 07:33:00 +0000 Subject: snapshot problem In-Reply-To: References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> Message-ID: <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> Could you share your whole nova.conf (only the uncommented options)? Is this option set in your env? #snapshot_image_format = Also when you manually created the snapshot did you do it as the nova user on the compute node? If not, could you retry? Zitat von Parsa Aminian : > Hi > kolla-ansible victoria version with ceph backend . > rbd info output : > rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': > size 20 GiB in 5120 objects > order 22 (4 MiB objects) > snapshot_count: 0 > id: b69aaf907284da > block_name_prefix: rbd_data.b69aaf907284da > format: 2 > features: layering, exclusive-lock, object-map, fast-diff, > deep-flatten > op_features: > flags: > create_timestamp: Fri May 20 00:04:47 2022 > access_timestamp: Sun Jun 12 16:26:02 2022 > --------------- > also live snapshot seems to work correctly without any error or any > downtime : > docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk > SNAPID NAME SIZE PROTECTED TIMESTAMP > 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 > > also on compute nova.conf, images_type is set on rbd . > > On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: > >> You should respond to the list so other users can try to support you. >> >> So nova is trying to live snapshot the instance: >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] >> > instance snapshotting >> > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning >> > live snapshot process >> >> But I don't see any 'rbd snap create' command. Either the rbd image >> doesn't support it or there is some setting to keep all rbd images >> "flat". Can you check any relevant configs you might have in nova? >> Also can you show the output of 'rbd info >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if >> the underlying rbd functions work as expected you could try to create >> a live snapshot manually: >> >> rbd snap create /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap >> >> And paste any relevant output here. >> >> Zitat von Parsa Aminian : >> >> > Its not working for any instances and all of them are paused . I enable >> > debug logs please check the logs : >> > >> > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock >> > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by >> > >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> > :: waited 0.000s inner >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute >> host >> > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> > 63426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot process >> > default default] Lazy-loading 'pci_devices' on Instance uuid >> > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 >> > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading >> > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 >> > obj_load_attr >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 >> > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event >> > > > => Paused> emit_event >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state >> > after lifecycle event "Paused"; current vm_state: active, current >> > task_state: image_pending_upload, current DB power_state: 1, VM >> > power_state: 3 handle_lifecycle_event >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> instance >> > has a pending task (image_pending_upload). Skip. >> > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd >> > (subprocess): qemu-img convert -t none -O raw -f raw >> > >> rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf >> > >> /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad >> > execute >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 >> > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event >> > > > => Stopped> emit_event >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> > INFO nova.compute.manager [-] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - >> - >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state >> > _get_power_state >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - >> - >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing >> > instance power state after lifecycle event "Stopped"; current vm_state: >> > active, current task_state: image_pending_upload, current DB power_state: >> > 1, VM power_state: 4 handle_lifecycle_event >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> > INFO nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - - >> - - >> > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During >> sync_power_state >> > the instance has a pending task (image_pending_upload). Skip. >> > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock >> > "25c8d676-e20a-4238 >> > -a45c-d51daa62b941" acquired by >> > >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> > :: waited 0.000s inner >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> instance >> > has a pending task (image_pending_upload). Skip. >> > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock >> > "25c8d676-e20a-4238-a45c-d51daa62b941" released by >> > >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> > :: held 0.001s inner >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 >> > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img convert >> > -t none -O raw -f raw >> > >> rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf >> > >> /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" >> > returned: 0 in 23.730s execute >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] >> Snapshot >> > extracted, beginning image upload >> > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event >> > > > => Started> emit_event >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state >> > _get_power_state >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power state >> > after lifecycle event "Started"; current vm_state: active, current >> > task_state: image_pending_upload, current DB power_state: 1, VM >> > power_state: 1 handle_lifecycle_event >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event >> > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock >> > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network info >> > cache for instance _get_instance_nw_info >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 >> > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache with >> > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "address": >> > "fa:16:3e:ca:00:d9", "network": {"id": >> > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", "label": >> > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": >> > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, >> > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": {}}], >> > "gateway": {"address": "141.11.42.1", "type": "gateway", "version": 4, >> > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", >> > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": >> 4, >> > "meta": {}}], "meta": {"injected": true, "tenant_id": >> > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, "physical_network": >> > "physnet1", "tunneled": false}}, "type": "ovs", "details": >> {"connectivity": >> > "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": >> > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", >> > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", "qbh_params": >> > null, "qbg_params": null, "active": true, "vnic_type": "normal", >> "profile": >> > {}, "preserve_on_delete": false, "meta": {}}] >> > update_instance_cache_with_nw_info >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 >> > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this >> > compute host and has allocations in placement: {'resources': {'VCPU': 1, >> > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to snapshot the >> > instance on the hypervisor. >> > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync for >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by >> > >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> > :: waited 0.000s inner >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute >> host >> > and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> 93fb420b3c604d4fae408b81135b58e9 >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> > >> > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: >> > >> >> Have you tried with debug logs? Has it worked with live snapshots >> >> before for other instances or did it never work and all snapshots were >> >> "cold"? >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> > Hi >> >> > kolla-ansible victoria version with ceph backend without volume >> >> > >> >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block wrote: >> >> > >> >> >> Hi, >> >> >> >> >> >> can you share more details about your environment? Which openstack >> >> >> version is it? What is the storage backend? In earlier releases there >> >> >> was an option: >> >> >> >> >> >> #disable_libvirt_livesnapshot = false >> >> >> >> >> >> but this option has been deprecated. But if you're on an older >> version >> >> >> that could explain it. >> >> >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> >> >> > When I snapshot from the instance , server will gone away and its >> not >> >> >> > reachable until the snapshot is complete here is the logs : >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> >> >> instance >> >> >> > has a pending task (image_pending_upload). Skip. >> >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> >> >> 93fb420b3c604d4fae408b81135b58e9 >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, beginning >> >> image >> >> >> > upload >> >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From gael.therond at bitswalk.com Mon Jun 13 08:33:06 2022 From: gael.therond at bitswalk.com (=?UTF-8?Q?Ga=C3=ABl_THEROND?=) Date: Mon, 13 Jun 2022 10:33:06 +0200 Subject: [VICTORIA][IRONIC] - Can inspect but not deploy Message-ID: Hi everyone! I'm dealing with a strange issue today, we deployed IRONIC on a VICTORIA platform, we activated the dnsmasq pxe filtering option at the inspector level, it works great as only IRONIC listed hosts are then served by the dnsmasq DHCP as those hosts are allowed using the dhcp hosts dir. BUT, I'm having a weird issue now. All my nodes are able to get an IP from the DHCP at the INSPECTION step, however, as soon as the inspection step is finished, the mac related file is once again filled with ",ignore" which prohibits further operations. This means as soon as I put that node as available and try to deploy an "instance" on it (provision the host), it doesn't work as dnsmasq reject the host boot DHCP requests. So, is there a way to instruct the ironic-conductor to edit/allow the host the way the inspector is able to manipulate this file? Did I missed something? Thanks a lot! -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Mon Jun 13 09:40:48 2022 From: pierre at stackhpc.com (Pierre Riteau) Date: Mon, 13 Jun 2022 11:40:48 +0200 Subject: [VICTORIA][IRONIC] - Can inspect but not deploy In-Reply-To: References: Message-ID: Hello Ga?l, Which network_interface are you using for your nodes? Is your provisioning network different from the inspection network? Pierre On Mon, 13 Jun 2022 at 10:36, Ga?l THEROND wrote: > Hi everyone! > > I'm dealing with a strange issue today, we deployed IRONIC on a VICTORIA > platform, we activated the dnsmasq pxe filtering option at the inspector > level, it works great as only IRONIC listed hosts are then served by the > dnsmasq DHCP as those hosts are allowed using the dhcp hosts dir. > > BUT, I'm having a weird issue now. > > All my nodes are able to get an IP from the DHCP at the INSPECTION step, > however, as soon as the inspection step is finished, the mac related file > is once again filled with ",ignore" which prohibits further operations. > > This means as soon as I put that node as available and try to deploy an > "instance" on it (provision the host), it doesn't work as dnsmasq reject > the host boot DHCP requests. > > So, is there a way to instruct the ironic-conductor to edit/allow the host > the way the inspector is able to manipulate this file? > > Did I missed something? > > Thanks a lot! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Mon Jun 13 10:06:04 2022 From: katonalala at gmail.com (Lajos Katona) Date: Mon, 13 Jun 2022 12:06:04 +0200 Subject: [neutron][networking-l2gw][puppet] Status of networking-l2gw project In-Reply-To: References: Message-ID: Hi Takashi Thanks for the headsup, I try to keep l2gw project maintained, but recently had no time to check its status. I check your patch. Lajos Takashi Kajinami ezt ?rta (id?pont: 2022. j?n. 8., Sze, 17:01): > Hello team, > > # I understand networking-l2gw is not under neutron stadium > # but adding [neutron] tag hoping it would attract more attention. > > We found a recent change in neutron[1] broke networking-l2gw[2], and > some jobs in puppet repos and rdo, with l2gw enabled, are failing. > > I proposed a potential fix for the bug[3], but I noticed the project > looks unmaintained according to the following points. > > - No change has been merged for more than 1 year > > - No release was created since Wallaby > > - The zuul setting still points the victoria job template > (openstack-python3-victoria-jobs-neutron) > > I'm wondering if anybody is actually interested in maintaining the project. > If not, I'll propose retiring the plugin support from puppet (and TripleO > because it requires puppet implementation). > > I appreciate any feedback about this. > > [1] https://review.opendev.org/c/openstack/neutron/+/837392 > [2] https://bugs.launchpad.net/networking-l2gw/+bug/1977980 > [3] https://review.opendev.org/c/x/networking-l2gw/+/845141 > > Thank you, > Takashi > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From iurygregory at gmail.com Mon Jun 13 10:15:40 2022 From: iurygregory at gmail.com (Iury Gregory) Date: Mon, 13 Jun 2022 12:15:40 +0200 Subject: [ironic] No weekly meeting today (June 13) Message-ID: Hello Ironicers, I'm still on my way back home from the OIS, I'll not be able to run the upstream meeting I've asked in the irc and people are ok with skipping today's meeting, see you next week! -- *Att[]'s* *Iury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic PTL * *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From gael.therond at bitswalk.com Mon Jun 13 10:41:01 2022 From: gael.therond at bitswalk.com (=?UTF-8?Q?Ga=C3=ABl_THEROND?=) Date: Mon, 13 Jun 2022 12:41:01 +0200 Subject: [VICTORIA][IRONIC] - Can inspect but not deploy In-Reply-To: References: Message-ID: Hi pierre, I?m using a dedicated interface but this interface is the same for all ironic networks inspection/provisioning/cleaning. This interface works fine for inspection, my only issue is the pxe_filter that the ironic inspector process allow during inspection correctly but then tag as disallowed again at the end of the inspection, shouldn?t the deploy process allow the Mac again before booting the node? I can correctly see the conductor instruct the node to boot up using pxe from the kvm console but the BootP process doesn?t load the IPA kernel/initramfs as the dnsmasq pxe discard the request (because of the mac being still tagged as ?,ignore ? within the hostdir file). I?m a bit disappointed by this behavior. Thanks a lot! Le lun. 13 juin 2022 ? 11:41, Pierre Riteau a ?crit : > Hello Ga?l, > > Which network_interface are you using for your nodes? Is your provisioning > network different from the inspection network? > > Pierre > > On Mon, 13 Jun 2022 at 10:36, Ga?l THEROND > wrote: > >> Hi everyone! >> >> I'm dealing with a strange issue today, we deployed IRONIC on a VICTORIA >> platform, we activated the dnsmasq pxe filtering option at the inspector >> level, it works great as only IRONIC listed hosts are then served by the >> dnsmasq DHCP as those hosts are allowed using the dhcp hosts dir. >> >> BUT, I'm having a weird issue now. >> >> All my nodes are able to get an IP from the DHCP at the INSPECTION step, >> however, as soon as the inspection step is finished, the mac related file >> is once again filled with ",ignore" which prohibits further operations. >> >> This means as soon as I put that node as available and try to deploy an >> "instance" on it (provision the host), it doesn't work as dnsmasq reject >> the host boot DHCP requests. >> >> So, is there a way to instruct the ironic-conductor to edit/allow the >> host the way the inspector is able to manipulate this file? >> >> Did I missed something? >> >> Thanks a lot! >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Mon Jun 13 11:14:03 2022 From: pierre at stackhpc.com (Pierre Riteau) Date: Mon, 13 Jun 2022 13:14:03 +0200 Subject: [VICTORIA][IRONIC] - Can inspect but not deploy In-Reply-To: References: Message-ID: Hi Ga?l, I am not talking about the physical network interface, but about the `network_interface` field on on your Ironic nodes: https://docs.openstack.org/ironic/latest/admin/multitenancy.html#network-interfaces Pierre On Mon, 13 Jun 2022 at 12:41, Ga?l THEROND wrote: > Hi pierre, > > I?m using a dedicated interface but this interface is the same for all > ironic networks inspection/provisioning/cleaning. > > This interface works fine for inspection, my only issue is the pxe_filter > that the ironic inspector process allow during inspection correctly but > then tag as disallowed again at the end of the inspection, shouldn?t the > deploy process allow the Mac again before booting the node? > > I can correctly see the conductor instruct the node to boot up using pxe > from the kvm console but the BootP process doesn?t load the IPA > kernel/initramfs as the dnsmasq pxe discard the request (because of the mac > being still tagged as ?,ignore ? within the hostdir file). > > I?m a bit disappointed by this behavior. > > Thanks a lot! > > Le lun. 13 juin 2022 ? 11:41, Pierre Riteau a > ?crit : > >> Hello Ga?l, >> >> Which network_interface are you using for your nodes? Is your >> provisioning network different from the inspection network? >> >> Pierre >> >> On Mon, 13 Jun 2022 at 10:36, Ga?l THEROND >> wrote: >> >>> Hi everyone! >>> >>> I'm dealing with a strange issue today, we deployed IRONIC on a VICTORIA >>> platform, we activated the dnsmasq pxe filtering option at the inspector >>> level, it works great as only IRONIC listed hosts are then served by the >>> dnsmasq DHCP as those hosts are allowed using the dhcp hosts dir. >>> >>> BUT, I'm having a weird issue now. >>> >>> All my nodes are able to get an IP from the DHCP at the INSPECTION step, >>> however, as soon as the inspection step is finished, the mac related file >>> is once again filled with ",ignore" which prohibits further operations. >>> >>> This means as soon as I put that node as available and try to deploy an >>> "instance" on it (provision the host), it doesn't work as dnsmasq reject >>> the host boot DHCP requests. >>> >>> So, is there a way to instruct the ironic-conductor to edit/allow the >>> host the way the inspector is able to manipulate this file? >>> >>> Did I missed something? >>> >>> Thanks a lot! >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From gael.therond at bitswalk.com Mon Jun 13 11:43:00 2022 From: gael.therond at bitswalk.com (=?UTF-8?Q?Ga=C3=ABl_THEROND?=) Date: Mon, 13 Jun 2022 13:43:00 +0200 Subject: [VICTORIA][IRONIC] - Can inspect but not deploy In-Reply-To: References: Message-ID: Aaaah yes, sorry my bad, as we do use Kolla-ansible I forget about this one, so we're using Flat network interface. Le lun. 13 juin 2022 ? 13:14, Pierre Riteau a ?crit : > Hi Ga?l, > > I am not talking about the physical network interface, but about the > `network_interface` field on on your Ironic nodes: > https://docs.openstack.org/ironic/latest/admin/multitenancy.html#network-interfaces > > Pierre > > On Mon, 13 Jun 2022 at 12:41, Ga?l THEROND > wrote: > >> Hi pierre, >> >> I?m using a dedicated interface but this interface is the same for all >> ironic networks inspection/provisioning/cleaning. >> >> This interface works fine for inspection, my only issue is the pxe_filter >> that the ironic inspector process allow during inspection correctly but >> then tag as disallowed again at the end of the inspection, shouldn?t the >> deploy process allow the Mac again before booting the node? >> >> I can correctly see the conductor instruct the node to boot up using pxe >> from the kvm console but the BootP process doesn?t load the IPA >> kernel/initramfs as the dnsmasq pxe discard the request (because of the mac >> being still tagged as ?,ignore ? within the hostdir file). >> >> I?m a bit disappointed by this behavior. >> >> Thanks a lot! >> >> Le lun. 13 juin 2022 ? 11:41, Pierre Riteau a >> ?crit : >> >>> Hello Ga?l, >>> >>> Which network_interface are you using for your nodes? Is your >>> provisioning network different from the inspection network? >>> >>> Pierre >>> >>> On Mon, 13 Jun 2022 at 10:36, Ga?l THEROND >>> wrote: >>> >>>> Hi everyone! >>>> >>>> I'm dealing with a strange issue today, we deployed IRONIC on a >>>> VICTORIA platform, we activated the dnsmasq pxe filtering option at the >>>> inspector level, it works great as only IRONIC listed hosts are then served >>>> by the dnsmasq DHCP as those hosts are allowed using the dhcp hosts dir. >>>> >>>> BUT, I'm having a weird issue now. >>>> >>>> All my nodes are able to get an IP from the DHCP at the INSPECTION >>>> step, however, as soon as the inspection step is finished, the mac related >>>> file is once again filled with ",ignore" which prohibits further >>>> operations. >>>> >>>> This means as soon as I put that node as available and try to deploy an >>>> "instance" on it (provision the host), it doesn't work as dnsmasq reject >>>> the host boot DHCP requests. >>>> >>>> So, is there a way to instruct the ironic-conductor to edit/allow the >>>> host the way the inspector is able to manipulate this file? >>>> >>>> Did I missed something? >>>> >>>> Thanks a lot! >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Mon Jun 13 12:29:10 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Mon, 13 Jun 2022 17:59:10 +0530 Subject: Regarding Policy.json entries for glance image update not working for a user In-Reply-To: References: Message-ID: hi Team, Any thoughts on this Regards Adivya Singh On Sat, Jun 11, 2022 at 12:40 AM Adivya Singh wrote: > Hi Team, > > I have a use case where I have to give a user restriction on updating the > image properties as a member. > > I have created a policy Json file and give the modify_image rule to the > particular role, but still it is not working > > "modify_image": "role:user", This role is created in OpenStack. > > but still it is failing while updating properties with a particular user > assigned to a role as "access denied" and unauthorized access > > Regards > Adivya Singh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From romain.chanu at univ-lyon1.fr Mon Jun 13 12:41:48 2022 From: romain.chanu at univ-lyon1.fr (CHANU ROMAIN) Date: Mon, 13 Jun 2022 12:41:48 +0000 Subject: [manila] CephFS NFS high availability cluster Message-ID: Hello, I added Manila service with CephFS NFS driver to my openstack cluster. Everything works fine but I would like to add 2 nfs-ganesha servers to ensure high availability to the service. I configured haproxy to forward 2049 to ganesha backend but Manila cephFS NFS provides only IP restriction and see only haproxy's IP address. To make it work you have to add haproxy to allowed ip but it means everyone can access the share. So currently the only way I found out is to use pacemaker to set public vip to a running nfs-ganesha node. Could you confirm that is not possible to provide an active/active nfs-ganesha cluster with manila cephfs NFS driver? Best regards, Romain -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4513 bytes Desc: not available URL: From gael.therond at bitswalk.com Mon Jun 13 12:44:08 2022 From: gael.therond at bitswalk.com (=?UTF-8?Q?Ga=C3=ABl_THEROND?=) Date: Mon, 13 Jun 2022 14:44:08 +0200 Subject: [VICTORIA][IRONIC] - Can inspect but not deploy In-Reply-To: References: Message-ID: Alright, I just tested to disable the pxe_filter option and it works, the host is then deployed using the glance image and neutron network settings correctly. I think there is either a bug or maybe something that I didn?t catch up but it?s pretty annoying. I?ll propose a patch to kolla-ansible as well as I discovered that you can?t disable this feature for now due to the template shape. Any ideas about why the conductor isn?t able to whitelist this node ? Thanks a lot! Le lun. 13 juin 2022 ? 13:43, Ga?l THEROND a ?crit : > Aaaah yes, sorry my bad, as we do use Kolla-ansible I forget about this > one, so we're using Flat network interface. > > Le lun. 13 juin 2022 ? 13:14, Pierre Riteau a > ?crit : > >> Hi Ga?l, >> >> I am not talking about the physical network interface, but about the >> `network_interface` field on on your Ironic nodes: >> https://docs.openstack.org/ironic/latest/admin/multitenancy.html#network-interfaces >> >> Pierre >> >> On Mon, 13 Jun 2022 at 12:41, Ga?l THEROND >> wrote: >> >>> Hi pierre, >>> >>> I?m using a dedicated interface but this interface is the same for all >>> ironic networks inspection/provisioning/cleaning. >>> >>> This interface works fine for inspection, my only issue is the >>> pxe_filter that the ironic inspector process allow during inspection >>> correctly but then tag as disallowed again at the end of the inspection, >>> shouldn?t the deploy process allow the Mac again before booting the node? >>> >>> I can correctly see the conductor instruct the node to boot up using pxe >>> from the kvm console but the BootP process doesn?t load the IPA >>> kernel/initramfs as the dnsmasq pxe discard the request (because of the mac >>> being still tagged as ?,ignore ? within the hostdir file). >>> >>> I?m a bit disappointed by this behavior. >>> >>> Thanks a lot! >>> >>> Le lun. 13 juin 2022 ? 11:41, Pierre Riteau a >>> ?crit : >>> >>>> Hello Ga?l, >>>> >>>> Which network_interface are you using for your nodes? Is your >>>> provisioning network different from the inspection network? >>>> >>>> Pierre >>>> >>>> On Mon, 13 Jun 2022 at 10:36, Ga?l THEROND >>>> wrote: >>>> >>>>> Hi everyone! >>>>> >>>>> I'm dealing with a strange issue today, we deployed IRONIC on a >>>>> VICTORIA platform, we activated the dnsmasq pxe filtering option at the >>>>> inspector level, it works great as only IRONIC listed hosts are then served >>>>> by the dnsmasq DHCP as those hosts are allowed using the dhcp hosts dir. >>>>> >>>>> BUT, I'm having a weird issue now. >>>>> >>>>> All my nodes are able to get an IP from the DHCP at the INSPECTION >>>>> step, however, as soon as the inspection step is finished, the mac related >>>>> file is once again filled with ",ignore" which prohibits further >>>>> operations. >>>>> >>>>> This means as soon as I put that node as available and try to deploy >>>>> an "instance" on it (provision the host), it doesn't work as dnsmasq reject >>>>> the host boot DHCP requests. >>>>> >>>>> So, is there a way to instruct the ironic-conductor to edit/allow the >>>>> host the way the inspector is able to manipulate this file? >>>>> >>>>> Did I missed something? >>>>> >>>>> Thanks a lot! >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Mon Jun 13 12:57:48 2022 From: mnaser at vexxhost.com (Mohammed Naser) Date: Mon, 13 Jun 2022 08:57:48 -0400 Subject: [qa][refstack][python-tempestconf] adding image Message-ID: Hi there, I'm wondering if the python-tempestconf team is open to publishing Docker images that contain `python-tempestconf`. This is something that we have a need for downstream but we'd be happy to contribute upstream. The main thing is that we'd need to setup an organization on Docker hub to publish things to. Is that something that the team is interested in? We can contribute it if that is the case. Thanks, Mohammed -- Mohammed Naser VEXXHOST, Inc. From rosmaita.fossdev at gmail.com Mon Jun 13 12:58:47 2022 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Mon, 13 Jun 2022 08:58:47 -0400 Subject: Regarding Policy.json entries for glance image update not working for a user In-Reply-To: References: Message-ID: On 6/13/22 8:29 AM, Adivya Singh wrote: > hi Team, > > Any thoughts on this H Adivya, Please supply some more information, for example: - which openstack release you are using - the full API request you are making to modify the image - the full API response you receive - whether the user with "role:user" is in the same project that owns the image - debug level log extract for this call if you have it - anything else that could be relevant, for example, have you modified any other policies, and if so, what values are you using now? cheers, brian > > Regards > Adivya Singh > > On Sat, Jun 11, 2022 at 12:40 AM Adivya Singh > wrote: > > Hi Team, > > I have a use case where I have to give a user restriction on > updating the image properties as a member. > > I have created a policy Json file and give the modify_image rule to > the particular role, but still it is not working > > "modify_image": "role:user", This role is created in OpenStack. > > but still it is failing while updating properties with a > particular?user assigned to a role as "access denied" and > unauthorized access > > Regards > Adivya Singh > From abishop at redhat.com Mon Jun 13 13:06:06 2022 From: abishop at redhat.com (Alan Bishop) Date: Mon, 13 Jun 2022 06:06:06 -0700 Subject: Regarding Policy.json entries for glance image update not working for a user In-Reply-To: References: Message-ID: On Mon, Jun 13, 2022 at 6:00 AM Brian Rosmaita wrote: > On 6/13/22 8:29 AM, Adivya Singh wrote: > > hi Team, > > > > Any thoughts on this > > H Adivya, > > Please supply some more information, for example: > > - which openstack release you are using > - the full API request you are making to modify the image > - the full API response you receive > - whether the user with "role:user" is in the same project that owns the > image > - debug level log extract for this call if you have it > - anything else that could be relevant, for example, have you modified > any other policies, and if so, what values are you using now? > Also bear in mind that the default policy_file name is "policy.yaml" (not .json). You either need to provide a policy.yaml file, or override the policy_file setting if you really want to use policy.json. Alan cheers, > brian > > > > > Regards > > Adivya Singh > > > > On Sat, Jun 11, 2022 at 12:40 AM Adivya Singh > > wrote: > > > > Hi Team, > > > > I have a use case where I have to give a user restriction on > > updating the image properties as a member. > > > > I have created a policy Json file and give the modify_image rule to > > the particular role, but still it is not working > > > > "modify_image": "role:user", This role is created in OpenStack. > > > > but still it is failing while updating properties with a > > particular user assigned to a role as "access denied" and > > unauthorized access > > > > Regards > > Adivya Singh > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Mon Jun 13 13:07:52 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Mon, 13 Jun 2022 18:37:52 +0530 Subject: 504 Gateway time out error while loading OpenStack Projects Message-ID: hi Team, We have Xena release installed on Ubuntu Openstack, Lately we are getting "504 Gateway Timeout error ", As the we moved from 1 node to 3 node Open Stack i do not see any error in haproxy , neither there is a time out when i ping external-VIP-IP configured. Apache 2 are also loaded, tried to delete the Horizon Container and created it again, but no errors, Still the same errors. Any guesses on that Regards Adivya Singh -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricolin at ricolky.com Mon Jun 13 13:10:55 2022 From: ricolin at ricolky.com (Rico Lin) Date: Mon, 13 Jun 2022 21:10:55 +0800 Subject: Propose to add Takashi Kajinami as heat core reviewer Message-ID: Hi all I want to suggest adding Takashi Kajinami as heat core reviewer. Who help provide valid reviews and fixes for issues on heat. Will be great to have Takashi Kajinami join heat core reviewer team and help. *Rico Lin* -------------- next part -------------- An HTML attachment was scrubbed... URL: From nblkhan.pool at gmail.com Mon Jun 13 13:14:42 2022 From: nblkhan.pool at gmail.com (Nabeel Tariq Khan) Date: Mon, 13 Jun 2022 18:14:42 +0500 Subject: [Magnum resize Master node root volume/docker volume or resize flavor] Message-ID: Hi, Is it possible to resize the root volume or docker volume or resize the flavor of master or minion node on openstack magnum? regards, Nabeel Tariq Khan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ramishra at redhat.com Mon Jun 13 13:47:48 2022 From: ramishra at redhat.com (Rabi Mishra) Date: Mon, 13 Jun 2022 19:17:48 +0530 Subject: Propose to add Takashi Kajinami as heat core reviewer In-Reply-To: References: Message-ID: On Mon, Jun 13, 2022 at 6:43 PM Rico Lin wrote: > Hi all > > I want to suggest adding Takashi Kajinami as heat > core reviewer. > Who help provide valid reviews and fixes for issues on heat. > Will be great to have Takashi Kajinami join heat core reviewer team and > help. > +1, Good addition, he is actively involved for some time and contributes to keep the project healthy . > > *Rico Lin* > -- Regards, Rabi Mishra -------------- next part -------------- An HTML attachment was scrubbed... URL: From tkajinam at redhat.com Wed Jun 8 14:54:31 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Wed, 8 Jun 2022 23:54:31 +0900 Subject: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version In-Reply-To: References: Message-ID: You'd need to check whether that oc_provisioning network is part of network_data and is mapped to the Controller role (might be a different role if you completely customize roles). If you want to use the provisioning network used by undercloud then it should be ctlplane instead. Although bugzilla is open, we maintain upstream bugs for OpenStack components not in bugzilla but in launchpad, so please report a bug in launchpad instead. https://bugs.launchpad.net/tripleo On Wed, Jun 8, 2022 at 11:48 PM Swogat Pradhan wrote: > Hi Lokendra, > I am not sure why this issue is caused, can you please check the > tripleo_dnsmasq service and the tripleo_ironic_pxe service in your director > node? IF the services are running fine then please report this here for > fast resolvement https://bugzilla.redhat.com/ > > With regards, > Swogat Pradhan > > On Tue, Jun 7, 2022 at 12:54 PM Lokendra Rathour < > lokendrarathour at gmail.com> wrote: > >> Hi Swogat, >> Yes after correcting the j2 template, as suggested by you, it started >> working. Thanks a lot once again. >> Now the Overcloud is deployed. >> >> Further, as a next requirement, we were also trying to add an *additional >> Node as a bare-metal Instance* on the overcloud. As it is the composable >> network approach, we add an entry in parameters default for ServiceNetMap, >> as like below in environments/network-environments.yaml >> >> parameter_defaults: >> # This section is where deployment-specific configuration is done >> ServiceNetMap: >> IronicApiNetwork: oc_provisioning >> IronicNetwork: oc_provisioning >> >> this is now passed as in the environment.yaml as parameter_defaults, but >> somehow we are not able to deploy the setup. it gives the error for >> containers. >> >> >> *2022-06-06 19:08:37.560800 | 525400fd-f24d-9703-b0e7-0000000084b2 | >> FATAL | Manage container systemd services and cleanup old systemd >> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >> overcloud-controller-0 | error={"changed": false, "msg": "Service >> ironic_pxe_tftp has not started yet"}* >> >> >> and if we remove this setting as stated above, the deployment is >> successful, but at the time of node provisioning it is not able to reach >> the TFTP server to pull files, mainly because of the composable network >> (oc_provisioning), we faced this issue in Train, so we did the above config >> changes and it working. But as it looks something more is needed in >> Wallaby for Baremetal provisioning as an instance. >> >> Please share your learnings with the reference to it. Thanks once again >> for your all-time support. >> reference link: Bare Metal Instances in Overcloud ? TripleO 3.0.0 >> documentation (openstack.org) >> >> >> -Lokendra >> >> >> On Wed, Jun 1, 2022 at 12:19 AM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Ok Swogat, >>> Thanks once again. >>> i will try this approach once and will let you know. >>> >>> >>> On Wed, 1 Jun 2022, 00:04 Swogat Pradhan, >>> wrote: >>> >>>> Hi Lokendra, >>>> Like i said, >>>> > NOTE: when passing --network-config parameter in node provision step, >>>> it creates a directory in /etc/os-net-config and in it creates a file >>>> > config.yaml, do check the indentation of that file. (in my case the >>>> indentation was wrong when i was using bonding everytime, so i had to >>>> > manually change the script and run a while loop and then my node >>>> provision step was successful) >>>> this parameter --network-config creates a config.yaml(network config) >>>> file in /etc/os-net-config directory in the overcloud nodes and then >>>> ansible tries to apply the network config from the generated config file. >>>> And in wallaby version if you are using bonding then the syntax in the >>>> generated conf is wrong (atleast was in my case) and then the ansible tries >>>> to apply the network config (with wrong syntax) so your overcloud nodes >>>> become unavailable. >>>> >>>> Please run node provision separately, and keep on monitoring >>>> "metalsmith list" command. Once IP is assigned to the overcloud nodes, ssh >>>> to the nodes and assign a password to heat-admin user so that even if the >>>> network becomes unavailable you still will be able to access the nodes via >>>> console access, then you can visit /etc/os-net-config directory and verify >>>> the config.yaml file. >>>> >>>> With regards, >>>> Swogat pradhan. >>>> >>>> On Tue, May 31, 2022 at 11:56 PM Lokendra Rathour < >>>> lokendrarathour at gmail.com> wrote: >>>> >>>>> Hi Swogat, >>>>> Thanks once again for your input it really helped much. >>>>> >>>>> instead of running mentioned those three provisioning steps i used >>>>> alternate method and passed directly in deploy command. now my current >>>>> deploy command is: >>>>> >>>>> openstack overcloud deploy --templates \ >>>>> --networks-file /home/stack/templates/custom_network_data.yaml \ >>>>> --vip-file /home/stack/templates/custom_vip_data.yaml \ >>>>> --baremetal-deployment >>>>> /home/stack/templates/overcloud-baremetal-deploy.yaml \ >>>>> --network-config \ >>>>> -e /home/stack/templates/environment.yaml \ >>>>> -e >>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml >>>>> \ >>>>> -e >>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml >>>>> \ >>>>> -e >>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml >>>>> \ >>>>> -e /home/stack/templates/ironic-config.yaml \ >>>>> -e >>>>> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ >>>>> -e >>>>> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ >>>>> -e /home/stack/containers-prepare-parameter.yaml >>>>> >>>>> The files as suggested by you are well created. >>>>> >>>>> But once we run the deployment I get this error for all nodes: >>>>> >>>>> 0:00:16.064781 | 1.16s >>>>> 2022-05-31 19:13:00.276954 | 525400ef-b928-9ded-fecc-000000000094 | >>>>> TASK | Run tripleo_os_net_config_module with network_config >>>>> 2022-05-31 19:40:30.061582 | 525400ef-b928-9ded-fecc-000000000094 | >>>>> FATAL | Run tripleo_os_net_config_module with network_config | >>>>> overcloud-controller-1 | error={"msg": "Data could not be sent to remote >>>>> host \"30.30.30.117\". Make sure this host can be reached over ssh: ssh: >>>>> connect to host 30.30.30.117 port 22: No route to host\r\n"} >>>>> 2022- >>>>> >>>>> >>>>> Baremetal node list are showing in as active. >>>>> >>>>> (undercloud) [stack at undercloud ~]$ openstack baremetal node list >>>>> /usr/lib64/python3.6/site-packages/_yaml/__init__.py:23: >>>>> DeprecationWarning: The _yaml extension module is now located at yaml._yaml >>>>> and its location is subject to change. To use the LibYAML-based parser and >>>>> emitter, import from `yaml`: `from yaml import CLoader as Loader, CDumper >>>>> as Dumper`. >>>>> DeprecationWarning >>>>> >>>>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>>>> | UUID | Name | Instance UUID >>>>> | Power State | Provisioning State | Maintenance | >>>>> >>>>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>>>> | 1a4d873c-f9f7-4504-a3af-92c11f954171 | node-a | >>>>> 901453a1-183f-4de8-aaab-0f38be2be455 | power on | active | >>>>> False | >>>>> | d18610fc-9532-410c-918e-8efc326c89f8 | node-b | >>>>> d059b94a-8357-4f8e-a0d8-15a24b0c1afe | power on | active | >>>>> False | >>>>> | b69f2d5a-5b18-4453-8843-15c6af79aca0 | node-c | >>>>> f196ef3a-7950-47b9-a5ae-751f06b18f75 | power on | active | >>>>> False | >>>>> | 8a38c584-f812-4ebc-a0b1-4299f0917637 | node-d | >>>>> 1636517c-2ab2-43d7-8205-9f02c5290207 | power on | active | >>>>> False | >>>>> >>>>> +--------------------------------------+--------+--------------------------------------+-------------+--------------------+-------------+ >>>>> >>>>> Some config is missing it seems, please check once and advise. >>>>> >>>>> >>>>> On Tue, May 31, 2022 at 11:40 AM Swogat Pradhan < >>>>> swogatpradhan22 at gmail.com> wrote: >>>>> >>>>>> Hi Lokendra, >>>>>> You need to generate another file also in the following step: openstack >>>>>> overcloud node provision --stack overcloud --overcloud-ssh-key >>>>>> /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml also you >>>>>> need to pass another parameter --network-config. >>>>>> example: >>>>>> openstack overcloud node provision --stack overcloud >>>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa *--network-config* *--output >>>>>> overcloud-baremetal-deployed.yaml* overcloud-baremetal-deploy.yaml >>>>>> >>>>>> And then all these output files will be passed on to the openstack >>>>>> overcloud deploy command. >>>>>> NOTE: when passing --network-config parameter in node provision step, >>>>>> it creates a directory in /etc/os-net-config and in it creates a file >>>>>> config.yaml, do check the indentation of that file. (in my case the >>>>>> indentation was wrong when i was using bondind everytime, so i had to >>>>>> manually change the script and run a while loop and then my node provision >>>>>> step was successful) >>>>>> >>>>>> On Tue, May 31, 2022 at 8:59 AM Lokendra Rathour < >>>>>> lokendrarathour at gmail.com> wrote: >>>>>> >>>>>>> Hi Swogat, >>>>>>> I tried generating the scripts as used by you in your deployments >>>>>>> using the >>>>>>> >>>>>>> >>>>>>> #openstack overcloud network provision --stack overcloud --output >>>>>>> networks-deployed-environment.yaml custom_network_data.yaml >>>>>>> # openstack overcloud network vip provision --stack overcloud >>>>>>> --output vip-deployed-environment.yaml custom_vip_data.yaml >>>>>>> # openstack overcloud node provision --stack overcloud >>>>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa >>>>>>> overcloud-baremetal-deploy.yaml >>>>>>> >>>>>>> and used the first two in the final deployment script, but it gives >>>>>>> the error: >>>>>>> >>>>>>> heatclient.exc.HTTPInternalServerError: ERROR: Internal Error >>>>>>> 2022-05-30 14:14:39.772 479668 ERROR >>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent >>>>>>> call last):\n', ' File "/usr/lib/python3.6/ted_stack\n >>>>>>> nested_stack.validate()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in >>>>>>> wrapper\n result = f(*args, ine 969, in validate\n result = >>>>>>> res.validate()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/openstack/neutron/port.py", >>>>>>> line 454site-packages/heat/engine/resources/openstack/neutron/neutron.py", >>>>>>> line 43, in validate\n res = super(NeutronResource, self).validate()\n', >>>>>>> ' File "/un return self.validate_template()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resource.py", line 1882, in >>>>>>> validate_template\n self.t.rpy", line 200, in >>>>>>> _validate_service_availability\n raise ex\n', >>>>>>> 'heat.common.exception.ResourceTypeUnavailable: HEAT-E99001 Service neutron >>>>>>> is not avaieutron network endpoint is not in service catalog.\n', '\nDuring >>>>>>> handling of the above exception, another exception occurred:\n\n', >>>>>>> 'Traceback (most recens/stack_resource.py", line 75, in >>>>>>> validate_nested_stack\n nested_stack.validate()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py"thon3.6/site-packages/heat/engine/stack.py", >>>>>>> line 969, in validate\n result = res.validate()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/ateResource, >>>>>>> self).validate()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", >>>>>>> line 65, in validate\n self.validources/stack_resource.py", line 81, in >>>>>>> validate_nested_stack\n ex, path=[self.stack.t.RESOURCES, path])\n', >>>>>>> 'heat.common.exception.StackValidationFaileeploy/overcloud/tripleo-heat-templates/deployed-server/deployed-server.yaml>: >>>>>>> HEAT-E99001 Service neutron is not available for resource type >>>>>>> OS::TripleO::vice catalog.\n', '\nDuring handling of the above exception, >>>>>>> another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' >>>>>>> File "/usr/lib/pline 320, in validate_nested_stack\n >>>>>>> nested_stack.validate()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in >>>>>>> wrappe/heat/engine/stack.py", line 969, in validate\n result = >>>>>>> res.validate()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/template_relidate()\n', >>>>>>> ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/stack_resource.py", >>>>>>> line 65, in validate\n self.validate_nested_stack()\n'.py", line 81, in >>>>>>> validate_nested_stack\n ex, path=[self.stack.t.RESOURCES, path])\n', >>>>>>> 'heat.common.exception.StackValidationFailed: >>>>>>> ResourceTypeUnavaimplates/puppet/compute-role.yaml>.resources.NovaCompute>>>>>> reason: neutron network endpoint is not in service catalog.\n', '\nDuring >>>>>>> handling of the above exception, >>>>>>> another/lib/python3.6/site-packages/heat/common/context.py", line 416, in >>>>>>> wrapped\n return func(self, ctx, *args, **kwargs)\n', ' File >>>>>>> "/usr/lib/python3.6/sirce_name, template_id)\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/service.py", line 756, in >>>>>>> _parse_template_and_validate_stack\n stack.v line 160, in wrapper\n >>>>>>> result = f(*args, **kwargs)\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/stack.py", line 969, in >>>>>>> validate\n resesources/stack_resource.py", line 65, in validate\n >>>>>>> self.validate_nested_stack()\n', ' File >>>>>>> "/usr/lib/python3.6/site-packages/heat/engine/resources/oph=[self.stack.t.RESOURCES, >>>>>>> path])\n', 'heat.common.exception.StackValidationFailed: >>>>>>> ResourceTypeUnavailable: >>>>>>> resources.Compute.resources.0.resources.NovaCompute: >>>>>>> HEAT-E9900::ControlPlanePort, reason: neutron network endpoint is not in >>>>>>> service catalog.\n']. >>>>>>> >>>>>>> Request you to check once, please. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, May 30, 2022 at 11:06 AM Lokendra Rathour < >>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>> >>>>>>>> Hi Swogat, >>>>>>>> Thanks once again. >>>>>>>> >>>>>>>> with the files as shown below I am running the overcloud deploy for >>>>>>>> wallaby using this command: >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud ~]$ cat >>>>>>>> deploy_overcloud_working_1.sh >>>>>>>> openstack overcloud deploy --templates \ >>>>>>>> -n /home/stack/templates/network_data.yaml \ >>>>>>>> -r /home/stack/templates/roles_data.yaml \ >>>>>>>> -e /home/stack/templates/environment.yaml \ >>>>>>>> -e /home/stack/templates/environments/network-isolation.yaml \ >>>>>>>> -e /home/stack/templates/environments/network-environment.yaml \ >>>>>>>> -e >>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-conductor.yaml >>>>>>>> \ >>>>>>>> -e >>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-inspector.yaml >>>>>>>> \ >>>>>>>> -e >>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml >>>>>>>> \ >>>>>>>> -e /home/stack/templates/ironic-config.yaml \ >>>>>>>> -e >>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml \ >>>>>>>> -e >>>>>>>> /usr/share/openstack-tripleo-heat-templates/environments/podman.yaml \ >>>>>>>> -e /home/stack/containers-prepare-parameter.yaml >>>>>>>> (undercloud) [stack at undercloud ~]$ >>>>>>>> >>>>>>>> >>>>>>>> This deployment is on ipv6 using triple0 wallaby, templates, as >>>>>>>> mentioned below, are generated using rendering steps and the >>>>>>>> network_data.yaml the roles_data.yaml >>>>>>>> Steps used to render the templates: >>>>>>>> cd /usr/share/openstack-tripleo-heat-templates/ >>>>>>>> ./tools/process-templates.py -o >>>>>>>> ~/openstack-tripleo-heat-templates-rendered_at_wallaby -n >>>>>>>> /home/stack/templates/network_data.yaml -r >>>>>>>> /home/stack/templates/roles_data.yaml >>>>>>>> >>>>>>>> *Now if i try adding the related to VIP port I do get the error as:* >>>>>>>> >>>>>>>> 2022-05-30 10:37:12.792 979387 WARNING >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template >>>>>>>> to file: >>>>>>>> /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/controller-role.yaml >>>>>>>> 2022-05-30 10:37:12.792 979387 WARNING >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] rendering j2 template >>>>>>>> to file: >>>>>>>> /home/stack/overcloud-deploy/overcloud/tripleo-heat-templates/puppet/compute-role.yaml >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured >>>>>>>> while running the command: ValueError: The environment is not a valid YAML >>>>>>>> mapping data type. >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent >>>>>>>> call last): >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, >>>>>>>> self).run(parsed_args) >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in >>>>>>>> run >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, >>>>>>>> self).run(parsed_args) >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = >>>>>>>> self.take_action(parsed_args) or 0 >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>>>> line 1189, in take_action >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud stack, parsed_args, >>>>>>>> new_tht_root, user_tht_root) >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>>>> line 227, in create_env_files >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud created_env_files, >>>>>>>> parsed_args, new_tht_root, user_tht_root) >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", >>>>>>>> line 204, in build_image_params >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud cleanup=(not >>>>>>>> parsed_args.no_cleanup)) >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 1929, in >>>>>>>> process_multiple_environments >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud env_path=env_path, >>>>>>>> include_env_in_files=include_env_in_files) >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/heatclient/common/template_utils.py", >>>>>>>> line 326, in process_environment_and_files >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud env = >>>>>>>> environment_format.parse(raw_env) >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud File >>>>>>>> "/usr/lib/python3.6/site-packages/heatclient/common/environment_format.py", >>>>>>>> line 50, in parse >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud raise >>>>>>>> ValueError(_('The environment is not a valid ' >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud ValueError: The >>>>>>>> environment is not a valid YAML mapping data type. >>>>>>>> 2022-05-30 10:37:14.455 979387 ERROR >>>>>>>> tripleoclient.v1.overcloud_deploy.DeployOvercloud >>>>>>>> 2022-05-30 10:37:14.457 979387 ERROR openstack [-] The environment >>>>>>>> is not a valid YAML mapping data type. >>>>>>>> 2022-05-30 10:37:14.457 979387 INFO osc_lib.shell [-] END return >>>>>>>> value: 1 >>>>>>>> (undercloud) [stack at undercloud ~]$ >>>>>>>> >>>>>>>> This is more of a syntax error where it is not able to understand >>>>>>>> the passed VIP data file: >>>>>>>> >>>>>>>> undercloud) [stack at undercloud ~]$ cat >>>>>>>> /home/stack/templates/vip-data-default-network-isolation.yaml >>>>>>>> - >>>>>>>> dns_name: overcloud >>>>>>>> network: internal_api >>>>>>>> - >>>>>>>> dns_name: overcloud >>>>>>>> network: external >>>>>>>> - >>>>>>>> dns_name: overcloud >>>>>>>> network: ctlplane >>>>>>>> - >>>>>>>> dns_name: overcloud >>>>>>>> network: oc_provisioning >>>>>>>> - >>>>>>>> dns_name: overcloud >>>>>>>> network: j3mgmt >>>>>>>> >>>>>>>> >>>>>>>> Please advise, also please note that similar templates generated in >>>>>>>> prior releases such as train/ussuri works perfectly. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Please check the list of *templates *files: >>>>>>>> >>>>>>>> drwxr-xr-x. 2 stack stack 68 May 30 09:22 environments >>>>>>>> -rw-r--r--. 1 stack stack 265 May 27 13:47 environment.yaml >>>>>>>> -rw-rw-r--. 1 stack stack 297 May 27 13:47 init-repo.yaml >>>>>>>> -rw-r--r--. 1 stack stack 570 May 27 13:47 ironic-config.yaml >>>>>>>> drwxrwxr-x. 4 stack stack 4096 May 27 13:53 network >>>>>>>> -rw-r--r--. 1 stack stack 6370 May 27 14:26 network_data.yaml >>>>>>>> -rw-r--r--. 1 stack stack 11137 May 27 13:53 roles_data.yaml >>>>>>>> -rw-r--r--. 1 stack stack 234 May 30 09:23 >>>>>>>> vip-data-default-network-isolation.yaml >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ cat environment.yaml >>>>>>>> >>>>>>>> parameter_defaults: >>>>>>>> OvercloudControllerFlavor: control >>>>>>>> OvercloudComputeFlavor: compute >>>>>>>> ControllerCount: 3 >>>>>>>> ComputeCount: 1 >>>>>>>> TimeZone: 'Asia/Kolkata' >>>>>>>> NtpServer: ['30.30.30.3'] >>>>>>>> NeutronBridgeMappings: datacentre:br-tenant >>>>>>>> NeutronFlatNetworks: datacentre >>>>>>>> (undercloud) [stack at undercloud templates]$ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ cat ironic-config.yaml >>>>>>>> >>>>>>>> parameter_defaults: >>>>>>>> NovaSchedulerDefaultFilters: >>>>>>>> - AggregateInstanceExtraSpecsFilter >>>>>>>> - AvailabilityZoneFilter >>>>>>>> - ComputeFilter >>>>>>>> - ComputeCapabilitiesFilter >>>>>>>> - ImagePropertiesFilter >>>>>>>> IronicEnabledHardwareTypes: >>>>>>>> - ipmi >>>>>>>> - redfish >>>>>>>> IronicEnabledPowerInterfaces: >>>>>>>> - ipmitool >>>>>>>> - redfish >>>>>>>> IronicEnabledManagementInterfaces: >>>>>>>> - ipmitool >>>>>>>> - redfish >>>>>>>> IronicCleaningDiskErase: metadata >>>>>>>> IronicIPXEEnabled: true >>>>>>>> IronicInspectorSubnets: >>>>>>>> - ip_range: 172.23.3.100,172.23.3.150 >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ cat network_data.yaml >>>>>>>> >>>>>>>> - name: J3Mgmt >>>>>>>> name_lower: j3mgmt >>>>>>>> vip: true >>>>>>>> vlan: 400 >>>>>>>> ipv6: true >>>>>>>> ipv6_subnet: 'fd80:fd00:fd00:4000::/64' >>>>>>>> ipv6_allocation_pools: [{'start': 'fd80:fd00:fd00:4000::10', >>>>>>>> 'end': 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] >>>>>>>> mtu: 9000 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> - name: InternalApi >>>>>>>> name_lower: internal_api >>>>>>>> vip: true >>>>>>>> vlan: 418 >>>>>>>> ipv6: true >>>>>>>> ipv6_subnet: 'fd00:fd00:fd00:2000::/64' >>>>>>>> ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:2000::10', >>>>>>>> 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] >>>>>>>> mtu: 9000 >>>>>>>> >>>>>>>> >>>>>>>> - name: External >>>>>>>> vip: true >>>>>>>> name_lower: external >>>>>>>> vlan: 408 >>>>>>>> ipv6: true >>>>>>>> gateway_ipv6: 'fd00:fd00:fd00:9900::1' >>>>>>>> ipv6_subnet: 'fd00:fd00:fd00:9900::/64' >>>>>>>> ipv6_allocation_pools: [{'start': 'fd00:fd00:fd00:9900::10', >>>>>>>> 'end': 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] >>>>>>>> mtu: 9000 >>>>>>>> >>>>>>>> >>>>>>>> - name: OCProvisioning >>>>>>>> vip: true >>>>>>>> name_lower: oc_provisioning >>>>>>>> vlan: 412 >>>>>>>> ip_subnet: '172.23.3.0/24' >>>>>>>> allocation_pools: [{'start': '172.23.3.10', 'end': '172.23.3.50'}] >>>>>>>> mtu: 9000 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ cat roles_data.yaml >>>>>>>> >>>>>>>> >>>>>>>> ############################################################################### >>>>>>>> # File generated by TripleO >>>>>>>> >>>>>>>> ############################################################################### >>>>>>>> >>>>>>>> ############################################################################### >>>>>>>> # Role: Controller >>>>>>>> # >>>>>>>> >>>>>>>> ############################################################################### >>>>>>>> - name: Controller >>>>>>>> description: | >>>>>>>> Controller role that has all the controller services loaded and >>>>>>>> handles >>>>>>>> Database, Messaging, and Network functions. >>>>>>>> CountDefault: 1 >>>>>>>> tags: >>>>>>>> - primary >>>>>>>> - controller >>>>>>>> # Create external Neutron bridge for SNAT (and floating IPs >>>>>>>> when using >>>>>>>> # ML2/OVS without DVR) >>>>>>>> - external_bridge >>>>>>>> networks: >>>>>>>> External: >>>>>>>> subnet: external_subnet >>>>>>>> InternalApi: >>>>>>>> subnet: internal_api_subnet >>>>>>>> OCProvisioning: >>>>>>>> subnet: oc_provisioning_subnet >>>>>>>> J3Mgmt: >>>>>>>> subnet: j3mgmt_subnet >>>>>>>> >>>>>>>> >>>>>>>> # For systems with both IPv4 and IPv6, you may specify a gateway >>>>>>>> network for >>>>>>>> # each, such as ['ControlPlane', 'External'] >>>>>>>> default_route_networks: ['External'] >>>>>>>> HostnameFormatDefault: '%stackname%-controller-%index%' >>>>>>>> RoleParametersDefault: >>>>>>>> OVNCMSOptions: "enable-chassis-as-gw" >>>>>>>> # Deprecated & backward-compatible values (FIXME: Make parameters >>>>>>>> consistent) >>>>>>>> # Set uses_deprecated_params to True if any deprecated params are >>>>>>>> used. >>>>>>>> uses_deprecated_params: True >>>>>>>> deprecated_param_extraconfig: 'controllerExtraConfig' >>>>>>>> deprecated_param_flavor: 'OvercloudControlFlavor' >>>>>>>> deprecated_param_image: 'controllerImage' >>>>>>>> deprecated_nic_config_name: 'controller.yaml' >>>>>>>> update_serial: 1 >>>>>>>> ServicesDefault: >>>>>>>> - OS::TripleO::Services::Aide >>>>>>>> - OS::TripleO::Services::AodhApi >>>>>>>> - OS::TripleO::Services::AodhEvaluator >>>>>>>> >>>>>>>> .. >>>>>>>> . >>>>>>>> >>>>>>>> >>>>>>>> ..############################################################################### >>>>>>>> # Role: Compute >>>>>>>> # >>>>>>>> >>>>>>>> ############################################################################### >>>>>>>> - name: Compute >>>>>>>> description: | >>>>>>>> Basic Compute Node role >>>>>>>> CountDefault: 1 >>>>>>>> # Create external Neutron bridge (unset if using ML2/OVS without >>>>>>>> DVR) >>>>>>>> tags: >>>>>>>> - compute >>>>>>>> - external_bridge >>>>>>>> networks: >>>>>>>> InternalApi: >>>>>>>> subnet: internal_api_subnet >>>>>>>> J3Mgmt: >>>>>>>> subnet: j3mgmt_subnet >>>>>>>> HostnameFormatDefault: '%stackname%-novacompute-%index%' >>>>>>>> RoleParametersDefault: >>>>>>>> FsAioMaxNumber: 1048576 >>>>>>>> TunedProfileName: "virtual-host" >>>>>>>> # Deprecated & backward-compatible values (FIXME: Make parameters >>>>>>>> consistent) >>>>>>>> # Set uses_deprecated_params to True if any deprecated params are >>>>>>>> used. >>>>>>>> # These deprecated_params only need to be used for existing roles >>>>>>>> and not for >>>>>>>> # composable roles. >>>>>>>> uses_deprecated_params: True >>>>>>>> deprecated_param_image: 'NovaImage' >>>>>>>> deprecated_param_extraconfig: 'NovaComputeExtraConfig' >>>>>>>> deprecated_param_metadata: 'NovaComputeServerMetadata' >>>>>>>> deprecated_param_scheduler_hints: 'NovaComputeSchedulerHints' >>>>>>>> deprecated_param_ips: 'NovaComputeIPs' >>>>>>>> deprecated_server_resource_name: 'NovaCompute' >>>>>>>> >>>>>>>> deprecated_nic_config_name: 'compute.yaml' >>>>>>>> update_serial: 25 >>>>>>>> ServicesDefault: >>>>>>>> - OS::TripleO::Services::Aide >>>>>>>> - OS::TripleO::Services::AuditD >>>>>>>> - OS::TripleO::Services::BootParams >>>>>>>> >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ cat >>>>>>>> environments/network-environment.yaml >>>>>>>> >>>>>>>> #This file is an example of an environment file for defining the >>>>>>>> isolated >>>>>>>> #networks and related parameters. >>>>>>>> resource_registry: >>>>>>>> # Network Interface templates to use (these files must exist). >>>>>>>> You can >>>>>>>> # override these by including one of the net-*.yaml environment >>>>>>>> files, >>>>>>>> # such as net-bond-with-vlans.yaml, or modifying the list here. >>>>>>>> # Port assignments for the Controller >>>>>>>> OS::TripleO::Controller::Net::SoftwareConfig: OS::Heat::None >>>>>>>> # Port assignments for the Compute >>>>>>>> OS::TripleO::Compute::Net::SoftwareConfig: OS::Heat::None >>>>>>>> >>>>>>>> >>>>>>>> parameter_defaults: >>>>>>>> # This section is where deployment-specific configuration is done >>>>>>>> # >>>>>>>> ServiceNetMap: >>>>>>>> IronicApiNetwork: oc_provisioning >>>>>>>> IronicNetwork: oc_provisioning >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> # This section is where deployment-specific configuration is done >>>>>>>> ControllerNetworkConfigTemplate: >>>>>>>> 'templates/bonds_vlans/bonds_vlans.j2' >>>>>>>> ComputeNetworkConfigTemplate: >>>>>>>> 'templates/bonds_vlans/bonds_vlans.j2' >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> # Customize the IP subnet to match the local environment >>>>>>>> J3MgmtNetCidr: 'fd80:fd00:fd00:4000::/64' >>>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>>> J3MgmtAllocationPools: [{'start': 'fd80:fd00:fd00:4000::10', >>>>>>>> 'end': 'fd80:fd00:fd00:4000:ffff:ffff:ffff:fffe'}] >>>>>>>> # Customize the VLAN ID to match the local environment >>>>>>>> J3MgmtNetworkVlanID: 400 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> # Customize the IP subnet to match the local environment >>>>>>>> InternalApiNetCidr: 'fd00:fd00:fd00:2000::/64' >>>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>>> InternalApiAllocationPools: [{'start': 'fd00:fd00:fd00:2000::10', >>>>>>>> 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] >>>>>>>> # Customize the VLAN ID to match the local environment >>>>>>>> InternalApiNetworkVlanID: 418 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> # Customize the IP subnet to match the local environment >>>>>>>> ExternalNetCidr: 'fd00:fd00:fd00:9900::/64' >>>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>>> # Leave room if the external network is also used for floating IPs >>>>>>>> ExternalAllocationPools: [{'start': 'fd00:fd00:fd00:9900::10', >>>>>>>> 'end': 'fd00:fd00:fd00:9900:ffff:ffff:ffff:fffe'}] >>>>>>>> # Gateway router for routable networks >>>>>>>> ExternalInterfaceDefaultRoute: 'fd00:fd00:fd00:9900::1' >>>>>>>> # Customize the VLAN ID to match the local environment >>>>>>>> ExternalNetworkVlanID: 408 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> # Customize the IP subnet to match the local environment >>>>>>>> OCProvisioningNetCidr: '172.23.3.0/24' >>>>>>>> # Customize the IP range to use for static IPs and VIPs >>>>>>>> OCProvisioningAllocationPools: [{'start': '172.23.3.10', 'end': >>>>>>>> '172.23.3.50'}] >>>>>>>> # Customize the VLAN ID to match the local environment >>>>>>>> OCProvisioningNetworkVlanID: 412 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> # List of Neutron network types for tenant networks (will be used >>>>>>>> in order) >>>>>>>> NeutronNetworkType: 'geneve,vlan' >>>>>>>> # Neutron VLAN ranges per network, for example >>>>>>>> 'datacentre:1:499,tenant:500:1000': >>>>>>>> NeutronNetworkVLANRanges: 'datacentre:1:1000' >>>>>>>> # Customize bonding options, e.g. "mode=4 lacp_rate=1 >>>>>>>> updelay=1000 miimon=100" >>>>>>>> # for Linux bonds w/LACP, or "bond_mode=active-backup" for OVS >>>>>>>> active/backup. >>>>>>>> BondInterfaceOvsOptions: "bond_mode=active-backup" >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ >>>>>>>> >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ cat >>>>>>>> environments/network-isolation.yaml >>>>>>>> >>>>>>>> # NOTE: This template is now deprecated, and is only included for >>>>>>>> compatibility >>>>>>>> # when upgrading a deployment where this template was originally >>>>>>>> used. For new >>>>>>>> # deployments, set "ipv6: true" on desired networks in >>>>>>>> network_data.yaml, and >>>>>>>> # include network-isolation.yaml. >>>>>>>> # >>>>>>>> # Enable the creation of Neutron networks for isolated Overcloud >>>>>>>> # traffic and configure each role to assign ports (related >>>>>>>> # to that role) on these networks. >>>>>>>> resource_registry: >>>>>>>> # networks as defined in network_data.yaml >>>>>>>> OS::TripleO::Network::J3Mgmt: ../network/j3mgmt_v6.yaml >>>>>>>> OS::TripleO::Network::InternalApi: ../network/internal_api_v6.yaml >>>>>>>> OS::TripleO::Network::External: ../network/external_v6.yaml >>>>>>>> OS::TripleO::Network::OCProvisioning: >>>>>>>> ../network/oc_provisioning.yaml >>>>>>>> >>>>>>>> >>>>>>>> # Port assignments for the VIPs >>>>>>>> OS::TripleO::Network::Ports::J3MgmtVipPort: >>>>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>>>> OS::TripleO::Network::Ports::InternalApiVipPort: >>>>>>>> ../network/ports/internal_api_v6.yaml >>>>>>>> OS::TripleO::Network::Ports::ExternalVipPort: >>>>>>>> ../network/ports/external_v6.yaml >>>>>>>> OS::TripleO::Network::Ports::OCProvisioningVipPort: >>>>>>>> ../network/ports/oc_provisioning.yaml >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> # Port assignments by role, edit role definition to assign >>>>>>>> networks to roles. >>>>>>>> # Port assignments for the Controller >>>>>>>> OS::TripleO::Controller::Ports::J3MgmtPort: >>>>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>>>> OS::TripleO::Controller::Ports::InternalApiPort: >>>>>>>> ../network/ports/internal_api_v6.yaml >>>>>>>> OS::TripleO::Controller::Ports::ExternalPort: >>>>>>>> ../network/ports/external_v6.yaml >>>>>>>> OS::TripleO::Controller::Ports::OCProvisioningPort: >>>>>>>> ../network/ports/oc_provisioning.yaml >>>>>>>> # Port assignments for the Compute >>>>>>>> OS::TripleO::Compute::Ports::J3MgmtPort: >>>>>>>> ../network/ports/j3mgmt_v6.yaml >>>>>>>> OS::TripleO::Compute::Ports::InternalApiPort: >>>>>>>> ../network/ports/internal_api_v6.yaml >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> parameter_defaults: >>>>>>>> # Enable IPv6 environment for Manila >>>>>>>> ManilaIPv6: True >>>>>>>> >>>>>>>> (undercloud) [stack at undercloud templates]$ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 24, 2022 at 5:04 PM Lokendra Rathour < >>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>> >>>>>>>>> Thanks, I'll check them out. >>>>>>>>> will let you know in case it works out. >>>>>>>>> >>>>>>>>> On Tue, May 24, 2022 at 2:37 PM Swogat Pradhan < >>>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> Please find the below templates: >>>>>>>>>> These are for openstack wallaby release: >>>>>>>>>> >>>>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>>>> custom_network_data.yaml >>>>>>>>>> - name: Storage >>>>>>>>>> name_lower: storage >>>>>>>>>> vip: true >>>>>>>>>> mtu: 1500 >>>>>>>>>> subnets: >>>>>>>>>> storage_subnet: >>>>>>>>>> ip_subnet: 172.25.202.0/26 >>>>>>>>>> allocation_pools: >>>>>>>>>> - start: 172.25.202.6 >>>>>>>>>> end: 172.25.202.20 >>>>>>>>>> vlan: 1105 >>>>>>>>>> - name: StorageMgmt >>>>>>>>>> name_lower: storage_mgmt >>>>>>>>>> vip: true >>>>>>>>>> mtu: 1500 >>>>>>>>>> subnets: >>>>>>>>>> storage_mgmt_subnet: >>>>>>>>>> ip_subnet: 172.25.202.64/26 >>>>>>>>>> allocation_pools: >>>>>>>>>> - start: 172.25.202.72 >>>>>>>>>> end: 172.25.202.87 >>>>>>>>>> vlan: 1106 >>>>>>>>>> - name: InternalApi >>>>>>>>>> name_lower: internal_api >>>>>>>>>> vip: true >>>>>>>>>> mtu: 1500 >>>>>>>>>> subnets: >>>>>>>>>> internal_api_subnet: >>>>>>>>>> ip_subnet: 172.25.201.192/26 >>>>>>>>>> allocation_pools: >>>>>>>>>> - start: 172.25.201.198 >>>>>>>>>> end: 172.25.201.212 >>>>>>>>>> vlan: 1104 >>>>>>>>>> - name: Tenant >>>>>>>>>> vip: false # Tenant network does not use VIPs >>>>>>>>>> mtu: 1500 >>>>>>>>>> name_lower: tenant >>>>>>>>>> subnets: >>>>>>>>>> tenant_subnet: >>>>>>>>>> ip_subnet: 172.25.202.128/26 >>>>>>>>>> allocation_pools: >>>>>>>>>> - start: 172.25.202.135 >>>>>>>>>> end: 172.25.202.150 >>>>>>>>>> vlan: 1108 >>>>>>>>>> - name: External >>>>>>>>>> name_lower: external >>>>>>>>>> vip: true >>>>>>>>>> mtu: 1500 >>>>>>>>>> subnets: >>>>>>>>>> external_subnet: >>>>>>>>>> ip_subnet: 172.25.201.128/26 >>>>>>>>>> allocation_pools: >>>>>>>>>> - start: 172.25.201.135 >>>>>>>>>> end: 172.25.201.150 >>>>>>>>>> gateway_ip: 172.25.201.129 >>>>>>>>>> vlan: 1103 >>>>>>>>>> >>>>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>>>> custom_vip_data.yaml >>>>>>>>>> - network: ctlplane >>>>>>>>>> #dns_name: overcloud >>>>>>>>>> ip_address: 172.25.201.91 >>>>>>>>>> subnet: ctlplane-subnet >>>>>>>>>> - network: external >>>>>>>>>> #dns_name: overcloud >>>>>>>>>> ip_address: 172.25.201.150 >>>>>>>>>> subnet: external_subnet >>>>>>>>>> - network: internal_api >>>>>>>>>> #dns_name: overcloud >>>>>>>>>> ip_address: 172.25.201.250 >>>>>>>>>> subnet: internal_api_subnet >>>>>>>>>> - network: storage >>>>>>>>>> #dns_name: overcloud >>>>>>>>>> ip_address: 172.25.202.50 >>>>>>>>>> subnet: storage_subnet >>>>>>>>>> - network: storage_mgmt >>>>>>>>>> #dns_name: overcloud >>>>>>>>>> ip_address: 172.25.202.90 >>>>>>>>>> subnet: storage_mgmt_subnet >>>>>>>>>> >>>>>>>>>> (undercloud) [stack at hkg2director workplace]$ cat >>>>>>>>>> overcloud-baremetal-deploy.yaml >>>>>>>>>> - name: Controller >>>>>>>>>> count: 4 >>>>>>>>>> defaults: >>>>>>>>>> networks: >>>>>>>>>> - network: ctlplane >>>>>>>>>> vif: true >>>>>>>>>> - network: external >>>>>>>>>> subnet: external_subnet >>>>>>>>>> - network: internal_api >>>>>>>>>> subnet: internal_api_subnet >>>>>>>>>> - network: storage >>>>>>>>>> subnet: storage_subnet >>>>>>>>>> - network: storage_mgmt >>>>>>>>>> subnet: storage_mgmt_subnet >>>>>>>>>> - network: tenant >>>>>>>>>> subnet: tenant_subnet >>>>>>>>>> network_config: >>>>>>>>>> template: /home/stack/templates/controller.j2 >>>>>>>>>> default_route_network: >>>>>>>>>> - external >>>>>>>>>> instances: >>>>>>>>>> - hostname: overcloud-controller-0 >>>>>>>>>> name: dc1-controller2 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-controller-1 >>>>>>>>>> name: dc2-controller2 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-controller-2 >>>>>>>>>> name: dc1-controller1 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-controller-no-ceph-3 >>>>>>>>>> name: dc2-ceph2 >>>>>>>>>> #provisioned: false >>>>>>>>>> #- hostname: overcloud-controller-3 >>>>>>>>>> #name: dc2-compute3 >>>>>>>>>> #provisioned: false >>>>>>>>>> >>>>>>>>>> - name: Compute >>>>>>>>>> count: 5 >>>>>>>>>> defaults: >>>>>>>>>> networks: >>>>>>>>>> - network: ctlplane >>>>>>>>>> vif: true >>>>>>>>>> - network: internal_api >>>>>>>>>> subnet: internal_api_subnet >>>>>>>>>> - network: tenant >>>>>>>>>> subnet: tenant_subnet >>>>>>>>>> - network: storage >>>>>>>>>> subnet: storage_subnet >>>>>>>>>> network_config: >>>>>>>>>> template: /home/stack/templates/compute.j2 >>>>>>>>>> instances: >>>>>>>>>> - hostname: overcloud-novacompute-0 >>>>>>>>>> name: dc2-compute1 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-novacompute-1 >>>>>>>>>> name: dc2-ceph1 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-novacompute-2 >>>>>>>>>> name: dc1-compute1 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-novacompute-3 >>>>>>>>>> name: dc1-compute2 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-novacompute-4 >>>>>>>>>> name: dc2-compute3 >>>>>>>>>> #provisioned: false >>>>>>>>>> >>>>>>>>>> - name: CephStorage >>>>>>>>>> count: 4 >>>>>>>>>> defaults: >>>>>>>>>> networks: >>>>>>>>>> - network: ctlplane >>>>>>>>>> vif: true >>>>>>>>>> - network: internal_api >>>>>>>>>> subnet: internal_api_subnet >>>>>>>>>> - network: storage >>>>>>>>>> subnet: storage_subnet >>>>>>>>>> - network: storage_mgmt >>>>>>>>>> subnet: storage_mgmt_subnet >>>>>>>>>> network_config: >>>>>>>>>> template: /home/stack/templates/ceph-storage.j2 >>>>>>>>>> instances: >>>>>>>>>> - hostname: overcloud-cephstorage-0 >>>>>>>>>> name: dc2-controller1 >>>>>>>>>> #provisioned: false >>>>>>>>>> # - hostname: overcloud-cephstorage-1 >>>>>>>>>> # name: dc2-ceph2 >>>>>>>>>> - hostname: overcloud-cephstorage-1 >>>>>>>>>> name: dc1-ceph1 >>>>>>>>>> # provisioned: false >>>>>>>>>> - hostname: overcloud-cephstorage-2 >>>>>>>>>> name: dc1-ceph2 >>>>>>>>>> #provisioned: false >>>>>>>>>> - hostname: overcloud-cephstorage-3 >>>>>>>>>> name: dc2-compute2 >>>>>>>>>> #provisioned: false >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> You must use these templates to provision network, vip and nodes. >>>>>>>>>> You must use the output files generated during the provisioning >>>>>>>>>> step in openstack overcloud deploy command using -e parameter. >>>>>>>>>> >>>>>>>>>> With regards, >>>>>>>>>> Swogat Pradhan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, May 23, 2022 at 8:33 PM Lokendra Rathour < >>>>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Swogat, >>>>>>>>>>> I tried checking your solution and my templates but could not >>>>>>>>>>> relate much. >>>>>>>>>>> But issue seems the same >>>>>>>>>>> >>>>>>>>>>> http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028401.html >>>>>>>>>>> >>>>>>>>>>> I tried somemore ways but looks like some issue with templates. >>>>>>>>>>> Can you please share the templates used to deploy the overcloud. >>>>>>>>>>> >>>>>>>>>>> Mysetup have 3 controller and 1 compute. >>>>>>>>>>> >>>>>>>>>>> Thanks once again for reading my mail. >>>>>>>>>>> >>>>>>>>>>> Waiting for your reply. >>>>>>>>>>> >>>>>>>>>>> -Lokendra >>>>>>>>>>> >>>>>>>>>>> On Fri, 20 May 2022, 08:25 Swogat Pradhan, < >>>>>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> Yes I was able to find the issue and fix it. >>>>>>>>>>>> The issue was with the overcloud-baremetal-deployed.yaml file i >>>>>>>>>>>> was trying to provision controller-0, controller-1 and controller-3 and >>>>>>>>>>>> kept controller-2 aside for later, but the tripleo scripts are built in >>>>>>>>>>>> such a way that they were taking controller- 0, 1 and 2 inplace of >>>>>>>>>>>> controller-3, so the network ports and vip were created for controller 0,1 >>>>>>>>>>>> and 2 but not for 3 , so this error was popping off. Also i would request >>>>>>>>>>>> you to check the jinja nic templates and once the node provisioning is done >>>>>>>>>>>> check the /etc/os-net-config/config.json/yaml file for syntax if using >>>>>>>>>>>> bonded nic template. >>>>>>>>>>>> If you need any more infor please let me know. >>>>>>>>>>>> >>>>>>>>>>>> With regards, >>>>>>>>>>>> Swogat Pradhan >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, May 20, 2022 at 8:01 AM Lokendra Rathour < >>>>>>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Swogat, >>>>>>>>>>>>> Thanks for raising this issue. >>>>>>>>>>>>> Did you find any solution? to this problem ? >>>>>>>>>>>>> >>>>>>>>>>>>> Please let me know it might be helpful >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Apr 19, 2022 at 12:43 PM Swogat Pradhan < >>>>>>>>>>>>> swogatpradhan22 at gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> I am currently trying to deploy openstack wallaby using >>>>>>>>>>>>>> tripleo arch. >>>>>>>>>>>>>> I created the network jinja templates, ran the following >>>>>>>>>>>>>> commands also: >>>>>>>>>>>>>> >>>>>>>>>>>>>> #openstack overcloud network provision --stack overcloud >>>>>>>>>>>>>> --output networks-deployed-environment.yaml custom_network_data.yaml >>>>>>>>>>>>>> # openstack overcloud network vip provision --stack >>>>>>>>>>>>>> overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml >>>>>>>>>>>>>> # openstack overcloud node provision --stack overcloud >>>>>>>>>>>>>> --overcloud-ssh-key /home/stack/sshkey/id_rsa >>>>>>>>>>>>>> overcloud-baremetal-deploy.yaml >>>>>>>>>>>>>> >>>>>>>>>>>>>> and used the environment files in the openstack overcloud >>>>>>>>>>>>>> deploy command: >>>>>>>>>>>>>> >>>>>>>>>>>>>> (undercloud) [stack at hkg2director ~]$ cat deploy.sh >>>>>>>>>>>>>> #!/bin/bash >>>>>>>>>>>>>> THT=/usr/share/openstack-tripleo-heat-templates/ >>>>>>>>>>>>>> CNF=/home/stack/ >>>>>>>>>>>>>> openstack overcloud deploy --templates $THT \ >>>>>>>>>>>>>> -r $CNF/templates/roles_data.yaml \ >>>>>>>>>>>>>> -n $CNF/workplace/custom_network_data.yaml \ >>>>>>>>>>>>>> -e ~/containers-prepare-parameter.yaml \ >>>>>>>>>>>>>> -e $CNF/templates/node-info.yaml \ >>>>>>>>>>>>>> -e $CNF/templates/scheduler-hints.yaml \ >>>>>>>>>>>>>> -e $CNF/workplace/networks-deployed-environment.yaml \ >>>>>>>>>>>>>> -e $CNF/workplace/vip-deployed-environment.yaml \ >>>>>>>>>>>>>> -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ >>>>>>>>>>>>>> -e $CNF/workplace/custom-net-bond-with-vlans.yaml >>>>>>>>>>>>>> >>>>>>>>>>>>>> Now when i run the ./deploy.sh script i encounter an error >>>>>>>>>>>>>> stating: >>>>>>>>>>>>>> >>>>>>>>>>>>>> ERROR openstack [-] Resource >>>>>>>>>>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>>>>>>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>>>>>>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>>>>>>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>>>>>>>>>> included with the deployment command.: >>>>>>>>>>>>>> tripleoclient.exceptions.InvalidConfiguration: Resource >>>>>>>>>>>>>> OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type >>>>>>>>>>>>>> OS::Neutron::Port and the Neutron service is not available when using >>>>>>>>>>>>>> ephemeral Heat. The generated environments from 'openstack overcloud >>>>>>>>>>>>>> baremetal provision' and 'openstack overcloud network provision' must be >>>>>>>>>>>>>> included with the deployment command. >>>>>>>>>>>>>> 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END >>>>>>>>>>>>>> return value: 1 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can someone tell me where the mistake is? >>>>>>>>>>>>>> >>>>>>>>>>>>>> With regards, >>>>>>>>>>>>>> Swogat Pradhan >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From christophe.sauthier at accenture.com Wed Jun 8 15:08:14 2022 From: christophe.sauthier at accenture.com (Sauthier, Christophe) Date: Wed, 8 Jun 2022 15:08:14 +0000 Subject: [External] [CLOUDKITTY] presentation used in the forum panels In-Reply-To: References: Message-ID: I just wanted to take a quick moment to thank you Rafael and Pierre for continuing the ongoing development on Cloudkitty ! Christophe ________________________________ De : Rafael Weing?rtner Envoy? : mercredi 8 juin 2022 09:59 ? : openstack-discuss Objet : [External] [CLOUDKITTY] presentation used in the forum panels This message is from an EXTERNAL SENDER - be CAUTIOUS, particularly with links and attachments. ________________________________ Hello CloudKitters! It was a pleasure to meet you guys during the OpenStack summit. Also, it was a good surprise to see that so many people already use CloudKitty. I really hope that we can all collaborate together to keep improving CloudKitty, and making it more and more solid. Attached is the presentation we used during the Forum, so you all can consult this material. Also, as always, if you need some guidance/help/or anything else, let us know. Link for the presentation online: https://docs.google.com/presentation/d/15e37xGKCJoTMwbarrGcSlLpECUkcJ2exazW_TztCehk/edit?usp=sharing -- Rafael Weing?rtner ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy. ______________________________________________________________________________________ www.accenture.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim+openstack.org at coote.org Thu Jun 9 08:58:46 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Thu, 9 Jun 2022 09:58:46 +0100 Subject: simple build fails to allocate network interface Message-ID: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> Hullo I?m trying to spin up a simple OS environment in a single vm and think that I?m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I?ve tried this on a couple of laptops, with the same result, so I suspect that it?s something to do with how I?ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I?ve got to get it working to log any bugs). I?m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try: `openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble` The network id is identified from this: ??" [vagrant at localhost devstack]$ openstack network list +--------------------------------------+---------+----------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+----------------------------------------------------------------------------+ | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 | +--------------------------------------+---------+??????????????????????????????????????+ ??? Using a public network fails with an error that this isn?t allowed. With the private network above, the failure throws out the following errors. ??" [vagrant at localhost devstack]$ sudo journalctl -f |grep ERROR Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2728, in _build_resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] yield resources Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling. ??" The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): ??" # -*- mode: ruby -*- # vi: set ft=ruby : $script = <<-SHELL dnf update -y #apt update #apt upgrade echo "updated" systemctl disable firewalld systemctl stop firewalld systemctl disable NetworkManager systemctl stop NetworkManager systemctl enable network systemctl start network #dnf config-manager --enable powertools #dnf install -y centos-release-openstack-yoga dnf update -y dnf install -y git python3-pip #dnf install -y openstack-packstack #packstack --provision-demo=n --os-neutron-ml2-type-drivers=vxlan,flat,vlan --gen-answer-file=packstack-answers.txt #sed -i -e 's:10.0.2.15:10.1.0.10:' packstack-answers.txt # not for centos 9 #dnf install -y network-scripts dnf install -y net-tools #cat /vagrant/answers.addon >> packstack-answers.txt # don't want this, but httpd won't start without it setenforce 0 pip3 install invoke #packstack --allinone cd /vagrant # permissions all wrong: ./go.sh # need su - vagrant cd /vagrant && ./go.sh (?) # cd devstack # not working: wrong user ./stack.sh SHELL Vagrant.configure(2) do |config| config.vm.box = "eurolinux-vagrant/centos-stream-9" #config.vm.box = "hashicorp/bionic64" # machines = { # 'node1.example.dd' #'node1.example.dd' => { :ip => '10.1.0.10'}, # 'node2.example.dd' => { :ip =>'10.1.0.12'}, # } config.hostmanager.enabled = true config.hostmanager.manage_host = true config.hostmanager.manage_guest = true config.hostmanager.ignore_private_ip = false config.hostmanager.include_offline = true config.ssh.pty = true config.vm.provision "shell", inline: $script config.vm.network "forwarded_port", guest: 80, host: 8080 # machines.each do | hostname, attrs| # config.vm.define hostname do |machine| # machine.vm.hostname = hostname # machine.vm.network :private_network, :ip => attrs[:ip] # machine.vm.provider "virtualbox" do | v | config.vm.provider "virtualbox" do | v | #v.memory = "4096" #v.memory = "8192" v.memory = "9216" v.cpus = "2" end #end # end end ??? I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files. Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details? Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From rafaelweingartner at gmail.com Thu Jun 9 18:08:26 2022 From: rafaelweingartner at gmail.com (=?UTF-8?Q?Rafael_Weing=C3=A4rtner?=) Date: Thu, 9 Jun 2022 15:08:26 -0300 Subject: [CLOUDKITTY][OpenInfra summit 2022][June 9th] Presentation on Ceilometer, Gnocchi, and CloudKitty Message-ID: Hello CloudKitters! It was a great opportunity to be able to present this talk for you all guys. It was a pleasure to see that so many people are interested in CloudKitty, Ceilometer, and Gnocchi. I really hope that we can all collaborate together to keep improving CloudKitty, and other related systems. Attached is the presentation we used during the presentation, so you all can consult this material. Also, as always, if you need some guidance, let us know. Furthermore, as I mentioned in the presentation, I will provide a step-by-step process on how to create a new metric using Ceilometer Dynamic pollster that can collect/generate/derive (however you want to call it) IOPs information for volumes. The new metric name is going to be called "dynamic_pollster.volume.services.volume.iops". Also, the idea is that we collect some attributes, and we use the attribute called "bootable", which indicates if the volume is bootable or not. Then, if the volume is bootable, we will charge for its IOPS; otherwise, we do not charge it. As I mentioned, this is just a very simple use case. It is not targeting anyone's need in production, but it can be used to get you going quite easily with the pipeline. Requirements: - Ceilometer up and running. Version 15.0.0 (or higher is recommended); - Gnocchi up and running. Version 4.3.0 (or higher). If you want to run this in production, this patch might be very important ( https://github.com/gnocchixyz/gnocchi/pull/1059); it has not been merged yet; - CloudKitty up and running. Version 10.0.0 (or higher). With all systems up and running, you will need to do the following: - Ceilometer - Add the new metric collection in the "polling.yaml" file. You can add it in any of the sections you already have there. Or, you can create its own, as follows: ``` - name: volume_iops_metric interval: 600 meters: - dynamic_pollster.volume.services.volume.iops ``` - After declaring the metrics there, you can create the pollster in the "pollster.d" folder in Ceilometer. The name of the file can be anything, as long as it ends with ".yaml"; for instance, "pollsters.d/volume-iops.yaml". Then, you will need to add the following content. Remember to replace the contents of "<>" with the values of your environment. ``` - name: "dynamic_pollster.volume.services.volume.iops" response_entries_key: "volumes" next_sample_url_attribute: "volumes_links | filter(lambda v: v.get('rel') == 'next', value) | list(value) | value[0] | value.get('href')" sample_type: "gauge" unit: "GB" value_attribute: "volume_type" endpoint_type: "volume" url_path: "/v3//volumes/detail?all_tenants=true&limit=10" headers: "Openstack-API-Version": "latest" project_id_attribute: "os-vol-tenant-attr:tenant_id" metadata_fields: - "status" - "name" - "volume_type" - "availability_zone" - "user_id" - "bootable | str(value)" - "attachments | value or [ { 'server_id': '' } ] | value[0] | value['server_id']" metadata_mapping: "availability_zone": "dynamic_availability_zone" "name": "dynamic_display_name" "volume_type": "dynamic_volume_type" "attachments | value or [{ 'server_id': '' }] | value[0] | value['server_id']": "dynamic_instance_id" "bootable | str(value)": "bootable" value_mapping: "": "" ``` - After creating that, you will need to customize the file "gnocchi_resources.yaml", which is the file that maps metrics to resources, and attributes that are pushed to the backend. You can also create your own "gnocchi_resources.yaml" file, and then use it in the pipeline configuration in the "pipeline.yaml" file, in the configuration of "publishers". Then, you would have something like "gnocchi://?resources_definition_file=%2Fetc%2Fceilometer%2Fcustom_gnocchi_resources.yaml", assuming the file is called "custom_gnocchi_resources.yaml". In this file, you will need to add the new metric and attribute to be pushed to the metrics backend. The following is an example of "volume" resource declaration in the "gnocchi_resource.yaml" file with the new metric and attribute that we want to capture. ``` - resource_type: volume metrics: dynamic_pollster.volume.services.volume.iops: attributes: display_name: resource_metadata.(dynamic_display_name|notification_display_name) bootable: resource_metadata.bootable ``` - After the above configuration, you will need to create the new attribute in the Gnocchi resource type. To achieve that, you can issue the following command. ``` gnocchi resource-type update -a bootable:string:false volume ``` - Then, you will need to restart the Ceilometer. - Now, you can issue "gnocchi resource show ", and you should see the new metric in the resource, and the new attribute as well. With Ceilometer and Gnocchi configured for the new metric, we can move on to CloudKitty. - The new metric needs to be declared in the "metrics.yml" file. The content should be something similar to the following (adapt as you need). ``` dynamic_pollster.volume.services.volume.iops: unit: IOPS alt_name: disk-volume-iops-usage-hours groupby: - id - user_id - project_id - volume_type - display_name - revision_start - instance_id - availability_zone - bootable metadata: [] extra_args: aggregation_method: max resource_type: volume use_all_resource_revisions: false ``` - Restart CloudKitty processor - With the new metric introduced in CloudKitty processor, you will need to create the rating configuration. In our case, we wanted to rate the volumes marked as bootable for IOPs. The volumes that are not bootable would not be rated. The following are the steps one needs to execute to create the hashmap rating rules. - Create the service ``` openstack rating hashmap service create disk-volume-iops-usage-hours ``` - Then, create the field ``` openstack rating hashmap field create ``` - After that, we can create the rating rule ``` openstack rating hashmap mapping create --field --value 'true' 1 --type flat ``` - Having done that, you should be able to rate all volumes by IOPS if they have the flag "bootable" set to true. Otherwise, they will not be rated. That is all that one needs to execute to replicate the example I showed during the presentation. If you guys have any doubts or difficulties when using CloudKitty, do not hesitate to ping me. -- Rafael Weing?rtner -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Gnocchi, Ceilometer, and CloudKitty, a dynamic and agnostic cloud monitoring and billing stack [OpenInfra summit 2022]..pdf Type: application/pdf Size: 761249 bytes Desc: not available URL: From kkchn.in at gmail.com Fri Jun 10 13:01:52 2022 From: kkchn.in at gmail.com (KK CHN) Date: Fri, 10 Jun 2022 18:31:52 +0530 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) Message-ID: List, I am trying Devstack installation on a Virtual Machine to try TROVE DbaaS. All the time ./stack.sh command stops with the same error message at this particular point + functions:_upload_image:121 : openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create cirros-0.5.2-x86_64-disk --public --container-format bare --disk-format qcow2 --property hw_rng_model=virtio Internal Server Error (HTTP 500) I have ./unstack.sh && ./clean.sh and re run ./stack.sh multiple time but no progress. Here always the script exit with error . Screenshot attached (using "stack" user created by the devstack/tool/ stack user script) my local.conf file as follows here 10.184.48.187 is my VMs IP Any hints to rectify this error most welcome!! ############################################################ [[local|localrc]] RECLONE=False HOST_IP=10.184.48.187 enable_plugin trove https://opendev.org/openstack/trove enable_plugin trove-dashboard https://opendev.org/openstack/trove-dashboard LIBS_FROM_GIT+=,python-troveclient DATABASE_PASSWORD=password ADMIN_PASSWORD=password SERVICE_PASSWORD=password SERVICE_TOKEN=password RABBIT_PASSWORD=password LOGFILE=$DEST/logs/stack.sh.log VERBOSE=True LOG_COLOR=False LOGDAYS=1 IPV4_ADDRS_SAFE_TO_USE=10.111.0.0/26 FIXED_RANGE=10.111.0.0/26 NETWORK_GATEWAY=10.111.0.1 FLOATING_RANGE=10.184.48.0/24 PUBLIC_NETWORK_GATEWAY=10.184.48.1 #pre requisites ENABLED_SERVICES=rabbit,mysql,key #nova enable_service horizon enable_service n-api enable_service n-cpu enable_service n-cond enable_service n-sch enable_service n-api-meta enable_service placement-api enable_service placement-client #Glance enable_service g-api enable_service g-reg #Cinder enable_service cinder enable_service c-api enable_service c-vol enable_service c-sch #Neutron enable_service q-svc enable_service q-agt enable_service q-dhcp enable_service q-l3 enable_service q-meta #enable DVR Q_AGENT=openvswitch Q_DVR_MODE=legacy Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch Q_ML2_TENANT_NETWORK_TYPE=vxlan Q_PLUGIN=ml2 #SWIFT ENABLED_SERVICES+=,swift SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 SWIFT_REPLICAS=1 stack at XenaCtrl1:/home/cloud/trove_devstack/devstack$ ################################################################# -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: DevstackError1.png Type: image/png Size: 228221 bytes Desc: not available URL: From p.aminian.server at gmail.com Mon Jun 13 09:59:41 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Mon, 13 Jun 2022 14:29:41 +0430 Subject: snapshot problem In-Reply-To: <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> Message-ID: I attached nova.conf On Mon, Jun 13, 2022 at 12:15 PM Eugen Block wrote: > Could you share your whole nova.conf (only the uncommented options)? > Is this option set in your env? > > #snapshot_image_format = > > Also when you manually created the snapshot did you do it as the nova > user on the compute node? If not, could you retry? > > > Zitat von Parsa Aminian : > > > Hi > > kolla-ansible victoria version with ceph backend . > > rbd info output : > > rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': > > size 20 GiB in 5120 objects > > order 22 (4 MiB objects) > > snapshot_count: 0 > > id: b69aaf907284da > > block_name_prefix: rbd_data.b69aaf907284da > > format: 2 > > features: layering, exclusive-lock, object-map, fast-diff, > > deep-flatten > > op_features: > > flags: > > create_timestamp: Fri May 20 00:04:47 2022 > > access_timestamp: Sun Jun 12 16:26:02 2022 > > --------------- > > also live snapshot seems to work correctly without any error or any > > downtime : > > docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls > > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk > > SNAPID NAME SIZE PROTECTED TIMESTAMP > > 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 > > > > also on compute nova.conf, images_type is set on rbd . > > > > On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: > > > >> You should respond to the list so other users can try to support you. > >> > >> So nova is trying to live snapshot the instance: > >> > >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - > >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > >> > instance snapshotting > >> > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning > >> > live snapshot process > >> > >> But I don't see any 'rbd snap create' command. Either the rbd image > >> doesn't support it or there is some setting to keep all rbd images > >> "flat". Can you check any relevant configs you might have in nova? > >> Also can you show the output of 'rbd info > >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if > >> the underlying rbd functions work as expected you could try to create > >> a live snapshot manually: > >> > >> rbd snap create > /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap > >> > >> And paste any relevant output here. > >> > >> Zitat von Parsa Aminian : > >> > >> > Its not working for any instances and all of them are paused . I > enable > >> > debug logs please check the logs : > >> > > >> > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync > for > >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: waited 0.000s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute > >> host > >> > and has allocations in placement: {'resources': {'VCPU': 1, > 'MEMORY_MB': > >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> > 63426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot process > >> > default default] Lazy-loading 'pci_devices' on Instance uuid > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading > >> > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > obj_load_attr > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > => Paused> emit_event > >> > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > >> > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power > state > >> > after lifecycle event "Paused"; current vm_state: active, current > >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> > power_state: 3 handle_lifecycle_event > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > >> instance > >> > has a pending task (image_pending_upload). Skip. > >> > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd > >> > (subprocess): qemu-img convert -t none -O raw -f raw > >> > > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> > > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad > >> > execute > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 > >> > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > => Stopped> emit_event > >> > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> > INFO nova.compute.manager [-] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > - - > >> - > >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> > _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > - - > >> - > >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing > >> > instance power state after lifecycle event "Stopped"; current > vm_state: > >> > active, current task_state: image_pending_upload, current DB > power_state: > >> > 1, VM power_state: 4 handle_lifecycle_event > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> > INFO nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - > - > >> - - > >> > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During > >> sync_power_state > >> > the instance has a pending task (image_pending_upload). Skip. > >> > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync > for > >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> > "25c8d676-e20a-4238 > >> > -a45c-d51daa62b941" acquired by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: waited 0.000s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > >> instance > >> > has a pending task (image_pending_upload). Skip. > >> > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> > "25c8d676-e20a-4238-a45c-d51daa62b941" released by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: held 0.001s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 > >> > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img > convert > >> > -t none -O raw -f raw > >> > > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> > > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" > >> > returned: 0 in 23.730s execute > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 > >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > >> Snapshot > >> > extracted, beginning image upload > >> > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > => Started> emit_event > >> > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) > >> > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> > _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power > state > >> > after lifecycle event "Started"; current vm_state: active, current > >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> > power_state: 1 handle_lifecycle_event > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event > >> > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock > >> > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network > info > >> > cache for instance _get_instance_nw_info > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 > >> > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache > with > >> > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > "address": > >> > "fa:16:3e:ca:00:d9", "network": {"id": > >> > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", "label": > >> > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": > >> > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, > >> > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": > {}}], > >> > "gateway": {"address": "141.11.42.1", "type": "gateway", "version": 4, > >> > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", > >> > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], > "version": > >> 4, > >> > "meta": {}}], "meta": {"injected": true, "tenant_id": > >> > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, "physical_network": > >> > "physnet1", "tunneled": false}}, "type": "ovs", "details": > >> {"connectivity": > >> > "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": > >> > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", > >> > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > "qbh_params": > >> > null, "qbg_params": null, "active": true, "vnic_type": "normal", > >> "profile": > >> > {}, "preserve_on_delete": false, "meta": {}}] > >> > update_instance_cache_with_nw_info > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 > >> > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > >> > compute host and has allocations in placement: {'resources': {'VCPU': > 1, > >> > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. > _remove_deleted_instances_allocations > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to snapshot > the > >> > instance on the hypervisor. > >> > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync > for > >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: waited 0.000s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute > >> host > >> > and has allocations in placement: {'resources': {'VCPU': 1, > 'MEMORY_MB': > >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager > >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> 93fb420b3c604d4fae408b81135b58e9 > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> > > >> > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: > >> > > >> >> Have you tried with debug logs? Has it worked with live snapshots > >> >> before for other instances or did it never work and all snapshots > were > >> >> "cold"? > >> >> > >> >> Zitat von Parsa Aminian : > >> >> > >> >> > Hi > >> >> > kolla-ansible victoria version with ceph backend without volume > >> >> > > >> >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block > wrote: > >> >> > > >> >> >> Hi, > >> >> >> > >> >> >> can you share more details about your environment? Which openstack > >> >> >> version is it? What is the storage backend? In earlier releases > there > >> >> >> was an option: > >> >> >> > >> >> >> #disable_libvirt_livesnapshot = false > >> >> >> > >> >> >> but this option has been deprecated. But if you're on an older > >> version > >> >> >> that could explain it. > >> >> >> > >> >> >> Zitat von Parsa Aminian : > >> >> >> > >> >> >> > When I snapshot from the instance , server will gone away and > its > >> not > >> >> >> > reachable until the snapshot is complete here is the logs : > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager > >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle > Event) > >> >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager > >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state > the > >> >> >> instance > >> >> >> > has a pending task (image_pending_upload). Skip. > >> >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] > [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle > Event) > >> >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver > >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> >> >> 93fb420b3c604d4fae408b81135b58e9 > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, > beginning > >> >> image > >> >> >> > upload > >> >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager > >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle > Event) > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> > >> > >> > >> > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- for nova.conf yes I changed ips and passwords for security reason : [root at R3SG4 ~]# cat /etc/kolla/nova-compute/nova.conf [DEFAULT] debug = False log_dir = /var/log/kolla/nova state_path = /var/lib/nova allow_resize_to_same_host = true compute_driver = libvirt.LibvirtDriver my_ip = ip instance_usage_audit = True instance_usage_audit_period = hour transport_url = rabbit://openstack:iyuiyiuyu at ip:5672// injected_network_template = /usr/lib/python3.6/site-packages/nova/virt/interfaces.template flat_injected = true force_config_drive = true config_drive_cdrom = True enable_instance_password = True dhcp_domain = resume_guests_state_on_host_boot = true reclaim_instance_interval = 86400 cpu_allocation_ratio = 3.5 ram_allocation_ratio = 1.0 disk_allocation_ratio = 1.0 resize_confirm_window = 10 [conductor] workers = 5 [vnc] novncproxy_host = ip novncproxy_port = 6080 server_listen = ip server_proxyclient_address = ip novncproxy_base_url = http://ip:6080/vnc_auto.html [oslo_concurrency] lock_path = /var/lib/nova/tmp [glance] api_servers = http://ip:9292 cafile = cafile = num_retries = 3 debug = False [neutron] metadata_proxy_shared_secret = jklhiuy service_metadata_proxy = true auth_url = http://ip:35357 auth_type = password cafile = project_domain_name = Default user_domain_id = default project_name = service username = neutron password = iupouopiupou region_name = RegionThree valid_interfaces = internal [libvirt] connection_uri = qemu+tcp://ip/system live_migration_inbound_addr = ip images_type = rbd images_rbd_pool = vms images_rbd_ceph_conf = /etc/ceph/ceph.conf rbd_user = cinder disk_cachemodes = network=writeback hw_disk_discard = unmap rbd_secret_uuid = 987798 virt_type = kvm inject_partition = -1 inject_password = true cpu_mode = custom cpu_model = Westmere [upgrade_levels] compute = auto [oslo_messaging_notifications] transport_url = rabbit://openstack:lkjhlkhlk at ip:5672// driver = messagingv2 topics = notifications [privsep_entrypoint] helper_command = sudo nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova.conf [guestfs] debug = False [placement] auth_type = password auth_url = http://ip:35357 username = placement password = oiuoiuoiu user_domain_name = Default project_name = service project_domain_name = Default region_name = RegionThree cafile = valid_interfaces = internal [notifications] notify_on_state_change = vm_and_task_state notification_format = unversioned [keystone_authtoken] www_authenticate_uri = http://ip:5000 auth_url = http://ip:35357 ------------- From p.aminian.server at gmail.com Mon Jun 13 10:01:15 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Mon, 13 Jun 2022 14:31:15 +0430 Subject: snapshot problem In-Reply-To: <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> Message-ID: snapshot_image_format doesn't exist on compute or controller nova.conf For the manual snapshot on the previous reply I ran it directly from ceph, but it's interesting from nova compute there is an error about connecting to CEPH cluster !!! also ceph -s is showing similar error [root at R3SG4 ~]# docker exec -u root -it nova_compute rbd snap create vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap2022-06-13 13:42:18.061 7f03dac23200 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory2022-06-13 13:42:18.061 7f03dac23200 -1 AuthRegistry(0x561d7014a790) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling ceph2022-06-13 13:42:18.065 7f03dac23200 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication rbd: couldn't connect to the cluster! On Mon, Jun 13, 2022 at 12:15 PM Eugen Block wrote: > Could you share your whole nova.conf (only the uncommented options)? > Is this option set in your env? > > #snapshot_image_format = > > Also when you manually created the snapshot did you do it as the nova > user on the compute node? If not, could you retry? > > > Zitat von Parsa Aminian : > > > Hi > > kolla-ansible victoria version with ceph backend . > > rbd info output : > > rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': > > size 20 GiB in 5120 objects > > order 22 (4 MiB objects) > > snapshot_count: 0 > > id: b69aaf907284da > > block_name_prefix: rbd_data.b69aaf907284da > > format: 2 > > features: layering, exclusive-lock, object-map, fast-diff, > > deep-flatten > > op_features: > > flags: > > create_timestamp: Fri May 20 00:04:47 2022 > > access_timestamp: Sun Jun 12 16:26:02 2022 > > --------------- > > also live snapshot seems to work correctly without any error or any > > downtime : > > docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls > > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk > > SNAPID NAME SIZE PROTECTED TIMESTAMP > > 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 > > > > also on compute nova.conf, images_type is set on rbd . > > > > On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: > > > >> You should respond to the list so other users can try to support you. > >> > >> So nova is trying to live snapshot the instance: > >> > >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - > >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > >> > instance snapshotting > >> > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning > >> > live snapshot process > >> > >> But I don't see any 'rbd snap create' command. Either the rbd image > >> doesn't support it or there is some setting to keep all rbd images > >> "flat". Can you check any relevant configs you might have in nova? > >> Also can you show the output of 'rbd info > >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if > >> the underlying rbd functions work as expected you could try to create > >> a live snapshot manually: > >> > >> rbd snap create > /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap > >> > >> And paste any relevant output here. > >> > >> Zitat von Parsa Aminian : > >> > >> > Its not working for any instances and all of them are paused . I > enable > >> > debug logs please check the logs : > >> > > >> > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync > for > >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: waited 0.000s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute > >> host > >> > and has allocations in placement: {'resources': {'VCPU': 1, > 'MEMORY_MB': > >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> > 63426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot process > >> > default default] Lazy-loading 'pci_devices' on Instance uuid > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading > >> > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > obj_load_attr > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > => Paused> emit_event > >> > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > >> > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power > state > >> > after lifecycle event "Paused"; current vm_state: active, current > >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> > power_state: 3 handle_lifecycle_event > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > >> instance > >> > has a pending task (image_pending_upload). Skip. > >> > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd > >> > (subprocess): qemu-img convert -t none -O raw -f raw > >> > > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> > > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad > >> > execute > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 > >> > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > => Stopped> emit_event > >> > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> > INFO nova.compute.manager [-] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > - - > >> - > >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> > _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > - - > >> - > >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing > >> > instance power state after lifecycle event "Stopped"; current > vm_state: > >> > active, current task_state: image_pending_upload, current DB > power_state: > >> > 1, VM power_state: 4 handle_lifecycle_event > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> > INFO nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - > - > >> - - > >> > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During > >> sync_power_state > >> > the instance has a pending task (image_pending_upload). Skip. > >> > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync > for > >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> > "25c8d676-e20a-4238 > >> > -a45c-d51daa62b941" acquired by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: waited 0.000s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > >> instance > >> > has a pending task (image_pending_upload). Skip. > >> > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> > "25c8d676-e20a-4238-a45c-d51daa62b941" released by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: held 0.001s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 > >> > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img > convert > >> > -t none -O raw -f raw > >> > > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> > > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" > >> > returned: 0 in 23.730s execute > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 > >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > >> Snapshot > >> > extracted, beginning image upload > >> > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 > >> > => Started> emit_event > >> > > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) > >> > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> > _get_power_state > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power > state > >> > after lifecycle event "Started"; current vm_state: active, current > >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> > power_state: 1 handle_lifecycle_event > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager > >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event > >> > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock > >> > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network > info > >> > cache for instance _get_instance_nw_info > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 > >> > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache > with > >> > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > "address": > >> > "fa:16:3e:ca:00:d9", "network": {"id": > >> > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", "label": > >> > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": > >> > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, > >> > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": > {}}], > >> > "gateway": {"address": "141.11.42.1", "type": "gateway", "version": 4, > >> > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", > >> > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], > "version": > >> 4, > >> > "meta": {}}], "meta": {"injected": true, "tenant_id": > >> > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, "physical_network": > >> > "physnet1", "tunneled": false}}, "type": "ovs", "details": > >> {"connectivity": > >> > "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": > >> > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", > >> > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > "qbh_params": > >> > null, "qbg_params": null, "active": true, "vnic_type": "normal", > >> "profile": > >> > {}, "preserve_on_delete": false, "meta": {}}] > >> > update_instance_cache_with_nw_info > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 > >> > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > >> > compute host and has allocations in placement: {'resources': {'VCPU': > 1, > >> > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. > _remove_deleted_instances_allocations > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager > >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to snapshot > the > >> > instance on the hypervisor. > >> > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync > for > >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> > > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> > :: waited 0.000s inner > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker > >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute > >> host > >> > and has allocations in placement: {'resources': {'VCPU': 1, > 'MEMORY_MB': > >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> > > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager > >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> 93fb420b3c604d4fae408b81135b58e9 > >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> > > >> > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: > >> > > >> >> Have you tried with debug logs? Has it worked with live snapshots > >> >> before for other instances or did it never work and all snapshots > were > >> >> "cold"? > >> >> > >> >> Zitat von Parsa Aminian : > >> >> > >> >> > Hi > >> >> > kolla-ansible victoria version with ceph backend without volume > >> >> > > >> >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block > wrote: > >> >> > > >> >> >> Hi, > >> >> >> > >> >> >> can you share more details about your environment? Which openstack > >> >> >> version is it? What is the storage backend? In earlier releases > there > >> >> >> was an option: > >> >> >> > >> >> >> #disable_libvirt_livesnapshot = false > >> >> >> > >> >> >> but this option has been deprecated. But if you're on an older > >> version > >> >> >> that could explain it. > >> >> >> > >> >> >> Zitat von Parsa Aminian : > >> >> >> > >> >> >> > When I snapshot from the instance , server will gone away and > its > >> not > >> >> >> > reachable until the snapshot is complete here is the logs : > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager > >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle > Event) > >> >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager > >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state > the > >> >> >> instance > >> >> >> > has a pending task (image_pending_upload). Skip. > >> >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] > [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle > Event) > >> >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver > >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> >> >> 93fb420b3c604d4fae408b81135b58e9 > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, > beginning > >> >> image > >> >> >> > upload > >> >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager > >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle > Event) > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> > >> > >> > >> > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Mon Jun 13 16:44:11 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 13 Jun 2022 11:44:11 -0500 Subject: [all][tc] Technical Committee next weekly meeting on 16 June 2022 at 1500 UTC Message-ID: <1815df38dec.b4d59475189195.5924549091605681735@ghanshyammann.com> Hello Everyone, The technical Committee's next weekly meeting is scheduled for 16 June 2022, at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, 15 June at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann From eblock at nde.ag Mon Jun 13 17:03:42 2022 From: eblock at nde.ag (Eugen Block) Date: Mon, 13 Jun 2022 17:03:42 +0000 Subject: snapshot problem In-Reply-To: References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> Message-ID: <20220613170342.Horde.qRaR_2eecVXaGrvWHuz6tC7@webmail.nde.ag> Hi, I don't have a containerized deployment so I don't have anything to compare, but my compute nodes have a ceph.conf and all required keyrings in /etc/ceph/, maybe someone else can confirm that it's missing and should be present in a kolla-ansible deployment. In the meantime you could provide those files to the container and retry the live snapshot. Zitat von Parsa Aminian : > snapshot_image_format doesn't exist on compute or controller nova.conf > For the manual snapshot on the previous reply I ran it directly from ceph, > but it's interesting from nova compute there is an error about connecting > to CEPH cluster !!! also ceph -s is showing similar error > [root at R3SG4 ~]# docker exec -u root -it nova_compute rbd snap create > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap2022-06-13 > 13:42:18.061 7f03dac23200 -1 auth: unable to find a keyring on > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: > (2) No such file or directory2022-06-13 13:42:18.061 7f03dac23200 -1 > AuthRegistry(0x561d7014a790) no keyring found at > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, > disabling ceph2022-06-13 13:42:18.065 7f03dac23200 -1 monclient: > authenticate NOTE: no keyring found; disabled cephx authentication > rbd: couldn't connect to the cluster! > > On Mon, Jun 13, 2022 at 12:15 PM Eugen Block wrote: > >> Could you share your whole nova.conf (only the uncommented options)? >> Is this option set in your env? >> >> #snapshot_image_format = >> >> Also when you manually created the snapshot did you do it as the nova >> user on the compute node? If not, could you retry? >> >> >> Zitat von Parsa Aminian : >> >> > Hi >> > kolla-ansible victoria version with ceph backend . >> > rbd info output : >> > rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': >> > size 20 GiB in 5120 objects >> > order 22 (4 MiB objects) >> > snapshot_count: 0 >> > id: b69aaf907284da >> > block_name_prefix: rbd_data.b69aaf907284da >> > format: 2 >> > features: layering, exclusive-lock, object-map, fast-diff, >> > deep-flatten >> > op_features: >> > flags: >> > create_timestamp: Fri May 20 00:04:47 2022 >> > access_timestamp: Sun Jun 12 16:26:02 2022 >> > --------------- >> > also live snapshot seems to work correctly without any error or any >> > downtime : >> > docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls >> > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk >> > SNAPID NAME SIZE PROTECTED TIMESTAMP >> > 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 >> > >> > also on compute nova.conf, images_type is set on rbd . >> > >> > On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: >> > >> >> You should respond to the list so other users can try to support you. >> >> >> >> So nova is trying to live snapshot the instance: >> >> >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - >> >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] >> >> > instance snapshotting >> >> > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning >> >> > live snapshot process >> >> >> >> But I don't see any 'rbd snap create' command. Either the rbd image >> >> doesn't support it or there is some setting to keep all rbd images >> >> "flat". Can you check any relevant configs you might have in nova? >> >> Also can you show the output of 'rbd info >> >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if >> >> the underlying rbd functions work as expected you could try to create >> >> a live snapshot manually: >> >> >> >> rbd snap create >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap >> >> >> >> And paste any relevant output here. >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> > Its not working for any instances and all of them are paused . I >> enable >> >> > debug logs please check the logs : >> >> > >> >> > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync >> for >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> >> > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by >> >> > >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> > :: waited 0.000s inner >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> >> > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute >> >> host >> >> > and has allocations in placement: {'resources': {'VCPU': 1, >> 'MEMORY_MB': >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> >> > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> > 63426b4c87a751afaf13b758e0 - default default] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot process >> >> > default default] Lazy-loading 'pci_devices' on Instance uuid >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 >> >> > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading >> >> > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> > obj_load_attr >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 >> >> > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event >> >> > > 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> > => Paused> emit_event >> >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> >> > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> >> > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state _get_power_state >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power >> state >> >> > after lifecycle event "Paused"; current vm_state: active, current >> >> > task_state: image_pending_upload, current DB power_state: 1, VM >> >> > power_state: 3 handle_lifecycle_event >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> >> > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> >> instance >> >> > has a pending task (image_pending_upload). Skip. >> >> > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd >> >> > (subprocess): qemu-img convert -t none -O raw -f raw >> >> > >> >> >> rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf >> >> > >> >> >> /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad >> >> > execute >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 >> >> > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event >> >> > > 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> > => Stopped> emit_event >> >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> >> > INFO nova.compute.manager [-] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa >> - - >> >> - >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state >> >> > _get_power_state >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> > DEBUG nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa >> - - >> >> - >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing >> >> > instance power state after lifecycle event "Stopped"; current >> vm_state: >> >> > active, current task_state: image_pending_upload, current DB >> power_state: >> >> > 1, VM power_state: 4 handle_lifecycle_event >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> >> > INFO nova.compute.manager [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - >> - >> >> - - >> >> > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During >> >> sync_power_state >> >> > the instance has a pending task (image_pending_upload). Skip. >> >> > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync >> for >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> >> > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock >> >> > "25c8d676-e20a-4238 >> >> > -a45c-d51daa62b941" acquired by >> >> > >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> > :: waited 0.000s inner >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> >> > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> >> instance >> >> > has a pending task (image_pending_upload). Skip. >> >> > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" released by >> >> > >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> > :: held 0.001s inner >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 >> >> > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img >> convert >> >> > -t none -O raw -f raw >> >> > >> >> >> rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf >> >> > >> >> >> /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" >> >> > returned: 0 in 23.730s execute >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 >> >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] >> >> Snapshot >> >> > extracted, beginning image upload >> >> > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event >> >> > > 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> > => Started> emit_event >> >> > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> >> > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state >> >> > _get_power_state >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power >> state >> >> > after lifecycle event "Started"; current vm_state: active, current >> >> > task_state: image_pending_upload, current DB power_state: 1, VM >> >> > power_state: 1 handle_lifecycle_event >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> >> > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event >> >> > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock >> >> > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network >> info >> >> > cache for instance _get_instance_nw_info >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 >> >> > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache >> with >> >> > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", >> "address": >> >> > "fa:16:3e:ca:00:d9", "network": {"id": >> >> > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", "label": >> >> > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": >> >> > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, >> >> > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": >> {}}], >> >> > "gateway": {"address": "141.11.42.1", "type": "gateway", "version": 4, >> >> > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", >> >> > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], >> "version": >> >> 4, >> >> > "meta": {}}], "meta": {"injected": true, "tenant_id": >> >> > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, "physical_network": >> >> > "physnet1", "tunneled": false}}, "type": "ovs", "details": >> >> {"connectivity": >> >> > "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": >> >> > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", >> >> > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", >> "qbh_params": >> >> > null, "qbg_params": null, "active": true, "vnic_type": "normal", >> >> "profile": >> >> > {}, "preserve_on_delete": false, "meta": {}}] >> >> > update_instance_cache_with_nw_info >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 >> >> > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this >> >> > compute host and has allocations in placement: {'resources': {'VCPU': >> 1, >> >> > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. >> _remove_deleted_instances_allocations >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> >> > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to snapshot >> the >> >> > instance on the hypervisor. >> >> > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering sync >> for >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> >> > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by >> >> > >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> > :: waited 0.000s inner >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> >> > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this compute >> >> host >> >> > and has allocations in placement: {'resources': {'VCPU': 1, >> 'MEMORY_MB': >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations >> >> > >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> >> > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> >> 93fb420b3c604d4fae408b81135b58e9 >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> > >> >> > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: >> >> > >> >> >> Have you tried with debug logs? Has it worked with live snapshots >> >> >> before for other instances or did it never work and all snapshots >> were >> >> >> "cold"? >> >> >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> >> >> > Hi >> >> >> > kolla-ansible victoria version with ceph backend without volume >> >> >> > >> >> >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block >> wrote: >> >> >> > >> >> >> >> Hi, >> >> >> >> >> >> >> >> can you share more details about your environment? Which openstack >> >> >> >> version is it? What is the storage backend? In earlier releases >> there >> >> >> >> was an option: >> >> >> >> >> >> >> >> #disable_libvirt_livesnapshot = false >> >> >> >> >> >> >> >> but this option has been deprecated. But if you're on an older >> >> version >> >> >> >> that could explain it. >> >> >> >> >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> >> >> >> >> > When I snapshot from the instance , server will gone away and >> its >> >> not >> >> >> >> > reachable until the snapshot is complete here is the logs : >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle >> Event) >> >> >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state >> the >> >> >> >> instance >> >> >> >> > has a pending task (image_pending_upload). Skip. >> >> >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] >> [instance: >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle >> Event) >> >> >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> >> >> >> 93fb420b3c604d4fae408b81135b58e9 >> >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, >> beginning >> >> >> image >> >> >> >> > upload >> >> >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] [instance: >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle >> Event) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From cboylan at sapwetik.org Mon Jun 13 17:14:57 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 13 Jun 2022 10:14:57 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: Message-ID: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> On Fri, Jun 10, 2022, at 6:01 AM, KK CHN wrote: > List, > > I am trying Devstack installation on a Virtual Machine to try TROVE DbaaS. > > All the time ./stack.sh command stops with the same error > message at this particular point > > + functions:_upload_image:121 : openstack > --os-cloud=devstack-admin --os-region-name=RegionOne image create > cirros-0.5.2-x86_64-disk --public --container-format bare --disk-format > qcow2 --property hw_rng_model=virtio > Internal Server Error (HTTP 500) You should examine the glance api logs to see why the service had an error. > > I have ./unstack.sh && ./clean.sh and re run ./stack.sh > multiple time but no progress. Here always the script exit with error > . Screenshot attached > (using "stack" user created by the devstack/tool/ stack user script) > > my local.conf file as follows > here 10.184.48.187 is my VMs IP > > Any hints to rectify this error most welcome!! > > ############################################################ > [[local|localrc]] > RECLONE=False > HOST_IP=10.184.48.187 > > enable_plugin trove https://opendev.org/openstack/trove > enable_plugin trove-dashboard https://opendev.org/openstack/trove-dashboard > > LIBS_FROM_GIT+=,python-troveclient > DATABASE_PASSWORD=password > ADMIN_PASSWORD=password > SERVICE_PASSWORD=password > SERVICE_TOKEN=password > RABBIT_PASSWORD=password > LOGFILE=$DEST/logs/stack.sh.log > VERBOSE=True > LOG_COLOR=False > LOGDAYS=1 > > IPV4_ADDRS_SAFE_TO_USE=10.111.0.0/26 > FIXED_RANGE=10.111.0.0/26 > NETWORK_GATEWAY=10.111.0.1 > FLOATING_RANGE=10.184.48.0/24 > PUBLIC_NETWORK_GATEWAY=10.184.48.1 > > #pre requisites > ENABLED_SERVICES=rabbit,mysql,key > > #nova > enable_service horizon > > enable_service n-api > > enable_service n-cpu > > enable_service n-cond > > enable_service n-sch > enable_service n-api-meta > enable_service placement-api > enable_service placement-client > > #Glance > > > enable_service g-api > enable_service g-reg > > #Cinder > enable_service cinder > enable_service c-api > enable_service c-vol > > enable_service c-sch > > #Neutron > > > enable_service q-svc > enable_service q-agt > > enable_service q-dhcp > enable_service q-l3 > enable_service q-meta > > #enable DVR > Q_AGENT=openvswitch > Q_DVR_MODE=legacy > Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch > Q_ML2_TENANT_NETWORK_TYPE=vxlan > Q_PLUGIN=ml2 > > #SWIFT > ENABLED_SERVICES+=,swift > SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 > SWIFT_REPLICAS=1 > stack at XenaCtrl1:/home/cloud/trove_devstack/devstack$ > > ################################################################# > > > > Attachments: > * DevstackError1.png From gouthampravi at gmail.com Mon Jun 13 18:12:28 2022 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Mon, 13 Jun 2022 23:42:28 +0530 Subject: [manila] CephFS NFS high availability cluster In-Reply-To: References: Message-ID: Hi Romain, On Mon, Jun 13, 2022 at 6:20 PM CHANU ROMAIN wrote: > > Hello, > > I added Manila service with CephFS NFS driver to my openstack cluster. > Everything works fine but I would like to add 2 nfs-ganesha servers to > ensure high availability to the service. > > I configured haproxy to forward 2049 to ganesha backend but Manila > cephFS NFS provides only IP restriction and see only haproxy's IP > address. To make it work you have to add haproxy to allowed ip but it > means everyone can access the share. True; HAProxy terminates client connections and NFS Ganesha sees the HAProxy's IP address instead of the client's IP address. This causes the client's mount operations to be denied since manila explicitly requests client restrictions to exports according to the share's access rules. Presumably, setting up haproxy in "transparent" mode may allow the client source IP to be preserved. We have found that this is infeasible in deployments such as the Red Hat OpenStack Platform. We're discussing with the nfs-ganesha community if they would support the PROXY protocol [1][2]. How are you orchestrating your nfs-ganesha solution with haproxy? Via a custom script? Or are you using cephadm? [3]. I ask also because the CephFS-NFS driver in manila currently communicates with NFS-Ganesha via dbus, and we're looking to support the use of new Ceph Manager APIs to setup, update and teardown exports in the Zed release - this should make configuring multiple NFS-Ganesha servers much more efficient and easy. [1] https://www.haproxy.com/blog/haproxy/proxy-protocol/ [2] https://github.com/nfs-ganesha/ntirpc/issues/252 [3] https://docs.ceph.com/en/latest/cephadm/services/nfs/ [4] https://docs.ceph.com/en/octopus/cephfs/fs-nfs-exports/#create-cephfs-export > > So currently the only way I found out is to use pacemaker to set public > vip to a running nfs-ganesha node. Could you confirm that is not > possible to provide an active/active nfs-ganesha cluster with manila > cephfs NFS driver? > > Best regards, > Romain From dms at danplanet.com Mon Jun 13 19:46:26 2022 From: dms at danplanet.com (Dan Smith) Date: Mon, 13 Jun 2022 12:46:26 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> (Clark Boylan's message of "Mon, 13 Jun 2022 10:14:57 -0700") References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> Message-ID: > You should examine the glance api logs to see why the service had an > error. This is actually not a glance problem, but a case of missing the dbcounter plugin when glance connects to the DB -- uploading an image is just the first thing we try to do, other than keystone stuff. I've been working with them on IRC and their devstack log clearly shows it is installed successfully, so I suspect something else is going on. They dropped out before we finished, but I'll continue chasing it down. --Dan From smooney at redhat.com Tue Jun 14 00:26:50 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 14 Jun 2022 01:26:50 +0100 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> Message-ID: <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> On Mon, 2022-06-13 at 12:46 -0700, Dan Smith wrote: > > You should examine the glance api logs to see why the service had an > > error. > > This is actually not a glance problem, but a case of missing the > dbcounter plugin when glance connects to the DB -- uploading an image is > just the first thing we try to do, other than keystone stuff. I've been > working with them on IRC and their devstack log clearly shows it is > installed successfully, so I suspect something else is going on. They > dropped out before we finished, but I'll continue chasing it down. i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow. in theory my ansibel role should hit it if its repoduceable reliable. i havent commit the change to my repo to disabel the perfromace counters so i shoudl be able to repoduce it either on my home server or laptop which ever does not have the change locally. ill let you know if i can and i can proably give you access to the vm if you want to take a look. > > --Dan > From adivya1.singh at gmail.com Tue Jun 14 04:56:32 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Tue, 14 Jun 2022 10:26:32 +0530 Subject: Regarding Policy.json entries for glance image update not working for a user In-Reply-To: References: Message-ID: Hi Brian, Please find the response > 1> i am using Xena release version 24.0.1 > > Now the scenario is line below, my customer wants to have their login > access on setting up the properties of an image to the public. now what i > did is > > 1> i created a role in openstack using the admin credential name as "user" > 2> i assigned that user to a role user. > 3> i assigned those user to those project id, which they want to access as > a user role > > Then i went to Glance container which is controller by lxc and made a > policy.yaml file as below > > root at aio1-glance-container-724aa778:/etc/glance# cat policy.yaml > > "modify_image": "role:user" > > then i went to utility container and try to set the properties of a image > using openstack command > > openstack image set --public > > and then i got this error > > HTTP 403 Forbidden: You are not authorized to complete publicize_image > action. > > Even when i am trying the upload image with this user , i get the above > error only > > export OS_ENDPOINT_TYPE=internalURL > export OS_INTERFACE=internalURL > export OS_USERNAME=adsingh > export OS_PASSWORD='adsingh' > export OS_PROJECT_NAME=adsingh > export OS_TENANT_NAME=adsingh > export OS_AUTH_TYPE=password > export OS_AUTH_URL=https://:5000/v3 > export OS_NO_CACHE=1 > export OS_USER_DOMAIN_NAME=Default > export OS_PROJECT_DOMAIN_NAME=Default > export OS_REGION_NAME=RegionOne > > Regards > Adivya Singh > > > > > On Mon, Jun 13, 2022 at 6:41 PM Alan Bishop wrote: > >> >> >> On Mon, Jun 13, 2022 at 6:00 AM Brian Rosmaita < >> rosmaita.fossdev at gmail.com> wrote: >> >>> On 6/13/22 8:29 AM, Adivya Singh wrote: >>> > hi Team, >>> > >>> > Any thoughts on this >>> >>> H Adivya, >>> >>> Please supply some more information, for example: >>> >>> - which openstack release you are using >>> - the full API request you are making to modify the image >>> - the full API response you receive >>> - whether the user with "role:user" is in the same project that owns the >>> image >>> - debug level log extract for this call if you have it >>> - anything else that could be relevant, for example, have you modified >>> any other policies, and if so, what values are you using now? >>> >> >> Also bear in mind that the default policy_file name is "policy.yaml" (not >> .json). You either >> need to provide a policy.yaml file, or override the policy_file setting >> if you really want to >> use policy.json. >> >> Alan >> >> cheers, >>> brian >>> >>> > >>> > Regards >>> > Adivya Singh >>> > >>> > On Sat, Jun 11, 2022 at 12:40 AM Adivya Singh >> > > wrote: >>> > >>> > Hi Team, >>> > >>> > I have a use case where I have to give a user restriction on >>> > updating the image properties as a member. >>> > >>> > I have created a policy Json file and give the modify_image rule to >>> > the particular role, but still it is not working >>> > >>> > "modify_image": "role:user", This role is created in OpenStack. >>> > >>> > but still it is failing while updating properties with a >>> > particular user assigned to a role as "access denied" and >>> > unauthorized access >>> > >>> > Regards >>> > Adivya Singh >>> > >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From tkajinam at redhat.com Tue Jun 14 05:24:33 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Tue, 14 Jun 2022 14:24:33 +0900 Subject: Regarding Policy.json entries for glance image update not working for a user In-Reply-To: References: Message-ID: Glance has a separate policy rule (publicize_image) for creating/updating public images., and you should define that policy rule instead of modify_image. https://docs.openstack.org/glance/xena/admin/policies.html ~~~ publicize_image - Create or update public images ~~~ AFAIK The modify_image policy defaults to rule:default and is allowed for any users as long as the target image is owned by that user. On Tue, Jun 14, 2022 at 2:01 PM Adivya Singh wrote: > Hi Brian, > > Please find the response > > >> 1> i am using Xena release version 24.0.1 >> >> Now the scenario is line below, my customer wants to have their login >> access on setting up the properties of an image to the public. now what i >> did is >> >> 1> i created a role in openstack using the admin credential name as "user" >> 2> i assigned that user to a role user. >> 3> i assigned those user to those project id, which they want to access >> as a user role >> >> Then i went to Glance container which is controller by lxc and made a >> policy.yaml file as below >> >> root at aio1-glance-container-724aa778:/etc/glance# cat policy.yaml >> >> "modify_image": "role:user" >> >> then i went to utility container and try to set the properties of a image >> using openstack command >> >> openstack image set --public >> >> and then i got this error >> >> HTTP 403 Forbidden: You are not authorized to complete publicize_image >> action. >> >> Even when i am trying the upload image with this user , i get the above >> error only >> >> export OS_ENDPOINT_TYPE=internalURL >> export OS_INTERFACE=internalURL >> export OS_USERNAME=adsingh >> export OS_PASSWORD='adsingh' >> export OS_PROJECT_NAME=adsingh >> export OS_TENANT_NAME=adsingh >> export OS_AUTH_TYPE=password >> export OS_AUTH_URL=https://:5000/v3 >> export OS_NO_CACHE=1 >> export OS_USER_DOMAIN_NAME=Default >> export OS_PROJECT_DOMAIN_NAME=Default >> export OS_REGION_NAME=RegionOne >> >> Regards >> Adivya Singh >> >> >> >> >> On Mon, Jun 13, 2022 at 6:41 PM Alan Bishop wrote: >> >>> >>> >>> On Mon, Jun 13, 2022 at 6:00 AM Brian Rosmaita < >>> rosmaita.fossdev at gmail.com> wrote: >>> >>>> On 6/13/22 8:29 AM, Adivya Singh wrote: >>>> > hi Team, >>>> > >>>> > Any thoughts on this >>>> >>>> H Adivya, >>>> >>>> Please supply some more information, for example: >>>> >>>> - which openstack release you are using >>>> - the full API request you are making to modify the image >>>> - the full API response you receive >>>> - whether the user with "role:user" is in the same project that owns >>>> the >>>> image >>>> - debug level log extract for this call if you have it >>>> - anything else that could be relevant, for example, have you modified >>>> any other policies, and if so, what values are you using now? >>>> >>> >>> Also bear in mind that the default policy_file name is "policy.yaml" >>> (not .json). You either >>> need to provide a policy.yaml file, or override the policy_file setting >>> if you really want to >>> use policy.json. >>> >>> Alan >>> >>> cheers, >>>> brian >>>> >>>> > >>>> > Regards >>>> > Adivya Singh >>>> > >>>> > On Sat, Jun 11, 2022 at 12:40 AM Adivya Singh < >>>> adivya1.singh at gmail.com >>>> > > wrote: >>>> > >>>> > Hi Team, >>>> > >>>> > I have a use case where I have to give a user restriction on >>>> > updating the image properties as a member. >>>> > >>>> > I have created a policy Json file and give the modify_image rule >>>> to >>>> > the particular role, but still it is not working >>>> > >>>> > "modify_image": "role:user", This role is created in OpenStack. >>>> > >>>> > but still it is failing while updating properties with a >>>> > particular user assigned to a role as "access denied" and >>>> > unauthorized access >>>> > >>>> > Regards >>>> > Adivya Singh >>>> > >>>> >>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kamil.madac at gmail.com Tue Jun 14 06:29:49 2022 From: kamil.madac at gmail.com (Kamil Madac) Date: Tue, 14 Jun 2022 08:29:49 +0200 Subject: [cinder] Failed to delete volume In-Reply-To: References: Message-ID: Hi Michal, We had similar issue in our kolla-ansible deployment. In our case we were not able to attach volume to the VM in 1 from 5 cases and it sometimes caused also inconsistencies in DB tables. We are also running Victoria release with Ceph backend. We found out that there were many disconnects/reconnects between nova and rabbitmq because of missing heartbeats which happened every 3 minutes. If attachment of volume happened in the time of reconnection, attachment was unsuccessful. What helped in our case was adding following lines to nova.conf: [oslo_messaging_rabbit] heartbeat_timeout_threshold = 0 In your case I would check cinder logs files for messaging errors whether you do not see regular reconnects to rabbitmq. Kamil Madac On Fri, Jun 10, 2022 at 2:08 PM Michal Arbet wrote: > Hi, > > Cinder is from victoria, backend is ceph, steps to reproduce ? Just delete > volume. > I think that it could be an error not in cinder but somehow a rabbitmq > issue. > > :/ > > Michal Arbet > Openstack Engineer > > Ultimum Technologies a.s. > Na Po???? 1047/26, 11000 Praha 1 > Czech Republic > > +420 604 228 897 > michal.arbet at ultimum.io > *https://ultimum.io * > > LinkedIn | Twitter > | Facebook > > > > ?t 7. 6. 2022 v 19:27 odes?latel Sofia Enriquez > napsal: > >> Hi Michal, >> >> I do not think we have such a bug on the Launchpad platform.[1] >> Could you create a new bug report indicating which backend you are using, >> which Cinder version you have, and if the error occurs after following some >> steps? >> >> Thanks in advance, >> Sofia >> >> [1] https://bugs.launchpad.net/cinder >> >> On Tue, Jun 7, 2022 at 4:28 AM Michal Arbet >> wrote: >> >>> Hi, >>> >>> I would like to ask if someone saw issue when volume failed to delete, >>> log below : >>> This is happening from time to time. >>> >>> LOG : https://paste.opendev.org/show/b1jRCxuBFnnVzf64i9yV/ >>> >>> Thanks, >>> >>> Michal Arbet >>> Openstack Engineer >>> >>> Ultimum Technologies a.s. >>> Na Po???? 1047/26, 11000 Praha 1 >>> Czech Republic >>> >>> +420 604 228 897 >>> michal.arbet at ultimum.io >>> *https://ultimum.io * >>> >>> LinkedIn | >>> Twitter | Facebook >>> >>> >> >> >> -- >> >> Sof?a Enriquez >> >> she/her >> >> Software Engineer >> >> Red Hat PnT >> >> IRC: @enriquetaso >> @RedHat Red Hat >> Red Hat >> >> >> >> -- Kamil Madac -------------- next part -------------- An HTML attachment was scrubbed... URL: From geguileo at redhat.com Tue Jun 14 07:38:46 2022 From: geguileo at redhat.com (Gorka Eguileor) Date: Tue, 14 Jun 2022 09:38:46 +0200 Subject: [cinder] Failed to delete volume In-Reply-To: References: Message-ID: <20220614073846.fd7pdx4hwe7joqpz@localhost> On 07/06, Michal Arbet wrote: > Hi, > > I would like to ask if someone saw issue when volume failed to delete, log > below : > This is happening from time to time. > > LOG : https://paste.opendev.org/show/b1jRCxuBFnnVzf64i9yV/ > > Thanks, > > Michal Arbet > Openstack Engineer > Hi, Error seems to be happening acquiring a lock with the Redis driver in Tooz, so it must be configured to be used by Cinder as the DLM. This is only necessary when the Cinder Volume service is configured as Active-Active. Are you running the volume service as Active-Active? If you are not you should be able to fix the issue by commenting the backend_url configuration option in the [coordination] section of the cinder.conf file. Cheers, Gorka. > Ultimum Technologies a.s. > Na Po???? 1047/26, 11000 Praha 1 > Czech Republic > > +420 604 228 897 > michal.arbet at ultimum.io > *https://ultimum.io * > > LinkedIn | Twitter > | Facebook > From fereshtehloghmani at gmail.com Tue Jun 14 07:50:44 2022 From: fereshtehloghmani at gmail.com (fereshteh loghmani) Date: Tue, 14 Jun 2022 12:20:44 +0430 Subject: evacuate problem Message-ID: hello i use masakari for migrate servers when the compute being down. i install these container on the region: masakari-monitors masakari-engine masakari-api hacluster-pacemaker hacluster-corosync and on the compute server i install "hacluster-pacemaker-remote". In openstack i created segment and I added 2 hosts (in this case the name of that 2 hosts are: R3SG5 & R3SG12). for testing evacuate function i shut off one of that compute that i added in host. (in this case i shutoff R3SG5) i attached the log that i found in this directory: /var/log/kolla/masakari/masakari-hostmonitor.log ********* INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG5' is 'online' (current: 'online'). INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG12' is 'online' (current: 'online'). WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication using 'eth0' is failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication is failed. INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG5' is 'offline' (current: 'offline'). INFO masakarimonitors.ha.masakari [-] Send a notification. {'notification': {'type': 'COMPUTE_HOST', 'hostname': 'R3SG5', 'generated_time': datetime.datetime(2022, 6, 14, 7, 6, 46, 138867), 'payload': {'event': 'STOPPED', 'cluster_status': 'OFFLINE', 'host_status': 'NORMAL'}}} INFO masakarimonitors.ha.masakari [-] Response: openstack.instance_ha.v1.notification.Notification(type=COMPUTE_HOST, hostname=R3SG5, generated_time=2022-06-14T07:06:46.138867, payload={'event': 'STOPPED', 'cluster_status': 'OFFLINE', 'host_status': 'NORMAL'}, id=105, notification_uuid=a7364095-cc7d-48f8-b963-c64ba147897c, source_host_uuid=6328f08c-c752-43d5-4689-801d91dd67ec, status=new, created_at=2022-06-14T07:06:47.000000, updated_at=None, location=Munch({'cloud': 'controller', 'region_name': 'RegionThree', 'zone': None, 'project': Munch({'id': 'a75a951b4537478e8cea39a932f830da', 'name': None, 'domain_id': None, 'domain_name': None})})) INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG12' is 'online' (current: 'online'). WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication using 'eth0' is failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication is failed. INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG5' is 'offline' (current: 'offline'). INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG12' is 'online' (current: 'online'). WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync communication using 'eth0' is failed.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. ***************** also i checked nova_scheduler logs and on that directory i receive error: *** ERROR oslo_messaging.rpc.server nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. ******* finally in OpenStack dashboard in the notification section tatus change from running to failed. after the error state that shows in the notification section, my VM that was on R3SG5 became to ERROR state and the VM still exists on R3SG5 and it doenst been migrated to R3SG12. could you please help me why evacuate function doesn't work correctly? -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Tue Jun 14 08:07:26 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 14 Jun 2022 10:07:26 +0200 Subject: simple build fails to allocate network interface In-Reply-To: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> References: <9A546D4D-7637-4021-9BBB-EDF756E54EAE@coote.org> Message-ID: <12175423.SmF1yDmi0e@p1> Hi, Dnia czwartek, 9 czerwca 2022 10:58:46 CEST tim+openstack.org at coote.org pisze: > Hullo > I?m trying to spin up a simple OS environment in a single vm and think that I?m making some silly mistake, but cannot spot it. Any hints would be greatly expected. I?ve tried this on a couple of laptops, with the same result, so I suspect that it?s something to do with how I?ve set up my vm, but I cannot identify it. (I think that there may be a few other devstack issues, but I?ve got to get it working to log any bugs). > > I?m using a vagrant box and VirtualBox on x86 Macs (Monterey), and the devstack install/build process. Everything seems to install, but when I try: > > `openstack server create --flavor 42 --image cirros-0.5.2-x86_64-disk --nic net-id=2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 --security-group default wibble` > > The network id is identified from this: > ??" > [vagrant at localhost devstack]$ openstack network list > +--------------------------------------+---------+----------------------------------------------------------------------------+ > | ID | Name | Subnets | > +--------------------------------------+---------+----------------------------------------------------------------------------+ > | 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f | public | 1bbbfa3c-8e0a-483f-a084-4e38241b3315, eecfc603-c057-4b28-90bd-950a47345410 | > | 2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63 | private | e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd, f915b88c-9988-4e1f-9060-a6295465699a | > | fab3fb15-cbd2-45fe-8451-3f0063d59454 | shared | 97e6d8e6-0f57-48bc-8f1d-e30587086153 | > +--------------------------------------+---------+??????????????????????????????????????+ > ??? > > Using a public network fails with an error that this isn?t allowed. > > With the private network above, the failure throws out the following errors. > ??" > [vagrant at localhost devstack]$ sudo journalctl -f |grep ERROR > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2728, in _build_resources > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] yield resources > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7510, in _create_guest_with_network > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] guest = self._create_guest( > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] next(self.gen) > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 556, in wait_for_instance_event > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._wait_for_instance_events( > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 468, in _wait_for_instance_events > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] actual_event = event.wait() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 433, in wait > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] instance_event = self.event.wait() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/event.py", line 125, in wait > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] result = hub.switch() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py", line 313, in switch > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] return self.greenlet.switch() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] eventlet.timeout.Timeout: 300 seconds > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] During handling of the above exception, another exception occurred: > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Traceback (most recent call last): > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/compute/manager.py", line 2487, in _build_and_run_instance > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self.driver.spawn(context, instance, image_meta, > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4344, in spawn > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] self._create_guest_with_network( > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 7528, in _create_guest_with_network > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] raise exception.VirtualInterfaceCreateException() > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] > Jun 08 15:33:52 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager [None req-01294678-b54a-4e40-964c-488ccf03d66c demo demo] [instance: 20e1b5bf-130a-48f9-af4e-9b4c7a475315] Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance 20e1b5bf-130a-48f9-af4e-9b4c7a475315 aborted: Failed to allocate the network(s), not rescheduling. > ??" > > The Vagrantfile (commented out lines relate to trying this with Ubunutu, which failed the same way, and attempts to use multiple vms): > ??" > # -*- mode: ruby -*- > # vi: set ft=ruby : > > $script = <<-SHELL > dnf update -y > #apt update > #apt upgrade > echo "updated" > systemctl disable firewalld > systemctl stop firewalld > systemctl disable NetworkManager > systemctl stop NetworkManager > systemctl enable network > systemctl start network > #dnf config-manager --enable powertools > #dnf install -y centos-release-openstack-yoga > dnf update -y > dnf install -y git python3-pip > #dnf install -y openstack-packstack > #packstack --provision-demo=n --os-neutron-ml2-type-drivers=vxlan,flat,vlan --gen-answer-file=packstack-answers.txt > #sed -i -e 's:10.0.2.15:10.1.0.10:' packstack-answers.txt > # not for centos 9 > #dnf install -y network-scripts > dnf install -y net-tools > #cat /vagrant/answers.addon >> packstack-answers.txt > # don't want this, but httpd won't start without it > setenforce 0 > pip3 install invoke > #packstack --allinone > cd /vagrant > # permissions all wrong: ./go.sh # need su - vagrant cd /vagrant && ./go.sh (?) > # cd devstack > # not working: wrong user ./stack.sh > SHELL > > Vagrant.configure(2) do |config| > > config.vm.box = "eurolinux-vagrant/centos-stream-9" > #config.vm.box = "hashicorp/bionic64" > > # machines = { > # 'node1.example.dd' > #'node1.example.dd' => { :ip => '10.1.0.10'}, > # 'node2.example.dd' => { :ip =>'10.1.0.12'}, > # } > > config.hostmanager.enabled = true > config.hostmanager.manage_host = true > config.hostmanager.manage_guest = true > config.hostmanager.ignore_private_ip = false > config.hostmanager.include_offline = true > > config.ssh.pty = true > > config.vm.provision "shell", inline: $script > > config.vm.network "forwarded_port", guest: 80, host: 8080 > > # machines.each do | hostname, attrs| > # config.vm.define hostname do |machine| > # machine.vm.hostname = hostname > # machine.vm.network :private_network, :ip => attrs[:ip] > > # machine.vm.provider "virtualbox" do | v | > config.vm.provider "virtualbox" do | v | > #v.memory = "4096" > #v.memory = "8192" > v.memory = "9216" > v.cpus = "2" > end > > #end > # end > end > ??? > > I think that the vm has enough RAM, although there is minimal swap being used, but I think that this is not significant as there is much more RAM used for caching files. > > Any obvious hints as to why the spin up fails to create the NIC - or somewhere to look for further and better details? > > Tim > This is some issue in neutron while provisioning port. Please check neutron server and neutron agents logs (depending on You backend it can be neutron-ovs-agent and neutron-dhcp-agent, or in case of OVN most of the things You will find in the neutron-server logs). -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From marios at redhat.com Tue Jun 14 10:33:35 2022 From: marios at redhat.com (Marios Andreou) Date: Tue, 14 Jun 2022 13:33:35 +0300 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) Message-ID: Hello o/ as proposed at [1][2] we are moving ahead with transitioning stable/victoria tripleo repos to End Of Life. In order to do this we need to have no open patches against stable/victoria. So please stop posting new patches for tripleo stable/victoria and if you have open patches please merge or abandon these by next Tuesday 21st June (for reference, list of repos with currently open patches at [3]). No problem if someone needs a few more days we can wait just please reach out here or on irc. If there are no such requests then I will abandon all open reviews on 21st and respin [2] to pick up any new commits between now and then. thanks for your help, marios [1] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028829.html [2] https://review.opendev.org/c/openstack/releases/+/845148 [3] (list of repos with open reviews): * https://review.opendev.org/q/project:openstack%252Fos-net-config+status:open+branch:stable/victoria * https://review.opendev.org/q/project:openstack%252Fpython-tripleoclient+status:open+branch:stable/victoria * https://review.opendev.org/q/project:openstack%252Ftripleo-ansible+status:open+branch:stable/victoria * https://review.opendev.org/q/project:openstack%252Ftripleo-common+status:open+branch:stable/victoria * https://review.opendev.org/q/project:openstack%252Ftripleo-heat-templates+status:open+branch:stable/victoria * https://review.opendev.org/q/project:openstack%252Ftripleo-image-elements+status:open+branch:stable/victoria * https://review.opendev.org/q/project:openstack%252Ftripleo-puppet-elements+status:open+branch:stable/victoria * https://review.opendev.org/q/project:openstack%252Ftripleo-validations+status:open+branch:stable/victoria From tkajinam at redhat.com Tue Jun 14 11:40:30 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Tue, 14 Jun 2022 20:40:30 +0900 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: References: Message-ID: I'd like to remind you that we should check *whether that patch has been backported to any older branches (ussuri, train)* we abandon any victoria backport. In the past, we abandoned backport without checking this and this resulted in broken commit id in some backports. (stable/train backports point to the commit id of ussuri backport, which was never merged). If the patch is already merged in ussuri or train then IMO we should try landing the victoria backport. If not, then please make sure you update the stable/train backport to remove any reference to the victoria backport. On Tue, Jun 14, 2022 at 7:41 PM Marios Andreou wrote: > Hello o/ > > as proposed at [1][2] we are moving ahead with transitioning > stable/victoria tripleo repos to End Of Life. > > In order to do this we need to have no open patches against > stable/victoria. > > So please stop posting new patches for tripleo stable/victoria and if > you have open patches please merge or abandon these by next Tuesday > 21st June (for reference, list of repos with currently open patches at > [3]). > > No problem if someone needs a few more days we can wait just please > reach out here or on irc. If there are no such requests then I will > abandon all open reviews on 21st and respin [2] to pick up any new > commits between now and then. > > thanks for your help, > > marios > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028829.html > [2] https://review.opendev.org/c/openstack/releases/+/845148 > [3] (list of repos with open reviews): > * > https://review.opendev.org/q/project:openstack%252Fos-net-config+status:open+branch:stable/victoria > * > https://review.opendev.org/q/project:openstack%252Fpython-tripleoclient+status:open+branch:stable/victoria > * > https://review.opendev.org/q/project:openstack%252Ftripleo-ansible+status:open+branch:stable/victoria > * > https://review.opendev.org/q/project:openstack%252Ftripleo-common+status:open+branch:stable/victoria > * > https://review.opendev.org/q/project:openstack%252Ftripleo-heat-templates+status:open+branch:stable/victoria > * > https://review.opendev.org/q/project:openstack%252Ftripleo-image-elements+status:open+branch:stable/victoria > * > https://review.opendev.org/q/project:openstack%252Ftripleo-puppet-elements+status:open+branch:stable/victoria > * > https://review.opendev.org/q/project:openstack%252Ftripleo-validations+status:open+branch:stable/victoria > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Tue Jun 14 11:59:42 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 14 Jun 2022 11:59:42 +0000 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: References: Message-ID: <20220614115942.wqiaqk7hpa2hzdlk@yuggoth.org> On 2022-06-14 20:40:30 +0900 (+0900), Takashi Kajinami wrote: [...] > In the past, we abandoned backport without checking this and this > resulted in broken commit id in some backports. (stable/train > backports point to the commit id of ussuri backport, which was > never merged). [...] Not sure if it's too heavy-handed, but one workaround would be to add a Depends-On footer in the commit message of each backport referencing the change for the branch ahead of it. Then, Zuul would only ever allow the backport to merge if the same change on all newer branches had already merged successfully. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From marios at redhat.com Tue Jun 14 12:22:03 2022 From: marios at redhat.com (Marios Andreou) Date: Tue, 14 Jun 2022 15:22:03 +0300 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: References: Message-ID: On Tue, Jun 14, 2022 at 2:40 PM Takashi Kajinami wrote: > > I'd like to remind you that we should check whether that patch has been backported to any older branches (ussuri, train) > we abandon any victoria backport. > > In the past, we abandoned backport without checking this and this resulted in broken commit id in some backports. > (stable/train backports point to the commit id of ussuri backport, which was never merged). > > If the patch is already merged in ussuri or train then IMO we should try landing the victoria backport. > If not, then please make sure you update the stable/train backport to remove any reference to the victoria backport. > > OK thanks for the reminder. Noted, I'll check this next week and ideally folks will be reading this thread and checking their patches ;) regards > On Tue, Jun 14, 2022 at 7:41 PM Marios Andreou wrote: >> >> Hello o/ >> >> as proposed at [1][2] we are moving ahead with transitioning >> stable/victoria tripleo repos to End Of Life. >> >> In order to do this we need to have no open patches against stable/victoria. >> >> So please stop posting new patches for tripleo stable/victoria and if >> you have open patches please merge or abandon these by next Tuesday >> 21st June (for reference, list of repos with currently open patches at >> [3]). >> >> No problem if someone needs a few more days we can wait just please >> reach out here or on irc. If there are no such requests then I will >> abandon all open reviews on 21st and respin [2] to pick up any new >> commits between now and then. >> >> thanks for your help, >> >> marios >> >> [1] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028829.html >> [2] https://review.opendev.org/c/openstack/releases/+/845148 >> [3] (list of repos with open reviews): >> * https://review.opendev.org/q/project:openstack%252Fos-net-config+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Fpython-tripleoclient+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-ansible+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-common+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-heat-templates+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-image-elements+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-puppet-elements+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-validations+status:open+branch:stable/victoria >> >> From tkajinam at redhat.com Tue Jun 14 12:22:37 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Tue, 14 Jun 2022 21:22:37 +0900 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: <20220614115942.wqiaqk7hpa2hzdlk@yuggoth.org> References: <20220614115942.wqiaqk7hpa2hzdlk@yuggoth.org> Message-ID: On Tue, Jun 14, 2022 at 9:04 PM Jeremy Stanley wrote: > On 2022-06-14 20:40:30 +0900 (+0900), Takashi Kajinami wrote: > [...] > > In the past, we abandoned backport without checking this and this > > resulted in broken commit id in some backports. (stable/train > > backports point to the commit id of ussuri backport, which was > > never merged). > [...] > > Not sure if it's too heavy-handed, but one workaround would be to > add a Depends-On footer in the commit message of each backport > referencing the change for the branch ahead of it. Then, Zuul would > only ever allow the backport to merge if the same change on all > newer branches had already merged successfully. > Yeah I think that would be a solution though it requires relatively big overhead, because we need to manually add that Depends-on line. I sometimes hope zuul can block backports depending on the "cherry picked from commit ..." line but it might be worth discussing as a potential enhancement with the zuul devs. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.aminian.server at gmail.com Tue Jun 14 06:50:37 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Tue, 14 Jun 2022 11:20:37 +0430 Subject: snapshot problem In-Reply-To: <20220613170342.Horde.qRaR_2eecVXaGrvWHuz6tC7@webmail.nde.ag> References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> <20220613170342.Horde.qRaR_2eecVXaGrvWHuz6tC7@webmail.nde.ag> Message-ID: yes you are right , I added ceph.client.admin.keyring to nova compute container and ceph cluster error gone i also try rbd snap create from nova compute container and its work without problem and vm is up . maybe the problem is related to glance .my glance backend is not on ceph I think maybe its the reason On Mon, Jun 13, 2022 at 9:38 PM Eugen Block wrote: > Hi, > > I don't have a containerized deployment so I don't have anything to > compare, but my compute nodes have a ceph.conf and all required > keyrings in /etc/ceph/, maybe someone else can confirm that it's > missing and should be present in a kolla-ansible deployment. In the > meantime you could provide those files to the container and retry the > live snapshot. > > > Zitat von Parsa Aminian : > > > snapshot_image_format doesn't exist on compute or controller nova.conf > > For the manual snapshot on the previous reply I ran it directly from > ceph, > > but it's interesting from nova compute there is an error about connecting > > to CEPH cluster !!! also ceph -s is showing similar error > > [root at R3SG4 ~]# docker exec -u root -it nova_compute rbd snap create > > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap2022-06-13 > > 13:42:18.061 7f03dac23200 -1 auth: unable to find a keyring on > > > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: > > (2) No such file or directory2022-06-13 13:42:18.061 7f03dac23200 -1 > > AuthRegistry(0x561d7014a790) no keyring found at > > > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, > > disabling ceph2022-06-13 13:42:18.065 7f03dac23200 -1 monclient: > > authenticate NOTE: no keyring found; disabled cephx authentication > > rbd: couldn't connect to the cluster! > > > > On Mon, Jun 13, 2022 at 12:15 PM Eugen Block wrote: > > > >> Could you share your whole nova.conf (only the uncommented options)? > >> Is this option set in your env? > >> > >> #snapshot_image_format = > >> > >> Also when you manually created the snapshot did you do it as the nova > >> user on the compute node? If not, could you retry? > >> > >> > >> Zitat von Parsa Aminian : > >> > >> > Hi > >> > kolla-ansible victoria version with ceph backend . > >> > rbd info output : > >> > rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': > >> > size 20 GiB in 5120 objects > >> > order 22 (4 MiB objects) > >> > snapshot_count: 0 > >> > id: b69aaf907284da > >> > block_name_prefix: rbd_data.b69aaf907284da > >> > format: 2 > >> > features: layering, exclusive-lock, object-map, fast-diff, > >> > deep-flatten > >> > op_features: > >> > flags: > >> > create_timestamp: Fri May 20 00:04:47 2022 > >> > access_timestamp: Sun Jun 12 16:26:02 2022 > >> > --------------- > >> > also live snapshot seems to work correctly without any error or any > >> > downtime : > >> > docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls > >> > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk > >> > SNAPID NAME SIZE PROTECTED TIMESTAMP > >> > 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 > >> > > >> > also on compute nova.conf, images_type is set on rbd . > >> > > >> > On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: > >> > > >> >> You should respond to the list so other users can try to support you. > >> >> > >> >> So nova is trying to live snapshot the instance: > >> >> > >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - > >> >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > >> >> > instance snapshotting > >> >> > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning > >> >> > live snapshot process > >> >> > >> >> But I don't see any 'rbd snap create' command. Either the rbd image > >> >> doesn't support it or there is some setting to keep all rbd images > >> >> "flat". Can you check any relevant configs you might have in nova? > >> >> Also can you show the output of 'rbd info > >> >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if > >> >> the underlying rbd functions work as expected you could try to create > >> >> a live snapshot manually: > >> >> > >> >> rbd snap create > >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap > >> >> > >> >> And paste any relevant output here. > >> >> > >> >> Zitat von Parsa Aminian : > >> >> > >> >> > Its not working for any instances and all of them are paused . I > >> enable > >> >> > debug logs please check the logs : > >> >> > > >> >> > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager > >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering > sync > >> for > >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> >> > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> >> > > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> > :: waited 0.000s inner > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> >> > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker > >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > compute > >> >> host > >> >> > and has allocations in placement: {'resources': {'VCPU': 1, > >> 'MEMORY_MB': > >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> >> > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager > >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > _get_power_state > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> > 63426b4c87a751afaf13b758e0 - default default] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot > process > >> >> > default default] Lazy-loading 'pci_devices' on Instance uuid > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> >> > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance > >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading > >> >> > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> > obj_load_attr > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> >> > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event > >> >> > >> 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> > => Paused> emit_event > >> >> > > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> >> > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) > >> >> > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager > >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > _get_power_state > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager > >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power > >> state > >> >> > after lifecycle event "Paused"; current vm_state: active, current > >> >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> >> > power_state: 3 handle_lifecycle_event > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> >> > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager > >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > >> >> instance > >> >> > has a pending task (image_pending_upload). Skip. > >> >> > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils > >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd > >> >> > (subprocess): qemu-img convert -t none -O raw -f raw > >> >> > > >> >> > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> >> > > >> >> > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad > >> >> > execute > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 > >> >> > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event > >> >> > >> 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> > => Stopped> emit_event > >> >> > > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> >> > INFO nova.compute.manager [-] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) > >> >> > DEBUG nova.compute.manager > [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > >> - - > >> >> - > >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking > state > >> >> > _get_power_state > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> > DEBUG nova.compute.manager > [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > >> - - > >> >> - > >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing > >> >> > instance power state after lifecycle event "Stopped"; current > >> vm_state: > >> >> > active, current task_state: image_pending_upload, current DB > >> power_state: > >> >> > 1, VM power_state: 4 handle_lifecycle_event > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> >> > INFO nova.compute.manager > [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - > >> - > >> >> - - > >> >> > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During > >> >> sync_power_state > >> >> > the instance has a pending task (image_pending_upload). Skip. > >> >> > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager > >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering > sync > >> for > >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> >> > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> >> > "25c8d676-e20a-4238 > >> >> > -a45c-d51daa62b941" acquired by > >> >> > > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> > :: waited 0.000s inner > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> >> > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the > >> >> instance > >> >> > has a pending task (image_pending_upload). Skip. > >> >> > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock > >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" released by > >> >> > > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> > :: held 0.001s inner > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 > >> >> > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils > >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img > >> convert > >> >> > -t none -O raw -f raw > >> >> > > >> >> > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> >> > > >> >> > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" > >> >> > returned: 0 in 23.730s execute > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 > >> >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > >> >> Snapshot > >> >> > extracted, beginning image upload > >> >> > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver > >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event > >> >> > >> 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> > => Started> emit_event > >> >> > > >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> >> > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager > >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) > >> >> > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> >> > _get_power_state > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power > >> state > >> >> > after lifecycle event "Started"; current vm_state: active, current > >> >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> >> > power_state: 1 handle_lifecycle_event > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> >> > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager > >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event > >> >> > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils > >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock > >> >> > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network > >> info > >> >> > cache for instance _get_instance_nw_info > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 > >> >> > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron > >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache > >> with > >> >> > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > >> "address": > >> >> > "fa:16:3e:ca:00:d9", "network": {"id": > >> >> > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", > "label": > >> >> > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": > >> >> > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, > >> >> > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": > >> {}}], > >> >> > "gateway": {"address": "141.11.42.1", "type": "gateway", > "version": 4, > >> >> > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", > >> >> > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], > >> "version": > >> >> 4, > >> >> > "meta": {}}], "meta": {"injected": true, "tenant_id": > >> >> > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, > "physical_network": > >> >> > "physnet1", "tunneled": false}}, "type": "ovs", "details": > >> >> {"connectivity": > >> >> > "l2", "port_filter": true, "ovs_hybrid_plug": true, > "datapath_type": > >> >> > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", > >> >> > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > >> "qbh_params": > >> >> > null, "qbg_params": null, "active": true, "vnic_type": "normal", > >> >> "profile": > >> >> > {}, "preserve_on_delete": false, "meta": {}}] > >> >> > update_instance_cache_with_nw_info > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 > >> >> > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on > this > >> >> > compute host and has allocations in placement: {'resources': > {'VCPU': > >> 1, > >> >> > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. > >> _remove_deleted_instances_allocations > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> >> > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager > >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to > snapshot > >> the > >> >> > instance on the hypervisor. > >> >> > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager > >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering > sync > >> for > >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> >> > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> >> > > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> > :: waited 0.000s inner > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> >> > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker > >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > compute > >> >> host > >> >> > and has allocations in placement: {'resources': {'VCPU': 1, > >> 'MEMORY_MB': > >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> >> > > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> >> > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager > >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> >> 93fb420b3c604d4fae408b81135b58e9 > >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> > > >> >> > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: > >> >> > > >> >> >> Have you tried with debug logs? Has it worked with live snapshots > >> >> >> before for other instances or did it never work and all snapshots > >> were > >> >> >> "cold"? > >> >> >> > >> >> >> Zitat von Parsa Aminian : > >> >> >> > >> >> >> > Hi > >> >> >> > kolla-ansible victoria version with ceph backend without volume > >> >> >> > > >> >> >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block > >> wrote: > >> >> >> > > >> >> >> >> Hi, > >> >> >> >> > >> >> >> >> can you share more details about your environment? Which > openstack > >> >> >> >> version is it? What is the storage backend? In earlier releases > >> there > >> >> >> >> was an option: > >> >> >> >> > >> >> >> >> #disable_libvirt_livesnapshot = false > >> >> >> >> > >> >> >> >> but this option has been deprecated. But if you're on an older > >> >> version > >> >> >> >> that could explain it. > >> >> >> >> > >> >> >> >> Zitat von Parsa Aminian : > >> >> >> >> > >> >> >> >> > When I snapshot from the instance , server will gone away and > >> its > >> >> not > >> >> >> >> > reachable until the snapshot is complete here is the logs : > >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager > >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] > [instance: > >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle > >> Event) > >> >> >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager > >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] > [instance: > >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state > >> the > >> >> >> >> instance > >> >> >> >> > has a pending task (image_pending_upload). Skip. > >> >> >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] > >> [instance: > >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle > >> Event) > >> >> >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver > >> >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> >> >> >> 93fb420b3c604d4fae408b81135b58e9 > >> >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] > [instance: > >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, > >> beginning > >> >> >> image > >> >> >> >> > upload > >> >> >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager > >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] > [instance: > >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle > >> Event) > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> > >> > >> > >> > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Tue Jun 14 12:53:30 2022 From: eblock at nde.ag (Eugen Block) Date: Tue, 14 Jun 2022 12:53:30 +0000 Subject: snapshot problem In-Reply-To: References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> <20220613170342.Horde.qRaR_2eecVXaGrvWHuz6tC7@webmail.nde.ag> Message-ID: <20220614125330.Horde.u5KxkrplHvtLIS5bmxnzeIM@webmail.nde.ag> You don't need the admin keyring on a compute node but for tests it can help. Basically it depends on your ceph auth config which keyring you need on the openstack nodes. If you already have ceph configured why don't you use it for all services? Of course live snapshots are not possible if your glance images are not within ceph (I don't think you mentioned that before). Zitat von Parsa Aminian : > yes you are right , I added ceph.client.admin.keyring to nova compute > container and ceph cluster error gone i also try rbd snap create from nova > compute container and its work without problem and vm is up . > maybe the problem is related to glance .my glance backend is not on ceph I > think maybe its the reason > > On Mon, Jun 13, 2022 at 9:38 PM Eugen Block wrote: > >> Hi, >> >> I don't have a containerized deployment so I don't have anything to >> compare, but my compute nodes have a ceph.conf and all required >> keyrings in /etc/ceph/, maybe someone else can confirm that it's >> missing and should be present in a kolla-ansible deployment. In the >> meantime you could provide those files to the container and retry the >> live snapshot. >> >> >> Zitat von Parsa Aminian : >> >> > snapshot_image_format doesn't exist on compute or controller nova.conf >> > For the manual snapshot on the previous reply I ran it directly from >> ceph, >> > but it's interesting from nova compute there is an error about connecting >> > to CEPH cluster !!! also ceph -s is showing similar error >> > [root at R3SG4 ~]# docker exec -u root -it nova_compute rbd snap create >> > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap2022-06-13 >> > 13:42:18.061 7f03dac23200 -1 auth: unable to find a keyring on >> > >> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: >> > (2) No such file or directory2022-06-13 13:42:18.061 7f03dac23200 -1 >> > AuthRegistry(0x561d7014a790) no keyring found at >> > >> /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, >> > disabling ceph2022-06-13 13:42:18.065 7f03dac23200 -1 monclient: >> > authenticate NOTE: no keyring found; disabled cephx authentication >> > rbd: couldn't connect to the cluster! >> > >> > On Mon, Jun 13, 2022 at 12:15 PM Eugen Block wrote: >> > >> >> Could you share your whole nova.conf (only the uncommented options)? >> >> Is this option set in your env? >> >> >> >> #snapshot_image_format = >> >> >> >> Also when you manually created the snapshot did you do it as the nova >> >> user on the compute node? If not, could you retry? >> >> >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> > Hi >> >> > kolla-ansible victoria version with ceph backend . >> >> > rbd info output : >> >> > rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': >> >> > size 20 GiB in 5120 objects >> >> > order 22 (4 MiB objects) >> >> > snapshot_count: 0 >> >> > id: b69aaf907284da >> >> > block_name_prefix: rbd_data.b69aaf907284da >> >> > format: 2 >> >> > features: layering, exclusive-lock, object-map, fast-diff, >> >> > deep-flatten >> >> > op_features: >> >> > flags: >> >> > create_timestamp: Fri May 20 00:04:47 2022 >> >> > access_timestamp: Sun Jun 12 16:26:02 2022 >> >> > --------------- >> >> > also live snapshot seems to work correctly without any error or any >> >> > downtime : >> >> > docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls >> >> > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk >> >> > SNAPID NAME SIZE PROTECTED TIMESTAMP >> >> > 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 >> >> > >> >> > also on compute nova.conf, images_type is set on rbd . >> >> > >> >> > On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: >> >> > >> >> >> You should respond to the list so other users can try to support you. >> >> >> >> >> >> So nova is trying to live snapshot the instance: >> >> >> >> >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> >> > 4dbffaa9c14e401c8c210e23ebe0ab7b ef940663426b4c87a751afaf13b758e0 - >> >> >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] >> >> >> > instance snapshotting >> >> >> > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning >> >> >> > live snapshot process >> >> >> >> >> >> But I don't see any 'rbd snap create' command. Either the rbd image >> >> >> doesn't support it or there is some setting to keep all rbd images >> >> >> "flat". Can you check any relevant configs you might have in nova? >> >> >> Also can you show the output of 'rbd info >> >> >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test if >> >> >> the underlying rbd functions work as expected you could try to create >> >> >> a live snapshot manually: >> >> >> >> >> >> rbd snap create >> >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap >> >> >> >> >> >> And paste any relevant output here. >> >> >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> >> >> > Its not working for any instances and all of them are paused . I >> >> enable >> >> >> > debug logs please check the logs : >> >> >> > >> >> >> > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering >> sync >> >> for >> >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> >> >> > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] Lock >> >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by >> >> >> > >> >> >> >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> >> > :: waited 0.000s inner >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> >> >> > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this >> compute >> >> >> host >> >> >> > and has allocations in placement: {'resources': {'VCPU': 1, >> >> 'MEMORY_MB': >> >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> >> >> > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state >> _get_power_state >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> >> > 63426b4c87a751afaf13b758e0 - default default] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot >> process >> >> >> > default default] Lazy-loading 'pci_devices' on Instance uuid >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 >> >> >> > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading >> >> >> > 'pci_devices' on Instance uuid 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> >> > obj_load_attr >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 >> >> >> > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting event >> >> >> > > >> 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> >> > => Paused> emit_event >> >> >> > >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> >> >> > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle Event) >> >> >> > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state >> _get_power_state >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> >> > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power >> >> state >> >> >> > after lifecycle event "Paused"; current vm_state: active, current >> >> >> > task_state: image_pending_upload, current DB power_state: 1, VM >> >> >> > power_state: 3 handle_lifecycle_event >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> >> >> > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> >> >> instance >> >> >> > has a pending task (image_pending_upload). Skip. >> >> >> > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd >> >> >> > (subprocess): qemu-img convert -t none -O raw -f raw >> >> >> > >> >> >> >> >> >> rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf >> >> >> > >> >> >> >> >> >> /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad >> >> >> > execute >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 >> >> >> > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting event >> >> >> > > >> 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> >> > => Stopped> emit_event >> >> >> > >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> >> >> > INFO nova.compute.manager [-] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle Event) >> >> >> > DEBUG nova.compute.manager >> [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa >> >> - - >> >> >> - >> >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking >> state >> >> >> > _get_power_state >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> >> > DEBUG nova.compute.manager >> [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa >> >> - - >> >> >> - >> >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing >> >> >> > instance power state after lifecycle event "Stopped"; current >> >> vm_state: >> >> >> > active, current task_state: image_pending_upload, current DB >> >> power_state: >> >> >> > 1, VM power_state: 4 handle_lifecycle_event >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> >> >> > INFO nova.compute.manager >> [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - >> >> - >> >> >> - - >> >> >> > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During >> >> >> sync_power_state >> >> >> > the instance has a pending task (image_pending_upload). Skip. >> >> >> > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering >> sync >> >> for >> >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> >> >> > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] Lock >> >> >> > "25c8d676-e20a-4238 >> >> >> > -a45c-d51daa62b941" acquired by >> >> >> > >> >> >> >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> >> > :: waited 0.000s inner >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> >> >> > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state the >> >> >> instance >> >> >> > has a pending task (image_pending_upload). Skip. >> >> >> > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] Lock >> >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" released by >> >> >> > >> >> >> >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> >> > :: held 0.001s inner >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 >> >> >> > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] CMD "qemu-img >> >> convert >> >> >> > -t none -O raw -f raw >> >> >> > >> >> >> >> >> >> rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf >> >> >> > >> >> >> >> >> >> /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" >> >> >> > returned: 0 in 23.730s execute >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 >> >> >> > default default] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] >> >> >> Snapshot >> >> >> > extracted, beginning image upload >> >> >> > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting event >> >> >> > > >> 25c8d676-e20a-4238-a45c-d51daa62b941 >> >> >> > => Started> emit_event >> >> >> > >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 >> >> >> > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle Event) >> >> >> > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state >> >> >> > _get_power_state >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance power >> >> state >> >> >> > after lifecycle event "Started"; current vm_state: active, current >> >> >> > task_state: image_pending_upload, current DB power_state: 1, VM >> >> >> > power_state: 1 handle_lifecycle_event >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 >> >> >> > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle Event >> >> >> > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired lock >> >> >> > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing network >> >> info >> >> >> > cache for instance _get_instance_nw_info >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 >> >> >> > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating instance_info_cache >> >> with >> >> >> > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", >> >> "address": >> >> >> > "fa:16:3e:ca:00:d9", "network": {"id": >> >> >> > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", >> "label": >> >> >> > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", "dns": >> >> >> > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": {}}, >> >> >> > {"address": "217.218.127.127", "type": "dns", "version": 4, "meta": >> >> {}}], >> >> >> > "gateway": {"address": "141.11.42.1", "type": "gateway", >> "version": 4, >> >> >> > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": "fixed", >> >> >> > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], >> >> "version": >> >> >> 4, >> >> >> > "meta": {}}], "meta": {"injected": true, "tenant_id": >> >> >> > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, >> "physical_network": >> >> >> > "physnet1", "tunneled": false}}, "type": "ovs", "details": >> >> >> {"connectivity": >> >> >> > "l2", "port_filter": true, "ovs_hybrid_plug": true, >> "datapath_type": >> >> >> > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", >> >> >> > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", >> >> "qbh_params": >> >> >> > null, "qbg_params": null, "active": true, "vnic_type": "normal", >> >> >> "profile": >> >> >> > {}, "preserve_on_delete": false, "meta": {}}] >> >> >> > update_instance_cache_with_nw_info >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 >> >> >> > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on >> this >> >> >> > compute host and has allocations in placement: {'resources': >> {'VCPU': >> >> 1, >> >> >> > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. >> >> _remove_deleted_instances_allocations >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> >> >> > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to >> snapshot >> >> the >> >> >> > instance on the hypervisor. >> >> >> > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering >> sync >> >> for >> >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 >> >> >> > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by >> >> >> > >> >> >> >> >> >> "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" >> >> >> > :: waited 0.000s inner >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 >> >> >> > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this >> compute >> >> >> host >> >> >> > and has allocations in placement: {'resources': {'VCPU': 1, >> >> 'MEMORY_MB': >> >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations >> >> >> > >> >> >> >> >> >> /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 >> >> >> > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> >> >> 93fb420b3c604d4fae408b81135b58e9 >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> >> > >> >> >> > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block wrote: >> >> >> > >> >> >> >> Have you tried with debug logs? Has it worked with live snapshots >> >> >> >> before for other instances or did it never work and all snapshots >> >> were >> >> >> >> "cold"? >> >> >> >> >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> >> >> >> >> > Hi >> >> >> >> > kolla-ansible victoria version with ceph backend without volume >> >> >> >> > >> >> >> >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block >> >> wrote: >> >> >> >> > >> >> >> >> >> Hi, >> >> >> >> >> >> >> >> >> >> can you share more details about your environment? Which >> openstack >> >> >> >> >> version is it? What is the storage backend? In earlier releases >> >> there >> >> >> >> >> was an option: >> >> >> >> >> >> >> >> >> >> #disable_libvirt_livesnapshot = false >> >> >> >> >> >> >> >> >> >> but this option has been deprecated. But if you're on an older >> >> >> version >> >> >> >> >> that could explain it. >> >> >> >> >> >> >> >> >> >> Zitat von Parsa Aminian : >> >> >> >> >> >> >> >> >> >> > When I snapshot from the instance , server will gone away and >> >> its >> >> >> not >> >> >> >> >> > reachable until the snapshot is complete here is the logs : >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting >> >> >> >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager >> >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] >> [instance: >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle >> >> Event) >> >> >> >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager >> >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] >> [instance: >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state >> >> the >> >> >> >> >> instance >> >> >> >> >> > has a pending task (image_pending_upload). Skip. >> >> >> >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] >> >> [instance: >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle >> >> Event) >> >> >> >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver >> >> >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 >> >> >> >> >> 93fb420b3c604d4fae408b81135b58e9 >> >> >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] >> [instance: >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, >> >> beginning >> >> >> >> image >> >> >> >> >> > upload >> >> >> >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager >> >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] >> [instance: >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle >> >> Event) >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From fungi at yuggoth.org Tue Jun 14 12:56:21 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 14 Jun 2022 12:56:21 +0000 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: References: <20220614115942.wqiaqk7hpa2hzdlk@yuggoth.org> Message-ID: <20220614125620.nqngnpqchlrg5zgb@yuggoth.org> On 2022-06-14 21:22:37 +0900 (+0900), Takashi Kajinami wrote: [...] > I sometimes hope zuul can block backports depending on the "cherry > picked from commit ..." line but it might be worth discussing as a > potential enhancement with the zuul devs. Definitely worth further discussion, but I suspect the possibility for cherry-picks from other repositories could make that challenging to differentiate. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From amotoki at gmail.com Tue Jun 14 13:22:57 2022 From: amotoki at gmail.com (Akihiro Motoki) Date: Tue, 14 Jun 2022 22:22:57 +0900 Subject: [neutron] bug deputy (week of Jun 6) Message-ID: Hi, I was a bug deputy in neutron. Here is a summary. ## Untriaged * https://bugs.launchpad.net/neutron/+bug/1978088 After ovs-agent restart, table=21 and table=22 on br-tun openflow table is missing This happens when a large number of agents are restarted, but it looks like deeper investigations are needed to understand the situation. (liuyolong and ralonsoh already helped) ## In Progress * https://bugs.launchpad.net/neutron/+bug/1977819 [OVS][QoS] "delete_minimum_bandwidth_queue" uses the incorrect Port UUID High, In Progress * https://bugs.launchpad.net/neutron/+bug/1978519 Create auto allocated topology encounter error when enable ndp_proxy service plugin ## RFE * https://bugs.launchpad.net/neutron/+bug/1978039 [ovn] Floating IP adds distributed attributes ## Fix Released * https://bugs.launchpad.net/neutron/+bug/1977752 [OVS][QoS] Minimum bandwidth backend enforcement does not provide max-rate values * https://bugs.launchpad.net/neutron/+bug/1977969 [OVN] DB sync tool: local variable 'members_to_verify' referenced before assignment" * https://bugs.launchpad.net/neutron/+bug/1978035 remove unused updated_at parameter for AgentCache.update * https://bugs.launchpad.net/neutron/+bug/1978389 Basic networking in Neutron - max VLAN should be 4094, not 4095 * https://bugs.launchpad.net/neutron/+bug/1977831 Response time increasing on new subnets over same network Thanks, Akihiro Motoki (amotoki) From dms at danplanet.com Tue Jun 14 13:37:47 2022 From: dms at danplanet.com (Dan Smith) Date: Tue, 14 Jun 2022 06:37:47 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> (Sean Mooney's message of "Tue, 14 Jun 2022 01:26:50 +0100") References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> Message-ID: > i hit the same issue the other day if you recall me asking about it. > didnt check my devstack logs to see if it installed or not but i can > see if i can repoduce it tomorrow. I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird things going on with their setup, so I'm not so sure. --Dan From JRACE at augusta.edu Tue Jun 14 13:42:11 2022 From: JRACE at augusta.edu (Race, Jonathan) Date: Tue, 14 Jun 2022 13:42:11 +0000 Subject: [Nova] Hardware Architecture Emulation - Summit forum - Emulation beyond just Local QEMU -outcomes Message-ID: Great to see everyone at the summit! For those not in attendance and interested, see the below etherpad that is being used to collect input and requirements as our team moves forward with work. I have also included the outcomes that I have identified. https://etherpad.opendev.org/p/Emulation_going_beyond_local_QEMU Berlin Forum Outcomes I appreciate all the feedback during the forum session, and it was great to have active participation on how to grow and shape this unique feature. We discussed the feature, what it allows, how to actually leverage it, and discussed about further refining the applicable use cases so that operators and users can effectively use it and to help drive interest. A prime example was of a company wanting to shift their infra from x86_64 to arm for the sake of power consumption, and whether this feature could lend its hand to perform some initial fact finding and testing. The answer is yes, you can spin up some of the infra (in your desired arch) on top of your existing deployment and perform your tests and validations. One of the great items that came out of the discussion was that this can not only be to support education, research, and training but it can act as a great 1st step within planning. Meaning when you are going from the drawing board to implementation, you can now model and design your network environments within the cloud for IT and OT systems. This is where refining the use cases really came to light, as our organization leverages OpenStack as a private cloud for hosting extremely customizable network environments (i.e. enterprise iT, IOT, Critical infrastructure, medical devices, etc.) While the feature itself is overall in the experimental phase, it is a great start. This etherpad will remain available and feel free to add to it, as we leverage this year to fact find and identify all further requirements going into the AA release cycle. A snapshot of the discussion points can be found below, and ill be posting the recorded session within the week. * Most operators and providers are running a few releases behind so getting accurate feedback at this time is still ongoing due to this feature only being available in yoga and newer. * Pursuit of additional architectures to be supported (riscv, etc.) * This will require custom firmware support * evaluate current AMI/AKI/ARI functionalities (add ability for leveraging glance if able, and add additional security) * Adding capability for trusted signers of OOVMF file descriptors * evaluating current work related to edk2 oovmf file descriptors * https://github.com/riscv-admin/riscv-uefi-edk2-docs * Expansion and documentation of a valid support matrix * something similar to: (arch x | feat 1 | supported | temptest test 1) * Evaluate requirements for leveraging virtio in relation to scsi/sata devices similar to pci passthrough capabilities Jonathan Race Cyber Range Engineer | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 (c) 762-716-7733 www.gacyberrange.org [cid:image001.png at 01D87FD1.CF558390] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 5865 bytes Desc: image001.png URL: From hjensas at redhat.com Tue Jun 14 14:13:19 2022 From: hjensas at redhat.com (Harald Jensas) Date: Tue, 14 Jun 2022 16:13:19 +0200 Subject: [VICTORIA][IRONIC] - Can inspect but not deploy In-Reply-To: References: Message-ID: <181eb30b-4389-e852-7b0b-da7c6c0ffe25@redhat.com> On 6/13/22 12:41, Ga?l THEROND wrote: > Hi pierre, > > I?m using a dedicated interface but this interface is the same for all > ironic networks inspection/provisioning/cleaning. > > This interface works fine for inspection, my only issue is the > pxe_filter that the ironic inspector process allow during inspection > correctly but then tag as disallowed again at the end of the inspection, > shouldn?t the deploy process allow the Mac again before booting the node? > The ironic dnsmasq pxe filter is only supposed to allow DHCP requests when a node is inspected. There are two DHCP services, one for inspector and the other one is typically hosted by neutron for provisioning/cleaning/rescue. Only the dnsmasq for inspector uses the hostdir with ',ignore' files. > I can correctly see the conductor instruct the node to boot up using pxe > from the kvm console but the BootP process doesn?t load the IPA > kernel/initramfs as the dnsmasq pxe discard the request (because of the > mac being still tagged as ?,ignore ? within the hostdir file). > When a node is provisioned/cleaned/rescued the DHCP instance in neutron should be the one providing the DHCP service, not the dnsmasq instance for inspector service. i.e the ',ignore' entries should remain as 'ignore' to ensure the node does not get a DHCP reply from inspector's DHCP service which would deliver the wrong DHCP options. > I?m a bit disappointed by this behavior. > > Thanks a lot! > > Le?lun. 13 juin 2022 ? 11:41, Pierre Riteau > a ?crit?: > > Hello Ga?l, > > Which network_interface are you using for your nodes? Is your > provisioning network different from the inspection network? > > Pierre > > On Mon, 13 Jun 2022 at 10:36, Ga?l THEROND > > wrote: > > Hi everyone! > > I'm dealing with a strange issue today, we deployed IRONIC on a > VICTORIA platform, we activated the dnsmasq pxe filtering option > at the inspector level, it works great as only IRONIC listed > hosts are then served by the dnsmasq DHCP as those hosts are > allowed using the dhcp hosts dir. > > BUT, I'm having a weird issue now. > > All my nodes are able to get an IP from the DHCP at the > INSPECTION step, however, as soon as the inspection step is > finished, the mac related file is once again filled with > ",ignore" which prohibits further operations. > > This means as soon as I put that node as available and try to > deploy an "instance" on it (provision the host), it doesn't work > as dnsmasq reject the host boot DHCP requests. > > So, is there a way to instruct the ironic-conductor to > edit/allow the host the way the inspector is able to manipulate > this file? > > Did I missed?something? > > Thanks a lot! > From smooney at redhat.com Tue Jun 14 15:15:57 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 14 Jun 2022 16:15:57 +0100 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> Message-ID: <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: > > i hit the same issue the other day if you recall me asking about it. > > didnt check my devstack logs to see if it installed or not but i can > > see if i can repoduce it tomorrow. > > I checked with the OP and they have it installed properly, > system-wide. I was assuming your issue might have been venv (or > ordering) related, but theirs was not. There are some other weird things > going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it. > > --Dan > From cboylan at sapwetik.org Tue Jun 14 17:13:57 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 14 Jun 2022 10:13:57 -0700 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: <20220614125620.nqngnpqchlrg5zgb@yuggoth.org> References: <20220614115942.wqiaqk7hpa2hzdlk@yuggoth.org> <20220614125620.nqngnpqchlrg5zgb@yuggoth.org> Message-ID: On Tue, Jun 14, 2022, at 5:56 AM, Jeremy Stanley wrote: > On 2022-06-14 21:22:37 +0900 (+0900), Takashi Kajinami wrote: > [...] >> I sometimes hope zuul can block backports depending on the "cherry >> picked from commit ..." line but it might be worth discussing as a >> potential enhancement with the zuul devs. > > Definitely worth further discussion, but I suspect the possibility > for cherry-picks from other repositories could make that challenging > to differentiate. I agree. I'm not sure this makes sense as a built in Zuul feature. However, you can always implement a backport sanity check job that fails if certain criteria are not met. > -- > Jeremy Stanley > > Attachments: > * signature.asc From adivya1.singh at gmail.com Tue Jun 14 18:18:02 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Tue, 14 Jun 2022 23:48:02 +0530 Subject: Regarding Policy.json entries for glance image update not working for a user In-Reply-To: References: Message-ID: Hi Takashi, when a user upload images which is a member , The image will be set to private. This is what he is asking for access to make it public, The above rule applies for only public images regards Adivya Singh On Tue, Jun 14, 2022 at 10:54 AM Takashi Kajinami wrote: > Glance has a separate policy rule (publicize_image) for creating/updating > public images., > and you should define that policy rule instead of modify_image. > > https://docs.openstack.org/glance/xena/admin/policies.html > ~~~ > publicize_image - Create or update public images > ~~~ > > AFAIK The modify_image policy defaults to rule:default and is allowed for > any users > as long as the target image is owned by that user. > > > On Tue, Jun 14, 2022 at 2:01 PM Adivya Singh > wrote: > >> Hi Brian, >> >> Please find the response >> >> >>> 1> i am using Xena release version 24.0.1 >>> >>> Now the scenario is line below, my customer wants to have their login >>> access on setting up the properties of an image to the public. now what i >>> did is >>> >>> 1> i created a role in openstack using the admin credential name as >>> "user" >>> 2> i assigned that user to a role user. >>> 3> i assigned those user to those project id, which they want to access >>> as a user role >>> >>> Then i went to Glance container which is controller by lxc and made a >>> policy.yaml file as below >>> >>> root at aio1-glance-container-724aa778:/etc/glance# cat policy.yaml >>> >>> "modify_image": "role:user" >>> >>> then i went to utility container and try to set the properties of a >>> image using openstack command >>> >>> openstack image set --public >>> >>> and then i got this error >>> >>> HTTP 403 Forbidden: You are not authorized to complete publicize_image >>> action. >>> >>> Even when i am trying the upload image with this user , i get the above >>> error only >>> >>> export OS_ENDPOINT_TYPE=internalURL >>> export OS_INTERFACE=internalURL >>> export OS_USERNAME=adsingh >>> export OS_PASSWORD='adsingh' >>> export OS_PROJECT_NAME=adsingh >>> export OS_TENANT_NAME=adsingh >>> export OS_AUTH_TYPE=password >>> export OS_AUTH_URL=https://:5000/v3 >>> export OS_NO_CACHE=1 >>> export OS_USER_DOMAIN_NAME=Default >>> export OS_PROJECT_DOMAIN_NAME=Default >>> export OS_REGION_NAME=RegionOne >>> >>> Regards >>> Adivya Singh >>> >>> >>> >>> >>> On Mon, Jun 13, 2022 at 6:41 PM Alan Bishop wrote: >>> >>>> >>>> >>>> On Mon, Jun 13, 2022 at 6:00 AM Brian Rosmaita < >>>> rosmaita.fossdev at gmail.com> wrote: >>>> >>>>> On 6/13/22 8:29 AM, Adivya Singh wrote: >>>>> > hi Team, >>>>> > >>>>> > Any thoughts on this >>>>> >>>>> H Adivya, >>>>> >>>>> Please supply some more information, for example: >>>>> >>>>> - which openstack release you are using >>>>> - the full API request you are making to modify the image >>>>> - the full API response you receive >>>>> - whether the user with "role:user" is in the same project that owns >>>>> the >>>>> image >>>>> - debug level log extract for this call if you have it >>>>> - anything else that could be relevant, for example, have you modified >>>>> any other policies, and if so, what values are you using now? >>>>> >>>> >>>> Also bear in mind that the default policy_file name is "policy.yaml" >>>> (not .json). You either >>>> need to provide a policy.yaml file, or override the policy_file setting >>>> if you really want to >>>> use policy.json. >>>> >>>> Alan >>>> >>>> cheers, >>>>> brian >>>>> >>>>> > >>>>> > Regards >>>>> > Adivya Singh >>>>> > >>>>> > On Sat, Jun 11, 2022 at 12:40 AM Adivya Singh < >>>>> adivya1.singh at gmail.com >>>>> > > wrote: >>>>> > >>>>> > Hi Team, >>>>> > >>>>> > I have a use case where I have to give a user restriction on >>>>> > updating the image properties as a member. >>>>> > >>>>> > I have created a policy Json file and give the modify_image rule >>>>> to >>>>> > the particular role, but still it is not working >>>>> > >>>>> > "modify_image": "role:user", This role is created in OpenStack. >>>>> > >>>>> > but still it is failing while updating properties with a >>>>> > particular user assigned to a role as "access denied" and >>>>> > unauthorized access >>>>> > >>>>> > Regards >>>>> > Adivya Singh >>>>> > >>>>> >>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Tue Jun 14 18:20:45 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Tue, 14 Jun 2022 23:50:45 +0530 Subject: 504 Gateway time out error while loading OpenStack Projects In-Reply-To: References: Message-ID: hi Team, Any input on this Regards Adivya Singh On Mon, Jun 13, 2022 at 6:37 PM Adivya Singh wrote: > hi Team, > > We have Xena release installed on Ubuntu Openstack, Lately we are getting > "504 Gateway Timeout error ", As the we moved from 1 node to 3 node Open > Stack > i do not see any error in haproxy , neither there is a time out when i > ping external-VIP-IP configured. > > Apache 2 are also loaded, tried to delete the Horizon Container and > created it again, but no errors, Still the same errors. > > Any guesses on that > > Regards > Adivya Singh > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haiwu.us at gmail.com Tue Jun 14 18:28:05 2022 From: haiwu.us at gmail.com (hai wu) Date: Tue, 14 Jun 2022 13:28:05 -0500 Subject: [keystone] Getting a token from existing token Message-ID: Is it possible to get a new token from an existing token, and at the same time, to have the new token expiration time extended? (token renewal from existing token) It seems the token expiration time is not extended when trying the same curl command as shown in section "Getting a token from a token" in this doc: https://docs.openstack.org/keystone/train/api_curl_examples.html. Quote: curl -i \ -H "Content-Type: application/json" \ -d ' { "auth": { "identity": { "methods": ["token"], "token": { "id": "'$OS_TOKEN'" } } } }' \ "http://localhost:5000/v3/auth/tokens" ; echo From ken at jots.org Tue Jun 14 18:59:09 2022 From: ken at jots.org (Ken D'Ambrosio) Date: Tue, 14 Jun 2022 14:59:09 -0400 Subject: New install on CentOS: what versions, etc.? Message-ID: <7ec0e14593137adf8a91193628d4e301@jots.org> Hey, all. We're currently running (I hope you're sitting) Juno, and we're thinking about rolling out a new OpenStack version. My boss has asked me if CentOS 7.9 a) would be supported for control/compute nodes, and, if so, if there are any long-term potential issues, and b) if there would be any concerns about versions of client OSes. Thanks! -Ken From noonedeadpunk at gmail.com Tue Jun 14 19:08:41 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Tue, 14 Jun 2022 21:08:41 +0200 Subject: New install on CentOS: what versions, etc.? In-Reply-To: <7ec0e14593137adf8a91193628d4e301@jots.org> References: <7ec0e14593137adf8a91193628d4e301@jots.org> Message-ID: Well, it depends of what new version you're thinking. As iirc last openstack version which would support CentOS 7 is Ussuri. After that you will have troubles with supported libvirt version and python bindings. As on CentsOS 7 you don't have python3 libvirt bindings, so likely you would need to run on python 2.7 which is already eol for couple of years. CentOS 8 Stream though will be working on Yoga but for Zed you will need CentOS 9 Stream. So all depends of what you consider new after Juno :D ??, 14 ???. 2022 ?., 21:02 Ken D'Ambrosio : > Hey, all. We're currently running (I hope you're sitting) Juno, and > we're thinking about rolling out a new OpenStack version. My boss has > asked me if CentOS 7.9 > a) would be supported for control/compute nodes, and, if so, if there > are any long-term potential issues, and > b) if there would be any concerns about versions of client OSes. > > Thanks! > > -Ken > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkopec at redhat.com Tue Jun 14 19:30:16 2022 From: mkopec at redhat.com (Martin Kopec) Date: Tue, 14 Jun 2022 21:30:16 +0200 Subject: [interop][refstack][summit] Interoperability testing feedback Message-ID: Hi everyone, we got great feedback during the summit, if you wanna check the topics we discussed, see this etherpad [1]. Feel also free to add additional concerns with the current interop processes. I was happy to see that many of you are thinking about interoperability and want to improve the testing in this regard. IWG has been seeking feedback from operators as well as users for some time now to make sure it helps with interoperability as much as possible. If you haven't already, please fill out this short (2-3 min) questionnaire [2]. Public Cloud SIG hosted an interesting session in terms of interoperability testing as well, see their etherpad [3]. I'm sure that you'll see further communication in that regard soon. I look forward to future collaboration between IWG and Public Cloud SIG. [1] https://etherpad.opendev.org/p/Berlin2022-Forum-InteropWG [2] https://forms.gle/YW95Hb2LoppaiLxv8 [3] https://etherpad.opendev.org/p/berlin-summit-future-public-cloud-sig Regards, -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkopec at redhat.com Tue Jun 14 20:10:06 2022 From: mkopec at redhat.com (Martin Kopec) Date: Tue, 14 Jun 2022 22:10:06 +0200 Subject: [qa][refstack][python-tempestconf] adding image In-Reply-To: References: Message-ID: Hi Mohammed, we are glad to hear that python-tempestconf is useful. We don't mind having python-tempestconf included in Docker images, although what exactly would we need to do? I'm afraid we can't commit to regular maintenance of the images. Cheers, On Mon, 13 Jun 2022 at 15:03, Mohammed Naser wrote: > Hi there, > > I'm wondering if the python-tempestconf team is open to publishing > Docker images that contain `python-tempestconf`. This is something > that we have a need for downstream but we'd be happy to contribute > upstream. > > The main thing is that we'd need to setup an organization on Docker > hub to publish things to. Is that something that the team is > interested in? We can contribute it if that is the case. > > Thanks, > Mohammed > > -- > Mohammed Naser > VEXXHOST, Inc. > > -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Tue Jun 14 20:27:01 2022 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Tue, 14 Jun 2022 16:27:01 -0400 Subject: Regarding Policy.json entries for glance image update not working for a user In-Reply-To: References: Message-ID: On 6/14/22 2:18 PM, Adivya Singh wrote: > Hi Takashi, > > when a user upload images which is a member , The image?will be set to > private. > > This is what he is asking for access to make it public,? The above rule > applies for only public images Alan and Takashi have both given you good advice: - By default, Glance assumes that your custom policy file is named "policy.yaml". If it doesn't have that name, Glance will assume it does not exist and will use the defaults defined in code. You can change the filename glance will look for in your glance-api.conf -- look for [oslo_policy]/policy_file - We recommend that you use YAML instead of JSON to write your policy file because YAML allows comments, which you will find useful in documenting any changes you make to the file - You want to keep the permissions on modify_image at their default value, because otherwise users won't be able to do simple things like add image properties to their own images - Some image properties can affect the system or other users. Glance will not allow *any* user to modify some system properties (for example, 'id', 'status'), and it requires additional permission along with modify_image to set 'public' or 'community' for image visibility. - It's also possible to configure property protections to require additional permission to CRUD specific properties (the default setting is *not* to do this). For your particular use case, where you want a specific user to be able to publicize_image, I would encourage you to think more carefully about what exactly you want to accomplish. Traditionally, images with 'public' visibility are provided by the cloud operator, and this gives image consumers some confidence that there's nothing malicious on the image. Public images are accessible to all users, and they will show up in the default image-list call for all users, so if a public image contains something nasty, it can spread very quickly. Glance provides four levels of image visibility: - private: only visible to users in the project that owns the image - shared: visible to users in the project that owns the image *plus* any projects that are added to the image as "members". (A shared image with no members is effectively a private image.) See [0] for info about how image sharing is designed and what API calls are associated with it. There are a bunch of policies around this; the defaults are basically what you'd expect, with the image owner being able to add and delete members, and image members being able to 'accept' or 'reject' shared images. - community: accessible to everyone, but only visible if you look for them. See [1] for an explanation of what that means. The ability to set 'community' visibility on an image is controlled by the "communitize_image" policy (default is admin-or-owner). - public: accessible to everyone, and easily visible to all users. Controlled by the "publicize_image" policy (default is admin-only). You're running your own cloud, so you can configure things however you like, but I encourage you to think carefully before handing out publicize_image permission, and consider whether one of the other visibilities can accomplish what you want. For more info, the introductory section on "Images" in the api-ref [2] has a useful discussion of image properties and image visibility. The final thing I want to stress is that you should be sure to test carefully any policies you define in a custom policy file. You are actually having a good problem, that is, someone can't do something you would like them to. The way worse problem happens when in addition to that someone being able to do what you want them to, a whole bunch of other users can also do that same thing. OK, so to get to your particular issue: - you don't want to change the "modify_image" policy in the way you proposed in your email, because no one (other than the person having the 'user role) will be able to do any kind of image updates. - if you decide to give that user publicize_image permissions, be careful how you do it. For example, "publicize_image": "role:user" won't allow an admin to make images public (unless you also give each admin the 'user' role). If you look at most of the policies in the Xena policy.yaml.sample, they begin "role:admin or ...". - the reason you were seeing the 403 when you tried to do openstack image set --public as the user with the 'user' property is that you were allowed to modify_image but when you tried to change the visibility, you did not have permission (because the default for that is role:admin) Hope this helps! Once you get this figured out, you may want to put up a patch to update the Glance documentation around policies. I think everything said above is in there somewhere, but it may not be in the most obvious places. Actually, there is one more thing. The above all applies to Xena, but there's been some work around policies in Yoga and more happening in Zed, so be sure to read the Glance release notes when you eventually upgrade. [0] https://specs.openstack.org/openstack/glance-specs/specs/api/v2/sharing-image-api-v2.html [1] https://specs.openstack.org/openstack/glance-specs/specs/api/v2/sharing-image-api-v2.html#sharing-images-with-all-users [2] https://docs.openstack.org/api-ref/image/v2/index.html#images > > regards > Adivya Singh > > On Tue, Jun 14, 2022 at 10:54 AM Takashi Kajinami > wrote: > > Glance has a separate policy rule (publicize_image) for > creating/updating public images., > and you should define that policy rule instead of modify_image. > > https://docs.openstack.org/glance/xena/admin/policies.html > > ~~~ > |publicize_image| - Create or update public images > ~~~ > > AFAIK The modify_image policy defaults to rule:default and is > allowed for any users > as long as the target image is owned by that user. > > > On Tue, Jun 14, 2022 at 2:01 PM Adivya Singh > > wrote: > > ?Hi Brian, > > Please find the response > > 1> i am using Xena release version 24.0.1 > > Now the scenario?is line below, my customer wants to have > their login access on setting up the properties of an image > to the public. now what i did is > > 1> i created a role in openstack using the admin credential > name as "user" > 2> i assigned that user to a role user. > 3> i assigned those user to those project id, which they > want to access as a user role > > Then i went to Glance container which is controller by lxc > and made a policy.yaml file as below > > root at aio1-glance-container-724aa778:/etc/glance# cat policy.yaml > > ?"modify_image": "role:user" > > then i went to utility container and try to set the > properties of a image using openstack command > > openstack image set --public > > and then i got this error > > HTTP 403 Forbidden: You are not authorized to complete > publicize_image action. > > Even when i am trying the upload image with this user , i > get the above error only > > export OS_ENDPOINT_TYPE=internalURL > export OS_INTERFACE=internalURL > export OS_USERNAME=adsingh > export OS_PASSWORD='adsingh' > export OS_PROJECT_NAME=adsingh > export OS_TENANT_NAME=adsingh > export OS_AUTH_TYPE=password > export OS_AUTH_URL=https://:5000/v3 > export OS_NO_CACHE=1 > export OS_USER_DOMAIN_NAME=Default > export OS_PROJECT_DOMAIN_NAME=Default > export OS_REGION_NAME=RegionOne > > Regards > Adivya Singh > > > > On Mon, Jun 13, 2022 at 6:41 PM Alan Bishop > > wrote: > > > > On Mon, Jun 13, 2022 at 6:00 AM Brian Rosmaita > > wrote: > > On 6/13/22 8:29 AM, Adivya Singh wrote: > > hi Team, > > > > Any thoughts on this > > H Adivya, > > Please supply some more information, for example: > > - which openstack release you are using > - the full API request you are making to modify the > image > - the full API response you receive > - whether the user with "role:user" is in the same > project that owns the > image > - debug level log extract for this call if you have it > - anything else that could be relevant, for example, > have you modified > any other policies, and if so, what values are you > using now? > > > Also bear in mind that the default policy_file name is > "policy.yaml" (not .json). You either > need to provide a policy.yaml file, or override the > policy_file setting if you really want to > use policy.json. > > Alan > > cheers, > brian > > > > > Regards > > Adivya Singh > > > > On Sat, Jun 11, 2022 at 12:40 AM Adivya Singh > > > >> wrote: > > > >? ? ?Hi Team, > > > >? ? ?I have a use case where I have to give a user > restriction on > >? ? ?updating the image properties as a member. > > > >? ? ?I have created a policy Json file and give > the modify_image rule to > >? ? ?the particular role, but still it is not working > > > >? ? ?"modify_image": "role:user", This role is > created in OpenStack. > > > >? ? ?but still it is failing while updating > properties with a > >? ? ?particular?user assigned to a role as "access > denied" and > >? ? ?unauthorized access > > > >? ? ?Regards > >? ? ?Adivya Singh > > > > From anyrude10 at gmail.com Tue Jun 14 12:54:20 2022 From: anyrude10 at gmail.com (Anirudh Gupta) Date: Tue, 14 Jun 2022 18:24:20 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure Message-ID: Hi Team, I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing the below error: 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | TASK | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | FATAL | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has not started yet"} 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | TIMING | tripleo_container_manage : Manage container systemd Sample Undercloud.conf is as follows: [DEFAULT] clean_nodes = true cleanup = false container_cli = podman container_healthcheck_disabled = true container_images_file = /home/stack/containers-prepare-parameter.yaml deployment_user = stack enable_ironic = true enable_ironic_inspector = true enable_neutron = true enable_routed_networks = false generate_service_certificate = false ipv6_address_mode = dhcpv6-stateful ipxe_enabled = true local_interface = enp8s0 local_ip = aaaa:aaaa:aaaa::1/64 subnets = ctlplane-subnet undercloud_admin_host = aaaa:aaaa:aaaa::1 undercloud_hostname = undercloud.com undercloud_ntp_servers = 30.30.30.3 undercloud_public_host = aaaa:aaaa:aaaa::1 undercloud_timezone = UTC [ctlplane-subnet] cidr = aaaa:aaaa:aaaa::/64 dhcp_end = aaaa:aaaa:aaaa::f dhcp_start = aaaa:aaaa:aaaa::a gateway = aaaa:aaaa:aaaa::1 inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 Can someone please help in this regard. Anirudh Gupta -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.aminian.server at gmail.com Tue Jun 14 17:27:21 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Tue, 14 Jun 2022 21:57:21 +0430 Subject: snapshot problem In-Reply-To: <20220614125330.Horde.u5KxkrplHvtLIS5bmxnzeIM@webmail.nde.ag> References: <20220612081038.Horde._xfxaaTmZ0fovOR01idXLyP@webmail.nde.ag> <20220612110208.Horde.CpZ6qNn0k0fnPwT4SwMk118@webmail.nde.ag> <20220612131436.Horde.n_8Sdwn8Jxu79Lbvu4WV3PH@webmail.nde.ag> <20220613073300.Horde.zitXQVnixZSoeq2NOodZs9l@webmail.nde.ag> <20220613170342.Horde.qRaR_2eecVXaGrvWHuz6tC7@webmail.nde.ag> <20220614125330.Horde.u5KxkrplHvtLIS5bmxnzeIM@webmail.nde.ag> Message-ID: sorry for my mistake yes you are right thanks for your help and Cooperation On Tue, Jun 14, 2022 at 5:28 PM Eugen Block wrote: > You don't need the admin keyring on a compute node but for tests it > can help. Basically it depends on your ceph auth config which keyring > you need on the openstack nodes. > If you already have ceph configured why don't you use it for all > services? Of course live snapshots are not possible if your glance > images are not within ceph (I don't think you mentioned that before). > > > Zitat von Parsa Aminian : > > > yes you are right , I added ceph.client.admin.keyring to nova compute > > container and ceph cluster error gone i also try rbd snap create from > nova > > compute container and its work without problem and vm is up . > > maybe the problem is related to glance .my glance backend is not on ceph > I > > think maybe its the reason > > > > On Mon, Jun 13, 2022 at 9:38 PM Eugen Block wrote: > > > >> Hi, > >> > >> I don't have a containerized deployment so I don't have anything to > >> compare, but my compute nodes have a ceph.conf and all required > >> keyrings in /etc/ceph/, maybe someone else can confirm that it's > >> missing and should be present in a kolla-ansible deployment. In the > >> meantime you could provide those files to the container and retry the > >> live snapshot. > >> > >> > >> Zitat von Parsa Aminian : > >> > >> > snapshot_image_format doesn't exist on compute or controller nova.conf > >> > For the manual snapshot on the previous reply I ran it directly from > >> ceph, > >> > but it's interesting from nova compute there is an error about > connecting > >> > to CEPH cluster !!! also ceph -s is showing similar error > >> > [root at R3SG4 ~]# docker exec -u root -it nova_compute rbd snap create > >> > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap2022-06-13 > >> > 13:42:18.061 7f03dac23200 -1 auth: unable to find a keyring on > >> > > >> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: > >> > (2) No such file or directory2022-06-13 13:42:18.061 7f03dac23200 -1 > >> > AuthRegistry(0x561d7014a790) no keyring found at > >> > > >> > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, > >> > disabling ceph2022-06-13 13:42:18.065 7f03dac23200 -1 monclient: > >> > authenticate NOTE: no keyring found; disabled cephx authentication > >> > rbd: couldn't connect to the cluster! > >> > > >> > On Mon, Jun 13, 2022 at 12:15 PM Eugen Block wrote: > >> > > >> >> Could you share your whole nova.conf (only the uncommented options)? > >> >> Is this option set in your env? > >> >> > >> >> #snapshot_image_format = > >> >> > >> >> Also when you manually created the snapshot did you do it as the nova > >> >> user on the compute node? If not, could you retry? > >> >> > >> >> > >> >> Zitat von Parsa Aminian : > >> >> > >> >> > Hi > >> >> > kolla-ansible victoria version with ceph backend . > >> >> > rbd info output : > >> >> > rbd image '25c8d676-e20a-4238-a45c-d51daa62b941_disk': > >> >> > size 20 GiB in 5120 objects > >> >> > order 22 (4 MiB objects) > >> >> > snapshot_count: 0 > >> >> > id: b69aaf907284da > >> >> > block_name_prefix: rbd_data.b69aaf907284da > >> >> > format: 2 > >> >> > features: layering, exclusive-lock, object-map, fast-diff, > >> >> > deep-flatten > >> >> > op_features: > >> >> > flags: > >> >> > create_timestamp: Fri May 20 00:04:47 2022 > >> >> > access_timestamp: Sun Jun 12 16:26:02 2022 > >> >> > --------------- > >> >> > also live snapshot seems to work correctly without any error or any > >> >> > downtime : > >> >> > docker exec -u root -it ceph-mgr-cephosd01 rbd snap ls > >> >> > vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk > >> >> > SNAPID NAME SIZE PROTECTED TIMESTAMP > >> >> > 344 test-snap 20 GiB Sun Jun 12 23:48:39 2022 > >> >> > > >> >> > also on compute nova.conf, images_type is set on rbd . > >> >> > > >> >> > On Sun, Jun 12, 2022 at 5:55 PM Eugen Block wrote: > >> >> > > >> >> >> You should respond to the list so other users can try to support > you. > >> >> >> > >> >> >> So nova is trying to live snapshot the instance: > >> >> >> > >> >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> >> > 4dbffaa9c14e401c8c210e23ebe0ab7b > ef940663426b4c87a751afaf13b758e0 - > >> >> >> > default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] > >> >> >> > instance snapshotting > >> >> >> > [...] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning > >> >> >> > live snapshot process > >> >> >> > >> >> >> But I don't see any 'rbd snap create' command. Either the rbd > image > >> >> >> doesn't support it or there is some setting to keep all rbd images > >> >> >> "flat". Can you check any relevant configs you might have in nova? > >> >> >> Also can you show the output of 'rbd info > >> >> >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk' ? Then to test > if > >> >> >> the underlying rbd functions work as expected you could try to > create > >> >> >> a live snapshot manually: > >> >> >> > >> >> >> rbd snap create > >> >> /25c8d676-e20a-4238-a45c-d51daa62b941_disk at test-snap > >> >> >> > >> >> >> And paste any relevant output here. > >> >> >> > >> >> >> Zitat von Parsa Aminian : > >> >> >> > >> >> >> > Its not working for any instances and all of them are paused . I > >> >> enable > >> >> >> > debug logs please check the logs : > >> >> >> > > >> >> >> > 2022-06-12 16:16:13.478 7 DEBUG nova.compute.manager > >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering > >> sync > >> >> for > >> >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> >> >> > 2022-06-12 16:16:13.506 7 DEBUG oslo_concurrency.lockutils [-] > Lock > >> >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> >> >> > > >> >> >> > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> >> > :: waited 0.000s inner > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> >> >> > 2022-06-12 16:16:43.562 7 DEBUG nova.compute.resource_tracker > >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > >> compute > >> >> >> host > >> >> >> > and has allocations in placement: {'resources': {'VCPU': 1, > >> >> 'MEMORY_MB': > >> >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> >> >> > 2022-06-12 16:25:55.104 7 DEBUG nova.compute.manager > >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> _get_power_state > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> >> > 2022-06-12 16:25:55.603 7 INFO nova.compute.manager > >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> >> > 63426b4c87a751afaf13b758e0 - default default] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Beginning live snapshot > >> process > >> >> >> > default default] Lazy-loading 'pci_devices' on Instance uuid > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 obj_load_attr > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> >> >> > 2022-06-12 16:25:57.250 7 DEBUG nova.objects.instance > >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Lazy-loading > >> >> >> > 'pci_devices' on Instance uuid > 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> >> > obj_load_attr > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/objects/instance.py:1101 > >> >> >> > 2022-06-12 16:25:57.317 7 DEBUG nova.virt.driver [-] Emitting > event > >> >> >> > >> >> 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> >> > => Paused> emit_event > >> >> >> > > >> >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> >> >> > 2022-06-12 16:25:57.318 7 INFO nova.compute.manager [-] > [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle > Event) > >> >> >> > 2022-06-12 16:25:57.389 7 DEBUG nova.compute.manager > >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> _get_power_state > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> >> > 2022-06-12 16:25:57.395 7 DEBUG nova.compute.manager > >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance > power > >> >> state > >> >> >> > after lifecycle event "Paused"; current vm_state: active, > current > >> >> >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> >> >> > power_state: 3 handle_lifecycle_event > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> >> >> > 2022-06-12 16:25:57.487 7 INFO nova.compute.manager > >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state > the > >> >> >> instance > >> >> >> > has a pending task (image_pending_upload). Skip. > >> >> >> > 2022-06-12 16:26:02.039 7 DEBUG oslo_concurrency.processutils > >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] Running cmd > >> >> >> > (subprocess): qemu-img convert -t none -O raw -f raw > >> >> >> > > >> >> >> > >> >> > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa62b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad > >> >> >> > execute > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:384 > >> >> >> > 2022-06-12 16:26:17.075 7 DEBUG nova.virt.driver [-] Emitting > event > >> >> >> > >> >> 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> >> > => Stopped> emit_event > >> >> >> > > >> >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> >> >> > INFO nova.compute.manager [-] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped (Lifecycle > Event) > >> >> >> > DEBUG nova.compute.manager > >> [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > >> >> - - > >> >> >> - > >> >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking > >> state > >> >> >> > _get_power_state > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> >> > DEBUG nova.compute.manager > >> [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa > >> >> - - > >> >> >> - > >> >> >> > - -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] > Synchronizing > >> >> >> > instance power state after lifecycle event "Stopped"; current > >> >> vm_state: > >> >> >> > active, current task_state: image_pending_upload, current DB > >> >> power_state: > >> >> >> > 1, VM power_state: 4 handle_lifecycle_event > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> >> >> > INFO nova.compute.manager > >> [req-f9f8cbf5-6208-4dca-aca6-48dee87f38fa - > >> >> - > >> >> >> - - > >> >> >> > -] [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] During > >> >> >> sync_power_state > >> >> >> > the instance has a pending task (image_pending_upload). Skip. > >> >> >> > 2022-06-12 16:26:18.539 7 DEBUG nova.compute.manager > >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering > >> sync > >> >> for > >> >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> >> >> > 2022-06-12 16:26:18.565 7 DEBUG oslo_concurrency.lockutils [-] > Lock > >> >> >> > "25c8d676-e20a-4238 > >> >> >> > -a45c-d51daa62b941" acquired by > >> >> >> > > >> >> >> > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> >> > :: waited 0.000s inner > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> >> >> > 2022-06-12 16:26:18.566 7 INFO nova.compute.manager [-] > [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During sync_power_state > the > >> >> >> instance > >> >> >> > has a pending task (image_pending_upload). Skip. > >> >> >> > 2022-06-12 16:26:18.566 7 DEBUG oslo_concurrency.lockutils [-] > Lock > >> >> >> > "25c8d676-e20a-4238-a45c-d51daa62b941" released by > >> >> >> > > >> >> >> > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> >> > :: held 0.001s inner > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:371 > >> >> >> > 2022-06-12 16:26:25.769 7 DEBUG oslo_concurrency.processutils > >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] CMD > "qemu-img > >> >> convert > >> >> >> > -t none -O raw -f raw > >> >> >> > > >> >> >> > >> >> > >> > rbd:vms/25c8d676-e20a-4238-a45c-d51daa6b941_disk:id=cinder:conf=/etc/ceph/ceph.conf > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/nova/instances/snapshots/tmpv21b_i59/8717dec4c99c4ef7bac752e2a48690ad" > >> >> >> > returned: 0 in 23.730s execute > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/processutils.py:423 > >> >> >> > default default] [instance: > 25c8d676-e20a-4238-a45c-d51daa62b941] > >> >> >> Snapshot > >> >> >> > extracted, beginning image upload > >> >> >> > 2022-06-12 16:26:27.981 7 DEBUG nova.virt.driver > >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] Emitting > event > >> >> >> > >> >> 25c8d676-e20a-4238-a45c-d51daa62b941 > >> >> >> > => Started> emit_event > >> >> >> > > >> >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/driver.py:1704 > >> >> >> > 2022-06-12 16:26:27.983 7 INFO nova.compute.manager > >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started (Lifecycle > Event) > >> >> >> > [instance: 25c8d676-e20a-4238-a45c-d51daa62b941] Checking state > >> >> >> > _get_power_state > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1569 > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Synchronizing instance > power > >> >> state > >> >> >> > after lifecycle event "Started"; current vm_state: active, > current > >> >> >> > task_state: image_pending_upload, current DB power_state: 1, VM > >> >> >> > power_state: 1 handle_lifecycle_event > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:1299 > >> >> >> > 2022-06-12 16:26:28.173 7 INFO nova.compute.manager > >> >> >> > [req-40444d74-f2fa-4569-87dd-375139938e81 - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Resumed (Lifecycle > Event > >> >> >> > 2022-06-12 16:29:00.859 7 DEBUG oslo_concurrency.lockutils > >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Acquired > lock > >> >> >> > "refresh_cache-25c8d676-e20a-4238-a45c-d51daa62b941" lock > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:266 > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Forcefully refreshing > network > >> >> info > >> >> >> > cache for instance _get_instance_nw_info > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:1833 > >> >> >> > 2022-06-12 16:29:03.278 7 DEBUG nova.network.neutron > >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Updating > instance_info_cache > >> >> with > >> >> >> > network_info: [{"id": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > >> >> "address": > >> >> >> > "fa:16:3e:ca:00:d9", "network": {"id": > >> >> >> > "b86c8304-a9bd-4b39-b7fc-f70ffe76f2a8", "bridge": "br-int", > >> "label": > >> >> >> > "External_Network", "subnets": [{"cidr": "141.11.42.0/24", > "dns": > >> >> >> > [{"address": "8.8.8.8", "type": "dns", "version": 4, "meta": > {}}, > >> >> >> > {"address": "217.218.127.127", "type": "dns", "version": 4, > "meta": > >> >> {}}], > >> >> >> > "gateway": {"address": "141.11.42.1", "type": "gateway", > >> "version": 4, > >> >> >> > "meta": {}}, "ips": [{"address": "141.11.42.37", "type": > "fixed", > >> >> >> > "version": 4, "meta": {}, "floating_ips": []}], "routes": [], > >> >> "version": > >> >> >> 4, > >> >> >> > "meta": {}}], "meta": {"injected": true, "tenant_id": > >> >> >> > "ef940663426b4c87a751afaf13b758e0", "mtu": 1500, > >> "physical_network": > >> >> >> > "physnet1", "tunneled": false}}, "type": "ovs", "details": > >> >> >> {"connectivity": > >> >> >> > "l2", "port_filter": true, "ovs_hybrid_plug": true, > >> "datapath_type": > >> >> >> > "system", "bridge_name": "br-int"}, "devname": "tapaa2fdd7d-ad", > >> >> >> > "ovs_interfaceid": "aa2fdd7d-ad18-4890-ad57-14bf9888d2c1", > >> >> "qbh_params": > >> >> >> > null, "qbg_params": null, "active": true, "vnic_type": "normal", > >> >> >> "profile": > >> >> >> > {}, "preserve_on_delete": false, "meta": {}}] > >> >> >> > update_instance_cache_with_nw_info > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/network/neutron.py:117 > >> >> >> > nstance 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on > >> this > >> >> >> > compute host and has allocations in placement: {'resources': > >> {'VCPU': > >> >> 1, > >> >> >> > 'MEMORY_MB': 1024, 'DISK_GB': 20}}. > >> >> _remove_deleted_instances_allocations > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> >> >> > 2022-06-12 16:33:37.595 7 INFO nova.compute.manager > >> >> >> > [req-5ecfdf74-7cf3-481a-aa12-140deae202f7 > >> >> >> 4dbffaa9c14e401c8c210e23ebe0ab7b > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Took 461.98 seconds to > >> snapshot > >> >> the > >> >> >> > instance on the hypervisor. > >> >> >> > 2022-06-12 16:36:16.459 7 DEBUG nova.compute.manager > >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Triggering > >> sync > >> >> for > >> >> >> > uuid 25c8d676-e20a-4238-a45c-d51daa62b941 _sync_power_states > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py:9693 > >> >> >> > Lock "25c8d676-e20a-4238-a45c-d51daa62b941" acquired by > >> >> >> > > >> >> >> > >> >> > >> > "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" > >> >> >> > :: waited 0.000s inner > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/oslo_concurrency/lockutils.py:359 > >> >> >> > 2022-06-12 16:37:05.365 7 DEBUG nova.compute.resource_tracker > >> >> >> > [req-2ecf34c3-72e7-4f33-89cb-9b250cd6d223 - - - - -] Instance > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941 actively managed on this > >> compute > >> >> >> host > >> >> >> > and has allocations in placement: {'resources': {'VCPU': 1, > >> >> 'MEMORY_MB': > >> >> >> > 1024, 'DISK_GB': 20}}. _remove_deleted_instances_allocations > >> >> >> > > >> >> >> > >> >> > >> > /var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/resource_tracker.py:1539 > >> >> >> > 2022-06-12 09:42:32.687 7 INFO nova.compute.manager > >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> >> >> 93fb420b3c604d4fae408b81135b58e9 > >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] [instance: > >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance snapshotting > >> >> >> > > >> >> >> > On Sun, Jun 12, 2022 at 3:36 PM Eugen Block > wrote: > >> >> >> > > >> >> >> >> Have you tried with debug logs? Has it worked with live > snapshots > >> >> >> >> before for other instances or did it never work and all > snapshots > >> >> were > >> >> >> >> "cold"? > >> >> >> >> > >> >> >> >> Zitat von Parsa Aminian : > >> >> >> >> > >> >> >> >> > Hi > >> >> >> >> > kolla-ansible victoria version with ceph backend without > volume > >> >> >> >> > > >> >> >> >> > On Sun, Jun 12, 2022 at 12:45 PM Eugen Block > >> >> wrote: > >> >> >> >> > > >> >> >> >> >> Hi, > >> >> >> >> >> > >> >> >> >> >> can you share more details about your environment? Which > >> openstack > >> >> >> >> >> version is it? What is the storage backend? In earlier > releases > >> >> there > >> >> >> >> >> was an option: > >> >> >> >> >> > >> >> >> >> >> #disable_libvirt_livesnapshot = false > >> >> >> >> >> > >> >> >> >> >> but this option has been deprecated. But if you're on an > older > >> >> >> version > >> >> >> >> >> that could explain it. > >> >> >> >> >> > >> >> >> >> >> Zitat von Parsa Aminian : > >> >> >> >> >> > >> >> >> >> >> > When I snapshot from the instance , server will gone away > and > >> >> its > >> >> >> not > >> >> >> >> >> > reachable until the snapshot is complete here is the logs > : > >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] instance > snapshotting > >> >> >> >> >> > 2022-06-12 09:42:34.755 7 INFO nova.compute.manager > >> >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] > >> [instance: > >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Paused (Lifecycle > >> >> Event) > >> >> >> >> >> > 2022-06-12 09:42:34.995 7 INFO nova.compute.manager > >> >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] > >> [instance: > >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] During > sync_power_state > >> >> the > >> >> >> >> >> instance > >> >> >> >> >> > has a pending task (image_pending_upload). Skip. > >> >> >> >> >> > 2022-06-12 09:42:57.749 7 INFO nova.compute.manager [-] > >> >> [instance: > >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Stopped > (Lifecycle > >> >> Event) > >> >> >> >> >> > 2022-06-12 09:43:06.102 7 INFO nova.virt.libvirt.driver > >> >> >> >> >> > [req-e79e4177-4712-4795-91da-853bc524fac0 > >> >> >> >> >> 93fb420b3c604d4fae408b81135b58e9 > >> >> >> >> >> > ef940663426b4c87a751afaf13b758e0 - default default] > >> [instance: > >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] Snapshot extracted, > >> >> beginning > >> >> >> >> image > >> >> >> >> >> > upload > >> >> >> >> >> > 2022-06-12 09:43:08.778 7 INFO nova.compute.manager > >> >> >> >> >> > [req-786946b1-3d22-489c-bf4d-8b1375b09ecb - - - - -] > >> [instance: > >> >> >> >> >> > 25c8d676-e20a-4238-a45c-d51daa62b941] VM Started > (Lifecycle > >> >> Event) > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> > >> > >> > >> > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Tue Jun 14 22:47:59 2022 From: zigo at debian.org (Thomas Goirand) Date: Wed, 15 Jun 2022 00:47:59 +0200 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> Message-ID: <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> On 6/13/22 00:10, Thomas Goirand wrote: > Hi, > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > However, OpenStack global requirements are still stuck at 3.2.0. Is > there any reason for it, or should we attempt to upgrade to 4.6.0? > > I'd really appreciate if someone (else than me) was driving this... > > Cheers, > > Thomas Goirand (zigo) > FYI, Nova fails with it: https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz Can someone from the Nova team investigate? Cheers, Thomas Goirand (zigo) From smooney at redhat.com Tue Jun 14 23:35:04 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 15 Jun 2022 00:35:04 +0100 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> Message-ID: On Wed, 2022-06-15 at 00:47 +0200, Thomas Goirand wrote: > On 6/13/22 00:10, Thomas Goirand wrote: > > Hi, > > > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > > However, OpenStack global requirements are still stuck at 3.2.0. Is > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > I'd really appreciate if someone (else than me) was driving this... > > > > Cheers, > > > > Thomas Goirand (zigo) > > > > FYI, Nova fails with it: > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > Can someone from the Nova team investigate? nova is not pinning this https://github.com/openstack/nova/blob/master/requirements.txt#L25= we test with whatever is allowed by uppper constriats so we need to look a this form a requirements team point of view anyjson seams to deplped on use2to3 which is breaking the script that updates requiremtns for me locally. if i remvoe that temporally and run .venv/bin/generate-constraints -b blacklist.txt -p python3.8 -r global-requirements.txt > upper-constraints.txt i get futher but then fail because i dont have all the bindep of every python lib in gloal-requiremetns install on my laptop ill try and check that again in a devstack vm which should have many of those deps installed. but something seams to be broken in the automation that updates the upper-constratis. the zuul periodic is at least sometime passing https://zuul.openstack.org/builds?job_name=propose-updates&project=openstack%2Frequirements&skip=0 but im not sure how many old deps we have and why the requiremnts repo is still pinning to 3.2.0 form november 2019 > > Cheers, > > Thomas Goirand (zigo) > From gmann at ghanshyammann.com Tue Jun 14 23:49:14 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Tue, 14 Jun 2022 18:49:14 -0500 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> Message-ID: <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> ---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand wrote ---- > On 6/13/22 00:10, Thomas Goirand wrote: > > Hi, > > > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > > However, OpenStack global requirements are still stuck at 3.2.0. Is > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > I'd really appreciate if someone (else than me) was driving this... > > > > Cheers, > > > > Thomas Goirand (zigo) > > > > FYI, Nova fails with it: > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > Can someone from the Nova team investigate? Nova failures are due to the error message change (it happens regularly and they change them in most versions) in jsonschema new version. I remember we faced this type of issue previously also and we updated nova tests not to assert the error message but seems like there are a few left which is failing with jsonschema 4.6.0. Along with these error message failures fixes and using the latest jsonschema, in nova, we use Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check what are backward-incompatible changes in Draft7Validator and bump this too? -gmann > > Cheers, > > Thomas Goirand (zigo) > > From smooney at redhat.com Tue Jun 14 23:54:23 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 15 Jun 2022 00:54:23 +0100 Subject: New install on CentOS: what versions, etc.? In-Reply-To: <7ec0e14593137adf8a91193628d4e301@jots.org> References: <7ec0e14593137adf8a91193628d4e301@jots.org> Message-ID: <1057e63b912984f4c2b5f712cbe56106de86cd8a.camel@redhat.com> On Tue, 2022-06-14 at 14:59 -0400, Ken D'Ambrosio wrote: > Hey, all. We're currently running (I hope you're sitting) Juno, and > we're thinking about rolling out a new OpenStack version. My boss has > asked me if CentOS 7.9 > a) would be supported for control/compute nodes, and, if so, if there > are any long-term potential issues, and most openstack testing in the centos family has move to centos 9 stream i woudl not presonally recommend deploying a new install in 7.9 google tells me censot 7 is eol on June 30, 2024, realisticaly that give you only 2 years of "supprot" but centos 7 woudl be useing python 2.7 or 3.6.8 python 3.6 is eol upstream so it will not recive any security fixes for any of these issues https://repology.org/project/python/cves?version=3.6.8 so i would start with at least centos 8 stream or 9 stream if you want to move to anything modern if you do use 7.9 the last openstack version redhat maintained on rhel 7 was queens thats proably the best tested/maitained version as a reult but if its a new install i really dont think you shoudl consider anything older then wallaby personally if i was rooling my own deployment or useign say the opentstack ansible or simialr to deploy i would go to the most recent stable openstack version avaiabel that is supproted by my installer tool of chice. so yoga in this case. there are far to many issues that have been adressed in the interveing time for me to really recommend deploying old releases unless you are going to pay a vendor to supprot the openstack distorbution for you i which case i would recoment the newest release they offer. > b) if there would be any concerns about versions of client OSes. if you are on centos 7 that likely means that you will not be able to deploy an openstack release with supprot for vtpm that will mean you will not be able to run windows 11 or windows server 2022 guest as they requrie vtpm 2.0 support i also belive that rhel 9 is not supprot on 7.9 host form a readhat point of view https://access.redhat.com/articles/973133 https://access.redhat.com/articles/973163 so yes using centos 7.9 release will limit the guests you can run > > Thanks! > > -Ken > From smooney at redhat.com Tue Jun 14 23:58:05 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 15 Jun 2022 00:58:05 +0100 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> Message-ID: <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote: > ---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand wrote ---- > > On 6/13/22 00:10, Thomas Goirand wrote: > > > Hi, > > > > > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > > > However, OpenStack global requirements are still stuck at 3.2.0. Is > > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > > > I'd really appreciate if someone (else than me) was driving this... > > > > > > Cheers, > > > > > > Thomas Goirand (zigo) > > > > > > > FYI, Nova fails with it: > > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > > > Can someone from the Nova team investigate? > > Nova failures are due to the error message change (it happens regularly and they change them in most versions) > in jsonschema new version. I remember we faced this type of issue previously also and we updated > nova tests not to assert the error message but seems like there are a few left which is failing with > jsonschema 4.6.0. > > Along with these error message failures fixes and using the latest jsonschema, in nova, we use > Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check > what are backward-incompatible changes in Draft7Validator and bump this too? well i think the reason this is on 3.2.0 iis nothing to do with nova i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/ The conflict is caused by: The user requested jsonschema===4.0.1 tripleo-common 16.4.0 depends on jsonschema>=3.2.0 rsd-lib 1.2.0 depends on jsonschema>=2.6.0 taskflow 4.7.0 depends on jsonschema>=3.2.0 zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 os-net-config 15.2.0 depends on jsonschema>=3.2.0 task-core 0.2.1 depends on jsonschema>=3.2.0 python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 warlock 1.3.3 depends on jsonschema<4 and >=0.7 https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9 it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0 > > -gmann > > > > > Cheers, > > > > Thomas Goirand (zigo) > > > > > From smooney at redhat.com Wed Jun 15 00:01:47 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 15 Jun 2022 01:01:47 +0100 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> Message-ID: On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote: > On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote: > > ---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand wrote ---- > > > On 6/13/22 00:10, Thomas Goirand wrote: > > > > Hi, > > > > > > > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > > > > However, OpenStack global requirements are still stuck at 3.2.0. Is > > > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > > > > > I'd really appreciate if someone (else than me) was driving this... > > > > > > > > Cheers, > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > FYI, Nova fails with it: > > > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > > > > > Can someone from the Nova team investigate? > > > > Nova failures are due to the error message change (it happens regularly and they change them in most versions) > > in jsonschema new version. I remember we faced this type of issue previously also and we updated > > nova tests not to assert the error message but seems like there are a few left which is failing with > > jsonschema 4.6.0. > > > > Along with these error message failures fixes and using the latest jsonschema, in nova, we use > > Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check > > what are backward-incompatible changes in Draft7Validator and bump this too? > > well i think the reason this is on 3.2.0 iis nothing to do with nova > i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/ > > The conflict is caused by: > The user requested jsonschema===4.0.1 > tripleo-common 16.4.0 depends on jsonschema>=3.2.0 > rsd-lib 1.2.0 depends on jsonschema>=2.6.0 > taskflow 4.7.0 depends on jsonschema>=3.2.0 > zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 > os-net-config 15.2.0 depends on jsonschema>=3.2.0 > task-core 0.2.1 depends on jsonschema>=3.2.0 > python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 > warlock 1.3.3 depends on jsonschema<4 and >=0.7 > > https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9 > > it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0 it look like there is already an issue open https://github.com/bcwaldon/warlock/issues/64 but they have not merged any comit sine 2020-02-09 so this looks like its at least parly unmaintaied > > > > > -gmann > > > > > > > > Cheers, > > > > > > Thomas Goirand (zigo) > > > > > > > > > From smooney at redhat.com Wed Jun 15 00:04:46 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 15 Jun 2022 01:04:46 +0100 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> Message-ID: On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote: > On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote: > > ---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand wrote ---- > > > On 6/13/22 00:10, Thomas Goirand wrote: > > > > Hi, > > > > > > > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > > > > However, OpenStack global requirements are still stuck at 3.2.0. Is > > > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > > > > > I'd really appreciate if someone (else than me) was driving this... > > > > > > > > Cheers, > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > FYI, Nova fails with it: > > > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > > > > > Can someone from the Nova team investigate? > > > > Nova failures are due to the error message change (it happens regularly and they change them in most versions) > > in jsonschema new version. I remember we faced this type of issue previously also and we updated > > nova tests not to assert the error message but seems like there are a few left which is failing with > > jsonschema 4.6.0. > > > > Along with these error message failures fixes and using the latest jsonschema, in nova, we use > > Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check > > what are backward-incompatible changes in Draft7Validator and bump this too? > > well i think the reason this is on 3.2.0 iis nothing to do with nova > i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/ > > The conflict is caused by: > The user requested jsonschema===4.0.1 > tripleo-common 16.4.0 depends on jsonschema>=3.2.0 > rsd-lib 1.2.0 depends on jsonschema>=2.6.0 > taskflow 4.7.0 depends on jsonschema>=3.2.0 > zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 > os-net-config 15.2.0 depends on jsonschema>=3.2.0 > task-core 0.2.1 depends on jsonschema>=3.2.0 > python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 > warlock 1.3.3 depends on jsonschema<4 and >=0.7 > > https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9 > > it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0 glance client seams to be the only real user of this https://codesearch.opendev.org/?q=warlock&i=nope&literal=nope&files=&excludeFiles=&repos= perhaps we could jsut remove the dependcy? > > > > > -gmann > > > > > > > > Cheers, > > > > > > Thomas Goirand (zigo) > > > > > > > > > From bshephar at redhat.com Wed Jun 15 05:22:12 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Wed, 15 Jun 2022 15:22:12 +1000 Subject: Propose to add Takashi Kajinami as heat core reviewer In-Reply-To: References: Message-ID: +1 for sure. Takashi is an asset to all of OpenStack! I would happily endorse his inclusion as a Core reviewer to the Heat project. Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Mon, Jun 13, 2022 at 11:55 PM Rabi Mishra wrote: > > > On Mon, Jun 13, 2022 at 6:43 PM Rico Lin wrote: > >> Hi all >> >> I want to suggest adding Takashi Kajinami as heat >> core reviewer. >> Who help provide valid reviews and fixes for issues on heat. >> Will be great to have Takashi Kajinami join heat core reviewer team and >> help. >> > > +1, Good addition, he is actively involved for some time and contributes > to keep the project healthy . > > >> >> *Rico Lin* >> > > > -- > Regards, > Rabi Mishra > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bshephar at redhat.com Wed Jun 15 06:00:24 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Wed, 15 Jun 2022 16:00:24 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hey Anirudh, You would need to look at the logs for the ironic_pxe_tftp container to see why it's failing. I assume the tftp container is not Up when you run this command? [stack at tripleo-director overcloud_playbooks]$ sudo podman ps --filter name=ironic_pxe -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0170be36e291 registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo kolla_start 12 days ago Up 30 hours ago (healthy) ironic_pxe_tftp e507f722bdf0 registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo kolla_start 12 days ago Up 30 hours ago (healthy) ironic_pxe_http Then check the logs to see what the error is: [stack at tripleo-director overcloud_playbooks]$ sudo podman logs ironic_pxe_tftp Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta wrote: > Hi Team, > > I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing the > below error: > > 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | TASK | > Manage container systemd services and cleanup old systemd healthchecks for > /var/lib/tripleo-config/container-startup-config/step_4 > 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | FATAL > | Manage container systemd services and cleanup old systemd healthchecks > for /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | > error={"changed": false, "msg": "Service ironic_pxe_tftp has not started > yet"} > 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | TIMING > | tripleo_container_manage : Manage container systemd > > Sample Undercloud.conf is as follows: > > [DEFAULT] > clean_nodes = true > cleanup = false > container_cli = podman > container_healthcheck_disabled = true > container_images_file = /home/stack/containers-prepare-parameter.yaml > deployment_user = stack > enable_ironic = true > enable_ironic_inspector = true > enable_neutron = true > enable_routed_networks = false > generate_service_certificate = false > ipv6_address_mode = dhcpv6-stateful > ipxe_enabled = true > local_interface = enp8s0 > local_ip = aaaa:aaaa:aaaa::1/64 > subnets = ctlplane-subnet > undercloud_admin_host = aaaa:aaaa:aaaa::1 > undercloud_hostname = undercloud.com > undercloud_ntp_servers = 30.30.30.3 > undercloud_public_host = aaaa:aaaa:aaaa::1 > undercloud_timezone = UTC > > [ctlplane-subnet] > cidr = aaaa:aaaa:aaaa::/64 > dhcp_end = aaaa:aaaa:aaaa::f > dhcp_start = aaaa:aaaa:aaaa::a > gateway = aaaa:aaaa:aaaa::1 > inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 > > Can someone please help in this regard. > > Anirudh Gupta > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralonsoh at redhat.com Wed Jun 15 07:38:44 2022 From: ralonsoh at redhat.com (Rodolfo Alonso Hernandez) Date: Wed, 15 Jun 2022 09:38:44 +0200 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> Message-ID: The jsonschema limitation in warlock [1] is removed in master branch but now we need a new release. Looking at the code of this project, is it possible to avoid importing this project and implement the required functionality? Code looks short and easy to be re-implemented (basically what I'm saying is to copy-paste this code into oslo.utils). [1]https://github.com/bcwaldon/warlock/issues/64 On Wed, Jun 15, 2022 at 2:13 AM Sean Mooney wrote: > On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote: > > On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote: > > > ---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand < > zigo at debian.org> wrote ---- > > > > On 6/13/22 00:10, Thomas Goirand wrote: > > > > > Hi, > > > > > > > > > > A few DDs are pushing me to upgrade jsonschema in Debian > Unstable. > > > > > However, OpenStack global requirements are still stuck at 3.2.0. > Is > > > > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > > > > > > > I'd really appreciate if someone (else than me) was driving > this... > > > > > > > > > > Cheers, > > > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > > > > FYI, Nova fails with it: > > > > > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > > > > > > > Can someone from the Nova team investigate? > > > > > > Nova failures are due to the error message change (it happens > regularly and they change them in most versions) > > > in jsonschema new version. I remember we faced this type of issue > previously also and we updated > > > nova tests not to assert the error message but seems like there are a > few left which is failing with > > > jsonschema 4.6.0. > > > > > > Along with these error message failures fixes and using the latest > jsonschema, in nova, we use > > > Draft4Validator from jsonschema and the latest validator is > Draft7Validator so we should check > > > what are backward-incompatible changes in Draft7Validator and bump > this too? > > > > well i think the reason this is on 3.2.0 iis nothing to do with nova > > i bumpt it manually in > https://review.opendev.org/c/openstack/requirements/+/845859/ > > > > The conflict is caused by: > > The user requested jsonschema===4.0.1 > > tripleo-common 16.4.0 depends on jsonschema>=3.2.0 > > rsd-lib 1.2.0 depends on jsonschema>=2.6.0 > > taskflow 4.7.0 depends on jsonschema>=3.2.0 > > zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 > > os-net-config 15.2.0 depends on jsonschema>=3.2.0 > > task-core 0.2.1 depends on jsonschema>=3.2.0 > > python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 > > warlock 1.3.3 depends on jsonschema<4 and >=0.7 > > > > > https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9 > > > > it looks like warlock is clamping it ot less then 4 which is why we are > stokc on 3.2.0 > glance client seams to be the only real user of this > > https://codesearch.opendev.org/?q=warlock&i=nope&literal=nope&files=&excludeFiles=&repos= > perhaps we could jsut remove the dependcy? > > > > > > > > -gmann > > > > > > > > > > > Cheers, > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chkumar at redhat.com Wed Jun 15 08:43:47 2022 From: chkumar at redhat.com (Chandan Kumar) Date: Wed, 15 Jun 2022 14:13:47 +0530 Subject: [qa][refstack][python-tempestconf] adding image In-Reply-To: References: Message-ID: Hello Mohammed, On Wed, Jun 15, 2022 at 1:47 AM Martin Kopec wrote: > > Hi Mohammed, > > we are glad to hear that python-tempestconf is useful. > We don't mind having python-tempestconf included in Docker images, although what exactly would we need to do? I'm afraid we can't commit to regular maintenance of the images. > The TripleO CI team publishes the tempest container on Quay for each release. 1. https://quay.io/repository/tripleomastercentos9/openstack-tempest It already contains python-tempestconf utility within the container. Here is the source for the same: https://opendev.org/openstack/tripleo-common/src/branch/master/container-images/tcib/base/os/tempest The above source is consumed by tripleo container image build utility, provided by python-tripleoclient to generate the dockerfile. Here is the dockerfile from one of the job: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-build-containers-centos-9-quay-push-master/ca7c761/logs/container-builds/a5802d5c-b997-4de0-ac68-654e0f9b4095/base/os/tempest/Dockerfile and other build artifacts: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-build-containers-centos-9-quay-push-master/ca7c761/logs/container-builds/a5802d5c-b997-4de0-ac68-654e0f9b4095/base/os/tempest/ Let me know if it helps. Thanks, Chandan Kumar From michal.arbet at ultimum.io Wed Jun 15 08:53:19 2022 From: michal.arbet at ultimum.io (Michal Arbet) Date: Wed, 15 Jun 2022 10:53:19 +0200 Subject: [cinder] Failed to delete volume In-Reply-To: <20220614073846.fd7pdx4hwe7joqpz@localhost> References: <20220614073846.fd7pdx4hwe7joqpz@localhost> Message-ID: Hi Gorka, Yeah, we are using active-active cinder deployment. - Not defined host - Not defined backend_host - Defined cluster And redis-sentinel as coordinator Binary Host Zone Status State Updated At RPC Version Object Version Cluster cinder-scheduler cn36x005 nova enabled :-) 2022-06-15 05:56:49 3.12 1.38 cinder-scheduler cv36x006 nova enabled :-) 2022-06-15 05:56:54 3.12 1.38 cinder-backup cn36x005 nova enabled :-) 2022-06-15 05:56:55 2.2 1.38 cinder-backup cv36x006 nova enabled :-) 2022-06-15 05:56:49 2.2 1.38 cinder-volume cn36x005 at rbd-2 nova enabled :-) 2022-06-15 05:56:50 3.16 1.38 cinder_cl1 at rbd-2 cinder-volume cv36x006 at rbd-2 nova enabled :-) 2022-06-15 05:56:53 3.16 1.38 cinder_cl1 at rbd-2 cinder-scheduler cn36x009 nova enabled :-) 2022-06-15 05:56:50 3.12 1.38 cinder-scheduler cv36x010 nova enabled :-) 2022-06-15 05:56:57 3.12 1.38 cinder-volume cn36x005 at rbd-1 nova enabled :-) 2022-06-15 05:56:54 3.16 1.38 cinder_cl1 at rbd-1 cinder-volume cv36x006 at rbd-1 nova enabled :-) 2022-06-15 05:56:53 3.16 1.38 cinder_cl1 at rbd-1 Michal Arbet Openstack Engineer Ultimum Technologies a.s. Na Po???? 1047/26, 11000 Praha 1 Czech Republic +420 604 228 897 michal.arbet at ultimum.io *https://ultimum.io * LinkedIn | Twitter | Facebook ?t 14. 6. 2022 v 9:38 odes?latel Gorka Eguileor napsal: > On 07/06, Michal Arbet wrote: > > Hi, > > > > I would like to ask if someone saw issue when volume failed to delete, > log > > below : > > This is happening from time to time. > > > > LOG : https://paste.opendev.org/show/b1jRCxuBFnnVzf64i9yV/ > > > > Thanks, > > > > Michal Arbet > > Openstack Engineer > > > > Hi, > > Error seems to be happening acquiring a lock with the Redis driver in > Tooz, so it must be configured to be used by Cinder as the DLM. > > This is only necessary when the Cinder Volume service is configured as > Active-Active. Are you running the volume service as Active-Active? > > If you are not you should be able to fix the issue by commenting the > backend_url configuration option in the [coordination] section of the > cinder.conf file. > > Cheers, > Gorka. > > > > Ultimum Technologies a.s. > > Na Po???? 1047/26, 11000 Praha 1 > > Czech Republic > > > > +420 604 228 897 > > michal.arbet at ultimum.io > > *https://ultimum.io * > > > > LinkedIn | > Twitter > > | Facebook > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bshephar at redhat.com Wed Jun 15 09:00:44 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Wed, 15 Jun 2022 19:00:44 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Looks like something wrong with the dnsmasq command the container is being launched with. What command is it trying to run? sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta wrote: > Hi Brendan, > > Thanks for your response. > > Please find the log below. > > [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp > > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > > [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe -a > CONTAINER ID IMAGE > COMMAND CREATED STATUS > PORTS NAMES > 02dacbc74cec > undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo > /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) > ironic_pxe_tftp > 1f8ca39fba32 > undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo > kolla_start 3 hours ago Up 3 hours ago (healthy) > ironic_pxe_http > > > Regards > > Anirudh Gupta > > On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard > wrote: > >> Hey Anirudh, >> >> You would need to look at the logs for the ironic_pxe_tftp container to >> see why it's failing. >> >> I assume the tftp container is not Up when you run this command? >> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps --filter >> name=ironic_pxe -a >> CONTAINER ID IMAGE >> COMMAND CREATED STATUS >> PORTS NAMES >> 0170be36e291 >> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >> kolla_start 12 days ago Up 30 hours ago (healthy) >> ironic_pxe_tftp >> e507f722bdf0 >> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >> kolla_start 12 days ago Up 30 hours ago (healthy) >> ironic_pxe_http >> >> Then check the logs to see what the error is: >> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >> ironic_pxe_tftp >> >> >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >> wrote: >> >>> Hi Team, >>> >>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing >>> the below error: >>> >>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | TASK >>> | Manage container systemd services and cleanup old systemd healthchecks >>> for /var/lib/tripleo-config/container-startup-config/step_4 >>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | >>> FATAL | Manage container systemd services and cleanup old systemd >>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>> not started yet"} >>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | >>> TIMING | tripleo_container_manage : Manage container systemd >>> >>> Sample Undercloud.conf is as follows: >>> >>> [DEFAULT] >>> clean_nodes = true >>> cleanup = false >>> container_cli = podman >>> container_healthcheck_disabled = true >>> container_images_file = /home/stack/containers-prepare-parameter.yaml >>> deployment_user = stack >>> enable_ironic = true >>> enable_ironic_inspector = true >>> enable_neutron = true >>> enable_routed_networks = false >>> generate_service_certificate = false >>> ipv6_address_mode = dhcpv6-stateful >>> ipxe_enabled = true >>> local_interface = enp8s0 >>> local_ip = aaaa:aaaa:aaaa::1/64 >>> subnets = ctlplane-subnet >>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>> undercloud_hostname = undercloud.com >>> undercloud_ntp_servers = 30.30.30.3 >>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>> undercloud_timezone = UTC >>> >>> [ctlplane-subnet] >>> cidr = aaaa:aaaa:aaaa::/64 >>> dhcp_end = aaaa:aaaa:aaaa::f >>> dhcp_start = aaaa:aaaa:aaaa::a >>> gateway = aaaa:aaaa:aaaa::1 >>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>> >>> Can someone please help in this regard. >>> >>> Anirudh Gupta >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From lokendrarathour at gmail.com Wed Jun 15 09:19:09 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Wed, 15 Jun 2022 14:49:09 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Shephard, Here is the o/p of the file: [root at undercloud ~]# sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json { "config_files": [ { "dest": "/", "merge": true, "preserve_properties": true, "source": "/var/lib/kolla/config_files/src/*" } ], "permissions": [ { "owner": "ironic:ironic", "path": "/var/log/ironic", "recurse": true }, { "owner": "ironic:ironic", "path": "/var/lib/ironic", "recurse": true } ] }[root at undercloud ~]# Thanks once agan. -Lokendra On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard wrote: > Looks like something wrong with the dnsmasq command the container is being > launched with. What command is it trying to run? > > sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta wrote: > >> Hi Brendan, >> >> Thanks for your response. >> >> Please find the log below. >> >> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >> >> dnsmasq: bad command line options: try --help >> dnsmasq: bad command line options: try --help >> dnsmasq: bad command line options: try --help >> dnsmasq: bad command line options: try --help >> dnsmasq: bad command line options: try --help >> dnsmasq: bad command line options: try --help >> >> [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe -a >> CONTAINER ID IMAGE >> COMMAND CREATED STATUS >> PORTS NAMES >> 02dacbc74cec >> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >> ironic_pxe_tftp >> 1f8ca39fba32 >> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >> kolla_start 3 hours ago Up 3 hours ago (healthy) >> ironic_pxe_http >> >> >> Regards >> >> Anirudh Gupta >> >> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard >> wrote: >> >>> Hey Anirudh, >>> >>> You would need to look at the logs for the ironic_pxe_tftp container to >>> see why it's failing. >>> >>> I assume the tftp container is not Up when you run this command? >>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps --filter >>> name=ironic_pxe -a >>> CONTAINER ID IMAGE >>> COMMAND CREATED STATUS >>> PORTS NAMES >>> 0170be36e291 >>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>> kolla_start 12 days ago Up 30 hours ago (healthy) >>> ironic_pxe_tftp >>> e507f722bdf0 >>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>> kolla_start 12 days ago Up 30 hours ago (healthy) >>> ironic_pxe_http >>> >>> Then check the logs to see what the error is: >>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>> ironic_pxe_tftp >>> >>> >>> >>> Brendan Shephard >>> >>> Software Engineer >>> >>> Red Hat APAC >>> >>> 193 N Quay >>> >>> Brisbane City QLD 4000 >>> @RedHat Red Hat >>> Red Hat >>> >>> >>> >>> >>> >>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>> wrote: >>> >>>> Hi Team, >>>> >>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing >>>> the below error: >>>> >>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>> TASK | Manage container systemd services and cleanup old systemd >>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>> FATAL | Manage container systemd services and cleanup old systemd >>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>> not started yet"} >>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>> TIMING | tripleo_container_manage : Manage container systemd >>>> >>>> Sample Undercloud.conf is as follows: >>>> >>>> [DEFAULT] >>>> clean_nodes = true >>>> cleanup = false >>>> container_cli = podman >>>> container_healthcheck_disabled = true >>>> container_images_file = /home/stack/containers-prepare-parameter.yaml >>>> deployment_user = stack >>>> enable_ironic = true >>>> enable_ironic_inspector = true >>>> enable_neutron = true >>>> enable_routed_networks = false >>>> generate_service_certificate = false >>>> ipv6_address_mode = dhcpv6-stateful >>>> ipxe_enabled = true >>>> local_interface = enp8s0 >>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>> subnets = ctlplane-subnet >>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>> undercloud_hostname = undercloud.com >>>> undercloud_ntp_servers = 30.30.30.3 >>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>> undercloud_timezone = UTC >>>> >>>> [ctlplane-subnet] >>>> cidr = aaaa:aaaa:aaaa::/64 >>>> dhcp_end = aaaa:aaaa:aaaa::f >>>> dhcp_start = aaaa:aaaa:aaaa::a >>>> gateway = aaaa:aaaa:aaaa::1 >>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>> >>>> Can someone please help in this regard. >>>> >>>> Anirudh Gupta >>>> >>>> -- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour -------------- next part -------------- An HTML attachment was scrubbed... URL: From bshephar at redhat.com Wed Jun 15 09:59:58 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Wed, 15 Jun 2022 19:59:58 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi, Looks like the command was in a different file in Wallaby, can you check: sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json That one should have the dnsmasq command it's trying to run. For example, here it is from my Wallaby environment: [stack at undercloud-0 ~]$ sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json | jq .command [ "/bin/bash", "-c", "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" ] Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour wrote: > Hi Shephard, > Here is the o/p of the file: > > [root at undercloud ~]# sudo cat > /var/lib/kolla/config_files/ironic_pxe_tftp.json > { > "config_files": [ > { > "dest": "/", > "merge": true, > "preserve_properties": true, > "source": "/var/lib/kolla/config_files/src/*" > } > ], > "permissions": [ > { > "owner": "ironic:ironic", > "path": "/var/log/ironic", > "recurse": true > }, > { > "owner": "ironic:ironic", > "path": "/var/lib/ironic", > "recurse": true > } > ] > }[root at undercloud ~]# > > > Thanks once agan. > > -Lokendra > > > On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard > wrote: > >> Looks like something wrong with the dnsmasq command the container is >> being launched with. What command is it trying to run? >> >> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >> wrote: >> >>> Hi Brendan, >>> >>> Thanks for your response. >>> >>> Please find the log below. >>> >>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>> >>> dnsmasq: bad command line options: try --help >>> dnsmasq: bad command line options: try --help >>> dnsmasq: bad command line options: try --help >>> dnsmasq: bad command line options: try --help >>> dnsmasq: bad command line options: try --help >>> dnsmasq: bad command line options: try --help >>> >>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe -a >>> CONTAINER ID IMAGE >>> COMMAND CREATED STATUS >>> PORTS NAMES >>> 02dacbc74cec >>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>> ironic_pxe_tftp >>> 1f8ca39fba32 >>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>> ironic_pxe_http >>> >>> >>> Regards >>> >>> Anirudh Gupta >>> >>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard >>> wrote: >>> >>>> Hey Anirudh, >>>> >>>> You would need to look at the logs for the ironic_pxe_tftp container >>>> to see why it's failing. >>>> >>>> I assume the tftp container is not Up when you run this command? >>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps --filter >>>> name=ironic_pxe -a >>>> CONTAINER ID IMAGE >>>> COMMAND CREATED STATUS >>>> PORTS NAMES >>>> 0170be36e291 >>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>> ironic_pxe_tftp >>>> e507f722bdf0 >>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>> ironic_pxe_http >>>> >>>> Then check the logs to see what the error is: >>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>> ironic_pxe_tftp >>>> >>>> >>>> >>>> Brendan Shephard >>>> >>>> Software Engineer >>>> >>>> Red Hat APAC >>>> >>>> 193 N Quay >>>> >>>> Brisbane City QLD 4000 >>>> @RedHat Red Hat >>>> Red Hat >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>> wrote: >>>> >>>>> Hi Team, >>>>> >>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing >>>>> the below error: >>>>> >>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>> TASK | Manage container systemd services and cleanup old systemd >>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>> FATAL | Manage container systemd services and cleanup old systemd >>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>> not started yet"} >>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>> TIMING | tripleo_container_manage : Manage container systemd >>>>> >>>>> Sample Undercloud.conf is as follows: >>>>> >>>>> [DEFAULT] >>>>> clean_nodes = true >>>>> cleanup = false >>>>> container_cli = podman >>>>> container_healthcheck_disabled = true >>>>> container_images_file = /home/stack/containers-prepare-parameter.yaml >>>>> deployment_user = stack >>>>> enable_ironic = true >>>>> enable_ironic_inspector = true >>>>> enable_neutron = true >>>>> enable_routed_networks = false >>>>> generate_service_certificate = false >>>>> ipv6_address_mode = dhcpv6-stateful >>>>> ipxe_enabled = true >>>>> local_interface = enp8s0 >>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>> subnets = ctlplane-subnet >>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>> undercloud_hostname = undercloud.com >>>>> undercloud_ntp_servers = 30.30.30.3 >>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>> undercloud_timezone = UTC >>>>> >>>>> [ctlplane-subnet] >>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>> gateway = aaaa:aaaa:aaaa::1 >>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>> >>>>> Can someone please help in this regard. >>>>> >>>>> Anirudh Gupta >>>>> >>>>> > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.acosta at luizalabs.com Wed Jun 15 10:57:23 2022 From: roberto.acosta at luizalabs.com (ROBERTO BARTZEN ACOSTA) Date: Wed, 15 Jun 2022 07:57:23 -0300 Subject: [Neutron] What is the best tool for automated data plane testing with openstack? Message-ID: Hey there, I'm using OpenStack(old version) with neutron OVN CMS-plugin and I need to migrate to the Yoga release (Ubuntu ovn v22.03.0 and ovs v2.17.0). I'm interested in validating a series of fixed issues on openvswitch and ovn in the releases related to the OpenStack Yoga version (e.g. https://github.com/ovn-org/ovn/commit/ccbbbd0584e52c2aa88436a431fe7a6deffd2221 ). What OpenStack automated testing tool do you use or recommend to validate the data plane? Best regards Roberto -- _?Esta mensagem ? direcionada apenas para os endere?os constantes no cabe?alho inicial. Se voc? n?o est? listado nos endere?os constantes no cabe?alho, pedimos-lhe que desconsidere completamente o conte?do dessa mensagem e cuja c?pia, encaminhamento e/ou execu??o das a??es citadas est?o imediatamente anuladas e proibidas?._ *?**?Apesar do Magazine Luiza tomar todas as precau??es razo?veis para assegurar que nenhum v?rus esteja presente nesse e-mail, a empresa n?o poder? aceitar a responsabilidade por quaisquer perdas ou danos causados por esse e-mail ou por seus anexos?.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From senrique at redhat.com Wed Jun 15 11:00:00 2022 From: senrique at redhat.com (Sofia Enriquez) Date: Wed, 15 Jun 2022 08:00:00 -0300 Subject: [cinder] Bug deputy report for week of 06-15-2022 Message-ID: This is a bug report from 06-08-2022 to 06-15-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium: - https://bugs.launchpad.net/cinder/+bug/1978294 "[NFS/NetApp] No feedback from Nova when extending attached Volumes." Assigned to Konrad Gub. - https://bugs.launchpad.net/os-brick/+bug/1967157 "Fails to extend in-use (non LUKS v1) encrypted volumes." Fix proposed to master. Low - https://bugs.launchpad.net/cinder/+bug/1978020 "Old-style Glance location URI sent when uploading a volume to an image." Fix proposed to master. - https://bugs.launchpad.net/cinder/+bug/1978290 "[IBM Storwize] Optimize SSH calls in create replicated volume." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1978729 "cinder-backup context.message_action is None on errors." Unassigned. Cheers, Sofia -- Sof?a Enriquez she/her Software Engineer Red Hat PnT IRC: @enriquetaso @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnasiadka at gmail.com Wed Jun 15 11:26:10 2022 From: mnasiadka at gmail.com (=?utf-8?Q?Micha=C5=82_Nasiadka?=) Date: Wed, 15 Jun 2022 13:26:10 +0200 Subject: [kolla] Weekly meeting 15th June 2022 cancelled Message-ID: <272663AF-AB88-4436-8126-F54EBF187E4C@gmail.com> Hello Koalas, Sorry for late notice - but the meeting today is cancelled - I can?t join (and as well most of the key members can?t). If you have any important questions - please ask them on #openstack-kolla. See you next week, Michal Nasiadka From jains8550 at gmail.com Wed Jun 15 11:35:12 2022 From: jains8550 at gmail.com (AJ_ sunny) Date: Wed, 15 Jun 2022 17:05:12 +0530 Subject: Poor upload speed of ubuntu 18.04 vm Message-ID: Hi team, I am getting very less upload speed for specific Ubuntu 18.04 vm in Openstack environment deployed through kolla ansible method Any idea why i am getting this issue for ubuntu 18.04? Thanks Arihant Jain -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Wed Jun 15 11:57:09 2022 From: eblock at nde.ag (Eugen Block) Date: Wed, 15 Jun 2022 11:57:09 +0000 Subject: 504 Gateway time out error while loading OpenStack Projects In-Reply-To: Message-ID: <20220615115709.Horde.b9lsBD6CdiKLFNBDa3Jy8AD@webmail.nde.ag> Hi, I remember having a similar issue a few years back in an older openstack release, probably Ocata. We had difficulties loading *all* instances in the admin tab in Horizon dashboard. It turned out to be a memcached misconfiguration. I don't recall the details, unfortunately. But I'd start with memcached. Zitat von Adivya Singh : > hi Team, > > We have Xena release installed on Ubuntu Openstack, Lately we are getting > "504 Gateway Timeout error ", As the we moved from 1 node to 3 node Open > Stack > i do not see any error in haproxy , neither there is a time out when i ping > external-VIP-IP configured. > > Apache 2 are also loaded, tried to delete the Horizon Container and created > it again, but no errors, Still the same errors. > > Any guesses on that > > Regards > Adivya Singh From Moiz.Mohammed at charter.com Wed Jun 15 13:35:47 2022 From: Moiz.Mohammed at charter.com (Mohammed, Moiz) Date: Wed, 15 Jun 2022 13:35:47 +0000 Subject: Migrate vmware to Openstack Message-ID: Hello, Is there any tool or are aware of to convert from Vmware to Openstack qcow2? Moiz Mohammed | Systems Engineer - Compute Infrastructure (704)-378-2934 O | (415) 866-3183 M | Oncall Phone: 1-866-577-0007 Ext 802 7815 Cresent Executive Dr, Floor#2 | Charlotte, NC 28217 [cid:image002.png at 01D3AA4F.EFCE7660] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 8747 bytes Desc: image001.png URL: From kkchn.in at gmail.com Wed Jun 15 07:01:00 2022 From: kkchn.in at gmail.com (KK CHN) Date: Wed, 15 Jun 2022 12:31:00 +0530 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: This is my latest devstack installation error.. No clues why it fails repeatedly; earlier it was during the uploading stage of the cirros image, internal server error ( HTTP 500 ) occurred. This was repeated multiple times. So I posted for the first time in this thread. In this iteration for starting afresh, I have removed the stack user and /opt/stack directory manually, and freshly created 'stack' user with sudo permissions and /opt/stack directory with stack:stack ownership, and fresh git cloned(git clone https://opendev.org/openstack/devstack ) to /opt/DEVSTACK_stack folder and created local.conf ( attached here) . Then executed ./stack.sh . Got the following error this time ... Any clues why it fails here ? Platform (on a VM with Debian uname output) : Linux XenaCtrl1 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux [image: Trove_installationError.png] On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney wrote: > On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: > > > i hit the same issue the other day if you recall me asking about it. > > > didnt check my devstack logs to see if it installed or not but i can > > > see if i can repoduce it tomorrow. > > > > I checked with the OP and they have it installed properly, > > system-wide. I was assuming your issue might have been venv (or > > ordering) related, but theirs was not. There are some other weird things > > going on with their setup, so I'm not so sure. > nope, was not useing a vnev or anything special, it just devstack > installed normaly more or less > but driven by the ansible roles. the run devstack rull litrally just opens > a shell and invokes > stack.sh so i dont think there was anything specific ot how i was invoking > it. > > > > > --Dan > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Trove_installationError.png Type: image/png Size: 88093 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: local.conf Type: application/octet-stream Size: 1309 bytes Desc: not available URL: From anyrude10 at gmail.com Wed Jun 15 08:22:38 2022 From: anyrude10 at gmail.com (Anirudh Gupta) Date: Wed, 15 Jun 2022 13:52:38 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Brendan, Thanks for your response. Please find the log below. [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp dnsmasq: bad command line options: try --help dnsmasq: bad command line options: try --help dnsmasq: bad command line options: try --help dnsmasq: bad command line options: try --help dnsmasq: bad command line options: try --help dnsmasq: bad command line options: try --help [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 02dacbc74cec undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) ironic_pxe_tftp 1f8ca39fba32 undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo kolla_start 3 hours ago Up 3 hours ago (healthy) ironic_pxe_http Regards Anirudh Gupta On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard wrote: > Hey Anirudh, > > You would need to look at the logs for the ironic_pxe_tftp container to > see why it's failing. > > I assume the tftp container is not Up when you run this command? > [stack at tripleo-director overcloud_playbooks]$ sudo podman ps --filter > name=ironic_pxe -a > CONTAINER ID IMAGE > COMMAND CREATED STATUS > PORTS NAMES > 0170be36e291 > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > kolla_start 12 days ago Up 30 hours ago (healthy) > ironic_pxe_tftp > e507f722bdf0 > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > kolla_start 12 days ago Up 30 hours ago (healthy) > ironic_pxe_http > > Then check the logs to see what the error is: > [stack at tripleo-director overcloud_playbooks]$ sudo podman logs > ironic_pxe_tftp > > > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta wrote: > >> Hi Team, >> >> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing >> the below error: >> >> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | TASK >> | Manage container systemd services and cleanup old systemd healthchecks >> for /var/lib/tripleo-config/container-startup-config/step_4 >> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | FATAL >> | Manage container systemd services and cleanup old systemd healthchecks >> for /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | >> error={"changed": false, "msg": "Service ironic_pxe_tftp has not started >> yet"} >> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | >> TIMING | tripleo_container_manage : Manage container systemd >> >> Sample Undercloud.conf is as follows: >> >> [DEFAULT] >> clean_nodes = true >> cleanup = false >> container_cli = podman >> container_healthcheck_disabled = true >> container_images_file = /home/stack/containers-prepare-parameter.yaml >> deployment_user = stack >> enable_ironic = true >> enable_ironic_inspector = true >> enable_neutron = true >> enable_routed_networks = false >> generate_service_certificate = false >> ipv6_address_mode = dhcpv6-stateful >> ipxe_enabled = true >> local_interface = enp8s0 >> local_ip = aaaa:aaaa:aaaa::1/64 >> subnets = ctlplane-subnet >> undercloud_admin_host = aaaa:aaaa:aaaa::1 >> undercloud_hostname = undercloud.com >> undercloud_ntp_servers = 30.30.30.3 >> undercloud_public_host = aaaa:aaaa:aaaa::1 >> undercloud_timezone = UTC >> >> [ctlplane-subnet] >> cidr = aaaa:aaaa:aaaa::/64 >> dhcp_end = aaaa:aaaa:aaaa::f >> dhcp_start = aaaa:aaaa:aaaa::a >> gateway = aaaa:aaaa:aaaa::1 >> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >> >> Can someone please help in this regard. >> >> Anirudh Gupta >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From lokendrarathour at gmail.com Wed Jun 15 10:20:07 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Wed, 15 Jun 2022 15:50:07 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Shephard, this is the command from my wallaby: [root at undercloud ~]# sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json { "cap_add": [ "NET_ADMIN", "NET_RAW", "SETUID" ], "command": [ "/bin/bash", "-c", "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" ], "environment": { "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" }, "healthcheck": { "test": "/openstack/healthcheck" }, "image": "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", "net": "host", "privileged": false, "restart": "always", "start_order": 90, "volumes": [ "/etc/hosts:/etc/hosts:ro", "/etc/localtime:/etc/localtime:ro", "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", "/dev/log:/dev/log", "/etc/puppet:/etc/puppet:ro", "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", "/var/lib/ironic:/var/lib/ironic:shared,z", "/var/log/containers/ironic:/var/log/ironic:z", "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" ] }[root at undercloud ~]# Comparing both, they look alike. please check once. On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard wrote: > Hi, > > Looks like the command was in a different file in Wallaby, can you check: > sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > > That one should have the dnsmasq command it's trying to run. For example, > here it is from my Wallaby environment: > [stack at undercloud-0 ~]$ sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > | jq .command > [ > "/bin/bash", > "-c", > "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp > --tftp-root=/var/lib/ironic/tftpboot" > ] > > > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < > lokendrarathour at gmail.com> wrote: > >> Hi Shephard, >> Here is the o/p of the file: >> >> [root at undercloud ~]# sudo cat >> /var/lib/kolla/config_files/ironic_pxe_tftp.json >> { >> "config_files": [ >> { >> "dest": "/", >> "merge": true, >> "preserve_properties": true, >> "source": "/var/lib/kolla/config_files/src/*" >> } >> ], >> "permissions": [ >> { >> "owner": "ironic:ironic", >> "path": "/var/log/ironic", >> "recurse": true >> }, >> { >> "owner": "ironic:ironic", >> "path": "/var/lib/ironic", >> "recurse": true >> } >> ] >> }[root at undercloud ~]# >> >> >> Thanks once agan. >> >> -Lokendra >> >> >> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard >> wrote: >> >>> Looks like something wrong with the dnsmasq command the container is >>> being launched with. What command is it trying to run? >>> >>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>> >>> Brendan Shephard >>> >>> Software Engineer >>> >>> Red Hat APAC >>> >>> 193 N Quay >>> >>> Brisbane City QLD 4000 >>> @RedHat Red Hat >>> Red Hat >>> >>> >>> >>> >>> >>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>> wrote: >>> >>>> Hi Brendan, >>>> >>>> Thanks for your response. >>>> >>>> Please find the log below. >>>> >>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>> >>>> dnsmasq: bad command line options: try --help >>>> dnsmasq: bad command line options: try --help >>>> dnsmasq: bad command line options: try --help >>>> dnsmasq: bad command line options: try --help >>>> dnsmasq: bad command line options: try --help >>>> dnsmasq: bad command line options: try --help >>>> >>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe -a >>>> CONTAINER ID IMAGE >>>> COMMAND CREATED STATUS >>>> PORTS NAMES >>>> 02dacbc74cec >>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>> ironic_pxe_tftp >>>> 1f8ca39fba32 >>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>> ironic_pxe_http >>>> >>>> >>>> Regards >>>> >>>> Anirudh Gupta >>>> >>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard >>>> wrote: >>>> >>>>> Hey Anirudh, >>>>> >>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>> to see why it's failing. >>>>> >>>>> I assume the tftp container is not Up when you run this command? >>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps --filter >>>>> name=ironic_pxe -a >>>>> CONTAINER ID IMAGE >>>>> COMMAND CREATED STATUS >>>>> PORTS NAMES >>>>> 0170be36e291 >>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>> ironic_pxe_tftp >>>>> e507f722bdf0 >>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>> ironic_pxe_http >>>>> >>>>> Then check the logs to see what the error is: >>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>> ironic_pxe_tftp >>>>> >>>>> >>>>> >>>>> Brendan Shephard >>>>> >>>>> Software Engineer >>>>> >>>>> Red Hat APAC >>>>> >>>>> 193 N Quay >>>>> >>>>> Brisbane City QLD 4000 >>>>> @RedHat Red Hat >>>>> Red Hat >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>>> wrote: >>>>> >>>>>> Hi Team, >>>>>> >>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>> facing the below error: >>>>>> >>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>> TASK | Manage container systemd services and cleanup old systemd >>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>> FATAL | Manage container systemd services and cleanup old systemd >>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>>> not started yet"} >>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>> TIMING | tripleo_container_manage : Manage container systemd >>>>>> >>>>>> Sample Undercloud.conf is as follows: >>>>>> >>>>>> [DEFAULT] >>>>>> clean_nodes = true >>>>>> cleanup = false >>>>>> container_cli = podman >>>>>> container_healthcheck_disabled = true >>>>>> container_images_file = /home/stack/containers-prepare-parameter.yaml >>>>>> deployment_user = stack >>>>>> enable_ironic = true >>>>>> enable_ironic_inspector = true >>>>>> enable_neutron = true >>>>>> enable_routed_networks = false >>>>>> generate_service_certificate = false >>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>> ipxe_enabled = true >>>>>> local_interface = enp8s0 >>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>> subnets = ctlplane-subnet >>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>> undercloud_hostname = undercloud.com >>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>> undercloud_timezone = UTC >>>>>> >>>>>> [ctlplane-subnet] >>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>> >>>>>> Can someone please help in this regard. >>>>>> >>>>>> Anirudh Gupta >>>>>> >>>>>> >> >> -- >> ~ Lokendra >> www.inertiaspeaks.com >> www.inertiagroups.com >> skype: lokendrarathour >> >> >> -- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour -------------- next part -------------- An HTML attachment was scrubbed... URL: From bshephar at redhat.com Wed Jun 15 12:38:01 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Wed, 15 Jun 2022 22:38:01 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hey, Ok, that command looks fine. What about that variable there? Do you get anything back when you run: sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml Mine returns: sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml 192.168.24.115 Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour wrote: > Hi Shephard, > > this is the command from my wallaby: > [root at undercloud ~]# sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > { > "cap_add": [ > "NET_ADMIN", > "NET_RAW", > "SETUID" > ], > "command": [ > "/bin/bash", > "-c", > "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp > --tftp-root=/var/lib/ironic/tftpboot" > ], > "environment": { > "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", > "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" > }, > "healthcheck": { > "test": "/openstack/healthcheck" > }, > "image": > "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", > "net": "host", > "privileged": false, > "restart": "always", > "start_order": 90, > "volumes": [ > "/etc/hosts:/etc/hosts:ro", > "/etc/localtime:/etc/localtime:ro", > "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", > "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", > "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", > > "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", > "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", > "/dev/log:/dev/log", > "/etc/puppet:/etc/puppet:ro", > > "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", > > "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", > "/var/lib/ironic:/var/lib/ironic:shared,z", > "/var/log/containers/ironic:/var/log/ironic:z", > "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" > ] > }[root at undercloud ~]# > > Comparing both, they look alike. > please check once. > > On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard > wrote: > >> Hi, >> >> Looks like the command was in a different file in Wallaby, can you check: >> sudo cat >> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >> >> That one should have the dnsmasq command it's trying to run. For example, >> here it is from my Wallaby environment: >> [stack at undercloud-0 ~]$ sudo cat >> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >> | jq .command >> [ >> "/bin/bash", >> "-c", >> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >> --log-facility=/var/log/ironic/dnsmasq.log --user=root >> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >> --tftp-root=/var/lib/ironic/tftpboot" >> ] >> >> >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Hi Shephard, >>> Here is the o/p of the file: >>> >>> [root at undercloud ~]# sudo cat >>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>> { >>> "config_files": [ >>> { >>> "dest": "/", >>> "merge": true, >>> "preserve_properties": true, >>> "source": "/var/lib/kolla/config_files/src/*" >>> } >>> ], >>> "permissions": [ >>> { >>> "owner": "ironic:ironic", >>> "path": "/var/log/ironic", >>> "recurse": true >>> }, >>> { >>> "owner": "ironic:ironic", >>> "path": "/var/lib/ironic", >>> "recurse": true >>> } >>> ] >>> }[root at undercloud ~]# >>> >>> >>> Thanks once agan. >>> >>> -Lokendra >>> >>> >>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard >>> wrote: >>> >>>> Looks like something wrong with the dnsmasq command the container is >>>> being launched with. What command is it trying to run? >>>> >>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>> >>>> Brendan Shephard >>>> >>>> Software Engineer >>>> >>>> Red Hat APAC >>>> >>>> 193 N Quay >>>> >>>> Brisbane City QLD 4000 >>>> @RedHat Red Hat >>>> Red Hat >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>> wrote: >>>> >>>>> Hi Brendan, >>>>> >>>>> Thanks for your response. >>>>> >>>>> Please find the log below. >>>>> >>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>> >>>>> dnsmasq: bad command line options: try --help >>>>> dnsmasq: bad command line options: try --help >>>>> dnsmasq: bad command line options: try --help >>>>> dnsmasq: bad command line options: try --help >>>>> dnsmasq: bad command line options: try --help >>>>> dnsmasq: bad command line options: try --help >>>>> >>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe >>>>> -a >>>>> CONTAINER ID IMAGE >>>>> COMMAND CREATED STATUS >>>>> PORTS NAMES >>>>> 02dacbc74cec >>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>> ironic_pxe_tftp >>>>> 1f8ca39fba32 >>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>> ironic_pxe_http >>>>> >>>>> >>>>> Regards >>>>> >>>>> Anirudh Gupta >>>>> >>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard >>>>> wrote: >>>>> >>>>>> Hey Anirudh, >>>>>> >>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>> to see why it's failing. >>>>>> >>>>>> I assume the tftp container is not Up when you run this command? >>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>> --filter name=ironic_pxe -a >>>>>> CONTAINER ID IMAGE >>>>>> COMMAND CREATED STATUS >>>>>> PORTS NAMES >>>>>> 0170be36e291 >>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>> ironic_pxe_tftp >>>>>> e507f722bdf0 >>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>> ironic_pxe_http >>>>>> >>>>>> Then check the logs to see what the error is: >>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>> ironic_pxe_tftp >>>>>> >>>>>> >>>>>> >>>>>> Brendan Shephard >>>>>> >>>>>> Software Engineer >>>>>> >>>>>> Red Hat APAC >>>>>> >>>>>> 193 N Quay >>>>>> >>>>>> Brisbane City QLD 4000 >>>>>> @RedHat Red Hat >>>>>> Red Hat >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>>>> wrote: >>>>>> >>>>>>> Hi Team, >>>>>>> >>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>> facing the below error: >>>>>>> >>>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>>> TASK | Manage container systemd services and cleanup old systemd >>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>>> FATAL | Manage container systemd services and cleanup old systemd >>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>>>> not started yet"} >>>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>>> TIMING | tripleo_container_manage : Manage container systemd >>>>>>> >>>>>>> Sample Undercloud.conf is as follows: >>>>>>> >>>>>>> [DEFAULT] >>>>>>> clean_nodes = true >>>>>>> cleanup = false >>>>>>> container_cli = podman >>>>>>> container_healthcheck_disabled = true >>>>>>> container_images_file = /home/stack/containers-prepare-parameter.yaml >>>>>>> deployment_user = stack >>>>>>> enable_ironic = true >>>>>>> enable_ironic_inspector = true >>>>>>> enable_neutron = true >>>>>>> enable_routed_networks = false >>>>>>> generate_service_certificate = false >>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>> ipxe_enabled = true >>>>>>> local_interface = enp8s0 >>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>> subnets = ctlplane-subnet >>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>> undercloud_hostname = undercloud.com >>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>> undercloud_timezone = UTC >>>>>>> >>>>>>> [ctlplane-subnet] >>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>> >>>>>>> Can someone please help in this regard. >>>>>>> >>>>>>> Anirudh Gupta >>>>>>> >>>>>>> >>> >>> -- >>> ~ Lokendra >>> www.inertiaspeaks.com >>> www.inertiagroups.com >>> skype: lokendrarathour >>> >>> >>> > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Danny.Webb at thehutgroup.com Wed Jun 15 14:00:11 2022 From: Danny.Webb at thehutgroup.com (Danny Webb) Date: Wed, 15 Jun 2022 14:00:11 +0000 Subject: Migrate vmware to Openstack In-Reply-To: References: Message-ID: Hi Moiz, you can use qemu-img to convert between formats. https://docs.openstack.org/image-guide/convert-images.html Cheers, Danny ________________________________ From: Mohammed, Moiz Sent: 15 June 2022 14:35 To: openstack-discuss at lists.openstack.org Subject: Migrate vmware to Openstack CAUTION: This email originates from outside THG ________________________________ Hello, Is there any tool or are aware of to convert from Vmware to Openstack qcow2? Moiz Mohammed | Systems Engineer ? Compute Infrastructure (704)-378-2934 O | (415) 866-3183 M | Oncall Phone: 1-866-577-0007 Ext 802 7815 Cresent Executive Dr, Floor#2 | Charlotte, NC 28217 [cid:image002.png at 01D3AA4F.EFCE7660] Danny Webb Principal OpenStack Engineer The Hut Group Tel: Email: Danny.Webb at thehutgroup.com For the purposes of this email, the "company" means The Hut Group Limited, a company registered in England and Wales (company number 6539496) whose registered office is at Fifth Floor, Voyager House, Chicago Avenue, Manchester Airport, M90 3DQ and/or any of its respective subsidiaries. Confidentiality Notice This e-mail is confidential and intended for the use of the named recipient only. If you are not the intended recipient please notify us by telephone immediately on +44(0)1606 811888 or return it to us by e-mail. Please then delete it from your system and note that any use, dissemination, forwarding, printing or copying is strictly prohibited. Any views or opinions are solely those of the author and do not necessarily represent those of the company. Encryptions and Viruses Please note that this e-mail and any attachments have not been encrypted. They may therefore be liable to be compromised. Please also note that it is your responsibility to scan this e-mail and any attachments for viruses. We do not, to the extent permitted by law, accept any liability (whether in contract, negligence or otherwise) for any virus infection and/or external compromise of security and/or confidentiality in relation to transmissions sent by e-mail. Monitoring Activity and use of the company's systems is monitored to secure its effective use and operation and for other lawful business purposes. Communications using these systems will also be monitored and may be recorded to secure effective use and operation and for other lawful business purposes. hgvyjuv -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 8747 bytes Desc: image001.png URL: From tim+openstack.org at coote.org Wed Jun 15 14:04:55 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Wed, 15 Jun 2022 15:04:55 +0100 Subject: split brains Message-ID: <240B1D88-09DE-498B-8203-52B69E8E5B31@coote.org> Hullo I?ve encountered a couple of firms that have run into challenges with Open Stack environments becoming ?split brain?, e.g. https://www.youtube.com/watch?v=JZwYZqx0RMk The narrative is that there?s an issue with the RabbitMQ messaging that is causing this. I?ve used RabbitMQ since it was being developed by various banks and I?ve successfully deployed it to control large scale IoT systems. I?ve always found it reliable as a MoM approach, although I?ve never tried the transactional message type that comes with AMQP 1.0 (I think that?s the version). Can anyone cast any light on what this type of scenario is referring to? tia Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim+openstack.org at coote.org Wed Jun 15 14:33:47 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Wed, 15 Jun 2022 15:33:47 +0100 Subject: Migrate vmware to Openstack In-Reply-To: <7D91B877-4E7E-492C-9AFB-1CFBDC3B3A0B@coote.org> References: <7D91B877-4E7E-492C-9AFB-1CFBDC3B3A0B@coote.org> Message-ID: Isn?t the bigger issue migrating the code that deploys the vms and other resources? Is there a common Terraform model? > On 15 Jun 2022, at 15:00, Danny Webb > wrote: > > Hi Moiz, > > you can use qemu-img to convert between formats. > > https://docs.openstack.org/image-guide/convert-images.html > > Cheers, > > Danny > > From: Mohammed, Moiz > > Sent: 15 June 2022 14:35 > To: openstack-discuss at lists.openstack.org > > Subject: Migrate vmware to Openstack > > > CAUTION: This email originates from outside THG > > > > Hello, > > Is there any tool or are aware of to convert from Vmware to Openstack qcow2? > > > Moiz Mohammed | Systems Engineer ? Compute Infrastructure > (704)-378-2934 O | (415) 866-3183 M | Oncall Phone: 1-866-577-0007 Ext 802 > 7815 Cresent Executive Dr, Floor#2 | Charlotte, NC 28217 > > > Danny Webb > Principal OpenStack Engineer > The Hut Group > > Tel: > Email: Danny.Webb at thehutgroup.com > > For the purposes of this email, the "company" means The Hut Group Limited, a company registered in England and Wales (company number 6539496) whose registered office is at Fifth Floor, Voyager House, Chicago Avenue, Manchester Airport, M90 3DQ and/or any of its respective subsidiaries. > > Confidentiality Notice > This e-mail is confidential and intended for the use of the named recipient only. If you are not the intended recipient please notify us by telephone immediately on +44(0)1606 811888 or return it to us by e-mail. Please then delete it from your system and note that any use, dissemination, forwarding, printing or copying is strictly prohibited. Any views or opinions are solely those of the author and do not necessarily represent those of the company. > > Encryptions and Viruses > Please note that this e-mail and any attachments have not been encrypted. They may therefore be liable to be compromised. Please also note that it is your responsibility to scan this e-mail and any attachments for viruses. We do not, to the extent permitted by law, accept any liability (whether in contract, negligence or otherwise) for any virus infection and/or external compromise of security and/or confidentiality in relation to transmissions sent by e-mail. > > Monitoring > Activity and use of the company's systems is monitored to secure its effective use and operation and for other lawful business purposes. Communications using these systems will also be monitored and may be recorded to secure effective use and operation and for other lawful business purposes. > hgvyjuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdemaced at redhat.com Wed Jun 15 15:18:07 2022 From: mdemaced at redhat.com (Maysa De Macedo Souza) Date: Wed, 15 Jun 2022 17:18:07 +0200 Subject: 504 Gateway time out error while loading OpenStack Projects In-Reply-To: <20220615115709.Horde.b9lsBD6CdiKLFNBDa3Jy8AD@webmail.nde.ag> References: <20220615115709.Horde.b9lsBD6CdiKLFNBDa3Jy8AD@webmail.nde.ag> Message-ID: Hello, Maybe increasing the value of a setting like memcache_pool_conn_get_timeout can help? Cheers, Maysa. On Wed, Jun 15, 2022 at 2:05 PM Eugen Block wrote: > Hi, > > I remember having a similar issue a few years back in an older > openstack release, probably Ocata. We had difficulties loading *all* > instances in the admin tab in Horizon dashboard. It turned out to be a > memcached misconfiguration. I don't recall the details, unfortunately. > But I'd start with memcached. > > > Zitat von Adivya Singh : > > > hi Team, > > > > We have Xena release installed on Ubuntu Openstack, Lately we are getting > > "504 Gateway Timeout error ", As the we moved from 1 node to 3 node Open > > Stack > > i do not see any error in haproxy , neither there is a time out when i > ping > > external-VIP-IP configured. > > > > Apache 2 are also loaded, tried to delete the Horizon Container and > created > > it again, but no errors, Still the same errors. > > > > Any guesses on that > > > > Regards > > Adivya Singh > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.vanommen at gmail.com Wed Jun 15 22:01:12 2022 From: john.vanommen at gmail.com (John van Ommen) Date: Wed, 15 Jun 2022 15:01:12 -0700 Subject: Migrate vmware to Openstack In-Reply-To: References: Message-ID: IMHO, VMWare Converter is great for this. On Wed, Jun 15, 2022 at 6:37 AM Mohammed, Moiz wrote: > > > Hello, > > Is there any tool or are aware of to convert from Vmware to Openstack > qcow2? > > > > > > *Moiz Mohammed* | * Systems Engineer ? Compute Infrastructure* > > (704)-378-2934 O | (415) 866-3183 M | Oncall Phone: 1-866-577-0007 Ext 802 > > 7815 Cresent Executive Dr, Floor#2 | Charlotte, NC 28217 > > [image: cid:image002.png at 01D3AA4F.EFCE7660] > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 8747 bytes Desc: not available URL: From gmann at ghanshyammann.com Wed Jun 15 23:12:07 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 15 Jun 2022 18:12:07 -0500 Subject: [all][tc] Technical Committee next weekly meeting on 16 June 2022 at 1500 UTC In-Reply-To: <1815df38dec.b4d59475189195.5924549091605681735@ghanshyammann.com> References: <1815df38dec.b4d59475189195.5924549091605681735@ghanshyammann.com> Message-ID: <18169a36dd4.121fc977825901.7956206769938276866@ghanshyammann.com> Hello Everyone, Below is the agenda for Today's TC IRC meeting schedule at 1500 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting == Agenda for tomorrow's TC meeting == * Roll call * Follow up on past action items * Gate health check ** Fixing Zuul config error in OpenStack *** https://etherpad.opendev.org/p/zuul-config-error-openstack ** (rosmaita) update on cinder-backup memory bloat issue * New ELK service dashboard: e-r service ** https://opensearch.logs.openstack.org/_dashboards/app/discover?security_tenant=global ** https://review.opendev.org/c/openstack/governance-sigs/+/835838 * RBAC feedback in ops meetup ** https://etherpad.opendev.org/p/ops-meetup-berlin-2022-planning#L53 ** https://etherpad.opendev.org/p/rbac-operator-feedback * TC + Board meeting ** https://docs.google.com/presentation/d/1yCiZy_9A6hURXD0BRdr33OlK7Ugch5EgSvFzL-jvDIg/edit#slide=id.g129c4257ac3_0_1456 * Open Reviews ** https://review.opendev.org/q/projects:openstack/governance+is:open ** (rosmaita) quick update on SLURP Release Notes -gmann ---- On Mon, 13 Jun 2022 11:44:11 -0500 Ghanshyam Mann wrote ---- > Hello Everyone, > > The technical Committee's next weekly meeting is scheduled for 16 June 2022, at 1500 UTC. > > If you would like to add topics for discussion, please add them to the below wiki page by > Wednesday, 15 June at 2100 UTC. > > https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting > > -gmann > > From manchandavishal143 at gmail.com Thu Jun 16 03:50:27 2022 From: manchandavishal143 at gmail.com (vishal manchanda) Date: Thu, 16 Jun 2022 09:20:27 +0530 Subject: [horizon][plugins] Horizon angular based plugins are broken after migrating to angularjs 1.8.2.2 Message-ID: Hello everyone, Horizon team recently migrated XStatic-Angular 1.5.8.0->1.8.2.2 [1] and after that many horizon angular based plugins start failing [2]. We already fixed these issues in horizon [3]. So I guess we need to fix the similar thing in horizon plugins. Why did we migrated Angular 1.5.8.0->1.8.2.2? The angularjs version is updated 1.5.8.0->1.8.2.2 to include the CVE fixed in the latest version. Also, there is a security bug reported for the same [4]. I have also created an etherpad to track the progress for failed horizon plugins [5]. Action items for the horizon plugins team either fixed their plugins or review/merge the patch pushed by horizon team members on priority basis to fix the gate. The npm jobs are failing in below horizon plugins : ? ironic-ui ? octavia-dashboard ? senlin-dashboard ? designate-dashboard ? vitrage-dashboard ? murano-dashboard ? zaqar-ui ? zun-ui ? magnum-ui In case of any queries, please feel free to reach out to Horizon channel #openstack-horizon. Thanks & Regards, Vishal Manchanda(irc: vishalmanchanda) [1] https://review.opendev.org/c/openstack/requirements/+/844099 [2] https://review.opendev.org/c/openstack/horizon/+/845733 [3] https://review.opendev.org/c/openstack/horizon/+/843346 [4] https://bugs.launchpad.net/horizon/+bug/1955556 [5] https://etherpad.opendev.org/p/Fix_Horizon_Plugins_With_Angularjs_v1.8.2.2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From fereshtehloghmani at gmail.com Thu Jun 16 06:38:43 2022 From: fereshtehloghmani at gmail.com (fereshteh loghmani) Date: Thu, 16 Jun 2022 11:08:43 +0430 Subject: raw and qcow2 format Message-ID: hello I have a question about disk format in OpenStack. I created a new server and shut off that server and with the command qemu-img info disk I found my "disk format" in qcow2 and it shows the "backing file format: raw". could you tell me why these 2 are different? (I use OpenStack without ceph and without volumes.) thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From swogatpradhan22 at gmail.com Thu Jun 16 07:01:33 2022 From: swogatpradhan22 at gmail.com (Swogat Pradhan) Date: Thu, 16 Jun 2022 12:31:33 +0530 Subject: How to setup murano and trove in a existing tripleo setup Message-ID: Hi, I have a running Openstack cloud based on Wallaby which was deployed in tripleo. I have a requirement to set up additional services like REAR, MURANO and TROVE. But murano and trove are not available in tripleo. How can I set it up? Should i manually install murano and trove and integrate it in the existing openstack. Will it in any way affect the setup? With Regards Swogat pradhan -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Thu Jun 16 07:07:41 2022 From: katonalala at gmail.com (Lajos Katona) Date: Thu, 16 Jun 2022 09:07:41 +0200 Subject: [Neutron] What is the best tool for automated data plane testing with openstack? In-Reply-To: References: Message-ID: Hi, In Openstack itself there is no such tool. There are several test tools which we use to test Openstack and Openstack components like Tempest (see [1]) and the plugin for it are full with networking related tests neutron-tempest-plugin (see [2]). The scenario tests in neutron-tempest-plugin tests some basic dataplane scenarios, but only in a functional manner, not with traffic load or such, as we used these tests in our CI, so we need quick feedback. To have some stress test you can check Rally (see [3]) but that is again not for dataplane testing, but to load the API of the components. You can find some Neutron specific rally scenarios in Neutron repo (see [4]). [1]: https://opendev.org/openstack/tempest [2]: https://opendev.org/openstack/neutron-tempest-plugin [3]: https://opendev.org/openstack/rally [4]: https://opendev.org/openstack/neutron/src/branch/master/rally-jobs Lajos Katona (lajoskatona) ROBERTO BARTZEN ACOSTA ezt ?rta (id?pont: 2022. j?n. 15., Sze, 13:04): > Hey there, > > I'm using OpenStack(old version) with neutron OVN CMS-plugin and I need to > migrate to the Yoga release (Ubuntu ovn v22.03.0 and ovs v2.17.0). > > I'm interested in validating a series of fixed issues on openvswitch and > ovn in the releases related to the OpenStack Yoga version (e.g. > https://github.com/ovn-org/ovn/commit/ccbbbd0584e52c2aa88436a431fe7a6deffd2221 > ). > > What OpenStack automated testing tool do you use or recommend to validate > the data plane? > > Best regards > Roberto > > > > *?Esta mensagem ? direcionada apenas para os endere?os constantes no > cabe?alho inicial. Se voc? n?o est? listado nos endere?os constantes no > cabe?alho, pedimos-lhe que desconsidere completamente o conte?do dessa > mensagem e cuja c?pia, encaminhamento e/ou execu??o das a??es citadas est?o > imediatamente anuladas e proibidas?.* > > *?Apesar do Magazine Luiza tomar todas as precau??es razo?veis para > assegurar que nenhum v?rus esteja presente nesse e-mail, a empresa n?o > poder? aceitar a responsabilidade por quaisquer perdas ou danos causados > por esse e-mail ou por seus anexos?.* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From swogatpradhan22 at gmail.com Thu Jun 16 07:33:02 2022 From: swogatpradhan22 at gmail.com (Swogat Pradhan) Date: Thu, 16 Jun 2022 13:03:02 +0530 Subject: Error while deploying ReaR | openstak wallaby | tripleo Message-ID: Hi, Steps to Reproduce: 1. Add /usr/share/openstack-tripleo-heat-templates/environments/backup-and-restore/rear.yaml as a environment file in openstack overcloud deploy command 2. Run the deployement, in step 0 or 1 the error should come up. Actual results: 2022-06-16 14:52:07.581055 | 48d539a1-1679-6c30-81d1-000000007f7d | TASK | Check backup server IP 2022-06-16 14:52:18.186942 | 48d539a1-1679-6c30-81d1-000000007f7d | FATAL | Check backup server IP | overcloud-controller-1 | error={"changed": true, "cmd": ["ping", "-c", "1", "192.168.24.1"], "delta": "0:00:10.011643", "end": "2022-06-16 06:52:18.119615", "msg": "non-zero return code", "rc": 1, "start": "2022-06-16 06:52:08.107972", "stderr": "", "stderr_lines": [], "stdout": "PING 192.168.24.1 (192.168.24.1) 56(84) bytes of data.\n\n--- 192.168.24.1 ping statistics ---\n1 packets transmitted, 0 received, 100% packet loss, time 0ms", "stdout_lines": ["PING 192.168.24.1 (192.168.24.1) 56(84) bytes of data.", "", "--- 192.168.24.1 ping statistics ---", "1 packets transmitted, 0 received, 100% packet loss, time 0ms"]} Expected results: ReaR in tripleo should be deployed without errors Additional info: There is no 192.168.24.0 subnet in my setup Should i make a host entry in all the overcloud nodes and undercloud node like below? 192.268.24.1 undercloud 192.268.24.1 overcloud-controller-1 With Regards, Swogat pradhan -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Thu Jun 16 08:48:23 2022 From: eblock at nde.ag (Eugen Block) Date: Thu, 16 Jun 2022 08:48:23 +0000 Subject: evacuate problem In-Reply-To: Message-ID: <20220616084823.Horde.OL4z4zkaIm5uhUGA6gFgsrP@webmail.nde.ag> Hi, I haven't used Masakari yet and I don't use kolla, but this message indicates that your pacemaker communication is not set up properly: > ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication is failed. I would start checking /etc/corosync/corosync.conf (or the respective file in a kolla/masakari deployment) if it matches your actual network setup. We use a designated network for the corosync traffic in our environment. If there aren't any other errors I would start this communication error first and see how far you get. Zitat von fereshteh loghmani : > hello > i use masakari for migrate servers when the compute being down. > i install these container on the region: > masakari-monitors > masakari-engine > masakari-api > hacluster-pacemaker > hacluster-corosync > and on the compute server i install "hacluster-pacemaker-remote". > In openstack i created segment and I added 2 hosts (in this case the name > of that 2 hosts are: R3SG5 & R3SG12). > for testing evacuate function i shut off one of that compute that i added > in host. > (in this case i shutoff R3SG5) > i attached the log that i found in this directory: > /var/log/kolla/masakari/masakari-hostmonitor.log > > ********* > INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG5' is > 'online' (current: 'online'). > INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG12' is > 'online' (current: 'online'). > WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication using 'eth0' is failed.: > oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while > running command. > ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication is failed. > INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG5' is > 'offline' (current: 'offline'). > INFO masakarimonitors.ha.masakari [-] Send a notification. > {'notification': {'type': 'COMPUTE_HOST', 'hostname': 'R3SG5', > 'generated_time': datetime.datetime(2022, 6, 14, 7, 6, 46, 138867), > 'payload': {'event': 'STOPPED', 'cluster_status': 'OFFLINE', 'host_status': > 'NORMAL'}}} > INFO masakarimonitors.ha.masakari [-] Response: > openstack.instance_ha.v1.notification.Notification(type=COMPUTE_HOST, > hostname=R3SG5, generated_time=2022-06-14T07:06:46.138867, > payload={'event': 'STOPPED', 'cluster_status': 'OFFLINE', 'host_status': > 'NORMAL'}, id=105, notification_uuid=a7364095-cc7d-48f8-b963-c64ba147897c, > source_host_uuid=6328f08c-c752-43d5-4689-801d91dd67ec, status=new, > created_at=2022-06-14T07:06:47.000000, updated_at=None, > location=Munch({'cloud': 'controller', 'region_name': 'RegionThree', > 'zone': None, 'project': Munch({'id': 'a75a951b4537478e8cea39a932f830da', > 'name': None, 'domain_id': None, 'domain_name': None})})) > INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG12' is > 'online' (current: 'online'). > WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication using 'eth0' is failed.: > oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while > running command. > ERROR masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication is failed. > INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG5' is > 'offline' (current: 'offline'). > INFO masakarimonitors.hostmonitor.host_handler.handle_host [-] 'R3SG12' is > 'online' (current: 'online'). > WARNING masakarimonitors.hostmonitor.host_handler.handle_host [-] Corosync > communication using 'eth0' is failed.: > oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while > running command. > ***************** > > > > also i checked nova_scheduler logs and on that directory i receive error: > > *** > ERROR oslo_messaging.rpc.server nova.exception.NoValidHost: No valid host > was found. There are not enough hosts available. > ******* > > finally in OpenStack dashboard in the notification section tatus change > from running to failed. after the error state that shows in the > notification section, my VM that was on R3SG5 became to ERROR state and the > VM still exists on R3SG5 and it doenst been migrated to R3SG12. > > could you please help me why evacuate function doesn't work correctly? From smooney at redhat.com Thu Jun 16 10:00:57 2022 From: smooney at redhat.com (Sean Mooney) Date: Thu, 16 Jun 2022 11:00:57 +0100 Subject: raw and qcow2 format In-Reply-To: References: Message-ID: On Thu, 2022-06-16 at 11:08 +0430, fereshteh loghmani wrote: > hello > I have a question about disk format in OpenStack. > I created a new server and shut off that server and with the command > qemu-img info disk I found my "disk format" in qcow2 and it shows the > "backing file format: raw". > could you tell me why these 2 are different? your nova config options and also perfromance i belve have in a raw backing file is faster but it uses more space https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.force_raw_images that defaults to true so we will use raw images for backing files https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.images_type default to default meaning the default is virt driver defiened for libvirt the default is qcow https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.use_cow_images defaults to true which allows qcow backing files to be used but the force flag overrides that. so if you disabel force_raw_images and your image in glance is qcow2 then the backing file would be qcow2 if the image was raw in glance the backing file woudl be raw. from a performance vs disk usage point of view i belive our defaults of using a raw backing file which is shared between multiple vms if they use the same image and qcow for the vm itself is correct if you want higher performce then the defautl then you can you can set images_type to raw if you want disbale the use of backing files you can set the images_type to flat so the answer basicaly is because that is what the config file currently has selected and you can change that behavor if you want too. > (I use OpenStack without ceph and without volumes.) > thank you From eblock at nde.ag Thu Jun 16 11:04:11 2022 From: eblock at nde.ag (Eugen Block) Date: Thu, 16 Jun 2022 11:04:11 +0000 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: <20220616110411.Horde.6VLs4vT4mEnXUkE_YYIGWL5@webmail.nde.ag> It can't reach the neutron service, that's what failing here. I'm not familiar with devstack, is there any way to verify the basic services before deploying trove or is it "all-in-one"? Anyway, in this case I'd recommend to check neutron logs and then fix it. Zitat von KK CHN : > This is my latest devstack installation error.. No clues why it fails > repeatedly; earlier it was during the uploading stage of the cirros image, > internal server error ( HTTP 500 ) occurred. This was repeated multiple > times. So I posted for the first time in this thread. > > In this iteration for starting afresh, I have removed the stack user and > /opt/stack directory manually, and freshly created 'stack' user with sudo > permissions and /opt/stack directory with stack:stack ownership, and > fresh git cloned(git clone https://opendev.org/openstack/devstack ) to > /opt/DEVSTACK_stack folder and created local.conf ( attached here) . > Then executed ./stack.sh . > > Got the following error this time ... Any clues why it fails here ? > Platform (on a VM with Debian uname output) : Linux XenaCtrl1 > 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux > > [image: Trove_installationError.png] > > On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney wrote: > >> On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: >> > > i hit the same issue the other day if you recall me asking about it. >> > > didnt check my devstack logs to see if it installed or not but i can >> > > see if i can repoduce it tomorrow. >> > >> > I checked with the OP and they have it installed properly, >> > system-wide. I was assuming your issue might have been venv (or >> > ordering) related, but theirs was not. There are some other weird things >> > going on with their setup, so I'm not so sure. >> nope, was not useing a vnev or anything special, it just devstack >> installed normaly more or less >> but driven by the ansible roles. the run devstack rull litrally just opens >> a shell and invokes >> stack.sh so i dont think there was anything specific ot how i was invoking >> it. >> >> > >> > --Dan >> > >> >> From lokendrarathour at gmail.com Wed Jun 15 14:11:51 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Wed, 15 Jun 2022 19:41:51 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Shephard, I am getting the local_ip (ipv6) of the undercloud : [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml [aaaa:aaaa:aaaa::1] is this because of some ipv6 reasons? On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard wrote: > Hey, > > Ok, that command looks fine. What about that variable there? Do you get > anything back when you run: > sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > > Mine returns: > sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > 192.168.24.115 > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < > lokendrarathour at gmail.com> wrote: > >> Hi Shephard, >> >> this is the command from my wallaby: >> [root at undercloud ~]# sudo cat >> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >> { >> "cap_add": [ >> "NET_ADMIN", >> "NET_RAW", >> "SETUID" >> ], >> "command": [ >> "/bin/bash", >> "-c", >> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >> --log-facility=/var/log/ironic/dnsmasq.log --user=root >> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >> --tftp-root=/var/lib/ironic/tftpboot" >> ], >> "environment": { >> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >> }, >> "healthcheck": { >> "test": "/openstack/healthcheck" >> }, >> "image": >> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >> "net": "host", >> "privileged": false, >> "restart": "always", >> "start_order": 90, >> "volumes": [ >> "/etc/hosts:/etc/hosts:ro", >> "/etc/localtime:/etc/localtime:ro", >> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >> >> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >> >> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >> >> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >> "/dev/log:/dev/log", >> "/etc/puppet:/etc/puppet:ro", >> >> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >> >> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >> "/var/lib/ironic:/var/lib/ironic:shared,z", >> "/var/log/containers/ironic:/var/log/ironic:z", >> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >> ] >> }[root at undercloud ~]# >> >> Comparing both, they look alike. >> please check once. >> >> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >> wrote: >> >>> Hi, >>> >>> Looks like the command was in a different file in Wallaby, can you check: >>> sudo cat >>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>> >>> That one should have the dnsmasq command it's trying to run. For >>> example, here it is from my Wallaby environment: >>> [stack at undercloud-0 ~]$ sudo cat >>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>> | jq .command >>> [ >>> "/bin/bash", >>> "-c", >>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>> --tftp-root=/var/lib/ironic/tftpboot" >>> ] >>> >>> >>> >>> Brendan Shephard >>> >>> Software Engineer >>> >>> Red Hat APAC >>> >>> 193 N Quay >>> >>> Brisbane City QLD 4000 >>> @RedHat Red Hat >>> Red Hat >>> >>> >>> >>> >>> >>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>> lokendrarathour at gmail.com> wrote: >>> >>>> Hi Shephard, >>>> Here is the o/p of the file: >>>> >>>> [root at undercloud ~]# sudo cat >>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>> { >>>> "config_files": [ >>>> { >>>> "dest": "/", >>>> "merge": true, >>>> "preserve_properties": true, >>>> "source": "/var/lib/kolla/config_files/src/*" >>>> } >>>> ], >>>> "permissions": [ >>>> { >>>> "owner": "ironic:ironic", >>>> "path": "/var/log/ironic", >>>> "recurse": true >>>> }, >>>> { >>>> "owner": "ironic:ironic", >>>> "path": "/var/lib/ironic", >>>> "recurse": true >>>> } >>>> ] >>>> }[root at undercloud ~]# >>>> >>>> >>>> Thanks once agan. >>>> >>>> -Lokendra >>>> >>>> >>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard >>>> wrote: >>>> >>>>> Looks like something wrong with the dnsmasq command the container is >>>>> being launched with. What command is it trying to run? >>>>> >>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>> >>>>> Brendan Shephard >>>>> >>>>> Software Engineer >>>>> >>>>> Red Hat APAC >>>>> >>>>> 193 N Quay >>>>> >>>>> Brisbane City QLD 4000 >>>>> @RedHat Red Hat >>>>> Red Hat >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>> wrote: >>>>> >>>>>> Hi Brendan, >>>>>> >>>>>> Thanks for your response. >>>>>> >>>>>> Please find the log below. >>>>>> >>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>> >>>>>> dnsmasq: bad command line options: try --help >>>>>> dnsmasq: bad command line options: try --help >>>>>> dnsmasq: bad command line options: try --help >>>>>> dnsmasq: bad command line options: try --help >>>>>> dnsmasq: bad command line options: try --help >>>>>> dnsmasq: bad command line options: try --help >>>>>> >>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe >>>>>> -a >>>>>> CONTAINER ID IMAGE >>>>>> COMMAND CREATED STATUS >>>>>> PORTS NAMES >>>>>> 02dacbc74cec >>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>> ironic_pxe_tftp >>>>>> 1f8ca39fba32 >>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>> ironic_pxe_http >>>>>> >>>>>> >>>>>> Regards >>>>>> >>>>>> Anirudh Gupta >>>>>> >>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>> bshephar at redhat.com> wrote: >>>>>> >>>>>>> Hey Anirudh, >>>>>>> >>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>> to see why it's failing. >>>>>>> >>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>> --filter name=ironic_pxe -a >>>>>>> CONTAINER ID IMAGE >>>>>>> COMMAND CREATED STATUS >>>>>>> PORTS NAMES >>>>>>> 0170be36e291 >>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>> ironic_pxe_tftp >>>>>>> e507f722bdf0 >>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>> ironic_pxe_http >>>>>>> >>>>>>> Then check the logs to see what the error is: >>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>> ironic_pxe_tftp >>>>>>> >>>>>>> >>>>>>> >>>>>>> Brendan Shephard >>>>>>> >>>>>>> Software Engineer >>>>>>> >>>>>>> Red Hat APAC >>>>>>> >>>>>>> 193 N Quay >>>>>>> >>>>>>> Brisbane City QLD 4000 >>>>>>> @RedHat Red Hat >>>>>>> Red Hat >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Team, >>>>>>>> >>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>> facing the below error: >>>>>>>> >>>>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>>>> TASK | Manage container systemd services and cleanup old systemd >>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>>>> FATAL | Manage container systemd services and cleanup old systemd >>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>>>>> not started yet"} >>>>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | >>>>>>>> TIMING | tripleo_container_manage : Manage container systemd >>>>>>>> >>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>> >>>>>>>> [DEFAULT] >>>>>>>> clean_nodes = true >>>>>>>> cleanup = false >>>>>>>> container_cli = podman >>>>>>>> container_healthcheck_disabled = true >>>>>>>> container_images_file = >>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>> deployment_user = stack >>>>>>>> enable_ironic = true >>>>>>>> enable_ironic_inspector = true >>>>>>>> enable_neutron = true >>>>>>>> enable_routed_networks = false >>>>>>>> generate_service_certificate = false >>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>> ipxe_enabled = true >>>>>>>> local_interface = enp8s0 >>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>> subnets = ctlplane-subnet >>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>> undercloud_hostname = undercloud.com >>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>> undercloud_timezone = UTC >>>>>>>> >>>>>>>> [ctlplane-subnet] >>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>> >>>>>>>> Can someone please help in this regard. >>>>>>>> >>>>>>>> Anirudh Gupta >>>>>>>> >>>>>>>> >>>> >>>> -- >>>> ~ Lokendra >>>> www.inertiaspeaks.com >>>> www.inertiagroups.com >>>> skype: lokendrarathour >>>> >>>> >>>> >> >> -- >> ~ Lokendra >> www.inertiaspeaks.com >> www.inertiagroups.com >> skype: lokendrarathour >> >> >> -- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour -------------- next part -------------- An HTML attachment was scrubbed... URL: From bshephar at redhat.com Wed Jun 15 23:45:22 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Thu, 16 Jun 2022 09:45:22 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hey, Looks like that is the problem. The [ ] around the IP address are causing the issue. If I try to run dnsmasq using exactly the output you get, it gives me the same error: [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot dnsmasq: bad command line options: try --help VS without the [ ] I can see it starts up normally. The settings in your undercloud.conf file look to be correct I believe. So I think there might be a bug here. I don't think we should be saving that value with the square brackets, or we would need to filter them out when we gather the value in that variable. I raised a bug for it here so that we can dig into this and find what needs fixing: https://bugs.launchpad.net/tripleo/+bug/1978892 In the meantime, if you edit that hieradata value, are you able to get that container started? Change this: [root at tripleo-director ~]# egrep -r 'tftp_bind_host' /etc/puppet/hieradata/ /etc/puppet/hieradata/service_configs.json: "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", To this: "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" Then restart the service: sudo systemctl restart tripleo_ironic_pxe_http.service tripleo_ironic_pxe_tftp.service Does that get the container running without the error? I did the same in my environment and can see that dnsmasq is running properly like that: [root at tripleo-director ~]# ps -ef | grep aaaa root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour wrote: > Hi Shephard, > I am getting the local_ip (ipv6) of the undercloud : > > [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml > [aaaa:aaaa:aaaa::1] > > is this because of some ipv6 reasons? > > > On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard > wrote: > >> Hey, >> >> Ok, that command looks fine. What about that variable there? Do you get >> anything back when you run: >> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >> >> Mine returns: >> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >> 192.168.24.115 >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Hi Shephard, >>> >>> this is the command from my wallaby: >>> [root at undercloud ~]# sudo cat >>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>> { >>> "cap_add": [ >>> "NET_ADMIN", >>> "NET_RAW", >>> "SETUID" >>> ], >>> "command": [ >>> "/bin/bash", >>> "-c", >>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>> --tftp-root=/var/lib/ironic/tftpboot" >>> ], >>> "environment": { >>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>> }, >>> "healthcheck": { >>> "test": "/openstack/healthcheck" >>> }, >>> "image": >>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>> "net": "host", >>> "privileged": false, >>> "restart": "always", >>> "start_order": 90, >>> "volumes": [ >>> "/etc/hosts:/etc/hosts:ro", >>> "/etc/localtime:/etc/localtime:ro", >>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>> >>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>> >>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>> >>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>> "/dev/log:/dev/log", >>> "/etc/puppet:/etc/puppet:ro", >>> >>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>> >>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>> "/var/log/containers/ironic:/var/log/ironic:z", >>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>> ] >>> }[root at undercloud ~]# >>> >>> Comparing both, they look alike. >>> please check once. >>> >>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >>> wrote: >>> >>>> Hi, >>>> >>>> Looks like the command was in a different file in Wallaby, can you >>>> check: >>>> sudo cat >>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>> >>>> That one should have the dnsmasq command it's trying to run. For >>>> example, here it is from my Wallaby environment: >>>> [stack at undercloud-0 ~]$ sudo cat >>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>> | jq .command >>>> [ >>>> "/bin/bash", >>>> "-c", >>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>> --tftp-root=/var/lib/ironic/tftpboot" >>>> ] >>>> >>>> >>>> >>>> Brendan Shephard >>>> >>>> Software Engineer >>>> >>>> Red Hat APAC >>>> >>>> 193 N Quay >>>> >>>> Brisbane City QLD 4000 >>>> @RedHat Red Hat >>>> Red Hat >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>> lokendrarathour at gmail.com> wrote: >>>> >>>>> Hi Shephard, >>>>> Here is the o/p of the file: >>>>> >>>>> [root at undercloud ~]# sudo cat >>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>> { >>>>> "config_files": [ >>>>> { >>>>> "dest": "/", >>>>> "merge": true, >>>>> "preserve_properties": true, >>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>> } >>>>> ], >>>>> "permissions": [ >>>>> { >>>>> "owner": "ironic:ironic", >>>>> "path": "/var/log/ironic", >>>>> "recurse": true >>>>> }, >>>>> { >>>>> "owner": "ironic:ironic", >>>>> "path": "/var/lib/ironic", >>>>> "recurse": true >>>>> } >>>>> ] >>>>> }[root at undercloud ~]# >>>>> >>>>> >>>>> Thanks once agan. >>>>> >>>>> -Lokendra >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard >>>>> wrote: >>>>> >>>>>> Looks like something wrong with the dnsmasq command the container is >>>>>> being launched with. What command is it trying to run? >>>>>> >>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>> >>>>>> Brendan Shephard >>>>>> >>>>>> Software Engineer >>>>>> >>>>>> Red Hat APAC >>>>>> >>>>>> 193 N Quay >>>>>> >>>>>> Brisbane City QLD 4000 >>>>>> @RedHat Red Hat >>>>>> Red Hat >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>>> wrote: >>>>>> >>>>>>> Hi Brendan, >>>>>>> >>>>>>> Thanks for your response. >>>>>>> >>>>>>> Please find the log below. >>>>>>> >>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>> >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> >>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>> name=ironic_pxe -a >>>>>>> CONTAINER ID IMAGE >>>>>>> COMMAND CREATED >>>>>>> STATUS PORTS NAMES >>>>>>> 02dacbc74cec >>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>> ironic_pxe_tftp >>>>>>> 1f8ca39fba32 >>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>> ironic_pxe_http >>>>>>> >>>>>>> >>>>>>> Regards >>>>>>> >>>>>>> Anirudh Gupta >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>> bshephar at redhat.com> wrote: >>>>>>> >>>>>>>> Hey Anirudh, >>>>>>>> >>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>> to see why it's failing. >>>>>>>> >>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>> --filter name=ironic_pxe -a >>>>>>>> CONTAINER ID IMAGE >>>>>>>> COMMAND CREATED STATUS >>>>>>>> PORTS NAMES >>>>>>>> 0170be36e291 >>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>> ironic_pxe_tftp >>>>>>>> e507f722bdf0 >>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>> ironic_pxe_http >>>>>>>> >>>>>>>> Then check the logs to see what the error is: >>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>> ironic_pxe_tftp >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Brendan Shephard >>>>>>>> >>>>>>>> Software Engineer >>>>>>>> >>>>>>>> Red Hat APAC >>>>>>>> >>>>>>>> 193 N Quay >>>>>>>> >>>>>>>> Brisbane City QLD 4000 >>>>>>>> @RedHat Red Hat >>>>>>>> Red Hat >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Team, >>>>>>>>> >>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>>> facing the below error: >>>>>>>>> >>>>>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>> | TASK | Manage container systemd services and cleanup old systemd >>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>> | FATAL | Manage container systemd services and cleanup old systemd >>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>>>>>> not started yet"} >>>>>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>> | TIMING | tripleo_container_manage : Manage container systemd >>>>>>>>> >>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>> >>>>>>>>> [DEFAULT] >>>>>>>>> clean_nodes = true >>>>>>>>> cleanup = false >>>>>>>>> container_cli = podman >>>>>>>>> container_healthcheck_disabled = true >>>>>>>>> container_images_file = >>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>> deployment_user = stack >>>>>>>>> enable_ironic = true >>>>>>>>> enable_ironic_inspector = true >>>>>>>>> enable_neutron = true >>>>>>>>> enable_routed_networks = false >>>>>>>>> generate_service_certificate = false >>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>> ipxe_enabled = true >>>>>>>>> local_interface = enp8s0 >>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>> subnets = ctlplane-subnet >>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>> undercloud_timezone = UTC >>>>>>>>> >>>>>>>>> [ctlplane-subnet] >>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>> >>>>>>>>> Can someone please help in this regard. >>>>>>>>> >>>>>>>>> Anirudh Gupta >>>>>>>>> >>>>>>>>> >>>>> >>>>> -- >>>>> ~ Lokendra >>>>> www.inertiaspeaks.com >>>>> www.inertiagroups.com >>>>> skype: lokendrarathour >>>>> >>>>> >>>>> >>> >>> -- >>> ~ Lokendra >>> www.inertiaspeaks.com >>> www.inertiagroups.com >>> skype: lokendrarathour >>> >>> >>> > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kkchn.in at gmail.com Thu Jun 16 06:21:58 2022 From: kkchn.in at gmail.com (KK CHN) Date: Thu, 16 Jun 2022 11:51:58 +0530 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: Update to my Devstack installation Error. I have crossed this Internal Error ( HTTP 500) and ended up with a new error for the Trove Devstack Installation attempts. ################################################## 2022-06-16 05:46:40.183 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:26 : total_kB=16393076 2022-06-16 05:46:40.188 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:29 : RAM_NEEDED=4 2022-06-16 05:46:40.192 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : '[' 16393076 -lt 4194304 ']' 2022-06-16 05:46:40.197 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : return 0 2022-06-16 05:46:40.202 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:236 : timeout 120 sh -c 'while ! sudo umount -f /tmp/dib_image.WMpKoawa; do sleep 1; done' 2022-06-16 05:46:40.225 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:241 : rm -rf --one-file-system /tmp/dib_image.WMpKoawa 2022-06-16 05:46:40.231 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 : exit 1 Error on exit # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ ############################################################################# I am able to login to the Horizon dash board inspite the ./stack.sh script Exit with the above error status.. And in the image section, I am able to view cirros-0.5.2-x86_64-disk image in the dashboard. But on the terminal when I issue the command DEVSTACK_TROVE/devstack$ openstack datastore version list mysql it says " Datastore 'mysql' cannot found. HTTP(404). #############Here the tty output stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ source ./openrc WARNING: setting legacy OS_TENANT_NAME to support cli tools. stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404) stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack server list stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack image list +--------------------------------------+--------------------------+--------+ | ID | Name | Status | +--------------------------------------+--------------------------+--------+ | be0c8769-286c-4b61-b989-a037e0e9601c | cirros-0.5.2-x86_64-disk | active | +--------------------------------------+--------------------------+--------+ stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404) ####################################################################################### So I ended up with the TROVE installation failing ? Or what does this error mean ? How can I fix this to get a devstack installed successfully with Trove-DbaaS feature up and running ? Any help / directions much appreciated .... [ How I rectify the earlier errors of HTTP 500 : I just created a VM again with same debian 11 image and 'stack' user created with /opt/stack dir and freshly git cloned devstack to DEVSTACK_TROVE/ Dir.. Then created the same local.conf file which I posted in my earlier mail. and reran ./stack.sh script . This time I was able to get the dashboard up and be able to login(earlier it's not possible to get the dashboard too) ] Krish On Wed, Jun 15, 2022 at 12:31 PM KK CHN wrote: > This is my latest devstack installation error.. No clues why it fails > repeatedly; earlier it was during the uploading stage of the cirros image, > internal server error ( HTTP 500 ) occurred. This was repeated multiple > times. So I posted for the first time in this thread. > > In this iteration for starting afresh, I have removed the stack user and > /opt/stack directory manually, and freshly created 'stack' user with sudo > permissions and /opt/stack directory with stack:stack ownership, and > fresh git cloned(git clone https://opendev.org/openstack/devstack ) to > /opt/DEVSTACK_stack folder and created local.conf ( attached here) . > Then executed ./stack.sh . > > Got the following error this time ... Any clues why it fails here ? > Platform (on a VM with Debian uname output) : Linux XenaCtrl1 > 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux > > [image: Trove_installationError.png] > > On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney wrote: > >> On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: >> > > i hit the same issue the other day if you recall me asking about it. >> > > didnt check my devstack logs to see if it installed or not but i can >> > > see if i can repoduce it tomorrow. >> > >> > I checked with the OP and they have it installed properly, >> > system-wide. I was assuming your issue might have been venv (or >> > ordering) related, but theirs was not. There are some other weird things >> > going on with their setup, so I'm not so sure. >> nope, was not useing a vnev or anything special, it just devstack >> installed normaly more or less >> but driven by the ansible roles. the run devstack rull litrally just >> opens a shell and invokes >> stack.sh so i dont think there was anything specific ot how i was >> invoking it. >> >> > >> > --Dan >> > >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Trove_installationError.png Type: image/png Size: 88093 bytes Desc: not available URL: From bshephar at redhat.com Thu Jun 16 06:49:40 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Thu, 16 Jun 2022 16:49:40 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hey, It looks like my reply email was too big and has been blocked. The problem is the square brackets in the IPv6 address being returned from the hiera command. Can you try this to workaround the problem: Change this: [root at tripleo-director ~]# egrep -r 'tftp_bind_host' /etc/puppet/hieradata/ /etc/puppet/hieradata/service_configs.json: "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", To this: "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" Then restart the service: sudo systemctl restart tripleo_ironic_pxe_http.service tripleo_ironic_pxe_tftp.service I raised a bug for this for you: https://bugs.launchpad.net/tripleo/+bug/1978892 Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour wrote: > Hi Shephard, > I am getting the local_ip (ipv6) of the undercloud : > > [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml > [aaaa:aaaa:aaaa::1] > > is this because of some ipv6 reasons? > > > On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard > wrote: > >> Hey, >> >> Ok, that command looks fine. What about that variable there? Do you get >> anything back when you run: >> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >> >> Mine returns: >> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >> 192.168.24.115 >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Hi Shephard, >>> >>> this is the command from my wallaby: >>> [root at undercloud ~]# sudo cat >>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>> { >>> "cap_add": [ >>> "NET_ADMIN", >>> "NET_RAW", >>> "SETUID" >>> ], >>> "command": [ >>> "/bin/bash", >>> "-c", >>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>> --tftp-root=/var/lib/ironic/tftpboot" >>> ], >>> "environment": { >>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>> }, >>> "healthcheck": { >>> "test": "/openstack/healthcheck" >>> }, >>> "image": >>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>> "net": "host", >>> "privileged": false, >>> "restart": "always", >>> "start_order": 90, >>> "volumes": [ >>> "/etc/hosts:/etc/hosts:ro", >>> "/etc/localtime:/etc/localtime:ro", >>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>> >>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>> >>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>> >>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>> "/dev/log:/dev/log", >>> "/etc/puppet:/etc/puppet:ro", >>> >>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>> >>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>> "/var/log/containers/ironic:/var/log/ironic:z", >>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>> ] >>> }[root at undercloud ~]# >>> >>> Comparing both, they look alike. >>> please check once. >>> >>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >>> wrote: >>> >>>> Hi, >>>> >>>> Looks like the command was in a different file in Wallaby, can you >>>> check: >>>> sudo cat >>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>> >>>> That one should have the dnsmasq command it's trying to run. For >>>> example, here it is from my Wallaby environment: >>>> [stack at undercloud-0 ~]$ sudo cat >>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>> | jq .command >>>> [ >>>> "/bin/bash", >>>> "-c", >>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>> --tftp-root=/var/lib/ironic/tftpboot" >>>> ] >>>> >>>> >>>> >>>> Brendan Shephard >>>> >>>> Software Engineer >>>> >>>> Red Hat APAC >>>> >>>> 193 N Quay >>>> >>>> Brisbane City QLD 4000 >>>> @RedHat Red Hat >>>> Red Hat >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>> lokendrarathour at gmail.com> wrote: >>>> >>>>> Hi Shephard, >>>>> Here is the o/p of the file: >>>>> >>>>> [root at undercloud ~]# sudo cat >>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>> { >>>>> "config_files": [ >>>>> { >>>>> "dest": "/", >>>>> "merge": true, >>>>> "preserve_properties": true, >>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>> } >>>>> ], >>>>> "permissions": [ >>>>> { >>>>> "owner": "ironic:ironic", >>>>> "path": "/var/log/ironic", >>>>> "recurse": true >>>>> }, >>>>> { >>>>> "owner": "ironic:ironic", >>>>> "path": "/var/lib/ironic", >>>>> "recurse": true >>>>> } >>>>> ] >>>>> }[root at undercloud ~]# >>>>> >>>>> >>>>> Thanks once agan. >>>>> >>>>> -Lokendra >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard >>>>> wrote: >>>>> >>>>>> Looks like something wrong with the dnsmasq command the container is >>>>>> being launched with. What command is it trying to run? >>>>>> >>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>> >>>>>> Brendan Shephard >>>>>> >>>>>> Software Engineer >>>>>> >>>>>> Red Hat APAC >>>>>> >>>>>> 193 N Quay >>>>>> >>>>>> Brisbane City QLD 4000 >>>>>> @RedHat Red Hat >>>>>> Red Hat >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>>> wrote: >>>>>> >>>>>>> Hi Brendan, >>>>>>> >>>>>>> Thanks for your response. >>>>>>> >>>>>>> Please find the log below. >>>>>>> >>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>> >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> dnsmasq: bad command line options: try --help >>>>>>> >>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>> name=ironic_pxe -a >>>>>>> CONTAINER ID IMAGE >>>>>>> COMMAND CREATED >>>>>>> STATUS PORTS NAMES >>>>>>> 02dacbc74cec >>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>> ironic_pxe_tftp >>>>>>> 1f8ca39fba32 >>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>> ironic_pxe_http >>>>>>> >>>>>>> >>>>>>> Regards >>>>>>> >>>>>>> Anirudh Gupta >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>> bshephar at redhat.com> wrote: >>>>>>> >>>>>>>> Hey Anirudh, >>>>>>>> >>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>> to see why it's failing. >>>>>>>> >>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>> --filter name=ironic_pxe -a >>>>>>>> CONTAINER ID IMAGE >>>>>>>> COMMAND CREATED STATUS >>>>>>>> PORTS NAMES >>>>>>>> 0170be36e291 >>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>> ironic_pxe_tftp >>>>>>>> e507f722bdf0 >>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>> ironic_pxe_http >>>>>>>> >>>>>>>> Then check the logs to see what the error is: >>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>> ironic_pxe_tftp >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Brendan Shephard >>>>>>>> >>>>>>>> Software Engineer >>>>>>>> >>>>>>>> Red Hat APAC >>>>>>>> >>>>>>>> 193 N Quay >>>>>>>> >>>>>>>> Brisbane City QLD 4000 >>>>>>>> @RedHat Red Hat >>>>>>>> Red Hat >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Team, >>>>>>>>> >>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>>> facing the below error: >>>>>>>>> >>>>>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>> | TASK | Manage container systemd services and cleanup old systemd >>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>> | FATAL | Manage container systemd services and cleanup old systemd >>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>>>>>> not started yet"} >>>>>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>> | TIMING | tripleo_container_manage : Manage container systemd >>>>>>>>> >>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>> >>>>>>>>> [DEFAULT] >>>>>>>>> clean_nodes = true >>>>>>>>> cleanup = false >>>>>>>>> container_cli = podman >>>>>>>>> container_healthcheck_disabled = true >>>>>>>>> container_images_file = >>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>> deployment_user = stack >>>>>>>>> enable_ironic = true >>>>>>>>> enable_ironic_inspector = true >>>>>>>>> enable_neutron = true >>>>>>>>> enable_routed_networks = false >>>>>>>>> generate_service_certificate = false >>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>> ipxe_enabled = true >>>>>>>>> local_interface = enp8s0 >>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>> subnets = ctlplane-subnet >>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>> undercloud_timezone = UTC >>>>>>>>> >>>>>>>>> [ctlplane-subnet] >>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>> >>>>>>>>> Can someone please help in this regard. >>>>>>>>> >>>>>>>>> Anirudh Gupta >>>>>>>>> >>>>>>>>> >>>>> >>>>> -- >>>>> ~ Lokendra >>>>> www.inertiaspeaks.com >>>>> www.inertiagroups.com >>>>> skype: lokendrarathour >>>>> >>>>> >>>>> >>> >>> -- >>> ~ Lokendra >>> www.inertiaspeaks.com >>> www.inertiagroups.com >>> skype: lokendrarathour >>> >>> >>> > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kkchn.in at gmail.com Thu Jun 16 07:03:46 2022 From: kkchn.in at gmail.com (KK CHN) Date: Thu, 16 Jun 2022 12:33:46 +0530 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: Please find https://pastebin.com/yHgFKcx4 Complete Error Log (Note the line : 464 also ) On Thu, Jun 16, 2022 at 11:51 AM KK CHN wrote: > Update to my Devstack installation Error. > > I have crossed this Internal Error ( HTTP 500) and ended up with a new > error for the Trove Devstack Installation attempts. > ################################################## > 2022-06-16 05:46:40.183 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:26 > : total_kB=16393076 > 2022-06-16 05:46:40.188 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:29 > : RAM_NEEDED=4 > 2022-06-16 05:46:40.192 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 > : '[' 16393076 -lt 4194304 ']' > 2022-06-16 05:46:40.197 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 > : return 0 > 2022-06-16 05:46:40.202 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:236 > : timeout 120 sh -c 'while ! sudo umount -f /tmp/dib_image.WMpKoawa; do > sleep 1; done' > 2022-06-16 05:46:40.225 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:241 > : rm -rf --one-file-system /tmp/dib_image.WMpKoawa > 2022-06-16 05:46:40.231 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 > : exit 1 > Error on exit > # Warning: iptables-legacy tables present, use iptables-legacy to see them > # Warning: iptables-legacy tables present, use iptables-legacy to see them > # Warning: iptables-legacy tables present, use iptables-legacy to see them > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ > > ############################################################################# > > I am able to login to the Horizon dash board inspite the ./stack.sh > script Exit with the above error status.. And in the image section, I am > able to view cirros-0.5.2-x86_64-disk > > image in the dashboard. > > But on the terminal when I issue the command DEVSTACK_TROVE/devstack$ > openstack datastore version list mysql it says > " Datastore 'mysql' cannot found. HTTP(404). > > #############Here the tty output > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ source ./openrc > WARNING: setting legacy OS_TENANT_NAME to support cli tools. > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore > version list mysql > Datastore 'mysql' cannot be found. (HTTP 404) > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack server list > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack image list > > +--------------------------------------+--------------------------+--------+ > | ID | Name | Status > | > > +--------------------------------------+--------------------------+--------+ > | be0c8769-286c-4b61-b989-a037e0e9601c | cirros-0.5.2-x86_64-disk | active > | > > +--------------------------------------+--------------------------+--------+ > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore > version list mysql > Datastore 'mysql' cannot be found. (HTTP 404) > > ####################################################################################### > > > So I ended up with the TROVE installation failing ? Or what does this > error mean ? How can I fix this to get a devstack installed successfully > with Trove-DbaaS feature up and running ? > > > Any help / directions much appreciated .... > > [ How I rectify the earlier errors of HTTP 500 : I just created a VM > again with same debian 11 image and 'stack' user created with /opt/stack > dir and freshly git cloned devstack to DEVSTACK_TROVE/ Dir.. > Then created the same local.conf file which I posted in my earlier mail. > and reran ./stack.sh script . This time I was able to get the > dashboard up and be able to login(earlier it's not possible to get the > dashboard too) ] > > Krish > > > > > > On Wed, Jun 15, 2022 at 12:31 PM KK CHN wrote: > >> This is my latest devstack installation error.. No clues why it >> fails repeatedly; earlier it was during the uploading stage of the cirros >> image, internal server error ( HTTP 500 ) occurred. This was repeated >> multiple times. So I posted for the first time in this thread. >> >> In this iteration for starting afresh, I have removed the stack user and >> /opt/stack directory manually, and freshly created 'stack' user with sudo >> permissions and /opt/stack directory with stack:stack ownership, and >> fresh git cloned(git clone https://opendev.org/openstack/devstack ) to >> /opt/DEVSTACK_stack folder and created local.conf ( attached here) . >> Then executed ./stack.sh . >> >> Got the following error this time ... Any clues why it fails here ? >> Platform (on a VM with Debian uname output) : Linux XenaCtrl1 >> 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux >> >> [image: Trove_installationError.png] >> >> On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney wrote: >> >>> On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: >>> > > i hit the same issue the other day if you recall me asking about it. >>> > > didnt check my devstack logs to see if it installed or not but i can >>> > > see if i can repoduce it tomorrow. >>> > >>> > I checked with the OP and they have it installed properly, >>> > system-wide. I was assuming your issue might have been venv (or >>> > ordering) related, but theirs was not. There are some other weird >>> things >>> > going on with their setup, so I'm not so sure. >>> nope, was not useing a vnev or anything special, it just devstack >>> installed normaly more or less >>> but driven by the ansible roles. the run devstack rull litrally just >>> opens a shell and invokes >>> stack.sh so i dont think there was anything specific ot how i was >>> invoking it. >>> >>> > >>> > --Dan >>> > >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Trove_installationError.png Type: image/png Size: 88093 bytes Desc: not available URL: From lokendrarathour at gmail.com Thu Jun 16 07:05:32 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Thu, 16 Jun 2022 12:35:32 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Shephard, Thanks, after changing the details as mentioned, Undercloud got installed successfully. Now as a part to test the introspection we added a single node and initiated the introspection on which we are getting errors. IP as per the inspector range is getting allocated, but soon after the IP allocation the introspection ILO gives the below error: [image: image.png] it says, Downloading NBP file.... PXE-E99: Unexpected network error. Underlcoud.conf: undercloud_debug = true clean_nodes = true cleanup = false container_cli = podman container_healthcheck_disabled = true container_images_file = /home/stack/containers-prepare-parameter.yaml deployment_user = stack enable_heat = true enable_ironic = true enable_ironic_inspector = true enable_neutron = true generate_service_certificate = false enable_routed_networks = false ipv6_address_mode = dhcpv6-stateful ipxe_enabled = true ironic_default_network_interface = neutron ironic_enabled_network_interfaces = neutron,flat local_interface = enp8s0 local_ip = aaaa:aaaa:aaaa::1/64 subnets = ctlplane-subnet undercloud_admin_host = aaaa:aaaa:aaaa::1 undercloud_public_host = aaaa:aaaa:aaaa::1 undercloud_hostname = undercloud.com undercloud_ntp_servers = 30.30.30.3 undercloud_timezone = UTC [ctlplane-subnet] cidr = aaaa:aaaa:aaaa::/64 dhcp_end = aaaa:aaaa:aaaa::19 dhcp_start = aaaa:aaaa:aaaa::5 gateway = aaaa:aaaa:aaaa::1 inspection_iprange = aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40 the ironic config in the container: [root at undercloud /]# vi /etc/ironic-inspector/dnsmasq.conf port=0 interface=br-ctlplane dhcp-range=set:ctlplane-subnet,aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40,64,10m dhcp-option-force=tag:ctlplane-subnet,option:mtu,1500 dhcp-sequential-ip dhcp-match=ipxe,175 dhcp-match=set:efi,option:client-arch,7 dhcp-match=set:efi,option:client-arch,9 dhcp-match=set:efi,option:client-arch,11 # dhcpv6s for Client System Architecture Type (61) dhcp-match=set:efi6,option6:61,0007 dhcp-match=set:efi6,option6:61,0009 dhcp-match=set:efi6,option6:61,0011 dhcp-userclass=set:ipxe6,iPXE # Client is already running iPXE; move to next stage of chainloading dhcp-boot=tag:ipxe,http://[aaaa:aaaa:aaaa::1]:8088/inspector.ipxe dhcp-option=tag:ipxe6,option6:bootfile-url,http:// [aaaa:aaaa:aaaa::1]:8088/inspector.ipxe # Client is PXE booting over EFI without iPXE ROM; send EFI version of iPXE chainloader dhcp-boot=tag:efi,tag:!ipxe,snponly.efi dhcp-option=tag:efi6,tag:!ipxe6,option6:bootfile-url,tftp://[aaaa:aaaa:aaaa::1]/snponly.efi # Client is running PXE over BIOS; send BIOS version of iPXE chainloader dhcp-boot=undionly.kpxe,localhost.localdomain,aaaa:aaaa:aaaa::1 dhcp-hostsdir=/var/lib/ironic-inspector/dhcp-hostsdir Please check and help me with the possible error and resolution. Best Regards, Lokendra On Thu, Jun 16, 2022 at 5:15 AM Brendan Shephard wrote: > Hey, > > Looks like that is the problem. The [ ] around the IP address are causing > the issue. If I try to run dnsmasq using exactly the output you get, it > gives me the same error: > [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > dnsmasq: bad command line options: try --help > > VS without the [ ] I can see it starts up normally. > > The settings in your undercloud.conf file look to be correct I believe. So > I think there might be a bug here. I don't think we should be saving that > value with the square brackets, or we would need to filter them out when we > gather the value in that variable. > > I raised a bug for it here so that we can dig into this and find what > needs fixing: > https://bugs.launchpad.net/tripleo/+bug/1978892 > > In the meantime, if you edit that hieradata value, are you able to get > that container started? > > Change this: > [root at tripleo-director ~]# egrep -r 'tftp_bind_host' > /etc/puppet/hieradata/ > /etc/puppet/hieradata/service_configs.json: > "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", > > To this: > "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" > > Then restart the service: > sudo systemctl restart tripleo_ironic_pxe_http.service > tripleo_ironic_pxe_tftp.service > > Does that get the container running without the error? I did the same in > my environment and can see that dnsmasq is running properly like that: > [root at tripleo-director ~]# ps -ef | grep aaaa > root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq > --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour < > lokendrarathour at gmail.com> wrote: > >> Hi Shephard, >> I am getting the local_ip (ipv6) of the undercloud : >> >> [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c >> /etc/puppet/hiera.yaml >> [aaaa:aaaa:aaaa::1] >> >> is this because of some ipv6 reasons? >> >> >> On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard >> wrote: >> >>> Hey, >>> >>> Ok, that command looks fine. What about that variable there? Do you get >>> anything back when you run: >>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>> >>> Mine returns: >>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>> 192.168.24.115 >>> >>> Brendan Shephard >>> >>> Software Engineer >>> >>> Red Hat APAC >>> >>> 193 N Quay >>> >>> Brisbane City QLD 4000 >>> @RedHat Red Hat >>> Red Hat >>> >>> >>> >>> >>> >>> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >>> lokendrarathour at gmail.com> wrote: >>> >>>> Hi Shephard, >>>> >>>> this is the command from my wallaby: >>>> [root at undercloud ~]# sudo cat >>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>> { >>>> "cap_add": [ >>>> "NET_ADMIN", >>>> "NET_RAW", >>>> "SETUID" >>>> ], >>>> "command": [ >>>> "/bin/bash", >>>> "-c", >>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>> --tftp-root=/var/lib/ironic/tftpboot" >>>> ], >>>> "environment": { >>>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>>> }, >>>> "healthcheck": { >>>> "test": "/openstack/healthcheck" >>>> }, >>>> "image": >>>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>>> "net": "host", >>>> "privileged": false, >>>> "restart": "always", >>>> "start_order": 90, >>>> "volumes": [ >>>> "/etc/hosts:/etc/hosts:ro", >>>> "/etc/localtime:/etc/localtime:ro", >>>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>>> >>>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>>> >>>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>>> >>>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>>> "/dev/log:/dev/log", >>>> "/etc/puppet:/etc/puppet:ro", >>>> >>>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>>> >>>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>>> "/var/log/containers/ironic:/var/log/ironic:z", >>>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>>> ] >>>> }[root at undercloud ~]# >>>> >>>> Comparing both, they look alike. >>>> please check once. >>>> >>>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Looks like the command was in a different file in Wallaby, can you >>>>> check: >>>>> sudo cat >>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>> >>>>> That one should have the dnsmasq command it's trying to run. For >>>>> example, here it is from my Wallaby environment: >>>>> [stack at undercloud-0 ~]$ sudo cat >>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>> | jq .command >>>>> [ >>>>> "/bin/bash", >>>>> "-c", >>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>> ] >>>>> >>>>> >>>>> >>>>> Brendan Shephard >>>>> >>>>> Software Engineer >>>>> >>>>> Red Hat APAC >>>>> >>>>> 193 N Quay >>>>> >>>>> Brisbane City QLD 4000 >>>>> @RedHat Red Hat >>>>> Red Hat >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>>> lokendrarathour at gmail.com> wrote: >>>>> >>>>>> Hi Shephard, >>>>>> Here is the o/p of the file: >>>>>> >>>>>> [root at undercloud ~]# sudo cat >>>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>> { >>>>>> "config_files": [ >>>>>> { >>>>>> "dest": "/", >>>>>> "merge": true, >>>>>> "preserve_properties": true, >>>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>>> } >>>>>> ], >>>>>> "permissions": [ >>>>>> { >>>>>> "owner": "ironic:ironic", >>>>>> "path": "/var/log/ironic", >>>>>> "recurse": true >>>>>> }, >>>>>> { >>>>>> "owner": "ironic:ironic", >>>>>> "path": "/var/lib/ironic", >>>>>> "recurse": true >>>>>> } >>>>>> ] >>>>>> }[root at undercloud ~]# >>>>>> >>>>>> >>>>>> Thanks once agan. >>>>>> >>>>>> -Lokendra >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard >>>>>> wrote: >>>>>> >>>>>>> Looks like something wrong with the dnsmasq command the container is >>>>>>> being launched with. What command is it trying to run? >>>>>>> >>>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>> >>>>>>> Brendan Shephard >>>>>>> >>>>>>> Software Engineer >>>>>>> >>>>>>> Red Hat APAC >>>>>>> >>>>>>> 193 N Quay >>>>>>> >>>>>>> Brisbane City QLD 4000 >>>>>>> @RedHat Red Hat >>>>>>> Red Hat >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Brendan, >>>>>>>> >>>>>>>> Thanks for your response. >>>>>>>> >>>>>>>> Please find the log below. >>>>>>>> >>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>>> >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> >>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>>> name=ironic_pxe -a >>>>>>>> CONTAINER ID IMAGE >>>>>>>> COMMAND CREATED >>>>>>>> STATUS PORTS NAMES >>>>>>>> 02dacbc74cec >>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>>> ironic_pxe_tftp >>>>>>>> 1f8ca39fba32 >>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>>> ironic_pxe_http >>>>>>>> >>>>>>>> >>>>>>>> Regards >>>>>>>> >>>>>>>> Anirudh Gupta >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>>> bshephar at redhat.com> wrote: >>>>>>>> >>>>>>>>> Hey Anirudh, >>>>>>>>> >>>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>>> to see why it's failing. >>>>>>>>> >>>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>>> --filter name=ironic_pxe -a >>>>>>>>> CONTAINER ID IMAGE >>>>>>>>> COMMAND CREATED STATUS >>>>>>>>> PORTS NAMES >>>>>>>>> 0170be36e291 >>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>> ironic_pxe_tftp >>>>>>>>> e507f722bdf0 >>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>> ironic_pxe_http >>>>>>>>> >>>>>>>>> Then check the logs to see what the error is: >>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>>> ironic_pxe_tftp >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Brendan Shephard >>>>>>>>> >>>>>>>>> Software Engineer >>>>>>>>> >>>>>>>>> Red Hat APAC >>>>>>>>> >>>>>>>>> 193 N Quay >>>>>>>>> >>>>>>>>> Brisbane City QLD 4000 >>>>>>>>> @RedHat Red Hat >>>>>>>>> Red Hat >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Team, >>>>>>>>>> >>>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>>>> facing the below error: >>>>>>>>>> >>>>>>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>>> | TASK | Manage container systemd services and cleanup old systemd >>>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>>> | FATAL | Manage container systemd services and cleanup old systemd >>>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>>>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>>>>>>> not started yet"} >>>>>>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>>> | TIMING | tripleo_container_manage : Manage container systemd >>>>>>>>>> >>>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>>> >>>>>>>>>> [DEFAULT] >>>>>>>>>> clean_nodes = true >>>>>>>>>> cleanup = false >>>>>>>>>> container_cli = podman >>>>>>>>>> container_healthcheck_disabled = true >>>>>>>>>> container_images_file = >>>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>>> deployment_user = stack >>>>>>>>>> enable_ironic = true >>>>>>>>>> enable_ironic_inspector = true >>>>>>>>>> enable_neutron = true >>>>>>>>>> enable_routed_networks = false >>>>>>>>>> generate_service_certificate = false >>>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>>> ipxe_enabled = true >>>>>>>>>> local_interface = enp8s0 >>>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>>> subnets = ctlplane-subnet >>>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>>> undercloud_timezone = UTC >>>>>>>>>> >>>>>>>>>> [ctlplane-subnet] >>>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>>> >>>>>>>>>> Can someone please help in this regard. >>>>>>>>>> >>>>>>>>>> Anirudh Gupta >>>>>>>>>> >>>>>>>>>> >>>>>> >>>>>> -- >>>>>> ~ Lokendra >>>>>> www.inertiaspeaks.com >>>>>> www.inertiagroups.com >>>>>> skype: lokendrarathour >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> ~ Lokendra >>>> www.inertiaspeaks.com >>>> www.inertiagroups.com >>>> skype: lokendrarathour >>>> >>>> >>>> >> >> -- >> ~ Lokendra >> www.inertiaspeaks.com >> www.inertiagroups.com >> skype: lokendrarathour >> >> >> -- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 201716 bytes Desc: not available URL: From bshephar at redhat.com Thu Jun 16 09:21:05 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Thu, 16 Jun 2022 19:21:05 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hey, Looks like your server is booting in Legacy BIOS mode but the PXE file Ironic is offering is for UEFI. Did you try booting in UEFI mode instead of Legacy BIOS? Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Thu, Jun 16, 2022 at 5:05 PM Lokendra Rathour wrote: > Hi Shephard, > Thanks, after changing the details as mentioned, Undercloud got installed > successfully. > Now as a part to test the introspection we added a single node and > initiated the introspection on which we are getting errors. > IP as per the inspector range is getting allocated, but soon after the IP > allocation the introspection ILO gives the below error: > [image: image.png] > it says, Downloading NBP file.... > PXE-E99: Unexpected network error. > > Underlcoud.conf: > > undercloud_debug = true > clean_nodes = true > cleanup = false > container_cli = podman > container_healthcheck_disabled = true > container_images_file = /home/stack/containers-prepare-parameter.yaml > deployment_user = stack > enable_heat = true > enable_ironic = true > enable_ironic_inspector = true > enable_neutron = true > generate_service_certificate = false > enable_routed_networks = false > ipv6_address_mode = dhcpv6-stateful > ipxe_enabled = true > ironic_default_network_interface = neutron > ironic_enabled_network_interfaces = neutron,flat > local_interface = enp8s0 > local_ip = aaaa:aaaa:aaaa::1/64 > subnets = ctlplane-subnet > undercloud_admin_host = aaaa:aaaa:aaaa::1 > undercloud_public_host = aaaa:aaaa:aaaa::1 > undercloud_hostname = undercloud.com > undercloud_ntp_servers = 30.30.30.3 > undercloud_timezone = UTC > [ctlplane-subnet] > cidr = aaaa:aaaa:aaaa::/64 > dhcp_end = aaaa:aaaa:aaaa::19 > dhcp_start = aaaa:aaaa:aaaa::5 > gateway = aaaa:aaaa:aaaa::1 > inspection_iprange = aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40 > > > the ironic config in the container: > > [root at undercloud /]# vi /etc/ironic-inspector/dnsmasq.conf > port=0 > interface=br-ctlplane > > dhcp-range=set:ctlplane-subnet,aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40,64,10m > dhcp-option-force=tag:ctlplane-subnet,option:mtu,1500 > dhcp-sequential-ip > dhcp-match=ipxe,175 > dhcp-match=set:efi,option:client-arch,7 > dhcp-match=set:efi,option:client-arch,9 > dhcp-match=set:efi,option:client-arch,11 > # dhcpv6s for Client System Architecture Type (61) > dhcp-match=set:efi6,option6:61,0007 > dhcp-match=set:efi6,option6:61,0009 > dhcp-match=set:efi6,option6:61,0011 > dhcp-userclass=set:ipxe6,iPXE > # Client is already running iPXE; move to next stage of chainloading > dhcp-boot=tag:ipxe,http://[aaaa:aaaa:aaaa::1]:8088/inspector.ipxe > dhcp-option=tag:ipxe6,option6:bootfile-url,http:// > [aaaa:aaaa:aaaa::1]:8088/inspector.ipxe > # Client is PXE booting over EFI without iPXE ROM; send EFI version of > iPXE chainloader > dhcp-boot=tag:efi,tag:!ipxe,snponly.efi > > dhcp-option=tag:efi6,tag:!ipxe6,option6:bootfile-url,tftp://[aaaa:aaaa:aaaa::1]/snponly.efi > # Client is running PXE over BIOS; send BIOS version of iPXE chainloader > dhcp-boot=undionly.kpxe,localhost.localdomain,aaaa:aaaa:aaaa::1 > > dhcp-hostsdir=/var/lib/ironic-inspector/dhcp-hostsdir > > > Please check and help me with the possible error and resolution. > > Best Regards, > Lokendra > > > > > On Thu, Jun 16, 2022 at 5:15 AM Brendan Shephard > wrote: > >> Hey, >> >> Looks like that is the problem. The [ ] around the IP address are causing >> the issue. If I try to run dnsmasq using exactly the output you get, it >> gives me the same error: >> [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground >> --log-facility=/var/log/ironic/dnsmasq.log --user=root >> --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 >> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >> >> dnsmasq: bad command line options: try --help >> >> VS without the [ ] I can see it starts up normally. >> >> The settings in your undercloud.conf file look to be correct I believe. >> So I think there might be a bug here. I don't think we should be saving >> that value with the square brackets, or we would need to filter them out >> when we gather the value in that variable. >> >> I raised a bug for it here so that we can dig into this and find what >> needs fixing: >> https://bugs.launchpad.net/tripleo/+bug/1978892 >> >> In the meantime, if you edit that hieradata value, are you able to get >> that container started? >> >> Change this: >> [root at tripleo-director ~]# egrep -r 'tftp_bind_host' >> /etc/puppet/hieradata/ >> /etc/puppet/hieradata/service_configs.json: >> "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", >> >> To this: >> "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" >> >> Then restart the service: >> sudo systemctl restart tripleo_ironic_pxe_http.service >> tripleo_ironic_pxe_tftp.service >> >> Does that get the container running without the error? I did the same in >> my environment and can see that dnsmasq is running properly like that: >> [root at tripleo-director ~]# ps -ef | grep aaaa >> root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq >> --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root >> --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 >> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Hi Shephard, >>> I am getting the local_ip (ipv6) of the undercloud : >>> >>> [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c >>> /etc/puppet/hiera.yaml >>> [aaaa:aaaa:aaaa::1] >>> >>> is this because of some ipv6 reasons? >>> >>> >>> On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard >>> wrote: >>> >>>> Hey, >>>> >>>> Ok, that command looks fine. What about that variable there? Do you get >>>> anything back when you run: >>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>> >>>> Mine returns: >>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>> 192.168.24.115 >>>> >>>> Brendan Shephard >>>> >>>> Software Engineer >>>> >>>> Red Hat APAC >>>> >>>> 193 N Quay >>>> >>>> Brisbane City QLD 4000 >>>> @RedHat Red Hat >>>> Red Hat >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >>>> lokendrarathour at gmail.com> wrote: >>>> >>>>> Hi Shephard, >>>>> >>>>> this is the command from my wallaby: >>>>> [root at undercloud ~]# sudo cat >>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>> { >>>>> "cap_add": [ >>>>> "NET_ADMIN", >>>>> "NET_RAW", >>>>> "SETUID" >>>>> ], >>>>> "command": [ >>>>> "/bin/bash", >>>>> "-c", >>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>> ], >>>>> "environment": { >>>>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>>>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>>>> }, >>>>> "healthcheck": { >>>>> "test": "/openstack/healthcheck" >>>>> }, >>>>> "image": >>>>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>>>> "net": "host", >>>>> "privileged": false, >>>>> "restart": "always", >>>>> "start_order": 90, >>>>> "volumes": [ >>>>> "/etc/hosts:/etc/hosts:ro", >>>>> "/etc/localtime:/etc/localtime:ro", >>>>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>>>> >>>>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>>>> >>>>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>>>> >>>>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>>>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>>>> "/dev/log:/dev/log", >>>>> "/etc/puppet:/etc/puppet:ro", >>>>> >>>>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>>>> >>>>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>>>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>>>> "/var/log/containers/ironic:/var/log/ironic:z", >>>>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>>>> ] >>>>> }[root at undercloud ~]# >>>>> >>>>> Comparing both, they look alike. >>>>> please check once. >>>>> >>>>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Looks like the command was in a different file in Wallaby, can you >>>>>> check: >>>>>> sudo cat >>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>> >>>>>> That one should have the dnsmasq command it's trying to run. For >>>>>> example, here it is from my Wallaby environment: >>>>>> [stack at undercloud-0 ~]$ sudo cat >>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>> | jq .command >>>>>> [ >>>>>> "/bin/bash", >>>>>> "-c", >>>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>>> ] >>>>>> >>>>>> >>>>>> >>>>>> Brendan Shephard >>>>>> >>>>>> Software Engineer >>>>>> >>>>>> Red Hat APAC >>>>>> >>>>>> 193 N Quay >>>>>> >>>>>> Brisbane City QLD 4000 >>>>>> @RedHat Red Hat >>>>>> Red Hat >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>>>> lokendrarathour at gmail.com> wrote: >>>>>> >>>>>>> Hi Shephard, >>>>>>> Here is the o/p of the file: >>>>>>> >>>>>>> [root at undercloud ~]# sudo cat >>>>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>> { >>>>>>> "config_files": [ >>>>>>> { >>>>>>> "dest": "/", >>>>>>> "merge": true, >>>>>>> "preserve_properties": true, >>>>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>>>> } >>>>>>> ], >>>>>>> "permissions": [ >>>>>>> { >>>>>>> "owner": "ironic:ironic", >>>>>>> "path": "/var/log/ironic", >>>>>>> "recurse": true >>>>>>> }, >>>>>>> { >>>>>>> "owner": "ironic:ironic", >>>>>>> "path": "/var/lib/ironic", >>>>>>> "recurse": true >>>>>>> } >>>>>>> ] >>>>>>> }[root at undercloud ~]# >>>>>>> >>>>>>> >>>>>>> Thanks once agan. >>>>>>> >>>>>>> -Lokendra >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard < >>>>>>> bshephar at redhat.com> wrote: >>>>>>> >>>>>>>> Looks like something wrong with the dnsmasq command the container >>>>>>>> is being launched with. What command is it trying to run? >>>>>>>> >>>>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>>> >>>>>>>> Brendan Shephard >>>>>>>> >>>>>>>> Software Engineer >>>>>>>> >>>>>>>> Red Hat APAC >>>>>>>> >>>>>>>> 193 N Quay >>>>>>>> >>>>>>>> Brisbane City QLD 4000 >>>>>>>> @RedHat Red Hat >>>>>>>> Red Hat >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Brendan, >>>>>>>>> >>>>>>>>> Thanks for your response. >>>>>>>>> >>>>>>>>> Please find the log below. >>>>>>>>> >>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>>>> >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> >>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>>>> name=ironic_pxe -a >>>>>>>>> CONTAINER ID IMAGE >>>>>>>>> COMMAND CREATED >>>>>>>>> STATUS PORTS NAMES >>>>>>>>> 02dacbc74cec >>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>>>> ironic_pxe_tftp >>>>>>>>> 1f8ca39fba32 >>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>>>> ironic_pxe_http >>>>>>>>> >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> Anirudh Gupta >>>>>>>>> >>>>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>>>> bshephar at redhat.com> wrote: >>>>>>>>> >>>>>>>>>> Hey Anirudh, >>>>>>>>>> >>>>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>>>> to see why it's failing. >>>>>>>>>> >>>>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>>>> --filter name=ironic_pxe -a >>>>>>>>>> CONTAINER ID IMAGE >>>>>>>>>> COMMAND CREATED STATUS >>>>>>>>>> PORTS NAMES >>>>>>>>>> 0170be36e291 >>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>> ironic_pxe_tftp >>>>>>>>>> e507f722bdf0 >>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>> ironic_pxe_http >>>>>>>>>> >>>>>>>>>> Then check the logs to see what the error is: >>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>>>> ironic_pxe_tftp >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Brendan Shephard >>>>>>>>>> >>>>>>>>>> Software Engineer >>>>>>>>>> >>>>>>>>>> Red Hat APAC >>>>>>>>>> >>>>>>>>>> 193 N Quay >>>>>>>>>> >>>>>>>>>> Brisbane City QLD 4000 >>>>>>>>>> @RedHat Red Hat >>>>>>>>>> Red Hat >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta < >>>>>>>>>> anyrude10 at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Team, >>>>>>>>>>> >>>>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>>>>> facing the below error: >>>>>>>>>>> >>>>>>>>>>> 2022-06-14 05:01:23.213708 | >>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TASK | Manage container systemd >>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>>>> 2022-06-14 05:03:22.912816 | >>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | FATAL | Manage container systemd >>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | >>>>>>>>>>> error={"changed": false, "msg": "Service ironic_pxe_tftp has not started >>>>>>>>>>> yet"} >>>>>>>>>>> 2022-06-14 05:03:22.914400 | >>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TIMING | tripleo_container_manage : >>>>>>>>>>> Manage container systemd >>>>>>>>>>> >>>>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>>>> >>>>>>>>>>> [DEFAULT] >>>>>>>>>>> clean_nodes = true >>>>>>>>>>> cleanup = false >>>>>>>>>>> container_cli = podman >>>>>>>>>>> container_healthcheck_disabled = true >>>>>>>>>>> container_images_file = >>>>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>>>> deployment_user = stack >>>>>>>>>>> enable_ironic = true >>>>>>>>>>> enable_ironic_inspector = true >>>>>>>>>>> enable_neutron = true >>>>>>>>>>> enable_routed_networks = false >>>>>>>>>>> generate_service_certificate = false >>>>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>>>> ipxe_enabled = true >>>>>>>>>>> local_interface = enp8s0 >>>>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>>>> subnets = ctlplane-subnet >>>>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>> undercloud_timezone = UTC >>>>>>>>>>> >>>>>>>>>>> [ctlplane-subnet] >>>>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>>>> >>>>>>>>>>> Can someone please help in this regard. >>>>>>>>>>> >>>>>>>>>>> Anirudh Gupta >>>>>>>>>>> >>>>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ~ Lokendra >>>>>>> www.inertiaspeaks.com >>>>>>> www.inertiagroups.com >>>>>>> skype: lokendrarathour >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> ~ Lokendra >>>>> www.inertiaspeaks.com >>>>> www.inertiagroups.com >>>>> skype: lokendrarathour >>>>> >>>>> >>>>> >>> >>> -- >>> ~ Lokendra >>> www.inertiaspeaks.com >>> www.inertiagroups.com >>> skype: lokendrarathour >>> >>> >>> > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 201716 bytes Desc: not available URL: From fungi at yuggoth.org Thu Jun 16 12:32:17 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 16 Jun 2022 12:32:17 +0000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: <20220616122309.z5zfhzdamogtlqkw@yuggoth.org> On 2022-06-16 16:49:40 +1000 (+1000), Brendan Shephard wrote: > It looks like my reply email was too big and has been blocked. [...] Not blocked, just held temporarily until a moderator could confirm it was a legitimate post (I okayed it a few minutes ago). I try to process the moderation queue at least once a day, but more volunteers are always welcome! ;) -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From josephine.seifert at secustack.com Thu Jun 16 13:56:43 2022 From: josephine.seifert at secustack.com (Josephine Seifert) Date: Thu, 16 Jun 2022 15:56:43 +0200 Subject: [Image Encryption] No meeting next week Message-ID: <93d61026-415f-07f1-ff7e-e835bd3fbe2e@secustack.com> Hi, there will be no meeting next monday. greetings Josephine (Luzi) -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 236 bytes Desc: OpenPGP digital signature URL: From tim+openstack.org at coote.org Thu Jun 16 14:12:55 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Thu, 16 Jun 2022 15:12:55 +0100 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: <0FE220F7-1C80-4604-8E77-ECB61CA6964A@coote.org> Isn?t there a standard library for parsing/encoding IP addresses per language used - this is a well known type problem, and if there isn?t a standard type, then the code is likely to have assumptions that are based on IPv4 and broken by IPv6 (e.g. that NAT is a good idea ;-) ) > On 16 Jun 2022, at 07:49, Brendan Shephard wrote: > > Hey, > > It looks like my reply email was too big and has been blocked. The problem is the square brackets in the IPv6 address being returned from the hiera command. Can you try this to workaround the problem: > Change this: > [root at tripleo-director ~]# egrep -r 'tftp_bind_host' /etc/puppet/hieradata/ > /etc/puppet/hieradata/service_configs.json: "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", > > To this: > "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" > > Then restart the service: > sudo systemctl restart tripleo_ironic_pxe_http.service tripleo_ironic_pxe_tftp.service > > I raised a bug for this for you: > https://bugs.launchpad.net/tripleo/+bug/1978892 > > Brendan Shephard > Software Engineer > Red Hat APAC > 193 N Quay > Brisbane City QLD 4000 > @RedHat Red Hat Red Hat > > > > On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour > wrote: > Hi Shephard, > I am getting the local_ip (ipv6) of the undercloud : > > [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > [aaaa:aaaa:aaaa::1] > > is this because of some ipv6 reasons? > > > On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard > wrote: > Hey, > > Ok, that command looks fine. What about that variable there? Do you get anything back when you run: > sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > > Mine returns: > sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > 192.168.24.115 > > Brendan Shephard > Software Engineer > Red Hat APAC > 193 N Quay > Brisbane City QLD 4000 > @RedHat Red Hat Red Hat > > > > On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour > wrote: > Hi Shephard, > > this is the command from my wallaby: > [root at undercloud ~]# sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > { > "cap_add": [ > "NET_ADMIN", > "NET_RAW", > "SETUID" > ], > "command": [ > "/bin/bash", > "-c", > "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" > ], > "environment": { > "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", > "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" > }, > "healthcheck": { > "test": "/openstack/healthcheck" > }, > "image": "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", > "net": "host", > "privileged": false, > "restart": "always", > "start_order": 90, > "volumes": [ > "/etc/hosts:/etc/hosts:ro", > "/etc/localtime:/etc/localtime:ro", > "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", > "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", > "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", > "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", > "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", > "/dev/log:/dev/log", > "/etc/puppet:/etc/puppet:ro", > "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", > "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", > "/var/lib/ironic:/var/lib/ironic:shared,z", > "/var/log/containers/ironic:/var/log/ironic:z", > "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" > ] > }[root at undercloud ~]# > > Comparing both, they look alike. > please check once. > > On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard > wrote: > Hi, > > Looks like the command was in a different file in Wallaby, can you check: > sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > > That one should have the dnsmasq command it's trying to run. For example, here it is from my Wallaby environment: > [stack at undercloud-0 ~]$ sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json | jq .command > [ > "/bin/bash", > "-c", > "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" > ] > > > > Brendan Shephard > Software Engineer > Red Hat APAC > 193 N Quay > Brisbane City QLD 4000 > @RedHat Red Hat Red Hat > > > > On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour > wrote: > Hi Shephard, > Here is the o/p of the file: > > [root at undercloud ~]# sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json > { > "config_files": [ > { > "dest": "/", > "merge": true, > "preserve_properties": true, > "source": "/var/lib/kolla/config_files/src/*" > } > ], > "permissions": [ > { > "owner": "ironic:ironic", > "path": "/var/log/ironic", > "recurse": true > }, > { > "owner": "ironic:ironic", > "path": "/var/lib/ironic", > "recurse": true > } > ] > }[root at undercloud ~]# > > > Thanks once agan. > > -Lokendra > > > On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard > wrote: > Looks like something wrong with the dnsmasq command the container is being launched with. What command is it trying to run? > > sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json > > Brendan Shephard > Software Engineer > Red Hat APAC > 193 N Quay > Brisbane City QLD 4000 > @RedHat Red Hat Red Hat > > > > On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta > wrote: > Hi Brendan, > > Thanks for your response. > > Please find the log below. > > [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp > > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > > [stack at undercloud t2u2v2w]$ sudo podman ps --filter name=ironic_pxe -a > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES > 02dacbc74cec undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) ironic_pxe_tftp > 1f8ca39fba32 undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo kolla_start 3 hours ago Up 3 hours ago (healthy) ironic_pxe_http > > Regards > Anirudh Gupta > > On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard > wrote: > Hey Anirudh, > > You would need to look at the logs for the ironic_pxe_tftp container to see why it's failing. > > I assume the tftp container is not Up when you run this command? > [stack at tripleo-director overcloud_playbooks]$ sudo podman ps --filter name=ironic_pxe -a > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES > 0170be36e291 registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo kolla_start 12 days ago Up 30 hours ago (healthy) ironic_pxe_tftp > e507f722bdf0 registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo kolla_start 12 days ago Up 30 hours ago (healthy) ironic_pxe_http > > Then check the logs to see what the error is: > [stack at tripleo-director overcloud_playbooks]$ sudo podman logs ironic_pxe_tftp > > > > Brendan Shephard > Software Engineer > Red Hat APAC > 193 N Quay > Brisbane City QLD 4000 > @RedHat Red Hat Red Hat > > > > On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta > wrote: > Hi Team, > > I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing the below error: > > 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | TASK | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 > 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | FATAL | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has not started yet"} > 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | TIMING | tripleo_container_manage : Manage container systemd > > Sample Undercloud.conf is as follows: > > [DEFAULT] > clean_nodes = true > cleanup = false > container_cli = podman > container_healthcheck_disabled = true > container_images_file = /home/stack/containers-prepare-parameter.yaml > deployment_user = stack > enable_ironic = true > enable_ironic_inspector = true > enable_neutron = true > enable_routed_networks = false > generate_service_certificate = false > ipv6_address_mode = dhcpv6-stateful > ipxe_enabled = true > local_interface = enp8s0 > local_ip = aaaa:aaaa:aaaa::1/64 > subnets = ctlplane-subnet > undercloud_admin_host = aaaa:aaaa:aaaa::1 > undercloud_hostname = undercloud.com > undercloud_ntp_servers = 30.30.30.3 > undercloud_public_host = aaaa:aaaa:aaaa::1 > undercloud_timezone = UTC > > [ctlplane-subnet] > cidr = aaaa:aaaa:aaaa::/64 > dhcp_end = aaaa:aaaa:aaaa::f > dhcp_start = aaaa:aaaa:aaaa::a > gateway = aaaa:aaaa:aaaa::1 > inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 > > Can someone please help in this regard. > > Anirudh Gupta > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlandy at redhat.com Thu Jun 16 14:57:27 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Thu, 16 Jun 2022 10:57:27 -0400 Subject: [TripleO] Gate blocker - tempest test error - 'Failed to attach network adapter device' Message-ID: Hello All, We have a new gate blocker on tripleo-ci-centos-9-standalone, scenario-010 and possibly other jobs. The jobs fail tempest test: tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic Details of the failure are in: https://bugs.launchpad.net/tripleo/+bug/1978969. We will reply to this thread as soon as we have more information and/or a fix. Please hold rechecks for now, Thank you, TripleO CI -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-francois.taltavull at elca.ch Thu Jun 16 15:45:55 2022 From: jean-francois.taltavull at elca.ch (=?utf-8?B?VGFsdGF2dWxsIEplYW4tRnJhbsOnb2lz?=) Date: Thu, 16 Jun 2022 15:45:55 +0000 Subject: [Ceph Rados Gateway] 403 when using S3 client In-Reply-To: <64208f0fd7344c0bab323b66c503f73a@elca.ch> References: <8b05dea37bd2412ab15d54ed806b2658@elca.ch> <612721648572028@mail.yandex.ru> <96f86768b73d456ba1937a9177c3831f@elca.ch> <64208f0fd7344c0bab323b66c503f73a@elca.ch> Message-ID: <54e6b45a4930418c95f0626ec78c3d1a@elca.ch> Hi Dmitriy, hi Jonathan, I finally managed to interact with RGW S3 API with ?s3cmd? client, but only if I add the option ?--signature-v2? to the command line. If I don?t, I get the message ?ERROR: S3 error: 403 (AccessDenied)?. The RGW is configured to use keystone as the users authority and it looks like the S3 auth requests including a version 4 signature were not supported. Is there a RGW or a Keystone configuration variable to enable S3 V4 signature ? Deployment characteristics: - OSA 23.2.0 - OpenStack Wallaby - Ceph and RGW Octopus Kind regards, Jean-Francois From: Taltavull Jean-Fran?ois Sent: mercredi, 30 mars 2022 11:01 To: 'Jonathan Rosser' ; openstack-discuss at lists.openstack.org Subject: RE: [Ceph Rados Gateway] 403 when using S3 client Hi Jonathan, The keystone URL is correct. HAProxy has been configured to handle this kind or URL. And everything works fine with the openstack client. From: Jonathan Rosser > Sent: mercredi, 30 mars 2022 10:44 To: openstack-discuss at lists.openstack.org Subject: Re: [Ceph Rados Gateway] 403 when using S3 client EXTERNAL MESSAGE - This email comes from outside ELCA companies. Hi Jean-Francois. I have the following difference to your config: rgw keystone url = http://xx.xx.xx.xx:5000 The normal OSA loadbalancer setup would have the keystone service on port 5000. Jonathan. On 30/03/2022 09:24, Taltavull Jean-Fran?ois wrote: Hi Dmitriy, I just tried with s3cmd but I still get a 403. Here is the rgw section of ceph.conf: rgw_keystone_url = http://xxxxx.xxxx.xxx/identity rgw_keystone_api_version = 3 rgw_keystone_admin_user = radosgw rgw_keystone_admin_password = xxxxxxxxxxxxxxxxxxxxxxxxx rgw_keystone_admin_project = service rgw_keystone_admin_domain = default rgw_keystone_accepted_roles = member, _member_, admin, swiftoperator rgw_keystone_accepted_admin_roles = ResellerAdmin rgw_keystone_implicit_tenants = true rgw_swift_account_in_url = true rgw_swift_versioning_enabled = true rgw_enable_apis = swift,s3 rgw_s3_auth_use_keystone = true From: Dmitriy Rabotyagov Sent: mardi, 29 mars 2022 18:49 To: openstack-discuss Subject: Re: [Ceph Rados Gateway] 403 when using S3 client EXTERNAL MESSAGE - This email comes from outside ELCA companies. - ??? Hi Jean-Francois. It's quite hard to understand what exactly could went wrong based on the information you've provided. Highly likely it's related to the RGW configuration itself and it's integration with keystone to be specific. Would be helpful if you could provide your ceph.conf regarding rgw configuration. I'm also not 100% sure if awscli does work with RGW... At least I always used s3cmd or rclone to interact with RGW S3 API. 29.03.2022, 16:36, "Taltavull Jean-Fran?ois" >: Hi All, I get an http 403 error code when I try to get the bucket list with Ubuntu (Focal) S3 client (awscli). S3 api has been activated in radosgw config file and EC2 credentials have been created and put in S3 client config file. Otherwise, everything is working fine with OpenStack client. My deployment: - OSA 23.2.0 - OpenStack Wallaby - Ceph and Rados GW Octopus Has any of you already experienced this kind of behaviour ? Many thanks, Jean-Francois -- Kind Regards, Dmitriy Rabotyagov -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephenfin at redhat.com Thu Jun 16 16:13:42 2022 From: stephenfin at redhat.com (Stephen Finucane) Date: Thu, 16 Jun 2022 17:13:42 +0100 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> Message-ID: <4d3f63840239c2533a060ed9596b57820cf3dfed.camel@redhat.com> On Wed, 2022-06-15 at 01:04 +0100, Sean Mooney wrote: > On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote: > > On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote: > > > ---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand wrote ---- > > > > On 6/13/22 00:10, Thomas Goirand wrote: > > > > > Hi, > > > > > > > > > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > > > > > However, OpenStack global requirements are still stuck at 3.2.0. Is > > > > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > > > > > > > I'd really appreciate if someone (else than me) was driving this... > > > > > > > > > > Cheers, > > > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > > > > FYI, Nova fails with it: > > > > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > > > > > > > Can someone from the Nova team investigate? > > > > > > Nova failures are due to the error message change (it happens regularly and they change them in most versions) > > > in jsonschema new version. I remember we faced this type of issue previously also and we updated > > > nova tests not to assert the error message but seems like there are a few left which is failing with > > > jsonschema 4.6.0. > > > > > > Along with these error message failures fixes and using the latest jsonschema, in nova, we use > > > Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check > > > what are backward-incompatible changes in Draft7Validator and bump this too? > > > > well i think the reason this is on 3.2.0 iis nothing to do with nova > > i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/ > > > > The conflict is caused by: > > The user requested jsonschema===4.0.1 > > tripleo-common 16.4.0 depends on jsonschema>=3.2.0 > > rsd-lib 1.2.0 depends on jsonschema>=2.6.0 > > taskflow 4.7.0 depends on jsonschema>=3.2.0 > > zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 > > os-net-config 15.2.0 depends on jsonschema>=3.2.0 > > task-core 0.2.1 depends on jsonschema>=3.2.0 > > python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 > > warlock 1.3.3 depends on jsonschema<4 and >=0.7 > > > > https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9 > > > > it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0 > glance client seams to be the only real user of this > https://codesearch.opendev.org/?q=warlock&i=nope&literal=nope&files=&excludeFiles=&repos= > perhaps we could jsut remove the dependcy? I've proposed vendoring this dependency in glanceclient [1]. I've also proposed a change to fix this in warlock [2] but given the lack of activity there, I doubt it'll merge anytime soon so the former sounds like a better option. Cheers, Stephen [1] https://review.opendev.org/c/openstack/python-glanceclient/+/846197 [2] https://github.com/bcwaldon/warlock/pull/65 > > > > > > > > -gmann > > > > > > > > > > > Cheers, > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > > > > > > From fungi at yuggoth.org Thu Jun 16 16:39:47 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 16 Jun 2022 16:39:47 +0000 Subject: [dev][infra][tact-sig] Updating Zuul's Default Ansible Version to Ansible v5 Message-ID: <20220616163947.37xhv6y5idchxtbw@yuggoth.org> Just a heads up, in case you haven't seen Clark's announcement on OpenDev's service-announce ML, the default Ansible version Zuul invokes to run jobs will be changing at the end of this month: https://lists.opendev.org/pipermail/service-announce/2022-June/000041.html Feel free to reach out with any concerns. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From miguel at mlavalle.com Thu Jun 16 17:00:26 2022 From: miguel at mlavalle.com (Miguel Lavalle) Date: Thu, 16 Jun 2022 12:00:26 -0500 Subject: [Neutron] What is the best tool for automated data plane testing with openstack? In-Reply-To: References: Message-ID: You might want to take a look at https://pyshaker.readthedocs.io/en/latest/ On Thu, Jun 16, 2022 at 2:08 AM Lajos Katona wrote: > Hi, > > In Openstack itself there is no such tool. > There are several test tools which we use to test Openstack and Openstack > components like > Tempest (see [1]) and the plugin for it are full with networking related > tests neutron-tempest-plugin (see [2]). > The scenario tests in neutron-tempest-plugin tests some basic dataplane > scenarios, but > only in a functional manner, not with traffic load or such, as we used > these tests in our CI, so > we need quick feedback. > To have some stress test you can check Rally (see [3]) but that is again > not for dataplane testing, > but to load the API of the components. You can find some Neutron specific > rally scenarios in > Neutron repo (see [4]). > > [1]: https://opendev.org/openstack/tempest > [2]: https://opendev.org/openstack/neutron-tempest-plugin > [3]: https://opendev.org/openstack/rally > [4]: https://opendev.org/openstack/neutron/src/branch/master/rally-jobs > > Lajos Katona (lajoskatona) > > ROBERTO BARTZEN ACOSTA ezt ?rta (id?pont: > 2022. j?n. 15., Sze, 13:04): > >> Hey there, >> >> I'm using OpenStack(old version) with neutron OVN CMS-plugin and I need >> to migrate to the Yoga release (Ubuntu ovn v22.03.0 and ovs v2.17.0). >> >> I'm interested in validating a series of fixed issues on openvswitch and >> ovn in the releases related to the OpenStack Yoga version (e.g. >> https://github.com/ovn-org/ovn/commit/ccbbbd0584e52c2aa88436a431fe7a6deffd2221 >> ). >> >> What OpenStack automated testing tool do you use or recommend to validate >> the data plane? >> >> Best regards >> Roberto >> >> >> >> *?Esta mensagem ? direcionada apenas para os endere?os constantes no >> cabe?alho inicial. Se voc? n?o est? listado nos endere?os constantes no >> cabe?alho, pedimos-lhe que desconsidere completamente o conte?do dessa >> mensagem e cuja c?pia, encaminhamento e/ou execu??o das a??es citadas est?o >> imediatamente anuladas e proibidas?.* >> >> *?Apesar do Magazine Luiza tomar todas as precau??es razo?veis para >> assegurar que nenhum v?rus esteja presente nesse e-mail, a empresa n?o >> poder? aceitar a responsabilidade por quaisquer perdas ou danos causados >> por esse e-mail ou por seus anexos?.* >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephenfin at redhat.com Thu Jun 16 17:53:05 2022 From: stephenfin at redhat.com (Stephen Finucane) Date: Thu, 16 Jun 2022 18:53:05 +0100 Subject: Upgrading to a more recent version of jsonschema In-Reply-To: <4d3f63840239c2533a060ed9596b57820cf3dfed.camel@redhat.com> References: <74f5fdba-8225-5f6a-a6f6-68853875d4f8@debian.org> <3a6170d4-e1fb-2988-e980-e8c152cb852b@debian.org> <181649f0df6.11d045b0f280764.1056849246214160471@ghanshyammann.com> <7fda4e895d6bb1d325c8b72522650c809bcc87f9.camel@redhat.com> <4d3f63840239c2533a060ed9596b57820cf3dfed.camel@redhat.com> Message-ID: <2707b10cbccab3e5a5a7930c1369727c896fde3a.camel@redhat.com> On Thu, 2022-06-16 at 17:13 +0100, Stephen Finucane wrote: > On Wed, 2022-06-15 at 01:04 +0100, Sean Mooney wrote: > > On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote: > > > On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote: > > > > ---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand wrote ---- > > > > > On 6/13/22 00:10, Thomas Goirand wrote: > > > > > > Hi, > > > > > > > > > > > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > > > > > > However, OpenStack global requirements are still stuck at 3.2.0. Is > > > > > > there any reason for it, or should we attempt to upgrade to 4.6.0? > > > > > > > > > > > > I'd really appreciate if someone (else than me) was driving this... > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > > > > > > > FYI, Nova fails with it: > > > > > https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz > > > > > > > > > > Can someone from the Nova team investigate? > > > > > > > > Nova failures are due to the error message change (it happens regularly and they change them in most versions) > > > > in jsonschema new version. I remember we faced this type of issue previously also and we updated > > > > nova tests not to assert the error message but seems like there are a few left which is failing with > > > > jsonschema 4.6.0. > > > > > > > > Along with these error message failures fixes and using the latest jsonschema, in nova, we use > > > > Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check > > > > what are backward-incompatible changes in Draft7Validator and bump this too? > > > > > > well i think the reason this is on 3.2.0 iis nothing to do with nova > > > i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/ > > > > > > The conflict is caused by: > > > The user requested jsonschema===4.0.1 > > > tripleo-common 16.4.0 depends on jsonschema>=3.2.0 > > > rsd-lib 1.2.0 depends on jsonschema>=2.6.0 > > > taskflow 4.7.0 depends on jsonschema>=3.2.0 > > > zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 > > > os-net-config 15.2.0 depends on jsonschema>=3.2.0 > > > task-core 0.2.1 depends on jsonschema>=3.2.0 > > > python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 > > > warlock 1.3.3 depends on jsonschema<4 and >=0.7 > > > > > > https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9 > > > > > > it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0 > > glance client seams to be the only real user of this > > https://codesearch.opendev.org/?q=warlock&i=nope&literal=nope&files=&excludeFiles=&repos= > > perhaps we could jsut remove the dependcy? > > I've proposed vendoring this dependency in glanceclient [1]. I've also proposed > a change to fix this in warlock [2] but given the lack of activity there, I > doubt it'll merge anytime soon so the former sounds like a better option. My efforts to collect *all* the projects continues. Just cut 2.0.0 of warlock so we should see this make it's way through the requirements machinery in the next few days. I'll abandon the glanceclient change now. Stephen > > Cheers, > Stephen > > [1] https://review.opendev.org/c/openstack/python-glanceclient/+/846197 > [2] https://github.com/bcwaldon/warlock/pull/65 > > > > > > > > > > > > -gmann > > > > > > > > > > > > > > Cheers, > > > > > > > > > > Thomas Goirand (zigo) > > > > > > > > > > > > > > > > > > > > > > From lokendrarathour at gmail.com Thu Jun 16 17:23:28 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Thu, 16 Jun 2022 22:53:28 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Team , Any support w.r.t the ipv6 based node provisioning for overcloud deployment. We tried but is getting failed with said error. -Lokendra On Thu, 16 Jun 2022, 14:51 Brendan Shephard, wrote: > Hey, > > Looks like your server is booting in Legacy BIOS mode but the PXE file > Ironic is offering is for UEFI. Did you try booting in UEFI mode instead of > Legacy BIOS? > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Thu, Jun 16, 2022 at 5:05 PM Lokendra Rathour < > lokendrarathour at gmail.com> wrote: > >> Hi Shephard, >> Thanks, after changing the details as mentioned, Undercloud got installed >> successfully. >> Now as a part to test the introspection we added a single node and >> initiated the introspection on which we are getting errors. >> IP as per the inspector range is getting allocated, but soon after the IP >> allocation the introspection ILO gives the below error: >> [image: image.png] >> it says, Downloading NBP file.... >> PXE-E99: Unexpected network error. >> >> Underlcoud.conf: >> >> undercloud_debug = true >> clean_nodes = true >> cleanup = false >> container_cli = podman >> container_healthcheck_disabled = true >> container_images_file = /home/stack/containers-prepare-parameter.yaml >> deployment_user = stack >> enable_heat = true >> enable_ironic = true >> enable_ironic_inspector = true >> enable_neutron = true >> generate_service_certificate = false >> enable_routed_networks = false >> ipv6_address_mode = dhcpv6-stateful >> ipxe_enabled = true >> ironic_default_network_interface = neutron >> ironic_enabled_network_interfaces = neutron,flat >> local_interface = enp8s0 >> local_ip = aaaa:aaaa:aaaa::1/64 >> subnets = ctlplane-subnet >> undercloud_admin_host = aaaa:aaaa:aaaa::1 >> undercloud_public_host = aaaa:aaaa:aaaa::1 >> undercloud_hostname = undercloud.com >> undercloud_ntp_servers = 30.30.30.3 >> undercloud_timezone = UTC >> [ctlplane-subnet] >> cidr = aaaa:aaaa:aaaa::/64 >> dhcp_end = aaaa:aaaa:aaaa::19 >> dhcp_start = aaaa:aaaa:aaaa::5 >> gateway = aaaa:aaaa:aaaa::1 >> inspection_iprange = aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40 >> >> >> the ironic config in the container: >> >> [root at undercloud /]# vi /etc/ironic-inspector/dnsmasq.conf >> port=0 >> interface=br-ctlplane >> >> >> dhcp-range=set:ctlplane-subnet,aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40,64,10m >> dhcp-option-force=tag:ctlplane-subnet,option:mtu,1500 >> dhcp-sequential-ip >> dhcp-match=ipxe,175 >> dhcp-match=set:efi,option:client-arch,7 >> dhcp-match=set:efi,option:client-arch,9 >> dhcp-match=set:efi,option:client-arch,11 >> # dhcpv6s for Client System Architecture Type (61) >> dhcp-match=set:efi6,option6:61,0007 >> dhcp-match=set:efi6,option6:61,0009 >> dhcp-match=set:efi6,option6:61,0011 >> dhcp-userclass=set:ipxe6,iPXE >> # Client is already running iPXE; move to next stage of chainloading >> dhcp-boot=tag:ipxe,http://[aaaa:aaaa:aaaa::1]:8088/inspector.ipxe >> dhcp-option=tag:ipxe6,option6:bootfile-url,http:// >> [aaaa:aaaa:aaaa::1]:8088/inspector.ipxe >> # Client is PXE booting over EFI without iPXE ROM; send EFI version of >> iPXE chainloader >> dhcp-boot=tag:efi,tag:!ipxe,snponly.efi >> >> dhcp-option=tag:efi6,tag:!ipxe6,option6:bootfile-url,tftp://[aaaa:aaaa:aaaa::1]/snponly.efi >> # Client is running PXE over BIOS; send BIOS version of iPXE chainloader >> dhcp-boot=undionly.kpxe,localhost.localdomain,aaaa:aaaa:aaaa::1 >> >> dhcp-hostsdir=/var/lib/ironic-inspector/dhcp-hostsdir >> >> >> Please check and help me with the possible error and resolution. >> >> Best Regards, >> Lokendra >> >> >> >> >> On Thu, Jun 16, 2022 at 5:15 AM Brendan Shephard >> wrote: >> >>> Hey, >>> >>> Looks like that is the problem. The [ ] around the IP address are >>> causing the issue. If I try to run dnsmasq using exactly the output you >>> get, it gives me the same error: >>> [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground >>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>> --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 >>> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >>> >>> dnsmasq: bad command line options: try --help >>> >>> VS without the [ ] I can see it starts up normally. >>> >>> The settings in your undercloud.conf file look to be correct I believe. >>> So I think there might be a bug here. I don't think we should be saving >>> that value with the square brackets, or we would need to filter them out >>> when we gather the value in that variable. >>> >>> I raised a bug for it here so that we can dig into this and find what >>> needs fixing: >>> https://bugs.launchpad.net/tripleo/+bug/1978892 >>> >>> In the meantime, if you edit that hieradata value, are you able to get >>> that container started? >>> >>> Change this: >>> [root at tripleo-director ~]# egrep -r 'tftp_bind_host' >>> /etc/puppet/hieradata/ >>> /etc/puppet/hieradata/service_configs.json: >>> "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", >>> >>> To this: >>> "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" >>> >>> Then restart the service: >>> sudo systemctl restart tripleo_ironic_pxe_http.service >>> tripleo_ironic_pxe_tftp.service >>> >>> Does that get the container running without the error? I did the same in >>> my environment and can see that dnsmasq is running properly like that: >>> [root at tripleo-director ~]# ps -ef | grep aaaa >>> root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq >>> --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root >>> --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 >>> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >>> >>> Brendan Shephard >>> >>> Software Engineer >>> >>> Red Hat APAC >>> >>> 193 N Quay >>> >>> Brisbane City QLD 4000 >>> @RedHat Red Hat >>> Red Hat >>> >>> >>> >>> >>> >>> On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour < >>> lokendrarathour at gmail.com> wrote: >>> >>>> Hi Shephard, >>>> I am getting the local_ip (ipv6) of the undercloud : >>>> >>>> [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c >>>> /etc/puppet/hiera.yaml >>>> [aaaa:aaaa:aaaa::1] >>>> >>>> is this because of some ipv6 reasons? >>>> >>>> >>>> On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard >>>> wrote: >>>> >>>>> Hey, >>>>> >>>>> Ok, that command looks fine. What about that variable there? Do you >>>>> get anything back when you run: >>>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>>> >>>>> Mine returns: >>>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>>> 192.168.24.115 >>>>> >>>>> Brendan Shephard >>>>> >>>>> Software Engineer >>>>> >>>>> Red Hat APAC >>>>> >>>>> 193 N Quay >>>>> >>>>> Brisbane City QLD 4000 >>>>> @RedHat Red Hat >>>>> Red Hat >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >>>>> lokendrarathour at gmail.com> wrote: >>>>> >>>>>> Hi Shephard, >>>>>> >>>>>> this is the command from my wallaby: >>>>>> [root at undercloud ~]# sudo cat >>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>> { >>>>>> "cap_add": [ >>>>>> "NET_ADMIN", >>>>>> "NET_RAW", >>>>>> "SETUID" >>>>>> ], >>>>>> "command": [ >>>>>> "/bin/bash", >>>>>> "-c", >>>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>>> ], >>>>>> "environment": { >>>>>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>>>>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>>>>> }, >>>>>> "healthcheck": { >>>>>> "test": "/openstack/healthcheck" >>>>>> }, >>>>>> "image": >>>>>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>>>>> "net": "host", >>>>>> "privileged": false, >>>>>> "restart": "always", >>>>>> "start_order": 90, >>>>>> "volumes": [ >>>>>> "/etc/hosts:/etc/hosts:ro", >>>>>> "/etc/localtime:/etc/localtime:ro", >>>>>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>>>>> >>>>>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>>>>> >>>>>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>>>>> >>>>>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>>>>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>>>>> "/dev/log:/dev/log", >>>>>> "/etc/puppet:/etc/puppet:ro", >>>>>> >>>>>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>>>>> >>>>>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>>>>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>>>>> "/var/log/containers/ironic:/var/log/ironic:z", >>>>>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>>>>> ] >>>>>> }[root at undercloud ~]# >>>>>> >>>>>> Comparing both, they look alike. >>>>>> please check once. >>>>>> >>>>>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Looks like the command was in a different file in Wallaby, can you >>>>>>> check: >>>>>>> sudo cat >>>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>>> >>>>>>> That one should have the dnsmasq command it's trying to run. For >>>>>>> example, here it is from my Wallaby environment: >>>>>>> [stack at undercloud-0 ~]$ sudo cat >>>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>>> | jq .command >>>>>>> [ >>>>>>> "/bin/bash", >>>>>>> "-c", >>>>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>>>> ] >>>>>>> >>>>>>> >>>>>>> >>>>>>> Brendan Shephard >>>>>>> >>>>>>> Software Engineer >>>>>>> >>>>>>> Red Hat APAC >>>>>>> >>>>>>> 193 N Quay >>>>>>> >>>>>>> Brisbane City QLD 4000 >>>>>>> @RedHat Red Hat >>>>>>> Red Hat >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>> >>>>>>>> Hi Shephard, >>>>>>>> Here is the o/p of the file: >>>>>>>> >>>>>>>> [root at undercloud ~]# sudo cat >>>>>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>>> { >>>>>>>> "config_files": [ >>>>>>>> { >>>>>>>> "dest": "/", >>>>>>>> "merge": true, >>>>>>>> "preserve_properties": true, >>>>>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>>>>> } >>>>>>>> ], >>>>>>>> "permissions": [ >>>>>>>> { >>>>>>>> "owner": "ironic:ironic", >>>>>>>> "path": "/var/log/ironic", >>>>>>>> "recurse": true >>>>>>>> }, >>>>>>>> { >>>>>>>> "owner": "ironic:ironic", >>>>>>>> "path": "/var/lib/ironic", >>>>>>>> "recurse": true >>>>>>>> } >>>>>>>> ] >>>>>>>> }[root at undercloud ~]# >>>>>>>> >>>>>>>> >>>>>>>> Thanks once agan. >>>>>>>> >>>>>>>> -Lokendra >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard < >>>>>>>> bshephar at redhat.com> wrote: >>>>>>>> >>>>>>>>> Looks like something wrong with the dnsmasq command the container >>>>>>>>> is being launched with. What command is it trying to run? >>>>>>>>> >>>>>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>>>> >>>>>>>>> Brendan Shephard >>>>>>>>> >>>>>>>>> Software Engineer >>>>>>>>> >>>>>>>>> Red Hat APAC >>>>>>>>> >>>>>>>>> 193 N Quay >>>>>>>>> >>>>>>>>> Brisbane City QLD 4000 >>>>>>>>> @RedHat Red Hat >>>>>>>>> Red Hat >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Brendan, >>>>>>>>>> >>>>>>>>>> Thanks for your response. >>>>>>>>>> >>>>>>>>>> Please find the log below. >>>>>>>>>> >>>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>>>>> >>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>> >>>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>>>>> name=ironic_pxe -a >>>>>>>>>> CONTAINER ID IMAGE >>>>>>>>>> COMMAND CREATED >>>>>>>>>> STATUS PORTS NAMES >>>>>>>>>> 02dacbc74cec >>>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>>>>> ironic_pxe_tftp >>>>>>>>>> 1f8ca39fba32 >>>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>>>>> ironic_pxe_http >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> >>>>>>>>>> Anirudh Gupta >>>>>>>>>> >>>>>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>>>>> bshephar at redhat.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hey Anirudh, >>>>>>>>>>> >>>>>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>>>>> to see why it's failing. >>>>>>>>>>> >>>>>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>>>>> --filter name=ironic_pxe -a >>>>>>>>>>> CONTAINER ID IMAGE >>>>>>>>>>> COMMAND CREATED STATUS >>>>>>>>>>> PORTS NAMES >>>>>>>>>>> 0170be36e291 >>>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>>> ironic_pxe_tftp >>>>>>>>>>> e507f722bdf0 >>>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>>> ironic_pxe_http >>>>>>>>>>> >>>>>>>>>>> Then check the logs to see what the error is: >>>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>>>>> ironic_pxe_tftp >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Brendan Shephard >>>>>>>>>>> >>>>>>>>>>> Software Engineer >>>>>>>>>>> >>>>>>>>>>> Red Hat APAC >>>>>>>>>>> >>>>>>>>>>> 193 N Quay >>>>>>>>>>> >>>>>>>>>>> Brisbane City QLD 4000 >>>>>>>>>>> @RedHat Red Hat >>>>>>>>>>> Red Hat >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta < >>>>>>>>>>> anyrude10 at gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Team, >>>>>>>>>>>> >>>>>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>>>>>> facing the below error: >>>>>>>>>>>> >>>>>>>>>>>> 2022-06-14 05:01:23.213708 | >>>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TASK | Manage container systemd >>>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>>>>> 2022-06-14 05:03:22.912816 | >>>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | FATAL | Manage container systemd >>>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | >>>>>>>>>>>> error={"changed": false, "msg": "Service ironic_pxe_tftp has not started >>>>>>>>>>>> yet"} >>>>>>>>>>>> 2022-06-14 05:03:22.914400 | >>>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TIMING | tripleo_container_manage : >>>>>>>>>>>> Manage container systemd >>>>>>>>>>>> >>>>>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>>>>> >>>>>>>>>>>> [DEFAULT] >>>>>>>>>>>> clean_nodes = true >>>>>>>>>>>> cleanup = false >>>>>>>>>>>> container_cli = podman >>>>>>>>>>>> container_healthcheck_disabled = true >>>>>>>>>>>> container_images_file = >>>>>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>>>>> deployment_user = stack >>>>>>>>>>>> enable_ironic = true >>>>>>>>>>>> enable_ironic_inspector = true >>>>>>>>>>>> enable_neutron = true >>>>>>>>>>>> enable_routed_networks = false >>>>>>>>>>>> generate_service_certificate = false >>>>>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>>>>> ipxe_enabled = true >>>>>>>>>>>> local_interface = enp8s0 >>>>>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>>>>> subnets = ctlplane-subnet >>>>>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>>> undercloud_timezone = UTC >>>>>>>>>>>> >>>>>>>>>>>> [ctlplane-subnet] >>>>>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>>>>> >>>>>>>>>>>> Can someone please help in this regard. >>>>>>>>>>>> >>>>>>>>>>>> Anirudh Gupta >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ~ Lokendra >>>>>>>> www.inertiaspeaks.com >>>>>>>> www.inertiagroups.com >>>>>>>> skype: lokendrarathour >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> -- >>>>>> ~ Lokendra >>>>>> www.inertiaspeaks.com >>>>>> www.inertiagroups.com >>>>>> skype: lokendrarathour >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> ~ Lokendra >>>> www.inertiaspeaks.com >>>> www.inertiagroups.com >>>> skype: lokendrarathour >>>> >>>> >>>> >> >> -- >> ~ Lokendra >> www.inertiaspeaks.com >> www.inertiagroups.com >> skype: lokendrarathour >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 201716 bytes Desc: not available URL: From melwittt at gmail.com Thu Jun 16 22:11:54 2022 From: melwittt at gmail.com (melanie witt) Date: Thu, 16 Jun 2022 15:11:54 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> On Tue Jun 14 2022 08:15:57 GMT-0700 (Pacific Daylight Time), Sean Mooney wrote: > On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: >>> i hit the same issue the other day if you recall me asking about it. >>> didnt check my devstack logs to see if it installed or not but i can >>> see if i can repoduce it tomorrow. >> >> I checked with the OP and they have it installed properly, >> system-wide. I was assuming your issue might have been venv (or >> ordering) related, but theirs was not. There are some other weird things >> going on with their setup, so I'm not so sure. > nope, was not useing a vnev or anything special, it just devstack installed normaly more or less > but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes > stack.sh so i dont think there was anything specific ot how i was invoking it. I saw this today while trying to re-install (stack.sh) an existing devstack (unstack.sh and clean.sh). I did a fresh devstack after this and it worked without any problems. So maybe the behavior is intermittent? The error in the g-api log [1] during service startup was: Jun 16 19:25:26 node1 devstack at g-api.service[421305]: ERROR glance.common.wsgi sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter The format of the above plugin message is : [2] which would mean the group is 'sqlalchemy.plugins' and the name is 'dbcounterplugin=dbcounterplugin=dbcounter'. I thought the latter looked strange. I thought the name would just be 'dbcounter'. The dbcounter package was installed: $ pip show dbcounter Name: dbcounter Version: 0.1 Summary: A teeny tiny dbcounter plugin for use with devstack Home-page: http://github.com/openstack/devstack Author: Dan Smith Author-email: dms at danplanet.com License: Apache Location: /usr/local/lib/python3.8/dist-packages Requires: Required-by: When I tried something like this, it imported fine: $ python3 Python 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import dbcounter >>> dbcounter >>> Is it possible dbcounter was not installed at the time glance was starting up? I didn't think so given the timestamp on the package dir: $ ls -l /usr/local/lib/python3.8/dist-packages/|grep dbcounter drwxr-sr-x 2 root staff 4096 Jun 16 19:14 dbcounter-0.1.dist-info -rw-r--r-- 1 root staff 4580 Jun 16 19:14 dbcounter.py and the error above happened at Jun 16 19:25:26. I couldn't figure out anything so finally I tried restarting the g-api service and it started without any error. Subsequent restart also had no error. -melwitt [1] https://paste.openstack.org/show/bEgbMLyUDH7zBW2vj43F/ [2] https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_37/lib/sqlalchemy/util/langhelpers.py#L344 From dms at danplanet.com Thu Jun 16 22:48:45 2022 From: dms at danplanet.com (Dan Smith) Date: Thu, 16 Jun 2022 15:48:45 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> (melanie witt's message of "Thu, 16 Jun 2022 15:11:54 -0700") References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> Message-ID: > Is it possible dbcounter was not installed at the time glance was > starting up? I didn't think so given the timestamp on the package dir: Certainly doesn't seem like it, based on when we do it in the devstack sequence either. > I couldn't figure out anything so finally I tried restarting the g-api > service and it started without any error. Subsequent restart also had > no error. Huh, interesting. I haven't been able to repro this locally so I haven't been able to poke at it. Restarting it and having it work certainly supports your sequencing theory. I guess I'll try to chase that possibility tomorrow. If lots of people start hitting this, we might want to just disable by default and enable the performance gathering in our jobs. It's already been helpful a couple times for debugging resource usage in the gate. --Dan From gmann at ghanshyammann.com Thu Jun 16 22:54:12 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 16 Jun 2022 17:54:12 -0500 Subject: [all][tc][policy] RBAC discussion policy pop-up meeting on Tuesday 21 June (biweekly meeting) Message-ID: <1816eb96466.da95d46091740.8408609993794671056@ghanshyammann.com> Hello Everyone, As discussed in the last meeting, we were waiting for the operator feedback from various places including ops meetup in Berlin[1]. We have a good amount of feedback and I have summarized those in our central feedback etherpad. If you have any feedback from your customer, feel free to add it to the ethrpad before our next meeting on 21 June. - https://etherpad.opendev.org/p/rbac-operator-feedback#L45 We will discuss the feedback and decide on the next step in our next policy popup meeting on Tuesday 21 June 2022 14:00 UTC. We will have this meeting biweekly, I am also attaching the .ics file in this email. Meeting Details: - Joining Info: https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popup_Team#Meeting - Agenda: https://etherpad.opendev.org/p/rbac-zed-ptg#L97 [1] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028878.html -gmann -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenStack-policy-popup-meeting.ics Type: application/octet-stream Size: 1896 bytes Desc: not available URL: From fungi at yuggoth.org Thu Jun 16 23:06:05 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 16 Jun 2022 23:06:05 +0000 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> Message-ID: <20220616230605.tf4fcef2opouwpfx@yuggoth.org> On 2022-06-16 15:11:54 -0700 (-0700), melanie witt wrote: [...] > Jun 16 19:25:26 node1 devstack at g-api.service[421305]: ERROR > glance.common.wsgi sqlalchemy.exc.NoSuchModuleError: Can't load plugin: > sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter > > The format of the above plugin message is : [2] which would > mean the group is 'sqlalchemy.plugins' and the name is > 'dbcounterplugin=dbcounterplugin=dbcounter'. I thought the latter looked > strange. I thought the name would just be 'dbcounter'. [...] It looks like copies of this may be getting repeatedly concatenated into the plugin variable: https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a215031ea85cb1f5e205/lib/databases/mysql#L231 -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Thu Jun 16 23:08:42 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 16 Jun 2022 23:08:42 +0000 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <20220616230605.tf4fcef2opouwpfx@yuggoth.org> References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> <20220616230605.tf4fcef2opouwpfx@yuggoth.org> Message-ID: <20220616230842.f7rgxij5hejfg2vm@yuggoth.org> On 2022-06-16 23:06:05 +0000 (+0000), Jeremy Stanley wrote: > On 2022-06-16 15:11:54 -0700 (-0700), melanie witt wrote: > [...] > > Jun 16 19:25:26 node1 devstack at g-api.service[421305]: ERROR > > glance.common.wsgi sqlalchemy.exc.NoSuchModuleError: Can't load plugin: > > sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter > > > > The format of the above plugin message is : [2] which would > > mean the group is 'sqlalchemy.plugins' and the name is > > 'dbcounterplugin=dbcounterplugin=dbcounter'. I thought the latter looked > > strange. I thought the name would just be 'dbcounter'. > [...] > > It looks like copies of this may be getting repeatedly concatenated > into the plugin variable: > > https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a215031ea85cb1f5e205/lib/databases/mysql#L231 Perhaps the shell is interpolating &plugin and filling it with the existing value of plugin each time that function is called, resulting in nested copies of the value? -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From dms at danplanet.com Thu Jun 16 23:56:21 2022 From: dms at danplanet.com (Dan Smith) Date: Thu, 16 Jun 2022 16:56:21 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <20220616230605.tf4fcef2opouwpfx@yuggoth.org> (Jeremy Stanley's message of "Thu, 16 Jun 2022 23:06:05 +0000") References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> <20220616230605.tf4fcef2opouwpfx@yuggoth.org> Message-ID: > It looks like copies of this may be getting repeatedly concatenated > into the plugin variable: > > https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a215031ea85cb1f5e205/lib/databases/mysql#L231 I don't think that's actually happening. If it was, restarting glance wouldn't help (as Melanie indicated) and it also should never really work. I was thinking that was more likely an artifact of the error message or something. Looking at the config files, they seem right. For example: https://zuul.opendev.org/t/openstack/build/ff5a7a6fc87840579b102b244f28200f/log/controller/logs/etc/glance/glance-api_conf.txt#21 --Dan From melwittt at gmail.com Fri Jun 17 02:02:46 2022 From: melwittt at gmail.com (melanie witt) Date: Thu, 16 Jun 2022 19:02:46 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <61f26a90-159d-9fe6-c79a-a1d5a5dac7ad@gmail.com> References: <61f26a90-159d-9fe6-c79a-a1d5a5dac7ad@gmail.com> Message-ID: <9394a5cd-557c-1616-c0b2-688e23316b4a@gmail.com> I had tried to send this earlier with devstack output attached without thinking about the large size of the text. Sorry :( On Thu Jun 16 2022 16:56:21 GMT-0700 (Pacific Daylight Time), Dan Smith wrote: >> It looks like copies of this may be getting repeatedly concatenated >> into the plugin variable: >> >> https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a215031ea85cb1f5e205/lib/databases/mysql#L231 > > I don't think that's actually happening. If it was, restarting glance > wouldn't help (as Melanie indicated) and it also should never really > work. I was thinking that was more likely an artifact of the error > message or something. Looking at the config files, they seem right. For > example: > > https://zuul.opendev.org/t/openstack/build/ff5a7a6fc87840579b102b244f28200f/log/controller/logs/etc/glance/glance-api_conf.txt#21 I didn't notice this before but before things error with glance, other projects had no problem loading the dbcounter plugin, keystone-manage: > +lib/keystone:init_keystone:484 /usr/local/bin/keystone-manage --config-file /etc/keystone/keystone.conf db_sync > INFO dbcounter [-] Registered counter for database keystone > DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=371448) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} > DEBUG dbcounter [-] [371448] Writer thread running {{(pid=371448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Yet glance fails to load it during two separate glance-manage calls: > +lib/glance:init_glance:489 /usr/local/bin/glance-manage --config-file /etc/glance/glance-api.conf db_sync > CRITICAL glance [-] Unhandled error: sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter [...] > +lib/glance:init_glance:492 /usr/local/bin/glance-manage --config-file /etc/glance/glance-api.conf db_load_metadefs > CRITICAL glance [-] Unhandled error: sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter Then cinder-manage loads it fine: > +lib/cinder:init_cinder:440 /usr/local/bin/cinder-manage --config-file /etc/cinder/cinder.conf db sync > INFO dbcounter [-] Registered counter for database cinder > DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=391785) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} > DEBUG dbcounter [-] [391785] Writer thread running {{(pid=391785) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Then nova-manage: > +lib/nova:init_nova:843 /usr/local/bin/nova-manage --config-file /etc/nova/nova.conf api_db sync > Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. > INFO dbcounter [-] Registered counter for database nova_api > DEBUG dbcounter [-] [394510] Writer thread running {{(pid=394510) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} [...] > INFO dbcounter [-] Registered counter for database nova_cell0 > DEBUG dbcounter [-] [394520] Writer thread running {{(pid=394520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} [...] > INFO dbcounter [-] Registered counter for database nova_cell1 > DEBUG dbcounter [-] [394531] Writer thread running {{(pid=394531) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Finally glance fails to load the plugin in the g-api service when an image upload is requested: > +functions:_upload_image:121 openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create cirros-0.5.2-x86_64-disk --public --container-format bare --disk-format qcow2 --property hw_rng_model=virtio > /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead > from cryptography.utils import int_from_bytes > /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead > from cryptography.utils import int_from_bytes > HttpException: 500: Server Error for url: http://192.168.42.11/image/v2/images, The server has either erred or is incapable of performing the requested operation.: 500 Internal Server Error With this ^ I realized that restarting g-api might not have been a good indicator of anything resolving because in this example the plugin load failure occurs while checking quota for an inbound request. I'm not sure whether services load the dbcounter passively when they start or if it's only when database accesses are initiated. I also don't understand how or why one project can't load the plugin but the rest can. And I wonder whether it's a coincidence that it was glance in the two examples we have so far. -melwitt From katonalala at gmail.com Fri Jun 17 04:18:03 2022 From: katonalala at gmail.com (Lajos Katona) Date: Fri, 17 Jun 2022 06:18:03 +0200 Subject: [neutron] Drivers meeting agenda - 17.06.2022. Message-ID: Hi Neutron Drivers, The agenda for tomorrow's drivers meeting is at [1]. [RFE] Improve performance of bulk port create/delete with networking-generic-switch (#linkhttps:// bugs.launchpad.net/neutron/+bug/1976270) [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda See you at the meeting tomorrow. Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From akekane at redhat.com Fri Jun 17 05:35:54 2022 From: akekane at redhat.com (Abhishek Kekane) Date: Fri, 17 Jun 2022 11:05:54 +0530 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <9394a5cd-557c-1616-c0b2-688e23316b4a@gmail.com> References: <61f26a90-159d-9fe6-c79a-a1d5a5dac7ad@gmail.com> <9394a5cd-557c-1616-c0b2-688e23316b4a@gmail.com> Message-ID: On Fri, Jun 17, 2022 at 7:37 AM melanie witt wrote: > I had tried to send this earlier with devstack output attached without > thinking about the large size of the text. Sorry :( > > On Thu Jun 16 2022 16:56:21 GMT-0700 (Pacific Daylight Time), Dan Smith > wrote: > >> It looks like copies of this may be getting repeatedly concatenated > >> into the plugin variable: > >> > >> > https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a215031ea85cb1f5e205/lib/databases/mysql#L231 > > > > I don't think that's actually happening. If it was, restarting glance > > wouldn't help (as Melanie indicated) and it also should never really > > work. I was thinking that was more likely an artifact of the error > > message or something. Looking at the config files, they seem right. For > > example: > > > > > https://zuul.opendev.org/t/openstack/build/ff5a7a6fc87840579b102b244f28200f/log/controller/logs/etc/glance/glance-api_conf.txt#21 > > I didn't notice this before but before things error with glance, other > projects had no problem loading the dbcounter plugin, keystone-manage: > > > +lib/keystone:init_keystone:484 > /usr/local/bin/keystone-manage --config-file /etc/keystone/keystone.conf > db_sync > > INFO dbcounter [-] Registered counter for database keystone > > DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to > STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION > {{(pid=371448) _check_effective_sql_mode > /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} > > DEBUG dbcounter [-] [371448] Writer thread running {{(pid=371448) > stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} > > Yet glance fails to load it during two separate glance-manage calls: > > > +lib/glance:init_glance:489 /usr/local/bin/glance-manage > --config-file /etc/glance/glance-api.conf db_sync > > CRITICAL glance [-] Unhandled error: sqlalchemy.exc.NoSuchModuleError: > Can't load plugin: > sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter > [...] > > +lib/glance:init_glance:492 /usr/local/bin/glance-manage > --config-file /etc/glance/glance-api.conf db_load_metadefs > > CRITICAL glance [-] Unhandled error: sqlalchemy.exc.NoSuchModuleError: > Can't load plugin: > sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter > > Then cinder-manage loads it fine: > > > +lib/cinder:init_cinder:440 /usr/local/bin/cinder-manage > --config-file /etc/cinder/cinder.conf db sync > > INFO dbcounter [-] Registered counter for database cinder > > DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to > STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION > {{(pid=391785) _check_effective_sql_mode > /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} > > DEBUG dbcounter [-] [391785] Writer thread running {{(pid=391785) > stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} > > Then nova-manage: > > > +lib/nova:init_nova:843 /usr/local/bin/nova-manage > --config-file /etc/nova/nova.conf api_db sync > > Modules with known eventlet monkey patching issues were imported prior > to eventlet monkey patching: urllib3. This warning can usually be ignored > if the caller is only importing and not executing nova code. > > INFO dbcounter [-] Registered counter for database nova_api > > DEBUG dbcounter [-] [394510] Writer thread running {{(pid=394510) > stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} > [...] > > INFO dbcounter [-] Registered counter for database nova_cell0 > > DEBUG dbcounter [-] [394520] Writer thread running {{(pid=394520) > stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} > [...] > > INFO dbcounter [-] Registered counter for database nova_cell1 > > DEBUG dbcounter [-] [394531] Writer thread running {{(pid=394531) > stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} > > Finally glance fails to load the plugin in the g-api service when an > image upload is requested: > > > +functions:_upload_image:121 openstack > --os-cloud=devstack-admin --os-region-name=RegionOne image create > cirros-0.5.2-x86_64-disk --public --container-format bare --disk-format > qcow2 --property hw_rng_model=virtio > > /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: > CryptographyDeprecationWarning: int_from_bytes is deprecated, use > int.from_bytes instead > > from cryptography.utils import int_from_bytes > > /usr/lib/python3/dist-packages/secretstorage/util.py:19: > CryptographyDeprecationWarning: int_from_bytes is deprecated, use > int.from_bytes instead > > from cryptography.utils import int_from_bytes > > HttpException: 500: Server Error for url: > http://192.168.42.11/image/v2/images, The server has either erred or is > incapable of performing the requested operation.: 500 Internal Server Error > > With this ^ I realized that restarting g-api might not have been a good > indicator of anything resolving because in this example the plugin load > failure occurs while checking quota for an inbound request. I'm not sure > whether services load the dbcounter passively when they start or if it's > only when database accesses are initiated. > > I also don't understand how or why one project can't load the plugin but > the rest can. And I wonder whether it's a coincidence that it was glance > in the two examples we have so far. > > -melwitt > > I was hitting with the same error but as a workaround so far Dan (Smith) suggested a re-stack with "MYSQL_GATHER_PERFORMANCE=False" and it worked. This setting will avoid trying to load that plugin - abhishekk -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Fri Jun 17 06:13:57 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 17 Jun 2022 08:13:57 +0200 Subject: [Neutron] What is the best tool for automated data plane testing with openstack? In-Reply-To: References: Message-ID: <206935683.H36mYAsI3Y@p1> Hi, For scenario tests there is also Tobiko project https://opendev.org/x/tobiko - we are using it in one periodic job in Neutron and we also are using it in our d/s CI. Dnia czwartek, 16 czerwca 2022 09:07:41 CEST Lajos Katona pisze: > Hi, > > In Openstack itself there is no such tool. > There are several test tools which we use to test Openstack and Openstack > components like > Tempest (see [1]) and the plugin for it are full with networking related > tests neutron-tempest-plugin (see [2]). > The scenario tests in neutron-tempest-plugin tests some basic dataplane > scenarios, but > only in a functional manner, not with traffic load or such, as we used > these tests in our CI, so > we need quick feedback. > To have some stress test you can check Rally (see [3]) but that is again > not for dataplane testing, > but to load the API of the components. You can find some Neutron specific > rally scenarios in > Neutron repo (see [4]). > > [1]: https://opendev.org/openstack/tempest > [2]: https://opendev.org/openstack/neutron-tempest-plugin > [3]: https://opendev.org/openstack/rally > [4]: https://opendev.org/openstack/neutron/src/branch/master/rally-jobs > > Lajos Katona (lajoskatona) > > ROBERTO BARTZEN ACOSTA ezt ?rta (id?pont: > 2022. j?n. 15., Sze, 13:04): > > > Hey there, > > > > I'm using OpenStack(old version) with neutron OVN CMS-plugin and I need to > > migrate to the Yoga release (Ubuntu ovn v22.03.0 and ovs v2.17.0). > > > > I'm interested in validating a series of fixed issues on openvswitch and > > ovn in the releases related to the OpenStack Yoga version (e.g. > > https://github.com/ovn-org/ovn/commit/ccbbbd0584e52c2aa88436a431fe7a6deffd2221 > > ). > > > > What OpenStack automated testing tool do you use or recommend to validate > > the data plane? > > > > Best regards > > Roberto > > > > > > > > *?Esta mensagem ? direcionada apenas para os endere?os constantes no > > cabe?alho inicial. Se voc? n?o est? listado nos endere?os constantes no > > cabe?alho, pedimos-lhe que desconsidere completamente o conte?do dessa > > mensagem e cuja c?pia, encaminhamento e/ou execu??o das a??es citadas est?o > > imediatamente anuladas e proibidas?.* > > > > *?Apesar do Magazine Luiza tomar todas as precau??es razo?veis para > > assegurar que nenhum v?rus esteja presente nesse e-mail, a empresa n?o > > poder? aceitar a responsabilidade por quaisquer perdas ou danos causados > > por esse e-mail ou por seus anexos?.* > > > -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From rdhasman at redhat.com Fri Jun 17 08:01:10 2022 From: rdhasman at redhat.com (Rajat Dhasmana) Date: Fri, 17 Jun 2022 13:31:10 +0530 Subject: [cinder] festival of XS reviews 17th June 2022 Message-ID: Hello Argonauts, We will be having our monthly festival of XS reviews today i.e. 17th June (Friday) from 1400-1600 UTC. Following are some additional details: Date: 17th June, 2022 Time: 1400-1600 UTC Meeting link: https://meetpad.opendev.org/cinder-festival-of-reviews etherpad: https://etherpad.opendev.org/p/cinder-festival-of-reviews Thanks and regards Rajat Dhasmana -------------- next part -------------- An HTML attachment was scrubbed... URL: From gibi at redhat.com Fri Jun 17 09:54:31 2022 From: gibi at redhat.com (Balazs Gibizer) Date: Fri, 17 Jun 2022 11:54:31 +0200 Subject: [nova]tempest-integrated-compute-centos-9-stream is broken since yesterday Message-ID: Hi, Please hold you rechecks as $subject is 100% fail on master with error: libvirt.libvirtError: internal error: unable to execute QEMU command 'netdev_add': File descriptor named '(null)' has not been found Cheers, gibi [1]https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-compute-centos-9-stream&skip=0 [2]https://bugs.launchpad.net/nova/+bug/1979047 From gibi at redhat.com Fri Jun 17 10:07:47 2022 From: gibi at redhat.com (Balazs Gibizer) Date: Fri, 17 Jun 2022 12:07:47 +0200 Subject: [nova]tempest-integrated-compute-centos-9-stream is broken since yesterday In-Reply-To: References: Message-ID: On Fri, Jun 17 2022 at 11:54:31 AM +02:00:00, Balazs Gibizer wrote: > Hi, > > Please hold you rechecks as $subject is 100% fail on master with > error: > > libvirt.libvirtError: internal error: unable to execute QEMU command > 'netdev_add': File descriptor named '(null)' has not been found We are turning the job to non-voting to unblock the gate [3]. > > Cheers, > gibi > > [1]https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-compute-centos-9-stream&skip=0 > [2]https://bugs.launchpad.net/nova/+bug/1979047 > [3] https://review.opendev.org/c/openstack/nova/+/846292 From arnaud.morin at gmail.com Fri Jun 17 10:14:52 2022 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Fri, 17 Jun 2022 10:14:52 +0000 Subject: [neutron] Drivers meeting agenda - 17.06.2022. In-Reply-To: References: Message-ID: hey team, Do you mind if I join and talk about: https://bugs.launchpad.net/neutron/+bug/1975603 ? Thanks On 17.06.22 - 06:18, Lajos Katona wrote: > Hi Neutron Drivers, > > The agenda for tomorrow's drivers meeting is at [1]. > [RFE] Improve performance of bulk port create/delete with > networking-generic-switch (#linkhttps:// > bugs.launchpad.net/neutron/+bug/1976270) > > [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda > > See you at the meeting tomorrow. > Lajos Katona (lajoskatona) From gibi at redhat.com Fri Jun 17 10:46:34 2022 From: gibi at redhat.com (Balazs Gibizer) Date: Fri, 17 Jun 2022 12:46:34 +0200 Subject: [nova] nova-live-migraton and nova-grenade-multinode is in bad shape on master Message-ID: Hi, Since yesterday we started seeing both $subject jobs to fail with [1]. [1]https://bugs.launchpad.net/tempest/+bug/1979052 From rlandy at redhat.com Fri Jun 17 11:13:22 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Fri, 17 Jun 2022 07:13:22 -0400 Subject: [TripleO] Gate blocker - tempest test error - 'Failed to attach network adapter device' In-Reply-To: References: Message-ID: On Thu, Jun 16, 2022 at 10:57 AM Ronelle Landy wrote: > Hello All, > > We have a new gate blocker on tripleo-ci-centos-9-standalone, scenario-010 > and possibly other jobs. The jobs fail tempest test: > tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic > > Details of the failure are in: > https://bugs.launchpad.net/tripleo/+bug/1978969. > > We will reply to this thread as soon as we have more information and/or a > fix. > A workaround has been merged to address the tripleo-ci-centos-9-standalone failure - https://review.opendev.org/c/openstack/tripleo-quickstart/+/846194 . We are following another gate blocker now: https://bugs.launchpad.net/tripleo/+bug/1978997 A workaround there is under test. > Please hold rechecks for now, > > Thank you, > TripleO CI > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gibi at redhat.com Fri Jun 17 11:42:42 2022 From: gibi at redhat.com (Balazs Gibizer) Date: Fri, 17 Jun 2022 13:42:42 +0200 Subject: [nova] nova-live-migraton and nova-grenade-multinode is in bad shape on master In-Reply-To: References: Message-ID: <6VDMDR.FIQQXC1QVYHU2@redhat.com> On Fri, Jun 17 2022 at 12:46:34 PM +02:00:00, Balazs Gibizer wrote: > Hi, > > Since yesterday we started seeing both $subject jobs to fail with [1]. A recently landed tempest change is being reverted[2] to fix it. > > > > [1]https://bugs.launchpad.net/tempest/+bug/1979052 > > [2] https://review.opendev.org/c/openstack/tempest/+/846251 From hjensas at redhat.com Fri Jun 17 12:23:09 2022 From: hjensas at redhat.com (Harald Jensas) Date: Fri, 17 Jun 2022 14:23:09 +0200 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi, The error in the screenshot, the booting node is not able to dowload the network boot program from the TFTP server on the undercloud. Can you: * Verify that the TFTP server is running? * Check the log file - /var/log/containers/ironic/dnsmasq.log * Try to capture the TFTP traffic on br-ctlplane to a PCAP file to see if we can use that to dignose the problem. * download that file manually? For example: curl -O tftp:///snponly.efi ? Best Regards Harald On 6/16/22 09:05, Lokendra Rathour wrote: > Hi Shephard, > Thanks, after changing the details as mentioned, Undercloud got > installed successfully. > Now as a part to test the introspection we added a single node and > initiated the introspection on which we are getting errors. > IP as per the inspector range is getting allocated, but soon after the > IP allocation the introspection?ILO gives?the below error: > image.png > it says, Downloading NBP file.... > PXE-E99: Unexpected network error. > > Underlcoud.conf: > > undercloud_debug = true > clean_nodes = true > cleanup = false > container_cli = podman > container_healthcheck_disabled = true > container_images_file = /home/stack/containers-prepare-parameter.yaml > deployment_user = stack > enable_heat = true > enable_ironic = true > enable_ironic_inspector = true > enable_neutron = true > generate_service_certificate = false > enable_routed_networks = false > ipv6_address_mode = dhcpv6-stateful > ipxe_enabled = true > ironic_default_network_interface = neutron > ironic_enabled_network_interfaces = neutron,flat > local_interface = enp8s0 > local_ip = aaaa:aaaa:aaaa::1/64 > subnets = ctlplane-subnet > undercloud_admin_host = aaaa:aaaa:aaaa::1 > undercloud_public_host = aaaa:aaaa:aaaa::1 > undercloud_hostname = undercloud.com > undercloud_ntp_servers = 30.30.30.3 > undercloud_timezone = UTC > [ctlplane-subnet] > cidr = aaaa:aaaa:aaaa::/64 > dhcp_end = aaaa:aaaa:aaaa::19 > dhcp_start = aaaa:aaaa:aaaa::5 > gateway = aaaa:aaaa:aaaa::1 > inspection_iprange = aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40 > > > the ironic config in the container: > > [root at undercloud /]# vi /etc/ironic-inspector/dnsmasq.conf > port=0 > interface=br-ctlplane > > dhcp-range=set:ctlplane-subnet,aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40,64,10m > dhcp-option-force=tag:ctlplane-subnet,option:mtu,1500 > dhcp-sequential-ip > dhcp-match=ipxe,175 > dhcp-match=set:efi,option:client-arch,7 > dhcp-match=set:efi,option:client-arch,9 > dhcp-match=set:efi,option:client-arch,11 > # dhcpv6s for Client System Architecture Type (61) > dhcp-match=set:efi6,option6:61,0007 > dhcp-match=set:efi6,option6:61,0009 > dhcp-match=set:efi6,option6:61,0011 > dhcp-userclass=set:ipxe6,iPXE > # Client is already running iPXE; move to next stage of chainloading > dhcp-boot=tag:ipxe,http://[aaaa:aaaa:aaaa::1]:8088/inspector.ipxe > dhcp-option=tag:ipxe6,option6:bootfile-url,http://[aaaa:aaaa:aaaa::1]:8088/inspector.ipxe > # Client is PXE booting over EFI without iPXE ROM; send EFI version of > iPXE chainloader > dhcp-boot=tag:efi,tag:!ipxe,snponly.efi > dhcp-option=tag:efi6,tag:!ipxe6,option6:bootfile-url,tftp://[aaaa:aaaa:aaaa::1]/snponly.efi > # Client is running PXE over BIOS; send BIOS version of iPXE chainloader > dhcp-boot=undionly.kpxe,localhost.localdomain,aaaa:aaaa:aaaa::1 > > dhcp-hostsdir=/var/lib/ironic-inspector/dhcp-hostsdir > > > Please check and help me with the possible error and resolution. > > Best Regards, > Lokendra > > > > > On Thu, Jun 16, 2022 at 5:15 AM Brendan Shephard > wrote: > > Hey, > > Looks like that is the problem. The [ ] around the IP address are > causing the issue. If I try to run dnsmasq using exactly the output > you get, it gives me the same error: > [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > dnsmasq: bad command line options: try --help > > VS without the [ ] I can see it starts up normally. > > The settings in your undercloud.conf file look to be correct I > believe. So I think there might be a?bug here. I don't think we > should be saving that value with the square brackets, or we would > need to filter them out when we gather the value in that variable. > > I raised a bug for it here so that we can dig into this and find > what needs fixing: > https://bugs.launchpad.net/tripleo/+bug/1978892 > > > In the meantime, if you edit that hieradata value, are you able to > get that container started? > > Change this: > [root at tripleo-director ~]# egrep -r 'tftp_bind_host' > /etc/puppet/hieradata/ > /etc/puppet/hieradata/service_configs.json: > ?"ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", > > To this: > ?"ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" > > Then restart the service: > sudo systemctl restart tripleo_ironic_pxe_http.service > tripleo_ironic_pxe_tftp.service > > Does that get the container running without the error? I did the > same in my environment and can see that dnsmasq is running properly > like that: > [root at tripleo-director ~]# ps -ef | grep aaaa > root ? ? ? 71180 ? 52675 ?0 19:24 pts/4 ? ?00:00:00 > /usr/sbin/dnsmasq --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > > @RedHat Red Hat > Red Hat > > > > > > > On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour > > wrote: > > Hi Shephard, > I am getting the local_ip (ipv6) of the undercloud : > > [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host > -c /etc/puppet/hiera.yaml > [aaaa:aaaa:aaaa::1] > > is this because of some ipv6 reasons? > > > On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard > > wrote: > > Hey, > > Ok, that command looks fine. What about that variable there? > Do you get anything back when you run: > sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > > Mine returns: > sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > 192.168.24.115 > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > > @RedHat Red Hat > Red Hat > > > > > > > On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour > > wrote: > > Hi Shephard, > > this is the command from my wallaby: > [root at undercloud ~]# sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > { > ? "cap_add": [ > ? ? "NET_ADMIN", > ? ? "NET_RAW", > ? ? "SETUID" > ? ], > ? "command": [ > ? ? "/bin/bash", > ? ? "-c", > ? ? "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq > --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=$BIND_HOST > --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" > ? ], > ? "environment": { > ? ? "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", > ? ? "TRIPLEO_CONFIG_HASH": > "9fb3e4e0e35ee35fdf74cfccb16a7543" > ? }, > ? "healthcheck": { > ? ? "test": "/openstack/healthcheck" > ? }, > ? "image": > "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", > ? "net": "host", > ? "privileged": false, > ? "restart": "always", > ? "start_order": 90, > ? "volumes": [ > ? ? "/etc/hosts:/etc/hosts:ro", > ? ? "/etc/localtime:/etc/localtime:ro", > > "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", > > "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", > > "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", > > "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", > ? ? "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", > ? ? "/dev/log:/dev/log", > ? ? "/etc/puppet:/etc/puppet:ro", > > "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", > > "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", > ? ? "/var/lib/ironic:/var/lib/ironic:shared,z", > ? ? "/var/log/containers/ironic:/var/log/ironic:z", > ? ? "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" > ? ] > }[root at undercloud ~]# > > Comparing both, they look alike. > please check once. > > On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard > > wrote: > > Hi, > > Looks like the command was in a different file in > Wallaby, can you check: > sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > > That one should have the dnsmasq command it's trying > to run. For example, here it is from my Wallaby > environment: > [stack at undercloud-0 ~]$ sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > | jq .command > [ > ? "/bin/bash", > ? "-c", > ? "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq > --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log > --user=root --conf-file=/dev/null > --listen-address=$BIND_HOST --port=0 --enable-tftp > --tftp-root=/var/lib/ironic/tftpboot" > ] > > > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > > @RedHat Red Hat > Red Hat > > > > > > > On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour > > wrote: > > Hi Shephard, > Here is the o/p of the file: > > [root at undercloud ~]# sudo cat > /var/lib/kolla/config_files/ironic_pxe_tftp.json > { > ? "config_files": [ > ? ? { > ? ? ? "dest": "/", > ? ? ? "merge": true, > ? ? ? "preserve_properties": true, > ? ? ? "source": "/var/lib/kolla/config_files/src/*" > ? ? } > ? ], > ? "permissions": [ > ? ? { > ? ? ? "owner": "ironic:ironic", > ? ? ? "path": "/var/log/ironic", > ? ? ? "recurse": true > ? ? }, > ? ? { > ? ? ? "owner": "ironic:ironic", > ? ? ? "path": "/var/lib/ironic", > ? ? ? "recurse": true > ? ? } > ? ] > }[root at undercloud ~]# > > > Thanks once agan. > > -Lokendra > > > On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard > > wrote: > > Looks like something wrong with the dnsmasq > command the container is being launched > with. What command is it trying to run? > > sudo cat > /var/lib/kolla/config_files/ironic_pxe_tftp.json > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > > @RedHat Red Hat > > Red Hat > > > > > > On Wed, Jun 15, 2022 at 6:22 PM Anirudh > Gupta > wrote: > > Hi Brendan, > > Thanks for your response. > > Please find the log below. > > [stack at undercloud t2u2v2w]$ sudo podman > logs ironic_pxe_tftp > > dnsmasq: bad command line options: try > --help > dnsmasq: bad command line options: try > --help > dnsmasq: bad command line options: try > --help > dnsmasq: bad command line options: try > --help > dnsmasq: bad command line options: try > --help > dnsmasq: bad command line options: try > --help > > [stack at undercloud t2u2v2w]$ ?sudo podman > ps --filter name=ironic_pxe -a > CONTAINER ID ?IMAGE > > ? ? ? ? ? ? ? ? ? ? ? ? COMMAND > ? ? ? CREATED ? ? ?STATUS > ? ? ? ? ? ? ? ?PORTS ? ? ? NAMES > 02dacbc74cec > ?undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo ?/bin/bash -c BIND... ?3 hours ago ?Exited (1) 3 hours ago (unhealthy) ? ? ? ? ? ? ?ironic_pxe_tftp > 1f8ca39fba32 > ?undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo ?kolla_start ? ? ? ? ? 3 hours ago ?Up 3 hours ago (healthy) ? ? ? ? ? ? ? ? ? ? ? ?ironic_pxe_http > > > Regards > > Anirudh Gupta > > > On Wed, Jun 15, 2022 at 11:30 AM Brendan > Shephard > wrote: > > Hey Anirudh, > > You would need to look at the logs > for the ironic_pxe_tftp container to > see why it's failing. > > I assume the tftp container is not > Up when you run this command? > [stack at tripleo-director > overcloud_playbooks]$ sudo podman ps > --filter name=ironic_pxe -a > CONTAINER ID ?IMAGE > > > COMMAND ? ? ?CREATED ? ? ?STATUS > ? ? ? ? ? ? ? ? PORTS ? ? ? NAMES > 0170be36e291 > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > ?kolla_start ?12 days ago ?Up 30 > hours ago (healthy) > ?ironic_pxe_tftp > e507f722bdf0 > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > ?kolla_start ?12 days ago ?Up 30 > hours ago (healthy) > ?ironic_pxe_http > > Then check the logs to see what the > error is: > [stack at tripleo-director > overcloud_playbooks]$ sudo podman > logs ironic_pxe_tftp > > > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > > @RedHat > Red Hat > > Red Hat > > > > > > > On Wed, Jun 15, 2022 at 7:53 AM > Anirudh Gupta > wrote: > > Hi Team, > > I am trying to deploy Openstack > Wallaby Undercloud on IPv6, but > facing the below error: > > 2022-06-14 05:01:23.213708 | > 52540083-cfa2-3f20-e9dc-00000000286f > | TASK | Manage container > systemd services and cleanup old > systemd healthchecks for > /var/lib/tripleo-config/container-startup-config/step_4 > 2022-06-14 05:03:22.912816 | > 52540083-cfa2-3f20-e9dc-00000000286f > | FATAL | Manage container > systemd services and cleanup old > systemd healthchecks for > /var/lib/tripleo-config/container-startup-config/step_4 > | undercloud | error={"changed": > false, "msg": "Service > ironic_pxe_tftp has not started > yet"} > 2022-06-14 05:03:22.914400 | > 52540083-cfa2-3f20-e9dc-00000000286f > | TIMING | > tripleo_container_manage : > Manage container systemd > > Sample Undercloud.conf is as > follows: > > [DEFAULT] > clean_nodes = true > cleanup = false > container_cli = podman > container_healthcheck_disabled = > true > container_images_file = > /home/stack/containers-prepare-parameter.yaml > deployment_user = stack > enable_ironic = true > enable_ironic_inspector = true > enable_neutron = true > enable_routed_networks = false > generate_service_certificate = false > ipv6_address_mode = dhcpv6-stateful > ipxe_enabled = true > local_interface = enp8s0 > local_ip = aaaa:aaaa:aaaa::1/64 > subnets = ctlplane-subnet > undercloud_admin_host = > aaaa:aaaa:aaaa::1 > undercloud_hostname = > undercloud.com > > undercloud_ntp_servers = 30.30.30.3 > undercloud_public_host = > aaaa:aaaa:aaaa::1 > undercloud_timezone = UTC > > [ctlplane-subnet] > cidr = aaaa:aaaa:aaaa::/64 > dhcp_end = aaaa:aaaa:aaaa::f > dhcp_start = aaaa:aaaa:aaaa::a > gateway = aaaa:aaaa:aaaa::1 > inspection_iprange = > aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 > > Can someone please help in this > regard. > > Anirudh Gupta > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > From bshephar at redhat.com Fri Jun 17 04:38:42 2022 From: bshephar at redhat.com (Brendan Shephard) Date: Fri, 17 Jun 2022 14:38:42 +1000 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hey, I'm not too sure what that issue could be there then. Probably best if you raise a bug so that you can attach logs and the team that looks after Ironic will be able to help you out. https://bugs.launchpad.net/tripleo If you can attach the logs from /var/log/containers/, we should be able to find some additional clues there. Brendan Shephard Software Engineer Red Hat APAC 193 N Quay Brisbane City QLD 4000 @RedHat Red Hat Red Hat On Fri, Jun 17, 2022 at 3:23 AM Lokendra Rathour wrote: > Hi Team , > Any support w.r.t the ipv6 based node provisioning for overcloud > deployment. > > We tried but is getting failed with said error. > > -Lokendra > > On Thu, 16 Jun 2022, 14:51 Brendan Shephard, wrote: > >> Hey, >> >> Looks like your server is booting in Legacy BIOS mode but the PXE file >> Ironic is offering is for UEFI. Did you try booting in UEFI mode instead of >> Legacy BIOS? >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Thu, Jun 16, 2022 at 5:05 PM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Hi Shephard, >>> Thanks, after changing the details as mentioned, Undercloud got >>> installed successfully. >>> Now as a part to test the introspection we added a single node and >>> initiated the introspection on which we are getting errors. >>> IP as per the inspector range is getting allocated, but soon after the >>> IP allocation the introspection ILO gives the below error: >>> [image: image.png] >>> it says, Downloading NBP file.... >>> PXE-E99: Unexpected network error. >>> >>> Underlcoud.conf: >>> >>> undercloud_debug = true >>> clean_nodes = true >>> cleanup = false >>> container_cli = podman >>> container_healthcheck_disabled = true >>> container_images_file = /home/stack/containers-prepare-parameter.yaml >>> deployment_user = stack >>> enable_heat = true >>> enable_ironic = true >>> enable_ironic_inspector = true >>> enable_neutron = true >>> generate_service_certificate = false >>> enable_routed_networks = false >>> ipv6_address_mode = dhcpv6-stateful >>> ipxe_enabled = true >>> ironic_default_network_interface = neutron >>> ironic_enabled_network_interfaces = neutron,flat >>> local_interface = enp8s0 >>> local_ip = aaaa:aaaa:aaaa::1/64 >>> subnets = ctlplane-subnet >>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>> undercloud_hostname = undercloud.com >>> undercloud_ntp_servers = 30.30.30.3 >>> undercloud_timezone = UTC >>> [ctlplane-subnet] >>> cidr = aaaa:aaaa:aaaa::/64 >>> dhcp_end = aaaa:aaaa:aaaa::19 >>> dhcp_start = aaaa:aaaa:aaaa::5 >>> gateway = aaaa:aaaa:aaaa::1 >>> inspection_iprange = aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40 >>> >>> >>> the ironic config in the container: >>> >>> [root at undercloud /]# vi /etc/ironic-inspector/dnsmasq.conf >>> port=0 >>> interface=br-ctlplane >>> >>> >>> dhcp-range=set:ctlplane-subnet,aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40,64,10m >>> dhcp-option-force=tag:ctlplane-subnet,option:mtu,1500 >>> dhcp-sequential-ip >>> dhcp-match=ipxe,175 >>> dhcp-match=set:efi,option:client-arch,7 >>> dhcp-match=set:efi,option:client-arch,9 >>> dhcp-match=set:efi,option:client-arch,11 >>> # dhcpv6s for Client System Architecture Type (61) >>> dhcp-match=set:efi6,option6:61,0007 >>> dhcp-match=set:efi6,option6:61,0009 >>> dhcp-match=set:efi6,option6:61,0011 >>> dhcp-userclass=set:ipxe6,iPXE >>> # Client is already running iPXE; move to next stage of chainloading >>> dhcp-boot=tag:ipxe,http://[aaaa:aaaa:aaaa::1]:8088/inspector.ipxe >>> dhcp-option=tag:ipxe6,option6:bootfile-url,http:// >>> [aaaa:aaaa:aaaa::1]:8088/inspector.ipxe >>> # Client is PXE booting over EFI without iPXE ROM; send EFI version of >>> iPXE chainloader >>> dhcp-boot=tag:efi,tag:!ipxe,snponly.efi >>> >>> dhcp-option=tag:efi6,tag:!ipxe6,option6:bootfile-url,tftp://[aaaa:aaaa:aaaa::1]/snponly.efi >>> # Client is running PXE over BIOS; send BIOS version of iPXE chainloader >>> dhcp-boot=undionly.kpxe,localhost.localdomain,aaaa:aaaa:aaaa::1 >>> >>> dhcp-hostsdir=/var/lib/ironic-inspector/dhcp-hostsdir >>> >>> >>> Please check and help me with the possible error and resolution. >>> >>> Best Regards, >>> Lokendra >>> >>> >>> >>> >>> On Thu, Jun 16, 2022 at 5:15 AM Brendan Shephard >>> wrote: >>> >>>> Hey, >>>> >>>> Looks like that is the problem. The [ ] around the IP address are >>>> causing the issue. If I try to run dnsmasq using exactly the output you >>>> get, it gives me the same error: >>>> [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground >>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>> --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 >>>> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >>>> >>>> dnsmasq: bad command line options: try --help >>>> >>>> VS without the [ ] I can see it starts up normally. >>>> >>>> The settings in your undercloud.conf file look to be correct I believe. >>>> So I think there might be a bug here. I don't think we should be saving >>>> that value with the square brackets, or we would need to filter them out >>>> when we gather the value in that variable. >>>> >>>> I raised a bug for it here so that we can dig into this and find what >>>> needs fixing: >>>> https://bugs.launchpad.net/tripleo/+bug/1978892 >>>> >>>> In the meantime, if you edit that hieradata value, are you able to get >>>> that container started? >>>> >>>> Change this: >>>> [root at tripleo-director ~]# egrep -r 'tftp_bind_host' >>>> /etc/puppet/hieradata/ >>>> /etc/puppet/hieradata/service_configs.json: >>>> "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", >>>> >>>> To this: >>>> "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" >>>> >>>> Then restart the service: >>>> sudo systemctl restart tripleo_ironic_pxe_http.service >>>> tripleo_ironic_pxe_tftp.service >>>> >>>> Does that get the container running without the error? I did the same >>>> in my environment and can see that dnsmasq is running properly like that: >>>> [root at tripleo-director ~]# ps -ef | grep aaaa >>>> root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq >>>> --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>> --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 >>>> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >>>> >>>> Brendan Shephard >>>> >>>> Software Engineer >>>> >>>> Red Hat APAC >>>> >>>> 193 N Quay >>>> >>>> Brisbane City QLD 4000 >>>> @RedHat Red Hat >>>> Red Hat >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour < >>>> lokendrarathour at gmail.com> wrote: >>>> >>>>> Hi Shephard, >>>>> I am getting the local_ip (ipv6) of the undercloud : >>>>> >>>>> [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c >>>>> /etc/puppet/hiera.yaml >>>>> [aaaa:aaaa:aaaa::1] >>>>> >>>>> is this because of some ipv6 reasons? >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard >>>>> wrote: >>>>> >>>>>> Hey, >>>>>> >>>>>> Ok, that command looks fine. What about that variable there? Do you >>>>>> get anything back when you run: >>>>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>>>> >>>>>> Mine returns: >>>>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>>>> 192.168.24.115 >>>>>> >>>>>> Brendan Shephard >>>>>> >>>>>> Software Engineer >>>>>> >>>>>> Red Hat APAC >>>>>> >>>>>> 193 N Quay >>>>>> >>>>>> Brisbane City QLD 4000 >>>>>> @RedHat Red Hat >>>>>> Red Hat >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >>>>>> lokendrarathour at gmail.com> wrote: >>>>>> >>>>>>> Hi Shephard, >>>>>>> >>>>>>> this is the command from my wallaby: >>>>>>> [root at undercloud ~]# sudo cat >>>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>>> { >>>>>>> "cap_add": [ >>>>>>> "NET_ADMIN", >>>>>>> "NET_RAW", >>>>>>> "SETUID" >>>>>>> ], >>>>>>> "command": [ >>>>>>> "/bin/bash", >>>>>>> "-c", >>>>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>>>> ], >>>>>>> "environment": { >>>>>>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>>>>>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>>>>>> }, >>>>>>> "healthcheck": { >>>>>>> "test": "/openstack/healthcheck" >>>>>>> }, >>>>>>> "image": >>>>>>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>>>>>> "net": "host", >>>>>>> "privileged": false, >>>>>>> "restart": "always", >>>>>>> "start_order": 90, >>>>>>> "volumes": [ >>>>>>> "/etc/hosts:/etc/hosts:ro", >>>>>>> "/etc/localtime:/etc/localtime:ro", >>>>>>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>>>>>> >>>>>>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>>>>>> >>>>>>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>>>>>> >>>>>>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>>>>>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>>>>>> "/dev/log:/dev/log", >>>>>>> "/etc/puppet:/etc/puppet:ro", >>>>>>> >>>>>>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>>>>>> >>>>>>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>>>>>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>>>>>> "/var/log/containers/ironic:/var/log/ironic:z", >>>>>>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>>>>>> ] >>>>>>> }[root at undercloud ~]# >>>>>>> >>>>>>> Comparing both, they look alike. >>>>>>> please check once. >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard < >>>>>>> bshephar at redhat.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Looks like the command was in a different file in Wallaby, can you >>>>>>>> check: >>>>>>>> sudo cat >>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>>>> >>>>>>>> That one should have the dnsmasq command it's trying to run. For >>>>>>>> example, here it is from my Wallaby environment: >>>>>>>> [stack at undercloud-0 ~]$ sudo cat >>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>>>> | jq .command >>>>>>>> [ >>>>>>>> "/bin/bash", >>>>>>>> "-c", >>>>>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>>>>> ] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Brendan Shephard >>>>>>>> >>>>>>>> Software Engineer >>>>>>>> >>>>>>>> Red Hat APAC >>>>>>>> >>>>>>>> 193 N Quay >>>>>>>> >>>>>>>> Brisbane City QLD 4000 >>>>>>>> @RedHat Red Hat >>>>>>>> Red Hat >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>>>>>> lokendrarathour at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Shephard, >>>>>>>>> Here is the o/p of the file: >>>>>>>>> >>>>>>>>> [root at undercloud ~]# sudo cat >>>>>>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>>>> { >>>>>>>>> "config_files": [ >>>>>>>>> { >>>>>>>>> "dest": "/", >>>>>>>>> "merge": true, >>>>>>>>> "preserve_properties": true, >>>>>>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>>>>>> } >>>>>>>>> ], >>>>>>>>> "permissions": [ >>>>>>>>> { >>>>>>>>> "owner": "ironic:ironic", >>>>>>>>> "path": "/var/log/ironic", >>>>>>>>> "recurse": true >>>>>>>>> }, >>>>>>>>> { >>>>>>>>> "owner": "ironic:ironic", >>>>>>>>> "path": "/var/lib/ironic", >>>>>>>>> "recurse": true >>>>>>>>> } >>>>>>>>> ] >>>>>>>>> }[root at undercloud ~]# >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks once agan. >>>>>>>>> >>>>>>>>> -Lokendra >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard < >>>>>>>>> bshephar at redhat.com> wrote: >>>>>>>>> >>>>>>>>>> Looks like something wrong with the dnsmasq command the container >>>>>>>>>> is being launched with. What command is it trying to run? >>>>>>>>>> >>>>>>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>>>>> >>>>>>>>>> Brendan Shephard >>>>>>>>>> >>>>>>>>>> Software Engineer >>>>>>>>>> >>>>>>>>>> Red Hat APAC >>>>>>>>>> >>>>>>>>>> 193 N Quay >>>>>>>>>> >>>>>>>>>> Brisbane City QLD 4000 >>>>>>>>>> @RedHat Red Hat >>>>>>>>>> Red Hat >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta < >>>>>>>>>> anyrude10 at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Brendan, >>>>>>>>>>> >>>>>>>>>>> Thanks for your response. >>>>>>>>>>> >>>>>>>>>>> Please find the log below. >>>>>>>>>>> >>>>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>>>>>> >>>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>>>> >>>>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>>>>>> name=ironic_pxe -a >>>>>>>>>>> CONTAINER ID IMAGE >>>>>>>>>>> COMMAND CREATED >>>>>>>>>>> STATUS PORTS NAMES >>>>>>>>>>> 02dacbc74cec >>>>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>>>>>> ironic_pxe_tftp >>>>>>>>>>> 1f8ca39fba32 >>>>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>>>>>> ironic_pxe_http >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> >>>>>>>>>>> Anirudh Gupta >>>>>>>>>>> >>>>>>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>>>>>> bshephar at redhat.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey Anirudh, >>>>>>>>>>>> >>>>>>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>>>>>> to see why it's failing. >>>>>>>>>>>> >>>>>>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>>>>>> --filter name=ironic_pxe -a >>>>>>>>>>>> CONTAINER ID IMAGE >>>>>>>>>>>> COMMAND CREATED STATUS >>>>>>>>>>>> PORTS NAMES >>>>>>>>>>>> 0170be36e291 >>>>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>>>> ironic_pxe_tftp >>>>>>>>>>>> e507f722bdf0 >>>>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>>>> ironic_pxe_http >>>>>>>>>>>> >>>>>>>>>>>> Then check the logs to see what the error is: >>>>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>>>>>> ironic_pxe_tftp >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Brendan Shephard >>>>>>>>>>>> >>>>>>>>>>>> Software Engineer >>>>>>>>>>>> >>>>>>>>>>>> Red Hat APAC >>>>>>>>>>>> >>>>>>>>>>>> 193 N Quay >>>>>>>>>>>> >>>>>>>>>>>> Brisbane City QLD 4000 >>>>>>>>>>>> @RedHat Red Hat >>>>>>>>>>>> Red Hat >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta < >>>>>>>>>>>> anyrude10 at gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Team, >>>>>>>>>>>>> >>>>>>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, >>>>>>>>>>>>> but facing the below error: >>>>>>>>>>>>> >>>>>>>>>>>>> 2022-06-14 05:01:23.213708 | >>>>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TASK | Manage container systemd >>>>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>>>>>> 2022-06-14 05:03:22.912816 | >>>>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | FATAL | Manage container systemd >>>>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | >>>>>>>>>>>>> error={"changed": false, "msg": "Service ironic_pxe_tftp has not started >>>>>>>>>>>>> yet"} >>>>>>>>>>>>> 2022-06-14 05:03:22.914400 | >>>>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TIMING | tripleo_container_manage : >>>>>>>>>>>>> Manage container systemd >>>>>>>>>>>>> >>>>>>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>>>>>> >>>>>>>>>>>>> [DEFAULT] >>>>>>>>>>>>> clean_nodes = true >>>>>>>>>>>>> cleanup = false >>>>>>>>>>>>> container_cli = podman >>>>>>>>>>>>> container_healthcheck_disabled = true >>>>>>>>>>>>> container_images_file = >>>>>>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>>>>>> deployment_user = stack >>>>>>>>>>>>> enable_ironic = true >>>>>>>>>>>>> enable_ironic_inspector = true >>>>>>>>>>>>> enable_neutron = true >>>>>>>>>>>>> enable_routed_networks = false >>>>>>>>>>>>> generate_service_certificate = false >>>>>>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>>>>>> ipxe_enabled = true >>>>>>>>>>>>> local_interface = enp8s0 >>>>>>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>>>>>> subnets = ctlplane-subnet >>>>>>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>>>> undercloud_timezone = UTC >>>>>>>>>>>>> >>>>>>>>>>>>> [ctlplane-subnet] >>>>>>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>>>>>> >>>>>>>>>>>>> Can someone please help in this regard. >>>>>>>>>>>>> >>>>>>>>>>>>> Anirudh Gupta >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ~ Lokendra >>>>>>>>> www.inertiaspeaks.com >>>>>>>>> www.inertiagroups.com >>>>>>>>> skype: lokendrarathour >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ~ Lokendra >>>>>>> www.inertiaspeaks.com >>>>>>> www.inertiagroups.com >>>>>>> skype: lokendrarathour >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> ~ Lokendra >>>>> www.inertiaspeaks.com >>>>> www.inertiagroups.com >>>>> skype: lokendrarathour >>>>> >>>>> >>>>> >>> >>> -- >>> ~ Lokendra >>> www.inertiaspeaks.com >>> www.inertiagroups.com >>> skype: lokendrarathour >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 201716 bytes Desc: not available URL: From kkchn.in at gmail.com Fri Jun 17 10:16:25 2022 From: kkchn.in at gmail.com (KK CHN) Date: Fri, 17 Jun 2022 15:46:25 +0530 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: I have tried the Trove Devstack installation on Ubuntu20.04 too.. But no success.. ( As I have made no progress in Debian 11 Trove-Devstak installation, I tried a number of times but all failed with the error ). To my experience.. The same error for Ubuntu 20.04 also. Followed the same stuff : https://docs.openstack.org/trove/latest/install/install-devstack.html The compete Error log here https://paste.openstack.org/show/bwSXZXHRtHWHnaOO2Fb4/ It reports an Error starting with 2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 because these package versions have conflicting dependencies. 2022-06-17 09:58:04.721 | 2022-06-17 09:58:04.721 | The conflict is caused by: 2022-06-17 09:58:04.721 | trove 17.1.0.dev31 depends on Jinja2>=2.10 2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2 How to overcome this Jinja dependency, where we need to edit this and what jinja suitable ? 2022-06-17 09:58:04.721 | And finally Exit with error 2022-06-17 09:58:07.560 | + /usr/local/lib/python3.8/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 : exit 1 Error on exit stack at kk:~/devstack$ uname -a Linux kk 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux stack at kk:~/devstack$ On Thu, Jun 16, 2022 at 11:51 AM KK CHN wrote: > Update to my Devstack installation Error. > > I have crossed this Internal Error ( HTTP 500) and ended up with a new > error for the Trove Devstack Installation attempts. > ################################################## > 2022-06-16 05:46:40.183 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:26 > : total_kB=16393076 > 2022-06-16 05:46:40.188 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:29 > : RAM_NEEDED=4 > 2022-06-16 05:46:40.192 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 > : '[' 16393076 -lt 4194304 ']' > 2022-06-16 05:46:40.197 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 > : return 0 > 2022-06-16 05:46:40.202 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:236 > : timeout 120 sh -c 'while ! sudo umount -f /tmp/dib_image.WMpKoawa; do > sleep 1; done' > 2022-06-16 05:46:40.225 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:241 > : rm -rf --one-file-system /tmp/dib_image.WMpKoawa > 2022-06-16 05:46:40.231 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 > : exit 1 > Error on exit > # Warning: iptables-legacy tables present, use iptables-legacy to see them > # Warning: iptables-legacy tables present, use iptables-legacy to see them > # Warning: iptables-legacy tables present, use iptables-legacy to see them > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ > > ############################################################################# > > I am able to login to the Horizon dash board inspite the ./stack.sh > script Exit with the above error status.. And in the image section, I am > able to view cirros-0.5.2-x86_64-disk > > image in the dashboard. > > But on the terminal when I issue the command DEVSTACK_TROVE/devstack$ > openstack datastore version list mysql it says > " Datastore 'mysql' cannot found. HTTP(404). > > #############Here the tty output > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ source ./openrc > WARNING: setting legacy OS_TENANT_NAME to support cli tools. > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore > version list mysql > Datastore 'mysql' cannot be found. (HTTP 404) > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack server list > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack image list > > +--------------------------------------+--------------------------+--------+ > | ID | Name | Status > | > > +--------------------------------------+--------------------------+--------+ > | be0c8769-286c-4b61-b989-a037e0e9601c | cirros-0.5.2-x86_64-disk | active > | > > +--------------------------------------+--------------------------+--------+ > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore > version list mysql > Datastore 'mysql' cannot be found. (HTTP 404) > > ####################################################################################### > > > So I ended up with the TROVE installation failing ? Or what does this > error mean ? How can I fix this to get a devstack installed successfully > with Trove-DbaaS feature up and running ? > > > Any help / directions much appreciated .... > > [ How I rectify the earlier errors of HTTP 500 : I just created a VM > again with same debian 11 image and 'stack' user created with /opt/stack > dir and freshly git cloned devstack to DEVSTACK_TROVE/ Dir.. > Then created the same local.conf file which I posted in my earlier mail. > and reran ./stack.sh script . This time I was able to get the > dashboard up and be able to login(earlier it's not possible to get the > dashboard too) ] > > Krish > > > > > > On Wed, Jun 15, 2022 at 12:31 PM KK CHN wrote: > >> This is my latest devstack installation error.. No clues why it >> fails repeatedly; earlier it was during the uploading stage of the cirros >> image, internal server error ( HTTP 500 ) occurred. This was repeated >> multiple times. So I posted for the first time in this thread. >> >> In this iteration for starting afresh, I have removed the stack user and >> /opt/stack directory manually, and freshly created 'stack' user with sudo >> permissions and /opt/stack directory with stack:stack ownership, and >> fresh git cloned(git clone https://opendev.org/openstack/devstack ) to >> /opt/DEVSTACK_stack folder and created local.conf ( attached here) . >> Then executed ./stack.sh . >> >> Got the following error this time ... Any clues why it fails here ? >> Platform (on a VM with Debian uname output) : Linux XenaCtrl1 >> 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux >> >> [image: Trove_installationError.png] >> >> On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney wrote: >> >>> On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: >>> > > i hit the same issue the other day if you recall me asking about it. >>> > > didnt check my devstack logs to see if it installed or not but i can >>> > > see if i can repoduce it tomorrow. >>> > >>> > I checked with the OP and they have it installed properly, >>> > system-wide. I was assuming your issue might have been venv (or >>> > ordering) related, but theirs was not. There are some other weird >>> things >>> > going on with their setup, so I'm not so sure. >>> nope, was not useing a vnev or anything special, it just devstack >>> installed normaly more or less >>> but driven by the ansible roles. the run devstack rull litrally just >>> opens a shell and invokes >>> stack.sh so i dont think there was anything specific ot how i was >>> invoking it. >>> >>> > >>> > --Dan >>> > >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Trove_installationError.png Type: image/png Size: 88093 bytes Desc: not available URL: From dms at danplanet.com Fri Jun 17 13:41:06 2022 From: dms at danplanet.com (Dan Smith) Date: Fri, 17 Jun 2022 06:41:06 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <61f26a90-159d-9fe6-c79a-a1d5a5dac7ad@gmail.com> (melanie witt's message of "Thu, 16 Jun 2022 18:32:47 -0700") References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> <3817c3a3-2a60-fdfc-06d9-c02cd47091a1@gmail.com> <20220616230605.tf4fcef2opouwpfx@yuggoth.org> <61f26a90-159d-9fe6-c79a-a1d5a5dac7ad@gmail.com> Message-ID: > With this ^ I realized that restarting g-api might not have been a > good indicator of anything resolving because in this example the > plugin load failure occurs while checking quota for an inbound > request. I'm not sure whether services load the dbcounter passively > when they start or if it's only when database accesses are initiated. I would have though it would be when the enginefacade is created, which I suspect happens at load time. However, perhaps the plugin bit doesn't occur until the first connection is made, where it needs to use the URL. Not sure. In nova, that would happen at startup time too, but probably not glance. > I also don't understand how or why one project can't load the plugin > but the rest can. And I wonder whether it's a coincidence that it was > glance in the two examples we have so far. Yeah, your confirmation that we've loaded it multiple times before rules out the ordering thing (as I expected) but maybe it rules back in it being something specific to glance. Maybe we should ping zzzeek to look. --Dan From katonalala at gmail.com Fri Jun 17 14:27:32 2022 From: katonalala at gmail.com (Lajos Katona) Date: Fri, 17 Jun 2022 16:27:32 +0200 Subject: [neutron] Drivers meeting agenda - 17.06.2022. In-Reply-To: References: Message-ID: Hi Arnaud, Sure I added it to the agenda ( https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda ) but today we have not enough drivers for the meeting so we have to postpone it to next week. Lajos Arnaud Morin ezt ?rta (id?pont: 2022. j?n. 17., P, 12:14): > hey team, > > Do you mind if I join and talk about: > https://bugs.launchpad.net/neutron/+bug/1975603 > > ? > > Thanks > > On 17.06.22 - 06:18, Lajos Katona wrote: > > Hi Neutron Drivers, > > > > The agenda for tomorrow's drivers meeting is at [1]. > > [RFE] Improve performance of bulk port create/delete with > > networking-generic-switch (#linkhttps:// > > bugs.launchpad.net/neutron/+bug/1976270) > > > > [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda > > > > See you at the meeting tomorrow. > > Lajos Katona (lajoskatona) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjoen at dds.nl Fri Jun 17 16:42:16 2022 From: tjoen at dds.nl (tjoen) Date: Fri, 17 Jun 2022 18:42:16 +0200 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: <58ae62ab-5757-0996-b066-5effcaa0fcd7@dds.nl> On 6/17/22 12:16, KK CHN wrote: ... > 2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 ... > 2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2 Has to be a packaging error: one "=" too many From fungi at yuggoth.org Fri Jun 17 16:48:13 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 17 Jun 2022 16:48:13 +0000 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <58ae62ab-5757-0996-b066-5effcaa0fcd7@dds.nl> References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> <58ae62ab-5757-0996-b066-5effcaa0fcd7@dds.nl> Message-ID: <20220617164813.gm37nevvimzcidfi@yuggoth.org> On 2022-06-17 18:42:16 +0200 (+0200), tjoen wrote: > On 6/17/22 12:16, KK CHN wrote: > ... > > 2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 > ... > > 2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2 > > Has to be a packaging error: one "=" too many The "===" is valid syntax for pip. It means "exactly this version" rather than merely "equivalent to this version." That particular line is currently coming from here: https://opendev.org/openstack/requirements/src/commit/46db00e/upper-constraints.txt#L227 -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From cboylan at sapwetik.org Fri Jun 17 16:55:54 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Fri, 17 Jun 2022 09:55:54 -0700 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> Message-ID: <6ae1cd9b-e6b0-4c70-98ee-c5c71ed0f6aa@www.fastmail.com> On Fri, Jun 17, 2022, at 3:16 AM, KK CHN wrote: > I have tried the Trove Devstack installation on Ubuntu20.04 too.. But > no success.. > ( As I have made no progress in Debian 11 Trove-Devstak installation, > I tried a number of times but all failed with the error ). > > To my experience.. The same error for Ubuntu 20.04 also. > Followed the same stuff : > https://docs.openstack.org/trove/latest/install/install-devstack.html > > The compete Error log here > https://paste.openstack.org/show/bwSXZXHRtHWHnaOO2Fb4/ > > It reports an Error starting with > 2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 > because these package versions have conflicting dependencies. > 2022-06-17 09:58:04.721 | > 2022-06-17 09:58:04.721 | The conflict is caused by: > 2022-06-17 09:58:04.721 | trove 17.1.0.dev31 depends on Jinja2>=2.10 > 2022-06-17 09:58:04.721 | The user requested (constraint) > jinja2===3.1.2How to overcome this Jinja dependency, where we need to > edit this and what jinja suitable ?2022-06-17 09:58:04.721 | > And finally Exit with error > > 2022-06-17 09:58:07.560 | + > /usr/local/lib/python3.8/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 > : exit 1 > Error on exit > stack at kk:~/devstack$ uname -a > Linux kk 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 > UTC 2021 x86_64 x86_64 x86_64 GNU/Linux > stack at kk:~/devstack$ > You are running devstack on Ubuntu 22.04 which installs diskimage-builder under python3.8. Then the trove devstack process attempts to build a trove database image with its agent installed. The image build run by diskimage-builder appears to be running under Ubuntu 18.04 which has python3.6 and pip 9.0.1 (this pip version is what I'm using to make this inference). Unfortunately, the openstack master constraints no longer support python3.6. In particular jinja2 3.1.2 requires python3.7 or newer which is why you get the error above about there being a version conflict. Pip should report this more clearly (and I think very new pip may), but that is what it is trying to tell you. You have requested a specific version that cannot be installed due to a python interpreter error. Depending on what you are trying to achieve you may be better off installing older versions of trove and openstack (using one of the stable branches of devstack). Or as an alternative you can update the trove diskimage-builder base image from Ubuntu 18.04 to 20.04 and fix whatever new problems arise from that. From p.aminian.server at gmail.com Fri Jun 17 17:27:03 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Fri, 17 Jun 2022 21:57:03 +0430 Subject: nova evacuate Message-ID: Hi On kolla ansible with ceph wallaby version ,evacuating instances from failed compute to another compute is associated with the no valid host error .The command that was executed for this case is as follows: nova host-evacuate --target_host TARGET_HOST FAILED_HOST The interesting thing is that this is done successfully from the horizon if the Shared Storage option is selected . how can I solve this issue ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ozzzo at yahoo.com Fri Jun 17 17:33:27 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Fri, 17 Jun 2022 17:33:27 +0000 (UTC) Subject: Swift issues in one cluster References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> Message-ID: <483887237.5247998.1655487207819@mail.yahoo.com> We have multiple Swift clusters, all configured the same. One of them started failing this week. The symptom is that Swift commands take a long time to execute and sometimes they fail: $ openstack container list Unable to establish connection to https://swift..:8080/v1/AUTH_: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') When we look at the logs we see lots of swift-proxy-server errors: (from Splunk): Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) Payload: swift-proxy-server: STDERR: BlockingIOError Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/greenio/base.py", line 397, in send Payload: swift-proxy-server: STDERR: return self._sock.send(b) When we look at network connections, we see haproxy stacking up (many lines of this): # netstat -ntup | sort -b -k2 -n -r | head -n +100 tcp 5976932 0 127.0.0.1:60738 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5976446 0 127.0.0.1:58480 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5973217 0 127.0.0.1:33244 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5973120 0 127.0.0.1:51836 127.0.0.1:8080 ESTABLISHED 13045/haproxy tcp 5971968 0 127.0.0.1:58516 127.0.0.1:8080 ESTABLISHED 13045/haproxy ... If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue? From gmann at ghanshyammann.com Fri Jun 17 19:23:27 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 17 Jun 2022 14:23:27 -0500 Subject: [all][tc] What's happening in Technical Committee: summary 17 June 2022: Reading: 5 min Message-ID: <181731ecd68.f1d10f7340637.5685661944033410386@ghanshyammann.com> Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on 16 June. Most of the meeting discussions are summarized in this email. Meeting full logs are available @https://meetings.opendev.org/meetings/tc/2022/tc.2022-06-16-15.00.log.html * Next TC weekly meeting will be on 23 June Thursday at 15:00 UTC, feel free to add the topic on the agenda[1] by 22 June. 2. What we completed this week: ========================= * The TC presented the "OpenStack Updates" to the Board of Directors meeting in Berlin. Due to COVID and no in-person board meeting, OpenStack community and Board did not have any interaction for 2.5 years. As Board meet in-person in Berlin summit, The TC presented the "OpenStack Updates" to them which is much appreciated by the Board. Along with updates, OpenStack's current challenges are acknowledged and discussed especially operator+develoepr less interaction, fewer contributors, and less diversity in contribution. You can see the slides here[2]. 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[3], many items are in-progress. Open Reviews ----------------- * Four open reviews for ongoing activities[4]. New release cadence "SLURP" open items -------------------------------------------------- 1. release notes strategy: Brian is working on this[5] and has some ideas on automating the releasenotes process which is under testing[5]. This work is required by m-3 of the 2023.1 release so we have time but he mentioned to do it well before that. Improve project governance --------------------------------- Slawek updated the framework proposal as per review comments and is under review[6]. New ELK service dashboard: e-r service ----------------------------------------------- No update this week. Consistent and Secure Default RBAC ------------------------------------------- We have operators feedback[7] and we will be discussing it in policy popup meeting on Tuesday 21 June[8] Please note that operator feedback is very important for us to proceed on 'scope' in SRBAC. If you are operator or know anyone around you, we request you to provide feedback. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[9]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[10]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[11][12]. Project updates ------------------- * None 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[13]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [14] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://docs.google.com/presentation/d/1yCiZy_9A6hURXD0BRdr33OlK7Ugch5EgSvFzL-jvDIg/edit#slide=id.g129c4257ac3_0_1456 [3] https://etherpad.opendev.org/p/tc-zed-tracker [4] https://review.opendev.org/q/projects:openstack/governance+status:open [5] https://review.opendev.org/c/openstack/project-team-guide/+/843457 [6] https://review.opendev.org/c/openstack/governance/+/839880 [7] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/029098.html [8] https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popup_Team#Meeting [9] https://review.opendev.org/c/openstack/governance/+/836888 [10] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.html [11] https://etherpad.opendev.org/p/zuul-config-error-openstack [12] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [13] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [14] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann From gmann at ghanshyammann.com Sat Jun 18 02:36:10 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 17 Jun 2022 21:36:10 -0500 Subject: [nova] nova-live-migraton and nova-grenade-multinode is in bad shape on master In-Reply-To: <6VDMDR.FIQQXC1QVYHU2@redhat.com> References: <6VDMDR.FIQQXC1QVYHU2@redhat.com> Message-ID: <18174aaf766.bac4310444523.2500031565366855669@ghanshyammann.com> ---- On Fri, 17 Jun 2022 06:42:42 -0500 Balazs Gibizer wrote ---- > > > On Fri, Jun 17 2022 at 12:46:34 PM +02:00:00, Balazs Gibizer > wrote: > > Hi, > > > > Since yesterday we started seeing both $subject jobs to fail with [1]. > > A recently landed tempest change is being reverted[2] to fix it. Instead of revert, fix is merged and job is passing now, please recheck your patch for this failure. https://review.opendev.org/c/openstack/tempest/+/846345/3 -gmann > > > > > > > > [1]https://bugs.launchpad.net/tempest/+bug/1979052 > > > > > [2] https://review.opendev.org/c/openstack/tempest/+/846251 > > > > > From zaitcev at redhat.com Sat Jun 18 03:05:56 2022 From: zaitcev at redhat.com (Pete Zaitcev) Date: Fri, 17 Jun 2022 22:05:56 -0500 Subject: Swift issues in one cluster In-Reply-To: <483887237.5247998.1655487207819@mail.yahoo.com> References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> Message-ID: <20220617220556.0be3fb91@niphredil.zaitcev.lan> On Fri, 17 Jun 2022 17:33:27 +0000 (UTC) Albert Braden wrote: > $ openstack container list > Unable to establish connection to https://swift..:8080/v1/AUTH_: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') Right away I have a question: why in the world are you connecting to 8080 with HTTPS? > (from Splunk): > Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) > Payload: swift-proxy-server: STDERR: BlockingIOError > Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) > Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) This looks quite fishy to me, because the os.read is in swift/common/utils.py and it's responsible for the mutex. > When we look at network connections, we see haproxy stacking up (many lines of this): > > # netstat -ntup | sort -b -k2 -n -r | head -n +100 > tcp 5976932 0 127.0.0.1:60738 127.0.0.1:8080 ESTABLISHED 13045/haproxy > tcp 5976446 0 127.0.0.1:58480 127.0.0.1:8080 ESTABLISHED 13045/haproxy > tcp 5973217 0 127.0.0.1:33244 127.0.0.1:8080 ESTABLISHED 13045/haproxy > tcp 5973120 0 127.0.0.1:51836 127.0.0.1:8080 ESTABLISHED 13045/haproxy > tcp 5971968 0 127.0.0.1:58516 127.0.0.1:8080 ESTABLISHED 13045/haproxy > ... > > If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue? Indeed if so many requests are established, you're in trouble. The best fix, I think, is to find the customer who's doing it and punish them. Otherwise, quotas and the ratelimiting middleware are your friends. There's also a possibility that your cluster is underperforming, although usually that results in 500 results first. But then again, at times users would "compensate" for issues by just launching way more requests, in effect DoS-ing the cluster even worse. -- Pete From tjoen at dds.nl Sat Jun 18 03:14:22 2022 From: tjoen at dds.nl (tjoen) Date: Sat, 18 Jun 2022 05:14:22 +0200 Subject: Devstack Installation Fails with Internal Server Error (HTTP 500) In-Reply-To: <20220617164813.gm37nevvimzcidfi@yuggoth.org> References: <559999b7-f47d-4c0b-ac48-65ff3d45437d@www.fastmail.com> <7592ce705d219818f7574c961f22b2acde834358.camel@redhat.com> <378ca4888789313a17e790c05369772728c8b207.camel@redhat.com> <58ae62ab-5757-0996-b066-5effcaa0fcd7@dds.nl> <20220617164813.gm37nevvimzcidfi@yuggoth.org> Message-ID: <09a3bf59-cf9b-1e6e-ec47-5545e04df472@dds.nl> On 6/17/22 18:48, Jeremy Stanley wrote: > On 2022-06-17 18:42:16 +0200 (+0200), tjoen wrote: >> On 6/17/22 12:16, KK CHN wrote: >>> 2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2 >> >> Has to be a packaging error: one "=" too many > > The "===" is valid syntax for pip. It means "exactly this version" OK, never too old to learn something new From melwittt at gmail.com Sat Jun 18 04:18:47 2022 From: melwittt at gmail.com (melanie witt) Date: Fri, 17 Jun 2022 21:18:47 -0700 Subject: [nova]tempest-integrated-compute-centos-9-stream is broken since yesterday In-Reply-To: References: Message-ID: On Fri Jun 17 2022 03:07:47 GMT-0700 (Pacific Daylight Time), Balazs Gibizer wrote: > > > On Fri, Jun 17 2022 at 11:54:31 AM +02:00:00, Balazs Gibizer > wrote: >> Hi, >> >> Please hold you rechecks as $subject is 100% fail on master with >> error: >> >> libvirt.libvirtError: internal error: unable to execute QEMU command >> 'netdev_add': File descriptor named '(null)' has not been found > > We are turning the job to non-voting to unblock the gate [3]. > >> >> Cheers, >> gibi >> >> [1]https://zuul.opendev.org/t/openstack/builds?job_name=tempest-integrated-compute-centos-9-stream&skip=0 >> [2]https://bugs.launchpad.net/nova/+bug/1979047 >> > [3] https://review.opendev.org/c/openstack/nova/+/846292 The patch ^ to make tempest-integrated-compute-centos-9-stream non-voting has merged and it is now safe to recheck your patches. Cheers, -melwitt From hiwkby at yahoo.com Sat Jun 18 07:17:13 2022 From: hiwkby at yahoo.com (Hirotaka Wakabayashi) Date: Sat, 18 Jun 2022 07:17:13 +0000 (UTC) Subject: [trove] Devstack Installation Fails with Internal Server Error References: <1900816356.4272780.1655536633140.ref@mail.yahoo.com> Message-ID: <1900816356.4272780.1655536633140@mail.yahoo.com> Hi, KK CHN # Added [trove] topic to the subject. Please see: # https://wiki.openstack.org/wiki/MailingListEtiquette#Subjects I use CentOS Stream9 but I have successfully installed devstack with Trove. Here is my Jinja2 version in my devstack environment. ``` $ pip list | grep Jinja Jinja2?????????????????????? 3.1.2 ``` > https://docs.openstack.org/trove/latest/install/install-devstack.html I will check the docs and update if needed. Thanks in advance, Hirotaka On Friday, June 17, 2022 at 10:45:54 PM GMT+9, wrote: Send openstack-discuss mailing list submissions to ??? openstack-discuss at lists.openstack.org To subscribe or unsubscribe via the World Wide Web, visit ??? http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss or, via email, send a message with subject or body 'help' to ??? openstack-discuss-request at lists.openstack.org You can reach the person managing the list at ??? openstack-discuss-owner at lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: ? 1. Re: Devstack Installation Fails with Internal Server Error ? ? ? (HTTP 500) (KK CHN) ---------------------------------------------------------------------- Message: 1 Date: Fri, 17 Jun 2022 15:46:25 +0530 From: KK CHN To: openstack-discuss at lists.openstack.org Subject: Re: Devstack Installation Fails with Internal Server Error ??? (HTTP 500) Message-ID: ??? Content-Type: text/plain; charset="utf-8" I have tried the Trove Devstack installation on Ubuntu20.04? too.. But no success.. ( As I have made no progress in Debian 11? Trove-Devstak installation, I tried a number of times but all failed with the error ). To my experience..? The same error for? Ubuntu 20.04? also. Followed the same? stuff : https://docs.openstack.org/trove/latest/install/install-devstack.html The compete Error log here https://paste.openstack.org/show/bwSXZXHRtHWHnaOO2Fb4/ It reports an Error starting with 2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 because these package versions have conflicting dependencies. 2022-06-17 09:58:04.721 | 2022-06-17 09:58:04.721 | The conflict is caused by: 2022-06-17 09:58:04.721 |? ? trove 17.1.0.dev31 depends on Jinja2>=2.10 2022-06-17 09:58:04.721 |? ? The user requested (constraint) jinja2===3.1.2 How to overcome this? Jinja? dependency, where we need to edit this and what jinja suitable ? 2022-06-17 09:58:04.721 | And finally? Exit with error 2022-06-17 09:58:07.560 | + /usr/local/lib/python3.8/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 :? exit 1 Error on exit stack at kk:~/devstack$ uname -a Linux kk 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux stack at kk:~/devstack$ On Thu, Jun 16, 2022 at 11:51 AM KK CHN wrote: > Update to my Devstack installation Error. > > I have crossed this Internal Error ( HTTP 500)? and ended up with a new > error for the Trove Devstack? Installation attempts. > ################################################## > 2022-06-16 05:46:40.183 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:26 > :? total_kB=16393076 > 2022-06-16 05:46:40.188 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:29 > :? RAM_NEEDED=4 > 2022-06-16 05:46:40.192 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 > :? '[' 16393076 -lt 4194304 ']' > 2022-06-16 05:46:40.197 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 > :? return 0 > 2022-06-16 05:46:40.202 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:236 > :? timeout 120 sh -c 'while ! sudo umount -f /tmp/dib_image.WMpKoawa; do > sleep 1; done' > 2022-06-16 05:46:40.225 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:241 > :? rm -rf --one-file-system /tmp/dib_image.WMpKoawa > 2022-06-16 05:46:40.231 | + > /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 > :? exit 1 > Error on exit > # Warning: iptables-legacy tables present, use iptables-legacy to see them > # Warning: iptables-legacy tables present, use iptables-legacy to see them > # Warning: iptables-legacy tables present, use iptables-legacy to see them > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ > > ############################################################################# > > I am able to login to the Horizon dash board inspite? the ./stack.sh > script? Exit with the above error status..? And? in the image section, I am > able to view? ? cirros-0.5.2-x86_64-disk > >? image in the dashboard. > > But? on the terminal when I issue the command? DEVSTACK_TROVE/devstack$ > openstack datastore version list mysql? it says >? " Datastore 'mysql' cannot found. HTTP(404). > > #############Here the tty? output > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ source ./openrc > WARNING: setting legacy OS_TENANT_NAME to support cli tools. > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore > version list mysql > Datastore 'mysql' cannot be found. (HTTP 404) > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack server list > > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack image? list > > +--------------------------------------+--------------------------+--------+ > | ID? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? | Name? ? ? ? ? ? ? ? ? ? | Status > | > > +--------------------------------------+--------------------------+--------+ > | be0c8769-286c-4b61-b989-a037e0e9601c | cirros-0.5.2-x86_64-disk | active > | > > +--------------------------------------+--------------------------+--------+ > stack at XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore > version list mysql > Datastore 'mysql' cannot be found. (HTTP 404) > > ####################################################################################### > > > So I? ended up with the TROVE installation failing? ?? Or what does this > error mean ?? How can I fix this to get a devstack installed successfully > with Trove-DbaaS feature up and running ? > > > Any help / directions much appreciated .... > > [ How I rectify the earlier errors of? HTTP 500 :? I just created a VM > again with same debian 11 image? and 'stack' user created with /opt/stack > dir and? freshly git cloned devstack to? ? ? DEVSTACK_TROVE/? ? Dir.. > Then created the same local.conf file? which I posted in my earlier mail. > and? reran? ? ./stack.sh? script .? This time I was able to get the > dashboard up and be able to login(earlier it's not possible to get the > dashboard too) ] > > Krish > > > > > > On Wed, Jun 15, 2022 at 12:31 PM KK CHN wrote: > >> This is my latest? devstack installation error..? No clues? why it >> fails? repeatedly; earlier it was during the uploading stage of the cirros >> image,? internal server error ( HTTP 500 ) occurred. This was repeated >> multiple times. So I posted for the first time in this thread. >> >>? In this iteration for starting afresh, I have removed the stack user and >> /opt/stack directory manually, and freshly created? 'stack' user with sudo >> permissions and? /opt/stack? directory with? stack:stack ownership,? and >> fresh git cloned(git clone https://opendev.org/openstack/devstack )? to >> /opt/DEVSTACK_stack? folder and created local.conf? ( attached here) . >> Then executed? ? ./stack.sh . >> >> Got the following error this time ...? Any clues why it fails here ? >> Platform (on a VM with Debian uname output) :? Linux XenaCtrl1 >> 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux >> >> [image: Trove_installationError.png] >> >> On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney wrote: >> >>> On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: >>> > > i hit the same issue the other day if you recall me asking about it. >>> > > didnt check my devstack logs to see if it installed or not but i can >>> > > see if i can repoduce it tomorrow. >>> > >>> > I checked with the OP and they have it installed properly, >>> > system-wide. I was assuming your issue might have been venv (or >>> > ordering) related, but theirs was not. There are some other weird >>> things >>> > going on with their setup, so I'm not so sure. >>> nope, was not useing a vnev or anything special, it just devstack >>> installed normaly more or less >>> but driven by the ansible roles. the run devstack rull litrally just >>> opens a shell and invokes >>> stack.sh so i dont think there was anything specific ot how i was >>> invoking it. >>> >>> > >>> > --Dan >>> > >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Trove_installationError.png Type: image/png Size: 88093 bytes Desc: not available URL: ------------------------------ Subject: Digest Footer _______________________________________________ openstack-discuss mailing list openstack-discuss at lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss ------------------------------ End of openstack-discuss Digest, Vol 44, Issue 110 ************************************************** From lokendrarathour at gmail.com Sat Jun 18 08:53:02 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Sat, 18 Jun 2022 14:23:02 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Herald, please find the response in line: * Verify that the TFTP server is running? *[Loke]* *TFTP Container is running but is in unhealthy state:* *[root at undercloud ironic]# podman ps -a | grep tftpb3fb26bcf190 undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo /bin/bash -c BIND... 4 hours ago Up 4 hours ago (unhealthy) ironic_pxe_tftp[root at undercloud ironic]#* * Check the log file - /var/log/containers/ironic/dnsmasq.log *[Loke]: ERROR* *[root at undercloud ironic]# tail -f dnsmasq.logJun 18 04:41:10 dnsmasq[11]: started, version 2.79 DNS disabledJun 18 04:41:10 dnsmasq[11]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotifyJun 18 04:41:10 dnsmasq-tftp[11]: TFTP root is /var/lib/ironic/tftpbootJun 18 08:09:21 dnsmasq-tftp[11]: failed sending /var/lib/ironic/tftpboot/snponly.efi to aaaa:aaaa:aaaa::4Jun 18 08:09:28 dnsmasq-tftp[11]: failed sending /var/lib/ironic/tftpboot/snponly.efi to aaaa:aaaa:aaaa::4Jun 18 08:09:35 dnsmasq-tftp[11]: failed sending /var/lib/ironic/tftpboot/snponly.efi to aaaa:aaaa:aaaa::4* * Try to capture the TFTP traffic on br-ctlplane to a PCAP file to see *[Loke]: * *attached the same as aaaa.pcap* *Also do note that there is a similar bug reported:* *https://bugs.launchpad.net/tripleo/+bug/1964397 * *the error in the bug is similar to what I have.* On Fri, Jun 17, 2022 at 5:53 PM Harald Jensas wrote: > Hi, > > The error in the screenshot, the booting node is not able to dowload the > network boot program from the TFTP server on the undercloud. > > Can you: > * Verify that the TFTP server is running? > * Check the log file - /var/log/containers/ironic/dnsmasq.log > * Try to capture the TFTP traffic on br-ctlplane to a PCAP file to see > if we can use that to dignose the problem. > * download that file manually? > For example: curl -O tftp:///snponly.efi ? > > > Best Regards > Harald > > On 6/16/22 09:05, Lokendra Rathour wrote: > > Hi Shephard, > > Thanks, after changing the details as mentioned, Undercloud got > > installed successfully. > > Now as a part to test the introspection we added a single node and > > initiated the introspection on which we are getting errors. > > IP as per the inspector range is getting allocated, but soon after the > > IP allocation the introspection ILO gives the below error: > > image.png > > it says, Downloading NBP file.... > > PXE-E99: Unexpected network error. > > > > Underlcoud.conf: > > > > undercloud_debug = true > > clean_nodes = true > > cleanup = false > > container_cli = podman > > container_healthcheck_disabled = true > > container_images_file = /home/stack/containers-prepare-parameter.yaml > > deployment_user = stack > > enable_heat = true > > enable_ironic = true > > enable_ironic_inspector = true > > enable_neutron = true > > generate_service_certificate = false > > enable_routed_networks = false > > ipv6_address_mode = dhcpv6-stateful > > ipxe_enabled = true > > ironic_default_network_interface = neutron > > ironic_enabled_network_interfaces = neutron,flat > > local_interface = enp8s0 > > local_ip = aaaa:aaaa:aaaa::1/64 > > subnets = ctlplane-subnet > > undercloud_admin_host = aaaa:aaaa:aaaa::1 > > undercloud_public_host = aaaa:aaaa:aaaa::1 > > undercloud_hostname = undercloud.com > > undercloud_ntp_servers = 30.30.30.3 > > undercloud_timezone = UTC > > [ctlplane-subnet] > > cidr = aaaa:aaaa:aaaa::/64 > > dhcp_end = aaaa:aaaa:aaaa::19 > > dhcp_start = aaaa:aaaa:aaaa::5 > > gateway = aaaa:aaaa:aaaa::1 > > inspection_iprange = aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40 > > > > > > the ironic config in the container: > > > > [root at undercloud /]# vi /etc/ironic-inspector/dnsmasq.conf > > port=0 > > interface=br-ctlplane > > > > > dhcp-range=set:ctlplane-subnet,aaaa:aaaa:aaaa::20,aaaa:aaaa:aaaa::40,64,10m > > dhcp-option-force=tag:ctlplane-subnet,option:mtu,1500 > > dhcp-sequential-ip > > dhcp-match=ipxe,175 > > dhcp-match=set:efi,option:client-arch,7 > > dhcp-match=set:efi,option:client-arch,9 > > dhcp-match=set:efi,option:client-arch,11 > > # dhcpv6s for Client System Architecture Type (61) > > dhcp-match=set:efi6,option6:61,0007 > > dhcp-match=set:efi6,option6:61,0009 > > dhcp-match=set:efi6,option6:61,0011 > > dhcp-userclass=set:ipxe6,iPXE > > # Client is already running iPXE; move to next stage of chainloading > > dhcp-boot=tag:ipxe,http://[aaaa:aaaa:aaaa::1]:8088/inspector.ipxe > > dhcp-option=tag:ipxe6,option6:bootfile-url,http:// > [aaaa:aaaa:aaaa::1]:8088/inspector.ipxe > > # Client is PXE booting over EFI without iPXE ROM; send EFI version of > > iPXE chainloader > > dhcp-boot=tag:efi,tag:!ipxe,snponly.efi > > > dhcp-option=tag:efi6,tag:!ipxe6,option6:bootfile-url,tftp://[aaaa:aaaa:aaaa::1]/snponly.efi > > # Client is running PXE over BIOS; send BIOS version of iPXE chainloader > > dhcp-boot=undionly.kpxe,localhost.localdomain,aaaa:aaaa:aaaa::1 > > > > dhcp-hostsdir=/var/lib/ironic-inspector/dhcp-hostsdir > > > > > > Please check and help me with the possible error and resolution. > > > > Best Regards, > > Lokendra > > > > > > > > > > On Thu, Jun 16, 2022 at 5:15 AM Brendan Shephard > > wrote: > > > > Hey, > > > > Looks like that is the problem. The [ ] around the IP address are > > causing the issue. If I try to run dnsmasq using exactly the output > > you get, it gives me the same error: > > [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground > > --log-facility=/var/log/ironic/dnsmasq.log --user=root > > --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 > > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > > > dnsmasq: bad command line options: try --help > > > > VS without the [ ] I can see it starts up normally. > > > > The settings in your undercloud.conf file look to be correct I > > believe. So I think there might be a bug here. I don't think we > > should be saving that value with the square brackets, or we would > > need to filter them out when we gather the value in that variable. > > > > I raised a bug for it here so that we can dig into this and find > > what needs fixing: > > https://bugs.launchpad.net/tripleo/+bug/1978892 > > > > > > In the meantime, if you edit that hieradata value, are you able to > > get that container started? > > > > Change this: > > [root at tripleo-director ~]# egrep -r 'tftp_bind_host' > > /etc/puppet/hieradata/ > > /etc/puppet/hieradata/service_configs.json: > > "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", > > > > To this: > > "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" > > > > Then restart the service: > > sudo systemctl restart tripleo_ironic_pxe_http.service > > tripleo_ironic_pxe_tftp.service > > > > Does that get the container running without the error? I did the > > same in my environment and can see that dnsmasq is running properly > > like that: > > [root at tripleo-director ~]# ps -ef | grep aaaa > > root 71180 52675 0 19:24 pts/4 00:00:00 > > /usr/sbin/dnsmasq --keep-in-foreground > > --log-facility=/var/log/ironic/dnsmasq.log --user=root > > --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 > > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > > > Brendan Shephard > > > > Software Engineer > > > > Red Hat APAC > > > > 193 N Quay > > > > Brisbane City QLD 4000 > > > > @RedHat Red Hat > > Red Hat > > > > > > > > > > > > > > On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour > > > > wrote: > > > > Hi Shephard, > > I am getting the local_ip (ipv6) of the undercloud : > > > > [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host > > -c /etc/puppet/hiera.yaml > > [aaaa:aaaa:aaaa::1] > > > > is this because of some ipv6 reasons? > > > > > > On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard > > > wrote: > > > > Hey, > > > > Ok, that command looks fine. What about that variable there? > > Do you get anything back when you run: > > sudo hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml > > > > Mine returns: > > sudo hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml > > 192.168.24.115 > > > > Brendan Shephard > > > > Software Engineer > > > > Red Hat APAC > > > > 193 N Quay > > > > Brisbane City QLD 4000 > > > > @RedHat Red Hat > > Red Hat > > > > > > > > > > > > > > On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour > > > > wrote: > > > > Hi Shephard, > > > > this is the command from my wallaby: > > [root at undercloud ~]# sudo cat > > > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > > { > > "cap_add": [ > > "NET_ADMIN", > > "NET_RAW", > > "SETUID" > > ], > > "command": [ > > "/bin/bash", > > "-c", > > "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c > > /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq > > --keep-in-foreground > > --log-facility=/var/log/ironic/dnsmasq.log --user=root > > --conf-file=/dev/null --listen-address=$BIND_HOST > > --port=0 --enable-tftp > --tftp-root=/var/lib/ironic/tftpboot" > > ], > > "environment": { > > "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", > > "TRIPLEO_CONFIG_HASH": > > "9fb3e4e0e35ee35fdf74cfccb16a7543" > > }, > > "healthcheck": { > > "test": "/openstack/healthcheck" > > }, > > "image": > > > "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", > > "net": "host", > > "privileged": false, > > "restart": "always", > > "start_order": 90, > > "volumes": [ > > "/etc/hosts:/etc/hosts:ro", > > "/etc/localtime:/etc/localtime:ro", > > > > > "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", > > > > > "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", > > > > > "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", > > > > > "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", > > "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", > > "/dev/log:/dev/log", > > "/etc/puppet:/etc/puppet:ro", > > > > > "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", > > > > > "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", > > "/var/lib/ironic:/var/lib/ironic:shared,z", > > "/var/log/containers/ironic:/var/log/ironic:z", > > > "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" > > ] > > }[root at undercloud ~]# > > > > Comparing both, they look alike. > > please check once. > > > > On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard > > > > wrote: > > > > Hi, > > > > Looks like the command was in a different file in > > Wallaby, can you check: > > sudo cat > > > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > > > > That one should have the dnsmasq command it's trying > > to run. For example, here it is from my Wallaby > > environment: > > [stack at undercloud-0 ~]$ sudo cat > > > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > > | jq .command > > [ > > "/bin/bash", > > "-c", > > "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c > > /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq > > --keep-in-foreground > > --log-facility=/var/log/ironic/dnsmasq.log > > --user=root --conf-file=/dev/null > > --listen-address=$BIND_HOST --port=0 --enable-tftp > > --tftp-root=/var/lib/ironic/tftpboot" > > ] > > > > > > > > Brendan Shephard > > > > Software Engineer > > > > Red Hat APAC > > > > 193 N Quay > > > > Brisbane City QLD 4000 > > > > @RedHat Red Hat > > Red Hat > > > > > > > > > > > > > > On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour > > > > wrote: > > > > Hi Shephard, > > Here is the o/p of the file: > > > > [root at undercloud ~]# sudo cat > > /var/lib/kolla/config_files/ironic_pxe_tftp.json > > { > > "config_files": [ > > { > > "dest": "/", > > "merge": true, > > "preserve_properties": true, > > "source": > "/var/lib/kolla/config_files/src/*" > > } > > ], > > "permissions": [ > > { > > "owner": "ironic:ironic", > > "path": "/var/log/ironic", > > "recurse": true > > }, > > { > > "owner": "ironic:ironic", > > "path": "/var/lib/ironic", > > "recurse": true > > } > > ] > > }[root at undercloud ~]# > > > > > > Thanks once agan. > > > > -Lokendra > > > > > > On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard > > > > wrote: > > > > Looks like something wrong with the dnsmasq > > command the container is being launched > > with. What command is it trying to run? > > > > sudo cat > > > /var/lib/kolla/config_files/ironic_pxe_tftp.json > > > > Brendan Shephard > > > > Software Engineer > > > > Red Hat APAC > > > > 193 N Quay > > > > Brisbane City QLD 4000 > > > > @RedHat Red Hat > > > > Red Hat > > > > > > > > > > > > On Wed, Jun 15, 2022 at 6:22 PM Anirudh > > Gupta > > wrote: > > > > Hi Brendan, > > > > Thanks for your response. > > > > Please find the log below. > > > > [stack at undercloud t2u2v2w]$ sudo podman > > logs ironic_pxe_tftp > > > > dnsmasq: bad command line options: try > > --help > > dnsmasq: bad command line options: try > > --help > > dnsmasq: bad command line options: try > > --help > > dnsmasq: bad command line options: try > > --help > > dnsmasq: bad command line options: try > > --help > > dnsmasq: bad command line options: try > > --help > > > > [stack at undercloud t2u2v2w]$ sudo podman > > ps --filter name=ironic_pxe -a > > CONTAINER ID IMAGE > > > > COMMAND > > CREATED STATUS > > PORTS NAMES > > 02dacbc74cec > > > undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo > /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) > ironic_pxe_tftp > > 1f8ca39fba32 > > > undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo > kolla_start 3 hours ago Up 3 hours ago (healthy) > ironic_pxe_http > > > > > > Regards > > > > Anirudh Gupta > > > > > > On Wed, Jun 15, 2022 at 11:30 AM Brendan > > Shephard > > wrote: > > > > Hey Anirudh, > > > > You would need to look at the logs > > for the ironic_pxe_tftp container to > > see why it's failing. > > > > I assume the tftp container is not > > Up when you run this command? > > [stack at tripleo-director > > overcloud_playbooks]$ sudo podman ps > > --filter name=ironic_pxe -a > > CONTAINER ID IMAGE > > > > > > COMMAND CREATED STATUS > > PORTS NAMES > > 0170be36e291 > > > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > < > http://registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > > > kolla_start 12 days ago Up 30 > > hours ago (healthy) > > ironic_pxe_tftp > > e507f722bdf0 > > > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > < > http://registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > > > kolla_start 12 days ago Up 30 > > hours ago (healthy) > > ironic_pxe_http > > > > Then check the logs to see what the > > error is: > > [stack at tripleo-director > > overcloud_playbooks]$ sudo podman > > logs ironic_pxe_tftp > > > > > > > > Brendan Shephard > > > > Software Engineer > > > > Red Hat APAC > > > > > 193 N Quay > > > > Brisbane City QLD 4000 > > > > @RedHat > > Red Hat > > < > https://www.linkedin.com/company/red-hat> > > Red Hat > > > > > > > > > > > > > > On Wed, Jun 15, 2022 at 7:53 AM > > Anirudh Gupta > > wrote: > > > > Hi Team, > > > > I am trying to deploy Openstack > > Wallaby Undercloud on IPv6, but > > facing the below error: > > > > 2022-06-14 05:01:23.213708 | > > > 52540083-cfa2-3f20-e9dc-00000000286f > > | TASK | Manage container > > systemd services and cleanup old > > systemd healthchecks for > > > /var/lib/tripleo-config/container-startup-config/step_4 > > 2022-06-14 05:03:22.912816 | > > > 52540083-cfa2-3f20-e9dc-00000000286f > > | FATAL | Manage container > > systemd services and cleanup old > > systemd healthchecks for > > > /var/lib/tripleo-config/container-startup-config/step_4 > > | undercloud | error={"changed": > > false, "msg": "Service > > ironic_pxe_tftp has not started > > yet"} > > 2022-06-14 05:03:22.914400 | > > > 52540083-cfa2-3f20-e9dc-00000000286f > > | TIMING | > > tripleo_container_manage : > > Manage container systemd > > > > Sample Undercloud.conf is as > > follows: > > > > [DEFAULT] > > clean_nodes = true > > cleanup = false > > container_cli = podman > > container_healthcheck_disabled = > > true > > container_images_file = > > > /home/stack/containers-prepare-parameter.yaml > > deployment_user = stack > > enable_ironic = true > > enable_ironic_inspector = true > > enable_neutron = true > > enable_routed_networks = false > > generate_service_certificate = > false > > ipv6_address_mode = > dhcpv6-stateful > > ipxe_enabled = true > > local_interface = enp8s0 > > local_ip = aaaa:aaaa:aaaa::1/64 > > subnets = ctlplane-subnet > > undercloud_admin_host = > > aaaa:aaaa:aaaa::1 > > undercloud_hostname = > > undercloud.com > > > > undercloud_ntp_servers = > 30.30.30.3 > > undercloud_public_host = > > aaaa:aaaa:aaaa::1 > > undercloud_timezone = UTC > > > > [ctlplane-subnet] > > cidr = aaaa:aaaa:aaaa::/64 > > dhcp_end = aaaa:aaaa:aaaa::f > > dhcp_start = aaaa:aaaa:aaaa::a > > gateway = aaaa:aaaa:aaaa::1 > > inspection_iprange = > > > aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 > > > > Can someone please help in this > > regard. > > > > Anirudh Gupta > > > > > > > > -- > > ~ Lokendra > > www.inertiaspeaks.com < > http://www.inertiaspeaks.com> > > www.inertiagroups.com < > http://www.inertiagroups.com> > > skype: lokendrarathour > > > > > > > > > > -- > > ~ Lokendra > > www.inertiaspeaks.com > > www.inertiagroups.com > > skype: lokendrarathour > > > > > > > > > > -- > > ~ Lokendra > > www.inertiaspeaks.com > > www.inertiagroups.com > > skype: lokendrarathour > > > > > > > > > > -- > > ~ Lokendra > > www.inertiaspeaks.com > > www.inertiagroups.com > > skype: lokendrarathour > > > > > > -- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: aaaa.pcap Type: application/octet-stream Size: 17879 bytes Desc: not available URL: From satish.txt at gmail.com Sun Jun 19 14:21:51 2022 From: satish.txt at gmail.com (Satish Patel) Date: Sun, 19 Jun 2022 10:21:51 -0400 Subject: [ceph-users] Suggestion to build ceph storage In-Reply-To: <21E60B31-5869-4EC5-A16F-8EAB949903D8@gmail.com> References: <21E60B31-5869-4EC5-A16F-8EAB949903D8@gmail.com> Message-ID: Sent from my iPhone > On Jun 18, 2022, at 10:47 AM, Anthony D'Atri wrote: > > ?Please do not CC the list. It was my mistake, sorry about that. > >> 15 Total servers and each server has a 12x18TB HDD (spinning disk) . We >> understand SSD/NvME would be best fit but it's way out of budget. > > NVMe SSDs can be surprisingly competitive when you consider IOPS/$, density, and the cost of the HBA you don?t need. Yes totally and I have only single slot to mount one NvME on motherboard. Let?s say I want to put single M.2 NvME then what size I would go for ? > >> Ceph recommends using a faster disk for wal/db if the data disk is slow and >> in my case I do have a slower disk for data. >> >> Question: >> 1. Let's say if i want to put a NvME disk for wal/db then what size i >> should buy. > > Since you specify 12xHDD, you?re thinking an add-in PCI card? Or do you have rear bays? I have raid controller connected to all drivers in side box. > > >> 2. Do I need to partition wal/db for each OSD or just a single >> partition can share for all OSDs? > > Each OSD. What size of partition I should create for each 18TB OSD? > >> 3. Can I put the OS on the same disk where the wal/db is going to sit ? > > Bad idea. I can understand but I have redundancy of 3 copy just in case of failure. And OS disk doesn?t hammer correct? > >> (This way i don't need to spend extra money for extra disk) > > Boot drives are cheap. I have no extra slot left that is why planning to share but I can explore more. > >> >> Any suggestions you have for this kind of storage would be much >> appreciated. >> _______________________________________________ >> ceph-users mailing list -- ceph-users at ceph.io >> To unsubscribe send an email to ceph-users-leave at ceph.io > From miguel at mlavalle.com Sun Jun 19 16:40:45 2022 From: miguel at mlavalle.com (Miguel Lavalle) Date: Sun, 19 Jun 2022 11:40:45 -0500 Subject: [neutron] bug deputy report June 13th to 19th Message-ID: Hi, Here's this week's bugs deputy report: Requires follow up ============== https://bugs.launchpad.net/neutron/+bug/1979089 l3ha router delete race condition if state changes to master. Waiting for more info from reporter High ==== https://bugs.launchpad.net/neutron/+bug/1978938 [9-stream][master][yoga] ovs/ovn fips jobs fails randomly with DNS issue. Assigned to Yatin Karel. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/846001 Medium ====== https://bugs.launchpad.net/neutron/+bug/1978571 VPNaaS: creating ipsec site connection fails (in master). Assigned to Bodo Petermann. Proposed fix: https://review.opendev.org/c/openstack/neutron-vpnaas/+/845759 https://bugs.launchpad.net/neutron/+bug/1979072 Duplicated port binding registers per port, due to live-migration failures. Assigned to Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/846422 https://bugs.launchpad.net/neutron/+bug/1979002 DVR router takes too long to learn octavia LB VIP. Unassigned. We may need help from Octavia team RFE === https://bugs.launchpad.net/neutron/+bug/1979044. [RFE] Adding the "rekey" parameter to the api for strongswan like "dpd_action". This is really for vpnass and requires further triaging Incomplete ======== https://bugs.launchpad.net/neutron/+bug/1978775 ovn: ovs-system: deferred action limit reached, drop recirc action. Waiting for more info from reporter -------------- next part -------------- An HTML attachment was scrubbed... URL: From dsneddon at redhat.com Sat Jun 18 17:23:57 2022 From: dsneddon at redhat.com (Dan Sneddon) Date: Sat, 18 Jun 2022 10:23:57 -0700 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: It is arguable that dnsmasq should accept the IP address in brackets, since that is a standard way of representing IPv6 addresses. It gets confusing, see these ServerFault answers for more detail: https://serverfault.com/questions/444554/what-does-mean-as-an-ip-address-bracket-colon-colon-bracket http://serverfault.com/questions/1026466/ddg#1026469 We originally had a problem where IPv6 addresses needed to be in brackets to support URL representation. In TripleO at one time we even had two versions of hiera values, with and without the brackets. I don?t know if that still exists or not. I wouldn?t hold my breath about getting such a change into dnsmasq. It is notoriously difficult to get patches accepted into dnsmasq, especially where UI is concerned. -Dan Sneddon On Thu, Jun 16, 2022 at 5:32 AM Brendan Shephard wrote: > Hey, > > Looks like that is the problem. The [ ] around the IP address are causing > the issue. If I try to run dnsmasq using exactly the output you get, it > gives me the same error: > [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > dnsmasq: bad command line options: try --help > > VS without the [ ] I can see it starts up normally. > > The settings in your undercloud.conf file look to be correct I believe. So > I think there might be a bug here. I don't think we should be saving that > value with the square brackets, or we would need to filter them out when we > gather the value in that variable. > > I raised a bug for it here so that we can dig into this and find what > needs fixing: > https://bugs.launchpad.net/tripleo/+bug/1978892 > > In the meantime, if you edit that hieradata value, are you able to get > that container started? > > Change this: > [root at tripleo-director ~]# egrep -r 'tftp_bind_host' > /etc/puppet/hieradata/ > /etc/puppet/hieradata/service_configs.json: > "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", > > To this: > "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" > > Then restart the service: > sudo systemctl restart tripleo_ironic_pxe_http.service > tripleo_ironic_pxe_tftp.service > > Does that get the container running without the error? I did the same in > my environment and can see that dnsmasq is running properly like that: > [root at tripleo-director ~]# ps -ef | grep aaaa > root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq > --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > > > > > > Brisbane City QLD > > 4000 > @RedHat Red Hat > Red Hat > > > > > > On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour < > lokendrarathour at gmail.com> wrote: > >> Hi Shephard, >> I am getting the local_ip (ipv6) of the undercloud : >> >> [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c >> /etc/puppet/hiera.yaml >> [aaaa:aaaa:aaaa::1] >> >> is this because of some ipv6 reasons? >> >> >> On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard >> wrote: >> >>> Hey, >>> >>> Ok, that command looks fine. What about that variable there? Do you get >>> anything back when you run: >>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>> >>> Mine returns: >>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>> 192.168.24.115 >>> >>> Brendan Shephard >>> >>> Software Engineer >>> >>> Red Hat APAC >>> >>> 193 N Quay >>> >>> >>> Brisbane City QLD >>> >>> 4000 >>> @RedHat >>> Red >>> Hat >>> Red >>> Hat >>> >>> >>> >>> >>> >>> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >>> lokendrarathour at gmail.com> wrote: >>> >>>> Hi Shephard, >>>> >>>> this is the command from my wallaby: >>>> [root at undercloud ~]# sudo cat >>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>> { >>>> "cap_add": [ >>>> "NET_ADMIN", >>>> "NET_RAW", >>>> "SETUID" >>>> ], >>>> "command": [ >>>> "/bin/bash", >>>> "-c", >>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>> --tftp-root=/var/lib/ironic/tftpboot" >>>> ], >>>> "environment": { >>>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>>> }, >>>> "healthcheck": { >>>> "test": "/openstack/healthcheck" >>>> }, >>>> "image": >>>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>>> "net": "host", >>>> "privileged": false, >>>> "restart": "always", >>>> "start_order": 90, >>>> "volumes": [ >>>> "/etc/hosts:/etc/hosts:ro", >>>> "/etc/localtime:/etc/localtime:ro", >>>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>>> >>>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>>> >>>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>>> >>>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>>> "/dev/log:/dev/log", >>>> "/etc/puppet:/etc/puppet:ro", >>>> >>>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>>> >>>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>>> "/var/log/containers/ironic:/var/log/ironic:z", >>>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>>> ] >>>> }[root at undercloud ~]# >>>> >>>> Comparing both, they look alike. >>>> please check once. >>>> >>>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Looks like the command was in a different file in Wallaby, can you >>>>> check: >>>>> sudo cat >>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>> >>>>> That one should have the dnsmasq command it's trying to run. For >>>>> example, here it is from my Wallaby environment: >>>>> [stack at undercloud-0 ~]$ sudo cat >>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>> | jq .command >>>>> [ >>>>> "/bin/bash", >>>>> "-c", >>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>> ] >>>>> >>>>> >>>>> >>>>> Brendan Shephard >>>>> >>>>> Software Engineer >>>>> >>>>> Red Hat APAC >>>>> >>>>> 193 N Quay >>>>> >>>>> >>>>> Brisbane City QLD >>>>> >>>>> 4000 >>>>> @RedHat Red Hat >>>>> Red Hat >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>>> lokendrarathour at gmail.com> wrote: >>>>> >>>>>> Hi Shephard, >>>>>> Here is the o/p of the file: >>>>>> >>>>>> [root at undercloud ~]# sudo cat >>>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>> { >>>>>> "config_files": [ >>>>>> { >>>>>> "dest": "/", >>>>>> "merge": true, >>>>>> "preserve_properties": true, >>>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>>> } >>>>>> ], >>>>>> "permissions": [ >>>>>> { >>>>>> "owner": "ironic:ironic", >>>>>> "path": "/var/log/ironic", >>>>>> "recurse": true >>>>>> }, >>>>>> { >>>>>> "owner": "ironic:ironic", >>>>>> "path": "/var/lib/ironic", >>>>>> "recurse": true >>>>>> } >>>>>> ] >>>>>> }[root at undercloud ~]# >>>>>> >>>>>> >>>>>> Thanks once agan. >>>>>> >>>>>> -Lokendra >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard >>>>>> wrote: >>>>>> >>>>>>> Looks like something wrong with the dnsmasq command the container is >>>>>>> being launched with. What command is it trying to run? >>>>>>> >>>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>> >>>>>>> Brendan Shephard >>>>>>> >>>>>>> >>>>>>> Software Engineer >>>>>>> >>>>>>> Red Hat APAC >>>>>>> >>>>>>> 193 N Quay >>>>>>> >>>>>>> >>>>>>> Brisbane City QLD >>>>>>> >>>>>>> 4000 >>>>>>> @RedHat Red Hat >>>>>>> Red Hat >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Brendan, >>>>>>>> >>>>>>>> Thanks for your response. >>>>>>>> >>>>>>>> Please find the log below. >>>>>>>> >>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>>> >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>> >>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>>> name=ironic_pxe -a >>>>>>>> CONTAINER ID IMAGE >>>>>>>> COMMAND CREATED >>>>>>>> STATUS PORTS NAMES >>>>>>>> 02dacbc74cec >>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>>> ironic_pxe_tftp >>>>>>>> 1f8ca39fba32 >>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>>> ironic_pxe_http >>>>>>>> >>>>>>>> >>>>>>>> Regards >>>>>>>> >>>>>>>> Anirudh Gupta >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>>> bshephar at redhat.com> wrote: >>>>>>>> >>>>>>>>> Hey Anirudh, >>>>>>>>> >>>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>>> to see why it's failing. >>>>>>>>> >>>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>>> --filter name=ironic_pxe -a >>>>>>>>> CONTAINER ID IMAGE >>>>>>>>> COMMAND CREATED STATUS >>>>>>>>> PORTS NAMES >>>>>>>>> 0170be36e291 >>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>> ironic_pxe_tftp >>>>>>>>> e507f722bdf0 >>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>> ironic_pxe_http >>>>>>>>> >>>>>>>>> Then check the logs to see what the error is: >>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>>> ironic_pxe_tftp >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Brendan Shephard >>>>>>>>> >>>>>>>>> Software Engineer >>>>>>>>> >>>>>>>>> >>>>>>>>> Red >>>>>>>>> Hat APAC >>>>>>>>> >>>>>>>>> 193 N Quay >>>>>>>>> >>>>>>>>> >>>>>>>>> Brisbane City QLD >>>>>>>>> >>>>>>>>> 4000 >>>>>>>>> @RedHat Red Hat >>>>>>>>> Red Hat >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Team, >>>>>>>>>> >>>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>>>> facing the below error: >>>>>>>>>> >>>>>>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>>> | TASK | Manage container systemd services and cleanup old systemd >>>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>>> | FATAL | Manage container systemd services and cleanup old systemd >>>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>>>>>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>>>>>>>>> not started yet"} >>>>>>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f >>>>>>>>>> | TIMING | tripleo_container_manage : Manage container systemd >>>>>>>>>> >>>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>>> >>>>>>>>>> [DEFAULT] >>>>>>>>>> clean_nodes = true >>>>>>>>>> cleanup = false >>>>>>>>>> container_cli = podman >>>>>>>>>> container_healthcheck_disabled = true >>>>>>>>>> container_images_file = >>>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>>> deployment_user = stack >>>>>>>>>> enable_ironic = true >>>>>>>>>> enable_ironic_inspector = true >>>>>>>>>> enable_neutron = true >>>>>>>>>> enable_routed_networks = false >>>>>>>>>> generate_service_certificate = false >>>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>>> ipxe_enabled = true >>>>>>>>>> local_interface = enp8s0 >>>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>>> subnets = ctlplane-subnet >>>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>>> undercloud_timezone = UTC >>>>>>>>>> >>>>>>>>>> [ctlplane-subnet] >>>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>>> >>>>>>>>>> Can someone please help in this regard. >>>>>>>>>> >>>>>>>>>> Anirudh Gupta >>>>>>>>>> >>>>>>>>>> >>>>>> >>>>>> -- >>>>>> ~ Lokendra >>>>>> www.inertiaspeaks.com >>>>>> www.inertiagroups.com >>>>>> skype: lokendrarathour >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> ~ Lokendra >>>> www.inertiaspeaks.com >>>> www.inertiagroups.com >>>> skype: lokendrarathour >>>> >>>> >>>> >> >> -- >> ~ Lokendra >> www.inertiaspeaks.com >> www.inertiagroups.com >> skype: lokendrarathour >> >> >> -- Dan Sneddon | Senior Principal Software Engineer dsneddon at redhat.com | redhat.com/cloud dsneddon:irc | @dxs:twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.datri at gmail.com Sun Jun 19 16:24:35 2022 From: anthony.datri at gmail.com (Anthony D'Atri) Date: Sun, 19 Jun 2022 09:24:35 -0700 Subject: [ceph-users] Suggestion to build ceph storage In-Reply-To: References: <21E60B31-5869-4EC5-A16F-8EAB949903D8@gmail.com> Message-ID: <1601C392-85DF-4BD9-BF0D-9490D4631ACD@gmail.com> >> >> >> ?Please do not CC the list. > > It was my mistake, sorry about that. You hadn?t, but you just did (doh!) Back in the old days one would solicit replies and post a summary afterward, that cut down on list volume. ceph-users gets a LOT of traffic and I don?t like to flood people. >>> 15 Total servers and each server has a 12x18TB HDD (spinning disk) . We >>> understand SSD/NvME would be best fit but it's way out of budget. >> >> NVMe SSDs can be surprisingly competitive when you consider IOPS/$, density, and the cost of the HBA you don?t need. > > Yes totally This TCO calculator is gold: https://www.snia.org/forums/cmsi/programs/TCOcalc > and I have only single slot to mount one NvME on motherboard. Let?s say I want to put single M.2 NvME then what size I would go for ? What kind of systems are these? Are you sure that the M.2 port is NVMe, not just SATA? Any option to add a BOSS card or rear cage for additonal 2.5? SFF or M.2 SSDs? Let?s work toward the sizing. First, you mention CephFS today, and RGW (object) later. This is important. * For CephFS it is HIGHLY recommended to place the metadata pool on non-rotational media, ideally keeping 4 copies * For RGW, unless your usage is really low and your objects mostly large (like 1GB+), you?ll likely want non-rotational storage for the index pool as well. Assuming that you?ll be running a recent Ceph release, Pacific or Quincy, you have more flexibility in DB+WAL sizing than with previous releases, due to fixed RocksDB level sizes. A hair too little, and your DB data spills over onto the slow device, and everybody has a bad day. This can also happen during DB compaction. RGW and CephFS tend to be heavier on metadata than RBD (Block storage), in part due to the file/volume sizes tending to be dramatically smaller, so one has a LOT more of them. This influences how many DB levels you want to try to keep on the faster storage. 18TB is a big spinner, and you may find that the SATA interface is a real bottleneck. With one object storage deployment I?ve worked with, HDDs were capped at 8TB because of the bottleneck. Then after failing to squeeze iops from a stone, the deployment finally dumped the spinners and went with QLC SSDs, with much success. Without any more detail about your workload, I?ll guesstimate that you want to plan for 100GB per OSD on a system with 12x big spinners (yikes), so leaving additional space for potential future index and metadata pools, I?d say go with at least a 3.84TB SKU. >>> >> >> Since you specify 12xHDD, you?re thinking an add-in PCI card? Or do you have rear bays? > > I have raid controller connected to all drivers in side box. Don?t use a RAID controller aka RoC HBA. They add a lot of expense, they?re buggy as all get-out, finicky, need very specific monitoring, and add latency to your I/O. If you go the route of wrapping every HDD in a single-drive R0 volume, your lifecycle will be that much more complex, and you?ll need to add cache RAM/flash on the HBA and a supercap for backup. Cha-ching. Traditional battery BBUs are usually warranted for only a year. If it fails, everything will be silently slower, and unless you monitor the BBU/supercap state, you won?t even know. Add that into the TCO calculation. The $$$ saved on a RAID HBA goes a certain way toward SSDs. With QLC you can fit 1.4PB of raw space in 1RU, HDDs can?t come close. RUs cost money, and when you run out, getting more if possible is expensive. For CephFS you?ll do best with MDS nodes with high-frequency, low-core-count CPU SKUs. OSD nodes make much better use out of more cores and are less sensitive to high-end clock. >> >> >>> 2. Do I need to partition wal/db for each OSD or just a single >>> partition can share for all OSDs? >> >> Each OSD. > > What size of partition I should create for each 18TB OSD? Without more detail on your workload, I?ll guesstimate 100GB for WAL+DB. Partition the rest of the space for CephFS and RGW metadata. Oh, and be prepared that a 12:1 ratio + other pools isn?t ideal, the NVMe device may be bottlenecked. > >> >>> 3. Can I put the OS on the same disk where the wal/db is going to sit ? >> >> Bad idea. > > I can understand but I have redundancy of 3 copy just in case of failure. And OS disk doesn?t hammer correct? It doesn?t, no. You could even PXE-boot your nodes, cf. croit.io The is the question of blast radius, though. >> >>> (This way i don't need to spend extra money for extra disk) >> >> Boot drives are cheap. > > I have no extra slot left that is why planning to share but I can explore more. Please let me know privately which chassis model this is. >> >>> >>> Any suggestions you have for this kind of storage would be much >>> appreciated. >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users at ceph.io >>> To unsubscribe send an email to ceph-users-leave at ceph.io >> From pspal83 at hotmail.com Mon Jun 20 11:33:59 2022 From: pspal83 at hotmail.com (pspal83 at hotmail.com) Date: Mon, 20 Jun 2022 11:33:59 +0000 (UTC) Subject: [sdk] Question about openstacksdk References: <709616738.4766104.1655724839572.ref@mail.yahoo.com> Message-ID: Hi All, I am new to OpenStack SDA API and I need help. While I am running my basic py file I am always getting the below error, Please suggest. Code: import openstack conn = openstack.connect(cloud='openstack') for server in conn.compute.servers():? ? print(server.to_dict()) Error: [pke at vm01 ~]$ python openstack-connect.py? Traceback (most recent call last):? File "openstack-connect.py", line 7, in ? ? for server in conn.compute.servers():? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__? ? proxy = self._make_proxy(instance)? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 262, in _make_proxy? ? found_version = temp_adapter.get_api_major_version()? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version? ? return self.session.get_api_major_version(auth or self.auth, **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1276, in get_api_major_version? ? return auth.get_api_major_version(self, **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version? ? data = get_endpoint_data(discover_versions=discover_versions)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data? ? service_catalog = self.get_access(session).service_catalog? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access? ? self.auth_ref = self.get_auth_ref(session)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref? ? return self._plugin.get_auth_ref(session, **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref? ? authenticated=False, log=False, **rkwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1149, in post? ? return self.request(url, 'POST', **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 986, in request? ? raise exceptions.from_response(resp, method, url)keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) RegardsPradeep Kumar On Monday, 20 June, 2022, 04:54:19 pm IST, pspal83 at hotmail.com wrote: Hi All, I am new to OpenStack SDA API and I need help. While I am running my basic py file I am always getting the below error, Please suggest. Code: import openstack conn = openstack.connect(cloud='openstack') for server in conn.compute.servers():? ? print(server.to_dict()) Error: [pke at vm01 ~]$ python openstack-connect.py? Traceback (most recent call last):? File "openstack-connect.py", line 7, in ? ? for server in conn.compute.servers():? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__? ? proxy = self._make_proxy(instance)? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 262, in _make_proxy? ? found_version = temp_adapter.get_api_major_version()? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version? ? return self.session.get_api_major_version(auth or self.auth, **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1276, in get_api_major_version? ? return auth.get_api_major_version(self, **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version? ? data = get_endpoint_data(discover_versions=discover_versions)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data? ? service_catalog = self.get_access(session).service_catalog? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access? ? self.auth_ref = self.get_auth_ref(session)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref? ? return self._plugin.get_auth_ref(session, **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref? ? authenticated=False, log=False, **rkwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1149, in post? ? return self.request(url, 'POST', **kwargs)? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 986, in request? ? raise exceptions.from_response(resp, method, url)keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) RegardsPradeep Kumar -------------- next part -------------- An HTML attachment was scrubbed... URL: From jani.heikkinen at unibas.ch Mon Jun 20 11:49:08 2022 From: jani.heikkinen at unibas.ch (Jani Heikkinen) Date: Mon, 20 Jun 2022 13:49:08 +0200 Subject: [sdk] Question about openstacksdk In-Reply-To: References: <709616738.4766104.1655724839572.ref@mail.yahoo.com> Message-ID: <7f9582c4-f132-4038-7e8e-584427fe79a6@unibas.ch> Dear Predeep, you need to authenticate against Keystone API. For example see https://docs.openstack.org/openstacksdk/wallaby/user/guides/connect.html Best, Jani On 6/20/22 13:33, pspal83 at hotmail.com wrote: > > Hi All, > > I am new to OpenStack SDA API and I need help. > > While I am running my basic py file I am always getting the below > error, Please suggest. > > > Code: > > import openstack > conn = openstack.connect(cloud='openstack') > for server in conn.compute.servers(): > print(server.to_dict()) > > > *Error:* > > [pke at vm01 ~]$ python openstack-connect.py > > Traceback (most recent call last): > ? File "openstack-connect.py", line 7, in > ? ? for server in conn.compute.servers(): > ? File > "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", > line 87, in __get__ > ? ? proxy = self._make_proxy(instance) > ? File > "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", > line 262, in _make_proxy > ? ? found_version = temp_adapter.get_api_major_version() > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", > line 354, in get_api_major_version > ? ? return self.session.get_api_major_version(auth or self.auth, **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", > line 1276, in get_api_major_version > ? ? return auth.get_api_major_version(self, **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", > line 500, in get_api_major_version > ? ? data = get_endpoint_data(discover_versions=discover_versions) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", > line 271, in get_endpoint_data > ? ? service_catalog = self.get_access(session).service_catalog > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", > line 134, in get_access > ? ? self.auth_ref = self.get_auth_ref(session) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", > line 208, in get_auth_ref > ? ? return self._plugin.get_auth_ref(session, **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", > line 188, in get_auth_ref > ? ? authenticated=False, log=False, **rkwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", > line 1149, in post > ? ? return self.request(url, 'POST', **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", > line 986, in request > ? ? raise exceptions.from_response(resp, method, url) > keystoneauth1.exceptions.http.Unauthorized: The request you have made > requires authentication. (HTTP 401) (Request-ID: > req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) > > > Regards > Pradeep Kumar > On Monday, 20 June, 2022, 04:54:19 pm IST, pspal83 at hotmail.com > wrote: > > > Hi All, > > I am new to OpenStack SDA API and I need help. > > While I am running my basic py file I am always getting the below > error, Please suggest. > > > Code: > > import openstack > conn = openstack.connect(cloud='openstack') > for server in conn.compute.servers(): > print(server.to_dict()) > > > *Error:* > > [pke at vm01 ~]$ python openstack-connect.py > > Traceback (most recent call last): > ? File "openstack-connect.py", line 7, in > ? ? for server in conn.compute.servers(): > ? File > "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", > line 87, in __get__ > ? ? proxy = self._make_proxy(instance) > ? File > "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", > line 262, in _make_proxy > ? ? found_version = temp_adapter.get_api_major_version() > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", > line 354, in get_api_major_version > ? ? return self.session.get_api_major_version(auth or self.auth, **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", > line 1276, in get_api_major_version > ? ? return auth.get_api_major_version(self, **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", > line 500, in get_api_major_version > ? ? data = get_endpoint_data(discover_versions=discover_versions) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", > line 271, in get_endpoint_data > ? ? service_catalog = self.get_access(session).service_catalog > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", > line 134, in get_access > ? ? self.auth_ref = self.get_auth_ref(session) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", > line 208, in get_auth_ref > ? ? return self._plugin.get_auth_ref(session, **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", > line 188, in get_auth_ref > ? ? authenticated=False, log=False, **rkwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", > line 1149, in post > ? ? return self.request(url, 'POST', **kwargs) > ? File > "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", > line 986, in request > ? ? raise exceptions.from_response(resp, method, url) > keystoneauth1.exceptions.http.Unauthorized: The request you have made > requires authentication. (HTTP 401) (Request-ID: > req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) > > > Regards > Pradeep Kumar -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtantsur at redhat.com Mon Jun 20 12:38:02 2022 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Mon, 20 Jun 2022 14:38:02 +0200 Subject: [horizon][plugins] Horizon angular based plugins are broken after migrating to angularjs 1.8.2.2 In-Reply-To: References: Message-ID: Hi folks! Could someone please provide some guidance on how to fix these issues? We (ironic) have ~0 javascript expertise in the team, I don't even know where to start. Thanks! Dmitry On Thu, Jun 16, 2022 at 5:55 AM vishal manchanda < manchandavishal143 at gmail.com> wrote: > Hello everyone, > > Horizon team recently migrated XStatic-Angular 1.5.8.0->1.8.2.2 [1] and > after that many horizon angular based plugins start failing [2]. We already > fixed these issues in horizon [3]. So I guess we need to fix the similar > thing in horizon plugins. > > Why did we migrated Angular 1.5.8.0->1.8.2.2? > The angularjs version is updated 1.5.8.0->1.8.2.2 to include the CVE fixed > in the latest version. Also, there is a security bug reported for the same > [4]. > > I have also created an etherpad to track the progress for failed horizon > plugins [5]. > > Action items for the horizon plugins team either fixed their plugins or > review/merge the patch pushed by horizon team members on priority basis to > fix the gate. > > The npm jobs are failing in below horizon plugins : > ? ironic-ui > ? octavia-dashboard > ? senlin-dashboard > ? designate-dashboard > ? vitrage-dashboard > ? murano-dashboard > ? zaqar-ui > ? zun-ui > ? magnum-ui > > In case of any queries, please feel free to reach out to Horizon channel > #openstack-horizon. > > Thanks & Regards, > Vishal Manchanda(irc: vishalmanchanda) > [1] https://review.opendev.org/c/openstack/requirements/+/844099 > [2] https://review.opendev.org/c/openstack/horizon/+/845733 > [3] https://review.opendev.org/c/openstack/horizon/+/843346 > [4] https://bugs.launchpad.net/horizon/+bug/1955556 > [5] > https://etherpad.opendev.org/p/Fix_Horizon_Plugins_With_Angularjs_v1.8.2.2 > -- Red Hat GmbH , Registered seat: Werner von Siemens Ring 14, D-85630 Grasbrunn, Germany Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,Managing Directors: Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Mon Jun 20 13:51:26 2022 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 20 Jun 2022 15:51:26 +0200 Subject: [largescale-sig] Next meeting: June 22nd, 15utc Message-ID: Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 15UTC. You can check how that time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20220622T15 Feel free to add topics to the agenda: https://etherpad.openstack.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez From pspal83 at hotmail.com Mon Jun 20 15:15:09 2022 From: pspal83 at hotmail.com (pspal83 at hotmail.com) Date: Mon, 20 Jun 2022 15:15:09 +0000 (UTC) Subject: AW: [sdk] Question about openstacksdk In-Reply-To: References: <709616738.4766104.1655724839572.ref@mail.yahoo.com> Message-ID: Hi All, Thanks,??Let me? check. I will update you on this very soon. Regards?Pradeep Sent from Yahoo Mail on Android Hi, ? your code works for me, at least with proper credentials in place. How do you keep your credentials for Keystone authentication? I?d strongly recommend using a ?clouds.yaml? (and possibly a ?secure.yaml? in the same directory) for that in favor of using environment variables. To check if your information is read and processed correctly, you may run this code that reads, formats, and displays (parts of) your connection object: ? import openstacksdk import yaml print(yaml.dump({"clouds": {"mycloud": {"auth": openstack.connect().config.config["auth"]}}}, indent=4)) ? This creates something like: In [2]: print(yaml.dump({"clouds": {"mycloud": {"auth": openstack.connect().config.config["auth"]}}}, indent=4)) clouds: ??? mycloud: ??????? auth: ?? ?????????auth_url:https://iam.eu-de.otc.t-systems.com/v3 ??????????? password: "SecretSecretSeccret" ??????????? project_name: eu-de_myproj ??????????? user_domain_name: OTC000000000010000***** ??????????? username: nilsmagnus ? (the actual config object is way larger) ? For completeness, could you also specify the version of your SDK? In [35]: print(openstack.version.pbr.version.VersionInfo("openstacksdk").version_string()) 0.99.0 ? Von: pspal83 at hotmail.com Gesendet: Montag, 20. Juni 2022 13:34 An: user-committee at lists.openstack.org Betreff: [sdk] Question about openstacksdk ? ? Hi All, ? I am new to OpenStack SDA API and I need help. ? While I am running my basic py file I am always getting the below error, Please suggest. ? ? Code: ? import openstack ? conn = openstack.connect(cloud='openstack') ? for serverin conn.compute.servers(): ? ?print(server.to_dict()) ? ? Error: ? [pke at vm01 ~]$ python openstack-connect.py? ? Traceback (most recent call last): ? File "openstack-connect.py", line 7, in ? ? for server in conn.compute.servers(): ? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__ ? ? proxy = self._make_proxy(instance) ? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 262, in _make_proxy ? ? found_version = temp_adapter.get_api_major_version() ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version ? ? return self.session.get_api_major_version(auth or self.auth, **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1276, in get_api_major_version ? ? return auth.get_api_major_version(self, **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version ? ? data = get_endpoint_data(discover_versions=discover_versions) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data ? ? service_catalog = self.get_access(session).service_catalog ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access ? ? self.auth_ref = self.get_auth_ref(session) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref ? ? return self._plugin.get_auth_ref(session, **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref ? ? authenticated=False, log=False, **rkwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1149, in post ? ? return self.request(url, 'POST', **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 986, in request ? ? raise exceptions.from_response(resp, method, url) keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) ? ? Regards Pradeep Kumar On Monday, 20 June, 2022, 04:54:19 pm IST,pspal83 at hotmail.com wrote: ? ? Hi All, ? I am new to OpenStack SDA API and I need help. ? While I am running my basic py file I am always getting the below error, Please suggest. ? ? Code: ? import openstack ? conn = openstack.connect(cloud='openstack') ? for serverin conn.compute.servers(): ? ?print(server.to_dict()) ? ? Error: ? [pke at vm01 ~]$ python openstack-connect.py? ? Traceback (most recent call last): ? File "openstack-connect.py", line 7, in ? ? for server in conn.compute.servers(): ? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__ ? ? proxy = self._make_proxy(instance) ? File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 262, in _make_proxy ? ? found_version = temp_adapter.get_api_major_version() ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version ? ? return self.session.get_api_major_version(auth or self.auth, **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1276, in get_api_major_version ? ? return auth.get_api_major_version(self, **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version ? ? data = get_endpoint_data(discover_versions=discover_versions) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data ? ? service_catalog = self.get_access(session).service_catalog ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access ? ? self.auth_ref = self.get_auth_ref(session) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref ? ? return self._plugin.get_auth_ref(session, **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref ? ? authenticated=False, log=False, **rkwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1149, in post ? ? return self.request(url, 'POST', **kwargs) ? File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 986, in request ? ? raise exceptions.from_response(resp, method, url) keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) ? ? Regards Pradeep Kumar -------------- next part -------------- An HTML attachment was scrubbed... URL: From Nils.Magnus at t-systems.com Mon Jun 20 14:02:10 2022 From: Nils.Magnus at t-systems.com (Nils.Magnus at t-systems.com) Date: Mon, 20 Jun 2022 14:02:10 +0000 Subject: AW: [sdk] Question about openstacksdk In-Reply-To: References: <709616738.4766104.1655724839572.ref@mail.yahoo.com> Message-ID: Hi, your code works for me, at least with proper credentials in place. How do you keep your credentials for Keystone authentication? I?d strongly recommend using a ?clouds.yaml? (and possibly a ?secure.yaml? in the same directory) for that in favor of using environment variables. To check if your information is read and processed correctly, you may run this code that reads, formats, and displays (parts of) your connection object: import openstacksdk import yaml print(yaml.dump({"clouds": {"mycloud": {"auth": openstack.connect().config.config["auth"]}}}, indent=4)) This creates something like: In [2]: print(yaml.dump({"clouds": {"mycloud": {"auth": openstack.connect().config.config["auth"]}}}, indent=4)) clouds: mycloud: auth: auth_url: https://iam.eu-de.otc.t-systems.com/v3 password: "SecretSecretSeccret" project_name: eu-de_myproj user_domain_name: OTC000000000010000***** username: nilsmagnus (the actual config object is way larger) For completeness, could you also specify the version of your SDK? In [35]: print(openstack.version.pbr.version.VersionInfo("openstacksdk").version_string()) 0.99.0 Von: pspal83 at hotmail.com Gesendet: Montag, 20. Juni 2022 13:34 An: user-committee at lists.openstack.org Betreff: [sdk] Question about openstacksdk Hi All, I am new to OpenStack SDA API and I need help. While I am running my basic py file I am always getting the below error, Please suggest. Code: import openstack conn = openstack.connect(cloud='openstack') for server in conn.compute.servers(): print(server.to_dict()) Error: [pke at vm01 ~]$ python openstack-connect.py Traceback (most recent call last): File "openstack-connect.py", line 7, in for server in conn.compute.servers(): File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__ proxy = self._make_proxy(instance) File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 262, in _make_proxy found_version = temp_adapter.get_api_major_version() File "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version return self.session.get_api_major_version(auth or self.auth, **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1276, in get_api_major_version return auth.get_api_major_version(self, **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version data = get_endpoint_data(discover_versions=discover_versions) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data service_catalog = self.get_access(session).service_catalog File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access self.auth_ref = self.get_auth_ref(session) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref return self._plugin.get_auth_ref(session, **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref authenticated=False, log=False, **rkwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1149, in post return self.request(url, 'POST', **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 986, in request raise exceptions.from_response(resp, method, url) keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) Regards Pradeep Kumar On Monday, 20 June, 2022, 04:54:19 pm IST, pspal83 at hotmail.com > wrote: Hi All, I am new to OpenStack SDA API and I need help. While I am running my basic py file I am always getting the below error, Please suggest. Code: import openstack conn = openstack.connect(cloud='openstack') for server in conn.compute.servers(): print(server.to_dict()) Error: [pke at vm01 ~]$ python openstack-connect.py Traceback (most recent call last): File "openstack-connect.py", line 7, in for server in conn.compute.servers(): File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 87, in __get__ proxy = self._make_proxy(instance) File "/usr/local/lib/python3.6/site-packages/openstack/service_description.py", line 262, in _make_proxy found_version = temp_adapter.get_api_major_version() File "/usr/local/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 354, in get_api_major_version return self.session.get_api_major_version(auth or self.auth, **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1276, in get_api_major_version return auth.get_api_major_version(self, **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 500, in get_api_major_version data = get_endpoint_data(discover_versions=discover_versions) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data service_catalog = self.get_access(session).service_catalog File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/base.py", line 134, in get_access self.auth_ref = self.get_auth_ref(session) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref return self._plugin.get_auth_ref(session, **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/identity/v3/base.py", line 188, in get_auth_ref authenticated=False, log=False, **rkwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 1149, in post return self.request(url, 'POST', **kwargs) File "/usr/local/lib/python3.6/site-packages/keystoneauth1/session.py", line 986, in request raise exceptions.from_response(resp, method, url) keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-0e898e09-0dbc-4beb-8414-8dcdc4f6631e) Regards Pradeep Kumar -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Mon Jun 20 16:47:08 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Mon, 20 Jun 2022 22:17:08 +0530 Subject: Regarding Floating IP is existing Setup Message-ID: hi Team, We got a issue in Xena release, where we set the environment in Ubuntu Platform, But later we get some issues in Floating IP not reachable. In a Network node, not all router namespace are Impacted and only few of them get affected, So we can not comment Network node has a issue. The L3 agent where the Router is tied up, Worked just fine, as other routers work Fine. and the one having issue in Floating IP, if i unassigned and assigned it starts working most of the time. Any thoughts on this Regards Adivya Singh -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.acosta at luizalabs.com Mon Jun 20 18:29:59 2022 From: roberto.acosta at luizalabs.com (ROBERTO BARTZEN ACOSTA) Date: Mon, 20 Jun 2022 15:29:59 -0300 Subject: [Neutron] What is the best tool for automated data plane testing with openstack? In-Reply-To: <206935683.H36mYAsI3Y@p1> References: <206935683.H36mYAsI3Y@p1> Message-ID: Hi, I'm validating how the shaker tool works for basic L2 and L3 tests. My idea is to build additional scenarios with extended network traffic and topologies. I will learn about the Tobiko project, thanks Slawek. Best regards, Roberto Em sex., 17 de jun. de 2022 ?s 03:14, Slawek Kaplonski escreveu: > Hi, > > For scenario tests there is also Tobiko project > https://opendev.org/x/tobiko - we are using it in one periodic job in > Neutron and we also are using it in our d/s CI. > > Dnia czwartek, 16 czerwca 2022 09:07:41 CEST Lajos Katona pisze: > > Hi, > > > > In Openstack itself there is no such tool. > > There are several test tools which we use to test Openstack and Openstack > > components like > > Tempest (see [1]) and the plugin for it are full with networking related > > tests neutron-tempest-plugin (see [2]). > > The scenario tests in neutron-tempest-plugin tests some basic dataplane > > scenarios, but > > only in a functional manner, not with traffic load or such, as we used > > these tests in our CI, so > > we need quick feedback. > > To have some stress test you can check Rally (see [3]) but that is again > > not for dataplane testing, > > but to load the API of the components. You can find some Neutron specific > > rally scenarios in > > Neutron repo (see [4]). > > > > [1]: https://opendev.org/openstack/tempest > > [2]: https://opendev.org/openstack/neutron-tempest-plugin > > [3]: https://opendev.org/openstack/rally > > [4]: https://opendev.org/openstack/neutron/src/branch/master/rally-jobs > > > > Lajos Katona (lajoskatona) > > > > ROBERTO BARTZEN ACOSTA ezt ?rta (id?pont: > > 2022. j?n. 15., Sze, 13:04): > > > > > Hey there, > > > > > > I'm using OpenStack(old version) with neutron OVN CMS-plugin and I > need to > > > migrate to the Yoga release (Ubuntu ovn v22.03.0 and ovs v2.17.0). > > > > > > I'm interested in validating a series of fixed issues on openvswitch > and > > > ovn in the releases related to the OpenStack Yoga version (e.g. > > > > https://github.com/ovn-org/ovn/commit/ccbbbd0584e52c2aa88436a431fe7a6deffd2221 > > > ). > > > > > > What OpenStack automated testing tool do you use or recommend to > validate > > > the data plane? > > > > > > Best regards > > > Roberto > > > > > > > > > > > > *?Esta mensagem ? direcionada apenas para os endere?os constantes no > > > cabe?alho inicial. Se voc? n?o est? listado nos endere?os constantes no > > > cabe?alho, pedimos-lhe que desconsidere completamente o conte?do dessa > > > mensagem e cuja c?pia, encaminhamento e/ou execu??o das a??es citadas > est?o > > > imediatamente anuladas e proibidas?.* > > > > > > *?Apesar do Magazine Luiza tomar todas as precau??es razo?veis para > > > assegurar que nenhum v?rus esteja presente nesse e-mail, a empresa n?o > > > poder? aceitar a responsabilidade por quaisquer perdas ou danos > causados > > > por esse e-mail ou por seus anexos?.* > > > > > > > > -- > Slawek Kaplonski > Principal Software Engineer > Red Hat -- _?Esta mensagem ? direcionada apenas para os endere?os constantes no cabe?alho inicial. Se voc? n?o est? listado nos endere?os constantes no cabe?alho, pedimos-lhe que desconsidere completamente o conte?do dessa mensagem e cuja c?pia, encaminhamento e/ou execu??o das a??es citadas est?o imediatamente anuladas e proibidas?._ *?**?Apesar do Magazine Luiza tomar todas as precau??es razo?veis para assegurar que nenhum v?rus esteja presente nesse e-mail, a empresa n?o poder? aceitar a responsabilidade por quaisquer perdas ou danos causados por esse e-mail ou por seus anexos?.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcin.juszkiewicz at linaro.org Mon Jun 20 20:00:37 2022 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Mon, 20 Jun 2022 22:00:37 +0200 Subject: [kolla][monasca] No more Monasca for AArch64? Or Ubuntu? Message-ID: <83f1520b-92a3-79c4-095e-2bb5b3e52013@linaro.org> Monasca bumped confluent-kafka-python to newer version. And that generates side effects on AArch64 (all distributions Kolla supports) and on Ubuntu 22.04 (any architecture). Why? #error "confluent-kafka-python requires librdkafka v1.9.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html" Debian/stable has 1.6.0, Ubuntu/22.04 has 1.8.0 versions. On x86-64 architecture we can fetch wheel files for CentOS Stream 8, CentOS Stream 9, Debian/stable and Ubuntu/20.04 as they use Python 3.6/3.8/3.9 versions for which there are wheel files provided by upstream. On Ubuntu 22.04 we have Python 3.10 for which there is no wheel file from upstream... Sure, on x86-64 we can use extra repository from upstream and build 'confluent-kafka-python' from source. And probably we will go that way later in a cycle. But on AArch64 we have no other way than disabling build of Monasca. There is no way to satisfy dependency :( https://review.opendev.org/c/openstack/kolla/+/846800 From skaplons at redhat.com Mon Jun 20 20:05:05 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Mon, 20 Jun 2022 22:05:05 +0200 Subject: [neutron] CI meeting 21.06 cancelled Message-ID: <32472236.RQBgVUplPS@p1> Hi, I will be on PTO this Tuesday and I will not be able to run Neutron CI meeting. We don't have anything really urgent what can't wait one week more so lets cancel this week's meeting. See You on the Neutron CI meeting next week. -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From gmann at ghanshyammann.com Mon Jun 20 20:10:21 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 20 Jun 2022 15:10:21 -0500 Subject: [all][tc] Technical Committee next weekly meeting on 23 June 2022 at 1500 UTC Message-ID: <18182bccff4.dd7e8c1e155311.2291980582584492323@ghanshyammann.com> Hello Everyone, The technical Committee's next weekly meeting is scheduled for 23 June 2022, at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, 22 June at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann From jesper at schmitz.computer Mon Jun 20 21:17:24 2022 From: jesper at schmitz.computer (Jesper Schmitz Mouridsen) Date: Mon, 20 Jun 2022 23:17:24 +0200 Subject: [cinder] cinder backup benji driver. Message-ID: Hi, I have been looking into using benji-backup[1] in a cinder-backup driver[2]. I have not tested it thoroughly enough yet, but both in devstack and in a multinode setup.(focal victoria). On devstack I get a newer sqlalchemy that satisfies Benji's requirements list. I have not found issues with downgrading Benji to use an earlier version of sqlalchemy found in canonical focal repos. But that also needs some testing. I asked the author of Benji why he chose a version as high as he did. So just a little info about this work. Any comments on the idea? Thanks. Jesper Schmitz Mouridsen. [1] https://benji-backup.me/ [2] https://github.com/jsm222/cinder-backup-benji-driver From kkchn.in at gmail.com Tue Jun 21 06:59:01 2022 From: kkchn.in at gmail.com (KK CHN) Date: Tue, 21 Jun 2022 12:29:01 +0530 Subject: Few Open Questions: Openstack vs Proprietary cloud stacks Message-ID: List, Very recently I came across a discussion with a banking corporation who are heavy users of VMware in all their Data Centre operations .. They asked me a lot of questions in comparison with openstack vs other proprietary vendors (such as VMware. ). As I am promoting OpenStack I need to clarify these questions raised by the organization. I request your inputs for these queries . 1. Does OpenStack support Hyper Converged Infrastructure ? Or any HCI vendors already with OpenStack solutions? Please share the URL/link of case studies of OpenStack HCI. Any documentation/Reference to install openstack as HCI solution with supported H/W OEM details . I can try out ... 2. Is there any performance benchmark for openstack to compare with other such cloud stacks ( URL/reference links kindly share ) 3. What is the market share of OpenStack all over the world ? ( If any statistics kindly share.) Any survey reports available how many large and medium organizations are using Openstack distributions for private/public cloud use in production ? 4. Does OpenStack new versions support VDI(Virtual Desktop Infrastructure) solutions out of the box.? pls give a reference link / url for the same. 5. What is the alternative for VROPS and VRA (including self service capabilities, Blueprint/Template based deployment) in OpenStack and any comparative study on the capabilities available. ( Please share inputs if any to refer. ) 6. What is the alternative of VMWARE Network Insights and VMWARE CodeStream and Vrealize Loginsights in OpenStack ? 7. Any major capability available in VMware cloud stack for which alternative is not available in OpenStack out of the box ? 8. Tools to maintain multiple OpenStack Cluster, any concept of Supervisory cluster available 9. Who all are the large organizations who deployed OpenStack as their alternative over proprietary solutions(eg: VMware cloud and virtualization products ..) 10. Does OpenInfra Foundation also provide direct support For OpenStack Deployment ? If yes Kindly share the link / url where to contact /approach for paid support ? in case the clients need paid support ? 11. Regarding security of cloud environments /infrastructure, what are the USPs of OpenStack cloud security over other competing cloud middleware providers ? and the vulnerability handling model/mechanism in production ? Thanks in advance for your invaluable inputs. Each and every input from each member is valuable for me to showcase as solid proofs to defend and promote the openstack over proprietary market leaders. Krish -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Tue Jun 21 07:42:34 2022 From: eblock at nde.ag (Eugen Block) Date: Tue, 21 Jun 2022 07:42:34 +0000 Subject: Regarding Floating IP is existing Setup In-Reply-To: Message-ID: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> Hi, this sounds very familiar to me, I had to deal with something similar a couple of times in a heavily used cluster with 2 control nodes. What does your setup look like, is it a HA setup? I would start checking the DHCP and L3 agents. After increasing dhcp_agents_per_network to 2 in neutron.conf and restarting the services this didn't occur again (yet). This would impact floating IPs as well, sometimes I had to disable and enable the affected router(s). If you only have one control node a different approach is necessary. Do you see a high load on the control node? Zitat von Adivya Singh : > hi Team, > > We got a issue in Xena release, where we set the environment in Ubuntu > Platform, But later we get some issues in Floating IP not reachable. > > In a Network node, not all router namespace are Impacted and only few of > them get affected, So we can not comment Network node has a issue. > > The L3 agent where the Router is tied up, Worked just fine, as other > routers work Fine. > > and the one having issue in Floating IP, if i unassigned and assigned it > starts working most of the time. > > Any thoughts on this > > Regards > Adivya Singh From jains8550 at gmail.com Tue Jun 21 07:47:51 2022 From: jains8550 at gmail.com (AJ_ sunny) Date: Tue, 21 Jun 2022 13:17:51 +0530 Subject: Need help on rabbitmq Message-ID: Hi team I am using kolla-ansible based openstack infra and getting below error in logs seems frequent rabbitmq disconnections <0.5023.16> closing AMQP connection <0.5023.16> (10.80.0.13:40356 -> 10.80.0.13:5672 - mod_wsgi:19:d3196668-57e5-46dc-8b69-78d73b5873a0): missed heartbeats from client, timeout: 60s AMQP server on 10.80.0.13:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Is this bug or any resolution for fixing this issue? Thanks Arihant Jain -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjeanner at redhat.com Tue Jun 21 07:48:10 2022 From: cjeanner at redhat.com (=?UTF-8?Q?C=c3=a9dric_Jeanneret?=) Date: Tue, 21 Jun 2022 09:48:10 +0200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: Hello All, Since there were not objection, and with James out, I took the liberty to add both Takashi and Brendan to the core members. Welcome folks, and, again, thanks for all of your contributions! Cheers, C. On 6/8/22 21:35, James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered > core on puppet-tripleo, and I'm proposing we expand that to all of > TripleO, and likewise for Brendan. > > They both have been contributing and reviewing for a little while now, > and I feel it's time to add them to the team. Please +1 or let me know > any concerns? before Friday June 15th. Thanks! > > -- > -- James Slagle > -- -- C?dric Jeanneret (He/Him/His) Sr. Software Engineer - OpenStack Platform Deployment Framework TC Red Hat EMEA https://www.redhat.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: From mrxlazuardin at gmail.com Tue Jun 21 08:23:51 2022 From: mrxlazuardin at gmail.com (Lazuardi Nasution) Date: Tue, 21 Jun 2022 15:23:51 +0700 Subject: Few Open Questions: Openstack vs Proprietary cloud stacks Message-ID: > > > Hi Krish, Please find inline comment based my experience on banking implementation. Best regards. > Date: Tue, 21 Jun 2022 12:29:01 +0530 > From: KK CHN > To: openstack-discuss at lists.openstack.org > Subject: Few Open Questions: Openstack vs Proprietary cloud stacks > Message-ID: > ecxyQ at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > List, > > Very recently I came across a discussion with a banking corporation who > are heavy users of VMware in all their Data Centre operations .. > > They asked me a lot of questions in comparison with openstack vs other > proprietary vendors (such as VMware. ). > > > As I am promoting OpenStack I need to clarify these questions raised by > the organization. > > I request your inputs for these queries . > > 1. Does OpenStack support Hyper Converged Infrastructure ? > Or any HCI vendors already with OpenStack solutions? Please share the > URL/link of case studies of OpenStack HCI. > Any documentation/Reference to install openstack as HCI solution with > supported H/W OEM details . I can try out ... > Yes, I have experienced with HCI by using OpenStack and Ceph. > 2. Is there any performance benchmark for openstack to compare with other > such cloud stacks ( URL/reference links kindly share ) > I have done some performance test by using CBTool which is the corr of SPEC Cloud IaaS benchmark. So, so the result can be compared with published SPEC Cloud IaaS result as long it has same benchmark parameters. > 3. What is the market share of OpenStack all over the world ? ( If any > statistics kindly share.) > Any survey reports available how many large and medium organizations are > using Openstack distributions for private/public cloud use in production ? > Maybe you can refer to OpenStack surveys. It seem that on any forms (including customized), OpenStack os popular for public cloud. > 4. Does OpenStack new versions support VDI(Virtual Desktop Infrastructure) > solutions out of the box.? pls give a reference link / url for the same. > Actually, VDI consists managed desktop VMs. I haven't use it but maybe Guacamole can be used for VDI on OpenStack. > 5. What is the alternative for VROPS and VRA (including self service > capabilities, Blueprint/Template based deployment) in OpenStack and any > comparative study on the capabilities available. ( Please share inputs if > any to refer. ) > OpenStack includes self service capability. Yes, it has template based deployment like Heat and catalog based deployment like Murano. > 6. What is the alternative of VMWARE Network Insights and VMWARE > CodeStream and Vrealize Loginsights in OpenStack ? > I use other open source products for monitoring and log analysis. Somethink like Prometheus, Loki, etc. > 7. Any major capability available in VMware cloud stack for which > alternative is not available in OpenStack out of the box ? > I have no idea about this. But it is hard to have apple to apple comparison. > 8. Tools to maintain multiple OpenStack Cluster, any concept of > Supervisory cluster available There are some cloud management products which are compatible with OpenStack. But for me, Ansible and/or BASH scripting are good enough. > 9. Who all are the large organizations who deployed OpenStack as their > alternative over proprietary solutions(eg: VMware cloud and virtualization > products ..) > I have no idea with this. > 10. Does OpenInfra Foundation also provide direct support For OpenStack > Deployment ? If yes Kindly share the link / url where to contact > /approach for paid support ? in case the clients need paid support ? > It seem OpenInfra doesn't, but OpenInfra member do. CMIIW. > > 11. Regarding security of cloud environments /infrastructure, what are the > USPs of OpenStack cloud security over other competing cloud middleware > providers ? and the vulnerability handling model/mechanism in production ? > On OpenStack, every tenants can be fully isolated. Anyway, some security products may have another approach to enhance the security. Floating IP concept is good for hiding non service (externally) instance. > Thanks in advance for your invaluable inputs. > > Each and every input from each member is valuable for me to showcase as > solid proofs to defend and promote the openstack over proprietary market > leaders. > > Krish > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.morin at gmail.com Tue Jun 21 09:50:01 2022 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Tue, 21 Jun 2022 09:50:01 +0000 Subject: [nova] custom metric for weigh scheduling Message-ID: Hey nova team, Some context: I want nova-scheduler to favor instances on some computes based on some hypervisor metrics/traits. E.G. I have 2 hypervisors hosting few instances. One of them is having an old kernel, that I will upgrade some day. The second is already having the correct target kernel. I want the scheduler to favor booting on the second host (good kernel). How can I achieve that without disabling the bad_kernel hypervisor? I was thinking about something in the weight scheduler. I was first thinking about using placement traits (having a custom traits) and reading traits from a weight scheduler. But it seems not doable out of the box. And it also seems not a good idea at all to call placement API from weight scheduler. Maybe I am missing something here because I was expecting to have the traits given by placement up to the nova scheduler. Second shoot, I found the metrics weight [1] which rely on the compute monitors. It seems less hacky, but I will still have to write custom monitor to retrieve such metrics. Am I missing an option here? Is there anyone doing such thing is the community? [1] https://github.com/openstack/nova/blob/master/nova/scheduler/weights/metrics.py [2] https://github.com/openstack/nova/tree/master/nova/compute/monitors Cheers, Arnaud. From lokendrarathour at gmail.com Tue Jun 21 10:27:52 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Tue, 21 Jun 2022 15:57:52 +0530 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: +1 to Both. On Thu, Jun 9, 2022 at 1:14 AM James Slagle wrote: > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- > -- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour -------------- next part -------------- An HTML attachment was scrubbed... URL: From arxcruz at redhat.com Tue Jun 21 11:26:03 2022 From: arxcruz at redhat.com (Arx Cruz) Date: Tue, 21 Jun 2022 13:26:03 +0200 Subject: [Tripleo] Gate blocker - Standalone 9 failing Message-ID: Hello, We have another gate blocker on standalone centos 9 job: https://bugs.launchpad.net/tripleo/+bug/1979300 Kind regards, -- Arx Cruz Software Engineer Red Hat EMEA arxcruz at redhat.com @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Tue Jun 21 11:55:51 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Tue, 21 Jun 2022 17:25:51 +0530 Subject: Regarding Floating IP is existing Setup In-Reply-To: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> References: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> Message-ID: hi Eugen, The current setup is 3 controller nodes, The Load is not much on each controller and the number of DHCP agent is always set to 2 as per the standard in the neutron.conf, The L3 agent seems to be stables as other router namespace works fine under it, Only few router Namespace get affected under the agent. Most of the template having issue , Have all instance having FLoating IP, a Stack with a single floating IP have chance of issue very less Regards Adivya Singh On Tue, Jun 21, 2022 at 1:18 PM Eugen Block wrote: > Hi, > > this sounds very familiar to me, I had to deal with something similar > a couple of times in a heavily used cluster with 2 control nodes. What > does your setup look like, is it a HA setup? I would start checking > the DHCP and L3 agents. After increasing dhcp_agents_per_network to 2 > in neutron.conf and restarting the services this didn't occur again > (yet). This would impact floating IPs as well, sometimes I had to > disable and enable the affected router(s). If you only have one > control node a different approach is necessary. Do you see a high load > on the control node? > > > Zitat von Adivya Singh : > > > hi Team, > > > > We got a issue in Xena release, where we set the environment in Ubuntu > > Platform, But later we get some issues in Floating IP not reachable. > > > > In a Network node, not all router namespace are Impacted and only few of > > them get affected, So we can not comment Network node has a issue. > > > > The L3 agent where the Router is tied up, Worked just fine, as other > > routers work Fine. > > > > and the one having issue in Floating IP, if i unassigned and assigned it > > starts working most of the time. > > > > Any thoughts on this > > > > Regards > > Adivya Singh > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hjensas at redhat.com Tue Jun 21 12:01:42 2022 From: hjensas at redhat.com (Harald Jensas) Date: Tue, 21 Jun 2022 14:01:42 +0200 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: <30392fa2-517c-d454-2483-e1ec51072e9d@redhat.com> On 6/21/22 09:19, Lokendra Rathour wrote: > Hi Team, > So, I am still not able to provision?overcloud baremetal nodes using > IPv6. Based on the previous discussion i tried changing some settings in > terms of IPV6 Notation, but no luck. > > Has anyone of us tried ipv6 based overcloud deployment on > Wallaby??Please suggest some way forward. > We run regressions downstream on Wallaby with IPv6 provisioning. Looking at the PCAP file, the TFTP requests arrives and the server is responding. Not sure why dnsmasq (the tftp-server) is not able to complete the transfer. It is possible adding 'log-debug' to the TFTP server configuration file[1] and restart the service 'tripleo_ironic_pxe_tftp.service' will surface more details? Is selinux enabled? Try disabling it with 'setenforce permissive'? Check the /var/log/audit/audit.log for denials? [1] /var/lib/config-data/puppet-generated/ironic/etc/ironic/dnsmasq-tftp-server.conf > Thanks once again for your support. > > -Lokendra > > On Sat, Jun 18, 2022 at 10:54 PM Dan Sneddon > wrote: > > It is arguable that dnsmasq should accept the IP address in > brackets, since that is a standard way of representing IPv6 > addresses. It gets confusing, see these ServerFault answers for more > detail: > > https://serverfault.com/questions/444554/what-does-mean-as-an-ip-address-bracket-colon-colon-bracket > > > http://serverfault.com/questions/1026466/ddg#1026469 > > > We originally had a problem where IPv6 addresses needed to be in > brackets to support URL representation. In TripleO at one time we > even had two versions of hiera values, with and without the > brackets. I don?t know if that still exists or not. > > I wouldn?t hold my breath about getting such a change into dnsmasq. > It is notoriously difficult to get patches accepted into dnsmasq, > especially where UI is concerned. > > -Dan Sneddon > > On Thu, Jun 16, 2022 at 5:32 AM Brendan Shephard > > wrote: > > Hey, > > Looks like that is the problem. The [ ] around the IP address > are causing the issue. If I try to run dnsmasq using exactly the > output you get, it gives me the same error: > [root at tripleo-director ~]# /usr/sbin/dnsmasq > --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log > --user=root --conf-file=/dev/null > --listen-address=[aaaa:aaaa:aaaa::1] --port=0 --enable-tftp > --tftp-root=/var/lib/ironic/tftpboot > > dnsmasq: bad command line options: try --help > > VS without the [ ] I can see it starts up normally. > > The settings in your undercloud.conf file look to be correct I > believe. So I think there might be a?bug here. I don't think we > should be saving that value with the square brackets, or we > would need to filter them out when we gather the value in that > variable. > > I raised a bug for it here so that we can dig into this and find > what needs fixing: > https://bugs.launchpad.net/tripleo/+bug/1978892 > > > In the meantime, if you edit that hieradata value, are you able > to get that container started? > > Change this: > [root at tripleo-director ~]# egrep -r 'tftp_bind_host' > /etc/puppet/hieradata/ > /etc/puppet/hieradata/service_configs.json: > ?"ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", > > To this: > ?"ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" > > Then restart the service: > sudo systemctl restart tripleo_ironic_pxe_http.service > tripleo_ironic_pxe_tftp.service > > Does that get the container running without the error? I did the > same in my environment and can see that dnsmasq is running > properly like that: > [root at tripleo-director ~]# ps -ef | grep aaaa > root ? ? ? 71180 ? 52675 ?0 19:24 pts/4 ? ?00:00:00 > /usr/sbin/dnsmasq --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log --user=root > --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 > --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > > > > Brisbane City QLD > > 4000 > > @RedHat Red Hat > Red Hat > > > > > > > On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour > > > wrote: > > Hi Shephard, > I am getting the local_ip (ipv6) of the undercloud : > > [root at undercloud stack]# sudo hiera > ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml > [aaaa:aaaa:aaaa::1] > > is this because of some ipv6 reasons? > > > On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard > > wrote: > > Hey, > > Ok, that command looks fine. What about that variable > there? Do you get anything back when you run: > sudo hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml > > Mine returns: > sudo hiera ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml > 192.168.24.115 > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > > Brisbane City QLD > > 4000 > > @RedHat > > Red Hat > > Red Hat > > > > > > > > On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour > > wrote: > > Hi Shephard, > > this is the command from my wallaby: > [root at undercloud ~]# sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > { > ? "cap_add": [ > ? ? "NET_ADMIN", > ? ? "NET_RAW", > ? ? "SETUID" > ? ], > ? "command": [ > ? ? "/bin/bash", > ? ? "-c", > ? ? "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host > -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq > --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log > --user=root --conf-file=/dev/null > --listen-address=$BIND_HOST --port=0 --enable-tftp > --tftp-root=/var/lib/ironic/tftpboot" > ? ], > ? "environment": { > ? ? "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", > ? ? "TRIPLEO_CONFIG_HASH": > "9fb3e4e0e35ee35fdf74cfccb16a7543" > ? }, > ? "healthcheck": { > ? ? "test": "/openstack/healthcheck" > ? }, > ? "image": > "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", > ? "net": "host", > ? "privileged": false, > ? "restart": "always", > ? "start_order": 90, > ? "volumes": [ > ? ? "/etc/hosts:/etc/hosts:ro", > ? ? "/etc/localtime:/etc/localtime:ro", > > "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", > > "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", > > "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", > > "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", > ? ? "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", > ? ? "/dev/log:/dev/log", > ? ? "/etc/puppet:/etc/puppet:ro", > > "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", > > "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", > ? ? "/var/lib/ironic:/var/lib/ironic:shared,z", > ? ? "/var/log/containers/ironic:/var/log/ironic:z", > > "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" > ? ] > }[root at undercloud ~]# > > Comparing both, they look alike. > please check once. > > On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard > > > wrote: > > Hi, > > Looks like the command was in a different file > in Wallaby, can you check: > sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > > That one should have the dnsmasq command it's > trying to run. For example, here it is from my > Wallaby environment: > [stack at undercloud-0 ~]$ sudo cat > /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json > | jq .command > [ > ? "/bin/bash", > ? "-c", > ? "BIND_HOST=$(hiera > ironic::pxe::tftp_bind_host -c > /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq > --keep-in-foreground > --log-facility=/var/log/ironic/dnsmasq.log > --user=root --conf-file=/dev/null > --listen-address=$BIND_HOST --port=0 > --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" > ] > > > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > > Brisbane City QLD > > 4000 > > @RedHat Red Hat > Red > Hat > > > > > > > > > On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour > > wrote: > > Hi Shephard, > Here is the o/p of the file: > > [root at undercloud ~]# sudo cat > /var/lib/kolla/config_files/ironic_pxe_tftp.json > { > ? "config_files": [ > ? ? { > ? ? ? "dest": "/", > ? ? ? "merge": true, > ? ? ? "preserve_properties": true, > ? ? ? "source": > "/var/lib/kolla/config_files/src/*" > ? ? } > ? ], > ? "permissions": [ > ? ? { > ? ? ? "owner": "ironic:ironic", > ? ? ? "path": "/var/log/ironic", > ? ? ? "recurse": true > ? ? }, > ? ? { > ? ? ? "owner": "ironic:ironic", > ? ? ? "path": "/var/lib/ironic", > ? ? ? "recurse": true > ? ? } > ? ] > }[root at undercloud ~]# > > > Thanks once agan. > > -Lokendra > > > On Wed, Jun 15, 2022 at 2:38 PM Brendan > Shephard > wrote: > > Looks like something wrong with the > dnsmasq command the container is being > launched with. What command is it trying > to run? > > sudo cat > /var/lib/kolla/config_files/ironic_pxe_tftp.json > > Brendan > Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > > Brisbane City QLD > > 4000 > > @RedHat Red > Hat > > Red Hat > > > > > > > On Wed, Jun 15, 2022 at 6:22 PM Anirudh > Gupta > wrote: > > Hi Brendan, > > Thanks for your response. > > Please find the log below. > > [stack at undercloud t2u2v2w]$ sudo > podman logs ironic_pxe_tftp > > dnsmasq: bad command line options: > try --help > dnsmasq: bad command line options: > try --help > dnsmasq: bad command line options: > try --help > dnsmasq: bad command line options: > try --help > dnsmasq: bad command line options: > try --help > dnsmasq: bad command line options: > try --help > > [stack at undercloud t2u2v2w]$ ?sudo > podman ps --filter name=ironic_pxe -a > CONTAINER ID ?IMAGE > > > COMMAND ? ? ? ? ? ? ? CREATED > ?STATUS > ?PORTS ? ? ? NAMES > 02dacbc74cec > ?undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo ?/bin/bash -c BIND... ?3 hours ago ?Exited (1) 3 hours ago (unhealthy) ? ? ? ? ? ? ?ironic_pxe_tftp > 1f8ca39fba32 > ?undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo ?kolla_start ? ? ? ? ? 3 hours ago ?Up 3 hours ago (healthy) ? ? ? ? ? ? ? ? ? ? ? ?ironic_pxe_http > > > Regards > > Anirudh Gupta > > > On Wed, Jun 15, 2022 at 11:30 AM > Brendan Shephard > > wrote: > > Hey Anirudh, > > You would need to look at the > logs for the ironic_pxe_tftp > container to see why it's failing. > > I assume the tftp container is > not Up when you run this command? > [stack at tripleo-director > overcloud_playbooks]$ sudo > podman ps --filter > name=ironic_pxe -a > CONTAINER ID ?IMAGE > > > ? ? ? ? ? COMMAND ? ? ?CREATED > ? ? ?STATUS > PORTS ? ? ? NAMES > 0170be36e291 > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > ?kolla_start ?12 days ago ?Up > 30 hours ago (healthy) > ? ?ironic_pxe_tftp > e507f722bdf0 > registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo > > ?kolla_start ?12 days ago ?Up > 30 hours ago (healthy) > ? ?ironic_pxe_http > > Then check the logs to see what > the error is: > [stack at tripleo-director > overcloud_playbooks]$ sudo > podman logs ironic_pxe_tftp > > > > > > > > Brendan Shephard > > Software Engineer > > > Red Hat APAC > > > 193 N Quay > > > Brisbane City QLD > > 4000 > > @RedHat > Red > Hat > > Red Hat > > > > > > > > On Wed, Jun 15, 2022 at 7:53 AM > Anirudh Gupta > > wrote: > > Hi Team, > > I am trying to deploy > Openstack Wallaby Undercloud > on IPv6, but facing the > below error: > > 2022-06-14 05:01:23.213708 | > 52540083-cfa2-3f20-e9dc-00000000286f > | TASK | Manage container > systemd services and cleanup > old systemd healthchecks for > /var/lib/tripleo-config/container-startup-config/step_4 > 2022-06-14 05:03:22.912816 | > 52540083-cfa2-3f20-e9dc-00000000286f > | FATAL | Manage container > systemd services and cleanup > old systemd healthchecks for > /var/lib/tripleo-config/container-startup-config/step_4 > | undercloud | > error={"changed": false, > "msg": "Service > ironic_pxe_tftp has not > started yet"} > 2022-06-14 05:03:22.914400 | > 52540083-cfa2-3f20-e9dc-00000000286f > | TIMING | > tripleo_container_manage : > Manage container systemd > > Sample Undercloud.conf is as > follows: > > [DEFAULT] > clean_nodes = true > cleanup = false > container_cli = podman > container_healthcheck_disabled > = true > container_images_file = > /home/stack/containers-prepare-parameter.yaml > deployment_user = stack > enable_ironic = true > enable_ironic_inspector = true > enable_neutron = true > enable_routed_networks = false > generate_service_certificate > = false > ipv6_address_mode = > dhcpv6-stateful > ipxe_enabled = true > local_interface = enp8s0 > local_ip = aaaa:aaaa:aaaa::1/64 > subnets = ctlplane-subnet > undercloud_admin_host = > aaaa:aaaa:aaaa::1 > undercloud_hostname = > undercloud.com > > undercloud_ntp_servers = > 30.30.30.3 > undercloud_public_host = > aaaa:aaaa:aaaa::1 > undercloud_timezone = UTC > > [ctlplane-subnet] > cidr = aaaa:aaaa:aaaa::/64 > dhcp_end = aaaa:aaaa:aaaa::f > dhcp_start = aaaa:aaaa:aaaa::a > gateway = aaaa:aaaa:aaaa::1 > inspection_iprange = > aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 > > Can someone please help in > this regard. > > Anirudh Gupta > > > > -- > ~ Lokendra > www.inertiaspeaks.com > > www.inertiagroups.com > > skype: lokendrarathour > > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > -- > Dan Sneddon???????? |? Senior Principal Software Engineer > dsneddon at redhat.com | redhat.com/cloud > > dsneddon:irc??????? |? @dxs:twitter > > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > From gibi at redhat.com Tue Jun 21 12:04:01 2022 From: gibi at redhat.com (Balazs Gibizer) Date: Tue, 21 Jun 2022 14:04:01 +0200 Subject: [nova] Review guide for PCI tracking for Placement patches Message-ID: Hi Nova, The first batch of patches are up for review for the PCI tracking for Placement feature. These mostly covers two aspects of the spec[1]: 1) renaming [pci]passthrough_whitelist to [pci]device_spec 2) pci inventory reporting to placement, excluding existing PCI allocation healing in placement This covers the first 4 sub chapters of Proposed Change chapter of the spec[1] up until "PCI alias configuration". I noted intentional deviations from the spec in the spec review [2] and I will push a follow up to the spec at some point fixing those. I tried to do it in small steps hence the long list of commits[3]: #2) pci inventory reporting to placement, excluding existing PCI allocation healing in placement 5827d56310 Stop if tracking is disable after it was enabled before a4b5788858 Support [pci]device_spec reconfiguration 10642c787a Reject devname based device_spec config b0ad05fb69 Ignore PCI devs with physical_network tag f5a34ee441 Reject mixed VF rc and trait config c60b26014f Reject PCI dependent device config 5cf7325221 Extend device_spec with resource_class and traits eff0df6a98 Basics for PCI Placement reporting #1) renaming [pci]passthrough_whitelist to [pci]device_spec adfe34080a Rename whitelist in tests ea955a0c15 Rename exception.PciConfigInvalidWhitelist to PciConfigInvalidSpec 55770e4c14 Rename [pci]passthrough_whitelist to device_spec There is a side track branching out from "adfe34080a Rename whitelist in tests" to clean up the device spec handling[4]: 514500b5a4 Move __str__ to the PciAddressSpec base class 3a6198c8fb Fix type annotation of pci.Whitelist class f70adbb613 Remove unused PF checking from get_function_by_ifname b7eef53b1d Clean up mapping input to address spec types 93bbd67101 Poison /sys access via various calls in test 467ef91a86 Remove dead code from PhysicalPciAddress 233212d30f Fix PciAddressSpec descendants to call super.__init__ ad5bd46f46 Unparent PciDeviceSpec from PciAddressSpec cef0d2de4c Extra tests for remote managed dev spec 2fa2825afb Add more test coverage for devname base dev spec adfe34080a Rename whitelist in tests This is not a mandatory part of the feature but I think they improve the code in hand and even fixing some small bugs. I will continue with adding allocation healing for existing PCI allocations. Any feedback is highly appreciated. Cheers, gibi [1] https://specs.openstack.org/openstack/nova-specs/specs/zed/approved/pci-device-tracking-in-placement.html [2] https://review.opendev.org/c/openstack/nova-specs/+/791047 [3] https://review.opendev.org/q/topic:bp/pci-device-tracking-in-placement [4] https://review.opendev.org/q/topic:bp/pci-device-spec-cleanup From allison at openinfra.dev Tue Jun 21 12:06:46 2022 From: allison at openinfra.dev (Allison Price) Date: Tue, 21 Jun 2022 14:06:46 +0200 Subject: Few Open Questions: Openstack vs Proprietary cloud stacks In-Reply-To: References: Message-ID: <37314381-3D50-4359-80CA-16C462A508DA@openinfra.dev> Hi Krish, I can answer a few of your questions in line. > On Jun 21, 2022, at 8:59 AM, KK CHN wrote: > > List, > > Very recently I came across a discussion with a banking corporation who are heavy users of VMware in all their Data Centre operations .. > > They asked me a lot of questions in comparison with openstack vs other proprietary vendors (such as VMware. ). > > > As I am promoting OpenStack I need to clarify these questions raised by the organization. > > I request your inputs for these queries . > > 1. Does OpenStack support Hyper Converged Infrastructure ? > Or any HCI vendors already with OpenStack solutions? Please share the URL/link of case studies of OpenStack HCI. > Any documentation/Reference to install openstack as HCI solution with supported H/W OEM details . I can try out ... > > 2. Is there any performance benchmark for openstack to compare with other such cloud stacks ( URL/reference links kindly share ) > > 3. What is the market share of OpenStack all over the world ? ( If any statistics kindly share.) > Any survey reports available how many large and medium organizations are using Openstack distributions for private/public cloud use in production ? There is not an exact report that shows OpenStack market share, but there are a lot of statistics about the growing footprint of OpenStack-powered clouds that we collect from the OpenStack user survey (openstack.org/user-survey). - There are currently over 25 million compute cores of OpenStack in production - There are over 180 public cloud data centers running OpenStack - The market for OpenStack is over $7.7 billion USD (source: 451 Research, Market Monitor, Open Source Software, OpenStack 2019) We are planning to publish a new report later this year highlighting the current landscape of OpenStack adoption. Since the survey is opt-in, it is not comprehensive of all OpenStack use cases, but it gives a pretty good picture of the evolution and maturity of deployments. > > 4. Does OpenStack new versions support VDI(Virtual Desktop Infrastructure) solutions out of the box.? pls give a reference link / url for the same. > > 5. What is the alternative for VROPS and VRA (including self service capabilities, Blueprint/Template based deployment) in OpenStack and any comparative study on the capabilities available. ( Please share inputs if any to refer. ) > > 6. What is the alternative of VMWARE Network Insights and VMWARE CodeStream and Vrealize Loginsights in OpenStack ? > > 7. Any major capability available in VMware cloud stack for which alternative is not available in OpenStack out of the box ? > > 8. Tools to maintain multiple OpenStack Cluster, any concept of Supervisory cluster available > > 9. Who all are the large organizations who deployed OpenStack as their alternative over proprietary solutions(eg: VMware cloud and virtualization products ..) > > 10. Does OpenInfra Foundation also provide direct support For OpenStack Deployment ? If yes Kindly share the link / url where to contact /approach for paid support ? in case the clients need paid support ? The OpenInfra Foundation doesn?t directly provide support, but we have an OpenStack Marketplace (openstack.org/marketplace) where we work with our members to highlight their OpenStack-powered products. This list is not comprehensive, but it?s a great directory of OpenStack-powered products. > > 11. Regarding security of cloud environments /infrastructure, what are the USPs of OpenStack cloud security over other competing cloud middleware providers ? and the vulnerability handling model/mechanism in production ? > > Thanks in advance for your invaluable inputs. > > Each and every input from each member is valuable for me to showcase as solid proofs to defend and promote the openstack over proprietary market leaders. > > Krish Please let me know if you have any questions on any of the above resources / data points that I shared. Allison > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ozcan.isik at cern.ch Tue Jun 21 12:12:21 2022 From: ozcan.isik at cern.ch (Oezcan Isik) Date: Tue, 21 Jun 2022 12:12:21 +0000 Subject: [Dev] [Skyline] Boot from Image directly Message-ID: Hi, Will Skyline support boot from image directly during instance creation. Instead of choosing a system disk (volume) every time. Kind Regards, -- Ozcan Isik Software Engineer STAG CERN IT -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Tue Jun 21 12:24:43 2022 From: eblock at nde.ag (Eugen Block) Date: Tue, 21 Jun 2022 12:24:43 +0000 Subject: Regarding Floating IP is existing Setup In-Reply-To: References: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> Message-ID: <20220621122443.Horde.O9wXKZPeDMvt8haAkQ43hTH@webmail.nde.ag> Do you have ha routers enabled? > Most of the template having issue , Have all instance having FLoating IP, a > Stack with a single floating IP have chance of issue very less I don't quite understand that sentence, could you please clarify? Zitat von Adivya Singh : > hi Eugen, > > The current setup is 3 controller nodes, The Load is not much on each > controller and the number of DHCP agent is always set to 2 as per the > standard in the neutron.conf, The L3 agent seems to be stables as other > router namespace works fine under it, Only few router Namespace get > affected under the agent. > > Most of the template having issue , Have all instance having FLoating IP, a > Stack with a single floating IP have chance of issue very less > > Regards > Adivya Singh > > On Tue, Jun 21, 2022 at 1:18 PM Eugen Block wrote: > >> Hi, >> >> this sounds very familiar to me, I had to deal with something similar >> a couple of times in a heavily used cluster with 2 control nodes. What >> does your setup look like, is it a HA setup? I would start checking >> the DHCP and L3 agents. After increasing dhcp_agents_per_network to 2 >> in neutron.conf and restarting the services this didn't occur again >> (yet). This would impact floating IPs as well, sometimes I had to >> disable and enable the affected router(s). If you only have one >> control node a different approach is necessary. Do you see a high load >> on the control node? >> >> >> Zitat von Adivya Singh : >> >> > hi Team, >> > >> > We got a issue in Xena release, where we set the environment in Ubuntu >> > Platform, But later we get some issues in Floating IP not reachable. >> > >> > In a Network node, not all router namespace are Impacted and only few of >> > them get affected, So we can not comment Network node has a issue. >> > >> > The L3 agent where the Router is tied up, Worked just fine, as other >> > routers work Fine. >> > >> > and the one having issue in Floating IP, if i unassigned and assigned it >> > starts working most of the time. >> > >> > Any thoughts on this >> > >> > Regards >> > Adivya Singh >> >> >> >> >> From ozzzo at yahoo.com Tue Jun 21 13:26:46 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Tue, 21 Jun 2022 13:26:46 +0000 (UTC) Subject: Swift issues in one cluster In-Reply-To: <20220617220556.0be3fb91@niphredil.zaitcev.lan> References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> Message-ID: <845416351.6689926.1655818006920@mail.yahoo.com> All of our endpoints are https: +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | | | keystone | identity | True | internal | https://api-int..:5000 | | | | swift | object-store | True | public | https://swift. .:8080/v1/AUTH_%(tenant_id)s | | | | swift | object-store | True | internal | https://swift..:8080/v1/AUTH_%(tenant_id)s | | | | keystone | identity | True | admin | https://api-int. .:35357 | | | | keystone | identity | True | public | https://api-ext. .:5000 | | | | swift | object-store | True | admin | https://swift. .:8080/v1 | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ I don't think this is causing the issue; all of our clusters are setup the same. We did think it was load at first, and got the 2 heaviest users to stop what they were doing, but that didn't make a difference. Our other QA cluster has similar load and identical hardware. When I look at the network graphs, I see traffic spiking up to 1G, but these are 25G interfaces, and none of the resources on the boxes are exhausted. CPU is 97% idle; memory is 30% used, disk is not full. It doesn't look like the problem is load-related. We see the haproxy connections stacking up even when load is low. What else could be causing this? On Friday, June 17, 2022, 11:12:36 PM EDT, Pete Zaitcev wrote: On Fri, 17 Jun 2022 17:33:27 +0000 (UTC) Albert Braden wrote: > $ openstack container list > Unable to establish connection to https://swift..:8080/v1/AUTH_: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') Right away I have a question: why in the world are you connecting to 8080 with HTTPS? > (from Splunk): > Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) > Payload: swift-proxy-server: STDERR: BlockingIOError > Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) > Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) This looks quite fishy to me, because the os.read is in swift/common/utils.py and it's responsible for the mutex. > When we look at network connections, we see haproxy stacking up (many lines of this): >? > # netstat -ntup | sort -b -k2 -n -r | head -n +100 > tcp? 5976932? ? ? 0 127.0.0.1:60738? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5976446? ? ? 0 127.0.0.1:58480? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973217? ? ? 0 127.0.0.1:33244? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973120? ? ? 0 127.0.0.1:51836? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5971968? ? ? 0 127.0.0.1:58516? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? >? ... > > If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue? Indeed if so many requests are established, you're in trouble. The best fix, I think, is to find the customer who's doing it and punish them. Otherwise, quotas and the ratelimiting middleware are your friends. There's also a possibility that your cluster is underperforming, although usually that results in 500 results first. But then again, at times users would "compensate" for issues by just launching way more requests, in effect DoS-ing the cluster even worse. -- Pete -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Tue Jun 21 13:45:35 2022 From: zigo at debian.org (Thomas Goirand) Date: Tue, 21 Jun 2022 15:45:35 +0200 Subject: Few Open Questions: Openstack vs Proprietary cloud stacks In-Reply-To: References: Message-ID: On 6/21/22 08:59, KK CHN wrote: > List, > > Very recently? I came across a discussion with a banking corporation who > are? heavy users of VMware in all their Data Centre operations .. > > They asked me a? lot of questions in comparison with openstack vs other > proprietary vendors (such as? VMware.?? ). > > > As I am promoting OpenStack? I need to clarify these questions raised by > the organization. > > I request your inputs for these queries . > > 1. Does OpenStack? support Hyper Converged Infrastructure ? > Or any HCI vendors already? with OpenStack solutions? Please share the > URL/link of case studies of OpenStack HCI. > Any documentation/Reference? to install openstack as HCI solution? with > supported H/W OEM details . I can try out ... OpenStack does, as you are free to deploy the way you want. However, from my point of view, this is a bad idea: it's hard to get guaranteed I/O if you mix Ceph OSD and compute roles. > 4. Does OpenStack new versions support VDI(Virtual Desktop > Infrastructure) solutions out of the box.? pls give a reference link / > url for the same. Yes. 2 ways to do it: give a full PCI device to your instance, or use virtualized GPU. Nova can do both. > 8. Tools to maintain multiple OpenStack Cluster Yes. > any concept of Supervisory cluster available I don't know what that is (and many of the VMWare concept you mentioned but I didn't quote). I hope the above helps. Cheers, Thomas Goirand (zigo) From lokendrarathour at gmail.com Tue Jun 21 07:19:14 2022 From: lokendrarathour at gmail.com (Lokendra Rathour) Date: Tue, 21 Jun 2022 12:49:14 +0530 Subject: [Tripleo] - IPv6 Wallaby Undercloud Installation failure In-Reply-To: References: Message-ID: Hi Team, So, I am still not able to provision overcloud baremetal nodes using IPv6. Based on the previous discussion i tried changing some settings in terms of IPV6 Notation, but no luck. Has anyone of us tried ipv6 based overcloud deployment on Wallaby? Please suggest some way forward. Thanks once again for your support. -Lokendra On Sat, Jun 18, 2022 at 10:54 PM Dan Sneddon wrote: > It is arguable that dnsmasq should accept the IP address in brackets, > since that is a standard way of representing IPv6 addresses. It gets > confusing, see these ServerFault answers for more detail: > > > https://serverfault.com/questions/444554/what-does-mean-as-an-ip-address-bracket-colon-colon-bracket > > http://serverfault.com/questions/1026466/ddg#1026469 > > We originally had a problem where IPv6 addresses needed to be in brackets > to support URL representation. In TripleO at one time we even had two > versions of hiera values, with and without the brackets. I don?t know if > that still exists or not. > > I wouldn?t hold my breath about getting such a change into dnsmasq. It is > notoriously difficult to get patches accepted into dnsmasq, especially > where UI is concerned. > > -Dan Sneddon > > On Thu, Jun 16, 2022 at 5:32 AM Brendan Shephard > wrote: > >> Hey, >> >> Looks like that is the problem. The [ ] around the IP address are causing >> the issue. If I try to run dnsmasq using exactly the output you get, it >> gives me the same error: >> [root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground >> --log-facility=/var/log/ironic/dnsmasq.log --user=root >> --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 >> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >> >> dnsmasq: bad command line options: try --help >> >> VS without the [ ] I can see it starts up normally. >> >> The settings in your undercloud.conf file look to be correct I believe. >> So I think there might be a bug here. I don't think we should be saving >> that value with the square brackets, or we would need to filter them out >> when we gather the value in that variable. >> >> I raised a bug for it here so that we can dig into this and find what >> needs fixing: >> https://bugs.launchpad.net/tripleo/+bug/1978892 >> >> In the meantime, if you edit that hieradata value, are you able to get >> that container started? >> >> Change this: >> [root at tripleo-director ~]# egrep -r 'tftp_bind_host' >> /etc/puppet/hieradata/ >> /etc/puppet/hieradata/service_configs.json: >> "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", >> >> To this: >> "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" >> >> Then restart the service: >> sudo systemctl restart tripleo_ironic_pxe_http.service >> tripleo_ironic_pxe_tftp.service >> >> Does that get the container running without the error? I did the same in >> my environment and can see that dnsmasq is running properly like that: >> [root at tripleo-director ~]# ps -ef | grep aaaa >> root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq >> --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root >> --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 >> --enable-tftp --tftp-root=/var/lib/ironic/tftpboot >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> >> >> >> >> >> Brisbane City QLD >> >> 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour < >> lokendrarathour at gmail.com> wrote: >> >>> Hi Shephard, >>> I am getting the local_ip (ipv6) of the undercloud : >>> >>> [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c >>> /etc/puppet/hiera.yaml >>> [aaaa:aaaa:aaaa::1] >>> >>> is this because of some ipv6 reasons? >>> >>> >>> On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard >>> wrote: >>> >>>> Hey, >>>> >>>> Ok, that command looks fine. What about that variable there? Do you get >>>> anything back when you run: >>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>> >>>> Mine returns: >>>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml >>>> 192.168.24.115 >>>> >>>> Brendan Shephard >>>> >>>> Software Engineer >>>> >>>> Red Hat APAC >>>> >>>> 193 N Quay >>>> >>>> >>>> Brisbane City QLD >>>> >>>> 4000 >>>> @RedHat >>>> Red >>>> Hat >>>> Red >>>> Hat >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < >>>> lokendrarathour at gmail.com> wrote: >>>> >>>>> Hi Shephard, >>>>> >>>>> this is the command from my wallaby: >>>>> [root at undercloud ~]# sudo cat >>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>> { >>>>> "cap_add": [ >>>>> "NET_ADMIN", >>>>> "NET_RAW", >>>>> "SETUID" >>>>> ], >>>>> "command": [ >>>>> "/bin/bash", >>>>> "-c", >>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>> ], >>>>> "environment": { >>>>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", >>>>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" >>>>> }, >>>>> "healthcheck": { >>>>> "test": "/openstack/healthcheck" >>>>> }, >>>>> "image": >>>>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", >>>>> "net": "host", >>>>> "privileged": false, >>>>> "restart": "always", >>>>> "start_order": 90, >>>>> "volumes": [ >>>>> "/etc/hosts:/etc/hosts:ro", >>>>> "/etc/localtime:/etc/localtime:ro", >>>>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro", >>>>> >>>>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro", >>>>> >>>>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro", >>>>> >>>>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", >>>>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", >>>>> "/dev/log:/dev/log", >>>>> "/etc/puppet:/etc/puppet:ro", >>>>> >>>>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro", >>>>> >>>>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", >>>>> "/var/lib/ironic:/var/lib/ironic:shared,z", >>>>> "/var/log/containers/ironic:/var/log/ironic:z", >>>>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" >>>>> ] >>>>> }[root at undercloud ~]# >>>>> >>>>> Comparing both, they look alike. >>>>> please check once. >>>>> >>>>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Looks like the command was in a different file in Wallaby, can you >>>>>> check: >>>>>> sudo cat >>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>> >>>>>> That one should have the dnsmasq command it's trying to run. For >>>>>> example, here it is from my Wallaby environment: >>>>>> [stack at undercloud-0 ~]$ sudo cat >>>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json >>>>>> | jq .command >>>>>> [ >>>>>> "/bin/bash", >>>>>> "-c", >>>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c >>>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground >>>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root >>>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp >>>>>> --tftp-root=/var/lib/ironic/tftpboot" >>>>>> ] >>>>>> >>>>>> >>>>>> >>>>>> Brendan Shephard >>>>>> >>>>>> Software Engineer >>>>>> >>>>>> Red Hat APAC >>>>>> >>>>>> 193 N Quay >>>>>> >>>>>> >>>>>> Brisbane City QLD >>>>>> >>>>>> 4000 >>>>>> @RedHat Red Hat >>>>>> Red Hat >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < >>>>>> lokendrarathour at gmail.com> wrote: >>>>>> >>>>>>> Hi Shephard, >>>>>>> Here is the o/p of the file: >>>>>>> >>>>>>> [root at undercloud ~]# sudo cat >>>>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>> { >>>>>>> "config_files": [ >>>>>>> { >>>>>>> "dest": "/", >>>>>>> "merge": true, >>>>>>> "preserve_properties": true, >>>>>>> "source": "/var/lib/kolla/config_files/src/*" >>>>>>> } >>>>>>> ], >>>>>>> "permissions": [ >>>>>>> { >>>>>>> "owner": "ironic:ironic", >>>>>>> "path": "/var/log/ironic", >>>>>>> "recurse": true >>>>>>> }, >>>>>>> { >>>>>>> "owner": "ironic:ironic", >>>>>>> "path": "/var/lib/ironic", >>>>>>> "recurse": true >>>>>>> } >>>>>>> ] >>>>>>> }[root at undercloud ~]# >>>>>>> >>>>>>> >>>>>>> Thanks once agan. >>>>>>> >>>>>>> -Lokendra >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard < >>>>>>> bshephar at redhat.com> wrote: >>>>>>> >>>>>>>> Looks like something wrong with the dnsmasq command the container >>>>>>>> is being launched with. What command is it trying to run? >>>>>>>> >>>>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json >>>>>>>> >>>>>>>> Brendan Shephard >>>>>>>> >>>>>>>> >>>>>>>> Software Engineer >>>>>>>> >>>>>>>> Red Hat APAC >>>>>>>> >>>>>>>> 193 N Quay >>>>>>>> >>>>>>>> >>>>>>>> Brisbane City QLD >>>>>>>> >>>>>>>> 4000 >>>>>>>> @RedHat Red Hat >>>>>>>> Red Hat >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Brendan, >>>>>>>>> >>>>>>>>> Thanks for your response. >>>>>>>>> >>>>>>>>> Please find the log below. >>>>>>>>> >>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp >>>>>>>>> >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> dnsmasq: bad command line options: try --help >>>>>>>>> >>>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter >>>>>>>>> name=ironic_pxe -a >>>>>>>>> CONTAINER ID IMAGE >>>>>>>>> COMMAND CREATED >>>>>>>>> STATUS PORTS NAMES >>>>>>>>> 02dacbc74cec >>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) >>>>>>>>> ironic_pxe_tftp >>>>>>>>> 1f8ca39fba32 >>>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo >>>>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy) >>>>>>>>> ironic_pxe_http >>>>>>>>> >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> Anirudh Gupta >>>>>>>>> >>>>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < >>>>>>>>> bshephar at redhat.com> wrote: >>>>>>>>> >>>>>>>>>> Hey Anirudh, >>>>>>>>>> >>>>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container >>>>>>>>>> to see why it's failing. >>>>>>>>>> >>>>>>>>>> I assume the tftp container is not Up when you run this command? >>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps >>>>>>>>>> --filter name=ironic_pxe -a >>>>>>>>>> CONTAINER ID IMAGE >>>>>>>>>> COMMAND CREATED STATUS >>>>>>>>>> PORTS NAMES >>>>>>>>>> 0170be36e291 >>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>> ironic_pxe_tftp >>>>>>>>>> e507f722bdf0 >>>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >>>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy) >>>>>>>>>> ironic_pxe_http >>>>>>>>>> >>>>>>>>>> Then check the logs to see what the error is: >>>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs >>>>>>>>>> ironic_pxe_tftp >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Brendan Shephard >>>>>>>>>> >>>>>>>>>> Software Engineer >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Red >>>>>>>>>> Hat APAC >>>>>>>>>> >>>>>>>>>> 193 N Quay >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Brisbane City QLD >>>>>>>>>> >>>>>>>>>> 4000 >>>>>>>>>> @RedHat Red Hat >>>>>>>>>> Red Hat >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta < >>>>>>>>>> anyrude10 at gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Team, >>>>>>>>>>> >>>>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>>>>>>>>>> facing the below error: >>>>>>>>>>> >>>>>>>>>>> 2022-06-14 05:01:23.213708 | >>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TASK | Manage container systemd >>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 >>>>>>>>>>> 2022-06-14 05:03:22.912816 | >>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | FATAL | Manage container systemd >>>>>>>>>>> services and cleanup old systemd healthchecks for >>>>>>>>>>> /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | >>>>>>>>>>> error={"changed": false, "msg": "Service ironic_pxe_tftp has not started >>>>>>>>>>> yet"} >>>>>>>>>>> 2022-06-14 05:03:22.914400 | >>>>>>>>>>> 52540083-cfa2-3f20-e9dc-00000000286f | TIMING | tripleo_container_manage : >>>>>>>>>>> Manage container systemd >>>>>>>>>>> >>>>>>>>>>> Sample Undercloud.conf is as follows: >>>>>>>>>>> >>>>>>>>>>> [DEFAULT] >>>>>>>>>>> clean_nodes = true >>>>>>>>>>> cleanup = false >>>>>>>>>>> container_cli = podman >>>>>>>>>>> container_healthcheck_disabled = true >>>>>>>>>>> container_images_file = >>>>>>>>>>> /home/stack/containers-prepare-parameter.yaml >>>>>>>>>>> deployment_user = stack >>>>>>>>>>> enable_ironic = true >>>>>>>>>>> enable_ironic_inspector = true >>>>>>>>>>> enable_neutron = true >>>>>>>>>>> enable_routed_networks = false >>>>>>>>>>> generate_service_certificate = false >>>>>>>>>>> ipv6_address_mode = dhcpv6-stateful >>>>>>>>>>> ipxe_enabled = true >>>>>>>>>>> local_interface = enp8s0 >>>>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64 >>>>>>>>>>> subnets = ctlplane-subnet >>>>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>> undercloud_hostname = undercloud.com >>>>>>>>>>> undercloud_ntp_servers = 30.30.30.3 >>>>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>>>>>>>>>> undercloud_timezone = UTC >>>>>>>>>>> >>>>>>>>>>> [ctlplane-subnet] >>>>>>>>>>> cidr = aaaa:aaaa:aaaa::/64 >>>>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f >>>>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a >>>>>>>>>>> gateway = aaaa:aaaa:aaaa::1 >>>>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>>>>>>>>>> >>>>>>>>>>> Can someone please help in this regard. >>>>>>>>>>> >>>>>>>>>>> Anirudh Gupta >>>>>>>>>>> >>>>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ~ Lokendra >>>>>>> www.inertiaspeaks.com >>>>>>> www.inertiagroups.com >>>>>>> skype: lokendrarathour >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> ~ Lokendra >>>>> www.inertiaspeaks.com >>>>> www.inertiagroups.com >>>>> skype: lokendrarathour >>>>> >>>>> >>>>> >>> >>> -- >>> ~ Lokendra >>> www.inertiaspeaks.com >>> www.inertiagroups.com >>> skype: lokendrarathour >>> >>> >>> -- > Dan Sneddon | Senior Principal Software Engineer > dsneddon at redhat.com | redhat.com/cloud > dsneddon:irc | @dxs:twitter > -- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbultel at redhat.com Tue Jun 21 14:21:44 2022 From: mbultel at redhat.com (Mathieu Bultel) Date: Tue, 21 Jun 2022 16:21:44 +0200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: indeed +1 to both On Tue, Jun 21, 2022 at 12:33 PM Lokendra Rathour wrote: > +1 to Both. > > On Thu, Jun 9, 2022 at 1:14 AM James Slagle > wrote: > >> >> I'm formally proposing that we add Brendan Shephard and Takashi Kajinami >> to the core reviewers team for TripleO. Takashi is already considered core >> on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and >> likewise for Brendan. >> >> They both have been contributing and reviewing for a little while now, >> and I feel it's time to add them to the team. Please +1 or let me know any >> concerns before Friday June 15th. Thanks! >> >> -- >> -- James Slagle >> -- >> > > > -- > ~ Lokendra > www.inertiaspeaks.com > www.inertiagroups.com > skype: lokendrarathour > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at optimcloud.com Tue Jun 21 14:25:05 2022 From: lists at optimcloud.com (Embedded Devel) Date: Tue, 21 Jun 2022 14:25:05 +0000 Subject: Few Open Questions: Openstack vs Proprietary cloud stacks In-Reply-To: References: Message-ID: <1655821428475.3572424077.3112503886@optimcloud.com> On Tuesday 21 June 2022 09:45:35 AM (-04:00), Thomas Goirand wrote: > On 6/21/22 08:59, KK CHN wrote: > > List, > > Very recently I came across a discussion with a banking corporation who are heavy users of VMware in all their Data Centre operations .. > > They asked me a lot of questions in comparison with openstack vs other proprietary vendors (such as VMware. ). > > As I am promoting OpenStack I need to clarify these questions raised by the organization. > > I request your inputs for these queries . > > 1. Does OpenStack support Hyper Converged Infrastructure ? > > Or any HCI vendors already with OpenStack solutions? Please share the URL/link of case studies of OpenStack HCI. > > Any documentation/Reference to install openstack as HCI solution with supported H/W OEM details . I can try out ... StarlingX - kubernetes/openstack-helm/ceph integrated into an installable iso.. > > OpenStack does, as you are free to deploy the way you want. However, from my point of view, this is a bad idea: it's hard to get guaranteed I/O if you mix Ceph OSD and compute roles. > > > 4. Does OpenStack new versions support VDI(Virtual Desktop Infrastructure) solutions out of the box.? pls give a reference link / url for the same. > > Yes. 2 ways to do it: give a full PCI device to your instance, or use virtualized GPU. Nova can do both. > > > 8. Tools to maintain multiple OpenStack Cluster > > Yes. > > > any concept of Supervisory cluster available > > I don't know what that is (and many of the VMWare concept you mentioned but I didn't quote). > > I hope the above helps. > > Cheers, > > Thomas Goirand (zigo) > > -- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com From eblock at nde.ag Tue Jun 21 14:28:30 2022 From: eblock at nde.ag (Eugen Block) Date: Tue, 21 Jun 2022 14:28:30 +0000 Subject: Regarding Floating IP is existing Setup In-Reply-To: References: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> <20220621122443.Horde.O9wXKZPeDMvt8haAkQ43hTH@webmail.nde.ag> Message-ID: <20220621142830.Horde.HNgnhw2tW4FPuyx_qq2s7Em@webmail.nde.ag> Could this be a race condition? I saw something similar, too, when a user tried to attach floating IPs via terraform or ansible (I'm not quite sure) at the same time to two different VMs. We could trace it in the Logs that the port was first attached to the first instance and then failed for the other. I don't recall if they use a workaround at the moment or if just works now, will need to check with them. If this is a race condition can you do anything about it in Heat? Like some sort of delay? It could be a bug, though. Zitat von Adivya Singh : > Hi Eugen, > > These are orchestration Heat templates, if there is a Single FLoating IP > creation using this templates it works fine, But if there are multiple > floating IP attachment it shows there is a issue on 2 to 3 floating IP > attached to the instance > > Regards > Adivya Singh > > On Tue, Jun 21, 2022 at 5:59 PM Eugen Block wrote: > >> Do you have ha routers enabled? >> >> > Most of the template having issue , Have all instance having FLoating >> IP, a >> > Stack with a single floating IP have chance of issue very less >> >> I don't quite understand that sentence, could you please clarify? >> >> Zitat von Adivya Singh : >> >> > hi Eugen, >> > >> > The current setup is 3 controller nodes, The Load is not much on each >> > controller and the number of DHCP agent is always set to 2 as per the >> > standard in the neutron.conf, The L3 agent seems to be stables as other >> > router namespace works fine under it, Only few router Namespace get >> > affected under the agent. >> > >> > Most of the template having issue , Have all instance having FLoating >> IP, a >> > Stack with a single floating IP have chance of issue very less >> > >> > Regards >> > Adivya Singh >> > >> > On Tue, Jun 21, 2022 at 1:18 PM Eugen Block wrote: >> > >> >> Hi, >> >> >> >> this sounds very familiar to me, I had to deal with something similar >> >> a couple of times in a heavily used cluster with 2 control nodes. What >> >> does your setup look like, is it a HA setup? I would start checking >> >> the DHCP and L3 agents. After increasing dhcp_agents_per_network to 2 >> >> in neutron.conf and restarting the services this didn't occur again >> >> (yet). This would impact floating IPs as well, sometimes I had to >> >> disable and enable the affected router(s). If you only have one >> >> control node a different approach is necessary. Do you see a high load >> >> on the control node? >> >> >> >> >> >> Zitat von Adivya Singh : >> >> >> >> > hi Team, >> >> > >> >> > We got a issue in Xena release, where we set the environment in Ubuntu >> >> > Platform, But later we get some issues in Floating IP not reachable. >> >> > >> >> > In a Network node, not all router namespace are Impacted and only few >> of >> >> > them get affected, So we can not comment Network node has a issue. >> >> > >> >> > The L3 agent where the Router is tied up, Worked just fine, as other >> >> > routers work Fine. >> >> > >> >> > and the one having issue in Floating IP, if i unassigned and assigned >> it >> >> > starts working most of the time. >> >> > >> >> > Any thoughts on this >> >> > >> >> > Regards >> >> > Adivya Singh >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> From ricolin at ricolky.com Tue Jun 21 14:45:30 2022 From: ricolin at ricolky.com (Rico Lin) Date: Tue, 21 Jun 2022 22:45:30 +0800 Subject: Propose to add Takashi Kajinami as heat core reviewer In-Reply-To: References: Message-ID: Added Takashi Kajinami to heat core now. And welcome on board Takashi :) *Rico Lin* On Wed, Jun 15, 2022 at 1:22 PM Brendan Shephard wrote: > +1 for sure. Takashi is an asset to all of OpenStack! I would happily > endorse his inclusion as a Core reviewer to the Heat project. > > > Brendan Shephard > > Software Engineer > > Red Hat APAC > > 193 N Quay > > Brisbane City QLD 4000 > @RedHat Red Hat > Red Hat > > > > > > On Mon, Jun 13, 2022 at 11:55 PM Rabi Mishra wrote: > >> >> >> On Mon, Jun 13, 2022 at 6:43 PM Rico Lin wrote: >> >>> Hi all >>> >>> I want to suggest adding Takashi Kajinami as heat >>> core reviewer. >>> Who help provide valid reviews and fixes for issues on heat. >>> Will be great to have Takashi Kajinami join heat core reviewer team and >>> help. >>> >> >> +1, Good addition, he is actively involved for some time and contributes >> to keep the project healthy . >> >> >>> >>> *Rico Lin* >>> >> >> >> -- >> Regards, >> Rabi Mishra >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From tkajinam at redhat.com Tue Jun 21 14:53:12 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Tue, 21 Jun 2022 23:53:12 +0900 Subject: Propose to add Takashi Kajinami as heat core reviewer In-Reply-To: References: Message-ID: Thank you for your warm feedback ! I'll keep my effort and do my best . :-D On Tue, Jun 21, 2022 at 11:45 PM Rico Lin wrote: > > Added Takashi Kajinami to heat core now. > > And welcome on board Takashi :) > > *Rico Lin* > > > On Wed, Jun 15, 2022 at 1:22 PM Brendan Shephard > wrote: > >> +1 for sure. Takashi is an asset to all of OpenStack! I would happily >> endorse his inclusion as a Core reviewer to the Heat project. >> >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat Red Hat >> Red Hat >> >> >> >> >> >> On Mon, Jun 13, 2022 at 11:55 PM Rabi Mishra wrote: >> >>> >>> >>> On Mon, Jun 13, 2022 at 6:43 PM Rico Lin wrote: >>> >>>> Hi all >>>> >>>> I want to suggest adding Takashi Kajinami as >>>> heat core reviewer. >>>> Who help provide valid reviews and fixes for issues on heat. >>>> Will be great to have Takashi Kajinami join heat core reviewer team and >>>> help. >>>> >>> >>> +1, Good addition, he is actively involved for some time and >>> contributes to keep the project healthy . >>> >>> >>>> >>>> *Rico Lin* >>>> >>> >>> >>> -- >>> Regards, >>> Rabi Mishra >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.goncharov at gmail.com Tue Jun 21 15:03:00 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Tue, 21 Jun 2022 17:03:00 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat Message-ID: Hi all, During the summit we had some nice conversations and I wanted to actually use the chance and try to formalise the discussion to gather broad opinions and ideally solutions. Sidenotes:>>>>> - I enjoy opendev - I really enjoy gerrit and find it an ultimate code review system - I can understand people not able to cope with additional complexity and prefer pull request pattern to the change pattern - I use PR change pattern for relatively simple projects (and lot of other huge projects do it this way) - I am not blaming and not going to blame anybody here End sidenotes>>>>>> - Some month ago gophercloud folks came to us and wanted to give source code under the foundation. Feedback from us was more or less: you either host on opendev and use gerrit or not at all. Members of gopher cloud rejected proceeding this way with detailed reasoning. I can fully understand reasonings and I also observe similar issues with auto-closed GitHub PRs (for SDK or OSC) and the fact that those changes in most of the cases never really reach gerrit. Especially in the SDK area it would have been very useful to get various SDKs under single team to spread the know how and synchronise their feature sets. Maybe even use shared credentials to public OpenStack clouds to verify and publish their coverage. - Some software solutions (especially those based on Golang) were designed with GitHub in mind and are actually not really working outside. For example in order to publish provider for HashiCorp Terraform into their registry your product must be hosted on GitHub (particularly you must upload build artifacts as release on GH). There is sadly no other possibility. - My company has created HashiCorp secretstore plugin for OpenStack and we are willing to donate this under the foundation to have proper ownership of the software (not to have our company name in the hosting path since it is pretty confusing). From features point of view there is still place to evolve and it would be cool to get other interested parties to participate. We would be able to OpenDev without conceptual issues and are going to request projects for that, but that still has one issue: publishing of binary artifacts - not immediately available in opendev (afaik). On GitHub we publish it as GH Release. As such the point I am trying to address is that we as a very big project should be bit more open to the world of tools around OpenStack (I mean explicitly **not** the OpenStack itself) and allow easy integration for those who want to extend OpenStack ecosystem. Depending on the particular application this may set certain limitations on the solution (i.e. must be on GitHub, must use Jira, whatever) which might be not compatible out of box with the OpenDev scheme. Do we as a open community want to set strict boundaries or can we ease some of those for the ecosystem around of OpenStack? - Would it be possible for the project hosted outside opendev be governed by OIF and somehow visually belong to OpenStack? (As a user I would definitely choose to use GitHub.com/openstack/project_x rather then GitHub.com/some_crappy_nic/project_x) - Can such project enjoy Zuul integration not being part of opendev git hosting? Technically it is surely possible (Ansible modules were tested this way), but requires in my eyes precise definition. P.S. For the case when zuul/github integration (management) is the only limiting factor I will be gladly supporting the integration and maintenance, since we have created an Ansible collection for managing of GitHub organisations/projects/teams/members (in the project-config spirit) which we use to manage our deployment (and I know others are using it as well). Tell me what you think and let us, please, focus on how to cope with different expectations rather which one is right and which is wrong. Thanks a lot, Artem From marios at redhat.com Tue Jun 21 15:09:58 2022 From: marios at redhat.com (Marios Andreou) Date: Tue, 21 Jun 2022 18:09:58 +0300 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: References: Message-ID: o/ thanks to all who helped move along or abandon patches in the last week (& special thanks to Takashi o/). There are currently no open stable/victoria tripleo gerrit reviews (I abandoned the last few patches a little while ago). As promised there is now v2 of [1] with new commits that merged in the last week (add "1..2" to the end of the url at [1] to get the diff). I think we are now ready to move forward so please comment on [1] either way regards, marios [1] https://review.opendev.org/c/openstack/releases/+/845148/ On Tue, Jun 14, 2022 at 1:33 PM Marios Andreou wrote: > > Hello o/ > > as proposed at [1][2] we are moving ahead with transitioning > stable/victoria tripleo repos to End Of Life. > > In order to do this we need to have no open patches against stable/victoria. > > So please stop posting new patches for tripleo stable/victoria and if > you have open patches please merge or abandon these by next Tuesday > 21st June (for reference, list of repos with currently open patches at > [3]). > > No problem if someone needs a few more days we can wait just please > reach out here or on irc. If there are no such requests then I will > abandon all open reviews on 21st and respin [2] to pick up any new > commits between now and then. > > thanks for your help, > > marios > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028829.html > [2] https://review.opendev.org/c/openstack/releases/+/845148 > [3] (list of repos with open reviews): > * https://review.opendev.org/q/project:openstack%252Fos-net-config+status:open+branch:stable/victoria > * https://review.opendev.org/q/project:openstack%252Fpython-tripleoclient+status:open+branch:stable/victoria > * https://review.opendev.org/q/project:openstack%252Ftripleo-ansible+status:open+branch:stable/victoria > * https://review.opendev.org/q/project:openstack%252Ftripleo-common+status:open+branch:stable/victoria > * https://review.opendev.org/q/project:openstack%252Ftripleo-heat-templates+status:open+branch:stable/victoria > * https://review.opendev.org/q/project:openstack%252Ftripleo-image-elements+status:open+branch:stable/victoria > * https://review.opendev.org/q/project:openstack%252Ftripleo-puppet-elements+status:open+branch:stable/victoria > * https://review.opendev.org/q/project:openstack%252Ftripleo-validations+status:open+branch:stable/victoria From marios at redhat.com Tue Jun 21 15:13:40 2022 From: marios at redhat.com (Marios Andreou) Date: Tue, 21 Jun 2022 18:13:40 +0300 Subject: [TripleO] no more patches for tripleo stable/victoria please (going EOL) In-Reply-To: References: Message-ID: I know we just discussed this in IRC but adding a note here for anyone else who may be interested. In the end we have 2 problematic ones [1] and [2] - i.e. the train cherrypick merged for those and it references the victoria commit from the cherrypicked from in the commit message. I realise this isn't ideal but at least it is better than last time - maybe we'll have 0 on the next one ;) thanks for raising it Takashi regards, marios [1] https://review.opendev.org/c/openstack/tripleo-common/+/837804 [2] https://review.opendev.org/c/openstack/tripleo-heat-templates/+/830281 On Tue, Jun 14, 2022 at 2:40 PM Takashi Kajinami wrote: > > I'd like to remind you that we should check whether that patch has been backported to any older branches (ussuri, train) > we abandon any victoria backport. > > In the past, we abandoned backport without checking this and this resulted in broken commit id in some backports. > (stable/train backports point to the commit id of ussuri backport, which was never merged). > > If the patch is already merged in ussuri or train then IMO we should try landing the victoria backport. > If not, then please make sure you update the stable/train backport to remove any reference to the victoria backport. > > > On Tue, Jun 14, 2022 at 7:41 PM Marios Andreou wrote: >> >> Hello o/ >> >> as proposed at [1][2] we are moving ahead with transitioning >> stable/victoria tripleo repos to End Of Life. >> >> In order to do this we need to have no open patches against stable/victoria. >> >> So please stop posting new patches for tripleo stable/victoria and if >> you have open patches please merge or abandon these by next Tuesday >> 21st June (for reference, list of repos with currently open patches at >> [3]). >> >> No problem if someone needs a few more days we can wait just please >> reach out here or on irc. If there are no such requests then I will >> abandon all open reviews on 21st and respin [2] to pick up any new >> commits between now and then. >> >> thanks for your help, >> >> marios >> >> [1] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/028829.html >> [2] https://review.opendev.org/c/openstack/releases/+/845148 >> [3] (list of repos with open reviews): >> * https://review.opendev.org/q/project:openstack%252Fos-net-config+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Fpython-tripleoclient+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-ansible+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-common+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-heat-templates+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-image-elements+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-puppet-elements+status:open+branch:stable/victoria >> * https://review.opendev.org/q/project:openstack%252Ftripleo-validations+status:open+branch:stable/victoria >> >> From fungi at yuggoth.org Tue Jun 21 15:29:28 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 21 Jun 2022 15:29:28 +0000 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: Message-ID: <20220621152928.35wtc6ijmagtpyq6@yuggoth.org> On 2022-06-21 17:03:00 +0200 (+0200), Artem Goncharov wrote: [...] > Some month ago gophercloud folks came to us and wanted to give > source code under the foundation. Feedback from us was more or > less: you either host on opendev and use gerrit or not at all. [...] I think there are some misconceptions or conflation of separate ideas here. First, I hadn't heard that gophercloud folks wanted to be legally represented by the Open Infrastructure Foundation. This is an entirely separate concern from whether or not the project does its development within the OpenDev Collaboratory. There are official OpenInfra projects which rely on Github, Jenkins, and other tools outside OpenDev (Kata Containers is a long-standing example there). If you want a project hosted *within* the OpenDev Collaboratory, that means Gerrit explicitly. The OpenDev community systems administrators run an integrated instance of Gerrit and Zuul, and all projects gated by OpenDev's Zuul deployment must be in OpenDev's Gerrit deployment. As noted, that's an entirely separate concern from whether the project is legally represented by the foundation. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From cboylan at sapwetik.org Tue Jun 21 15:34:18 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 21 Jun 2022 08:34:18 -0700 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: Message-ID: On Tue, Jun 21, 2022, at 8:03 AM, Artem Goncharov wrote: > Hi all, > > During the summit we had some nice conversations and I wanted to > actually use the chance and try to formalise the discussion to gather > broad opinions and ideally solutions. > > Sidenotes:>>>>> > - I enjoy opendev > - I really enjoy gerrit and find it an ultimate code review system > - I can understand people not able to cope with additional complexity > and prefer pull request pattern to the change pattern I think we should be careful with the assertion that Gerrit is more complex. It is different, but I'm not sure it is more complex. Pushing to Gerrit isn't any more complex than pushing to Github. Github even has the extra step of needing to open a pull request explicitly. But it is different and does require learning a different process. > - I use PR change pattern for relatively simple projects (and lot of > other huge projects do it this way) > - I am not blaming and not going to blame anybody here > End sidenotes>>>>>> > > - Some month ago gophercloud folks came to us and wanted to give source > code under the foundation. Feedback from us was more or less: you > either host on opendev and use gerrit or not at all. Members of gopher > cloud rejected proceeding this way with detailed reasoning. I can fully > understand reasonings and I also observe similar issues with > auto-closed GitHub PRs (for SDK or OSC) and the fact that those changes > in most of the cases never really reach gerrit. Especially in the SDK > area it would have been very useful to get various SDKs under single > team to spread the know how and synchronise their feature sets. Maybe > even use shared credentials to public OpenStack clouds to verify and > publish their coverage. > > - Some software solutions (especially those based on Golang) were > designed with GitHub in mind and are actually not really working > outside. For example in order to publish provider for HashiCorp > Terraform into their registry your product must be hosted on GitHub > (particularly you must upload build artifacts as release on GH). There > is sadly no other possibility. This isn't the only instance we've run into this. The CNCF's Cloud Native landscape (https://landscape.cncf.io/) requires open source projects be hosted on Github (not not proprietary systems....). From my perspective, I think this calls out why it is important for us to continue to push back against these rules. Software is built in a number of places and if we force people to use Github (and other proprietary tools) we move away from part of the mission here. > > - My company has created HashiCorp secretstore plugin for OpenStack and > we are willing to donate this under the foundation to have proper > ownership of the software (not to have our company name in the hosting > path since it is pretty confusing). From features point of view there > is still place to evolve and it would be cool to get other interested > parties to participate. We would be able to OpenDev without conceptual > issues and are going to request projects for that, but that still has > one issue: publishing of binary artifacts - not immediately available > in opendev (afaik). On GitHub we publish it as GH Release. OpenDev supports publishing of binary artifacts through a number of mechanisms. OpenStack wheel packages are pushed to pypi, docker images are pushed to Docker Hub and Quay and so on, tarballs.opendev.org hosts other random artifacts as well. It sounds like you specifically need Github Releases though. Can you not trigger releases against mirrored content? I think that should work? You might need to be more specific about what the actual needs are here. > > As such the point I am trying to address is that we as a very big > project should be bit more open to the world of tools around OpenStack > (I mean explicitly **not** the OpenStack itself) and allow easy > integration for those who want to extend OpenStack ecosystem. Depending > on the particular application this may set certain limitations on the > solution (i.e. must be on GitHub, must use Jira, whatever) which might > be not compatible out of box with the OpenDev scheme. Do we as a open > community want to set strict boundaries or can we ease some of those > for the ecosystem around of OpenStack? > > - Would it be possible for the project hosted outside opendev be > governed by OIF and somehow visually belong to OpenStack? (As a user I > would definitely choose to use GitHub.com/openstack/project_x rather > then GitHub.com/some_crappy_nic/project_x) OIF does have projects that are not hosted on OpenDev. Whether or not OpenStack wants to have projects hosted outside of OpenDev is probably the major item to sort out here. > - Can such project enjoy Zuul integration not being part of opendev git > hosting? Technically it is surely possible (Ansible modules were tested > this way), but requires in my eyes precise definition. The OpenDev team tried to provide CI integration with Github provided projects (some Kata repos) and found that we weren't able to do it at a level that was maintainable. Github itself poses a number of challenges, but we also found that projects there being even more disconnected were even less likely to be involved in helping themselves which is something that we currently rely on to make this collaboratory successful. All that to say we (the OpenDev team) cannot provide CI resources to Github repos. We do provide third party CI for a small number of projects with the expectation that people on the OpenDev hosted project side are able to care and feed for that. I think this calls out the fundamental disconnect here. We use open source tools because we believe that open source needs and should be built with open source tools. That does require a little bit of investment from the project side to ensure the tooling functions as needed and is maintained. > > P.S. For the case when zuul/github integration (management) is the only > limiting factor I will be gladly supporting the integration and > maintenance, since we have created an Ansible collection for managing > of GitHub organisations/projects/teams/members (in the project-config > spirit) which we use to manage our deployment (and I know others are > using it as well). > > > Tell me what you think and let us, please, focus on how to cope with > different expectations rather which one is right and which is wrong. > > Thanks a lot, > Artem From tiagohp at gmail.com Tue Jun 21 17:21:19 2022 From: tiagohp at gmail.com (Tiago Pires) Date: Tue, 21 Jun 2022 14:21:19 -0300 Subject: Neutron/OVN + IPv6 Message-ID: Hi all, I am trying to figure out how to enable workloads on overlay to use IPv6 addresses and communicate with external networks. I have to use Neutron ML2/OVN in my scenario and I have doubts regarding how the external networks(Internet) can know how to reach the VMs using IPv6 address behind a regular neutron router. I was checking this link https://cloudbau.github.io/openstack/neutron/networking/2016/05/17/neutron-ipv6.html that uses the BGP speaker on neutron in order to advertise the /64 subnets to a border router, that option should work fine but Neutron/BGP Speaker is not supported with OVN as I could check on the documentation. How are you using IPv6 integration (Neutron/OVN) to advertise the /64 subnets? Tiago Pires -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim+openstack.org at coote.org Tue Jun 21 18:35:47 2022 From: tim+openstack.org at coote.org (tim+openstack.org at coote.org) Date: Tue, 21 Jun 2022 19:35:47 +0100 Subject: Neutron/OVN + IPv6 In-Reply-To: References: Message-ID: <544CB4E9-AB62-48D2-96D5-2FFD2A57018F@coote.org> I think that you?d normally advertise much larger subnets than /64 and then subnet those for different domains. Routing in ipv6 should be much more straightforward as there should be no NAT, which ought to make it much easier to work out what?s talking to what. > On 21 Jun 2022, at 18:21, Tiago Pires wrote: > > Hi all, > > I am trying to figure out how to enable workloads on overlay to use IPv6 addresses and communicate with external networks. > I have to use Neutron ML2/OVN in my scenario and I have doubts regarding how the external networks(Internet) can know how to reach the VMs using IPv6 address behind a regular neutron router. > I was checking this link https://cloudbau.github.io/openstack/neutron/networking/2016/05/17/neutron-ipv6.html that uses the BGP speaker on neutron in order to advertise the /64 subnets to a border router, that option should work fine but Neutron/BGP Speaker is not supported with OVN as I could check on the documentation. > How are you using IPv6 integration (Neutron/OVN) to advertise the /64 subnets? > > Tiago Pires -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgolovat at redhat.com Tue Jun 21 19:22:15 2022 From: sgolovat at redhat.com (Sergii Golovatiuk) Date: Tue, 21 Jun 2022 21:22:15 +0200 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: +1 ??, 8 ???. 2022 ?. ? 21:40, James Slagle : > > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami > to the core reviewers team for TripleO. Takashi is already considered core > on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- > -- Sergii Golovatiuk Senior Software Developer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From tiagohp at gmail.com Tue Jun 21 20:44:23 2022 From: tiagohp at gmail.com (Tiago Pires) Date: Tue, 21 Jun 2022 17:44:23 -0300 Subject: Neutron/OVN + IPv6 In-Reply-To: <544CB4E9-AB62-48D2-96D5-2FFD2A57018F@coote.org> References: <544CB4E9-AB62-48D2-96D5-2FFD2A57018F@coote.org> Message-ID: Hi Tim, >From the border routers to the Internet, it is not a problem to summarize the /64 in /48 for example. My point is how to know the next-hop neutron router's IP that has a subnet /64 behind it? When you have many routers, we need a dynamic mechanism for it. I think the neutron ML2/OVN gap is to not support IPv6 Prefix delegation that would be the better solution for it. And Neutron BGP dynamic with OVN is not supported as I could check in the documentation. So, I don't know if there is another way to use IPv6 using the integration with Neutron and OVN. Regards, Tiago Pires Em ter., 21 de jun. de 2022 ?s 15:35, escreveu: > I think that you?d normally advertise much larger subnets than /64 and > then subnet those for different domains. Routing in ipv6 should be much > more straightforward as there should be no NAT, which ought to make it much > easier to work out what?s talking to what. > > On 21 Jun 2022, at 18:21, Tiago Pires wrote: > > Hi all, > > I am trying to figure out how to enable workloads on overlay to use IPv6 > addresses and communicate with external networks. > I have to use Neutron ML2/OVN in my scenario and I have doubts regarding > how the external networks(Internet) can know how to reach the VMs using > IPv6 address behind a regular neutron router. > I was checking this link > https://cloudbau.github.io/openstack/neutron/networking/2016/05/17/neutron-ipv6.html > that uses the BGP speaker on neutron in order to advertise the /64 subnets > to a border router, that option should work fine but Neutron/BGP Speaker is > not supported with OVN as I could check on the documentation. > How are you using IPv6 integration (Neutron/OVN) to advertise the /64 > subnets? > > Tiago Pires > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kkchn.in at gmail.com Wed Jun 22 05:18:22 2022 From: kkchn.in at gmail.com (KK CHN) Date: Wed, 22 Jun 2022 10:48:22 +0530 Subject: Few Open Questions: Openstack vs Proprietary cloud stacks In-Reply-To: <37314381-3D50-4359-80CA-16C462A508DA@openinfra.dev> References: <37314381-3D50-4359-80CA-16C462A508DA@openinfra.dev> Message-ID: On Tue, Jun 21, 2022 at 5:36 PM Allison Price wrote: > Hi Krish, > > I can answer a few of your questions in line. > > On Jun 21, 2022, at 8:59 AM, KK CHN wrote: > > List, > > Very recently I came across a discussion with a banking corporation who > are heavy users of VMware in all their Data Centre operations .. > > They asked me a lot of questions in comparison with openstack vs other > proprietary vendors (such as VMware. ). > > > As I am promoting OpenStack I need to clarify these questions raised by > the organization. > > I request your inputs for these queries . > > 1. Does OpenStack support Hyper Converged Infrastructure ? > Or any HCI vendors already with OpenStack solutions? Please share the > URL/link of case studies of OpenStack HCI. > Any documentation/Reference to install openstack as HCI solution with > supported H/W OEM details . I can try out ... > > 2. Is there any performance benchmark for openstack to compare with other > such cloud stacks ( URL/reference links kindly share ) > > 3. What is the market share of OpenStack all over the world ? ( If any > statistics kindly share.) > Any survey reports available how many large and medium organizations are > using Openstack distributions for private/public cloud use in production ? > > > There is not an exact report that shows OpenStack market share, but there > are a lot of statistics about the growing footprint of OpenStack-powered > clouds that we collect from the OpenStack user survey ( > openstack.org/user-survey). > > - There are currently over 25 million compute cores of OpenStack in > production > - There are over 180 public cloud data centers running OpenStack > - The market for OpenStack is over $7.7 billion USD (source: 451 Research, > Market Monitor, Open Source Software, OpenStack 2019) > > We are planning to publish a new report later this year highlighting the > current landscape of OpenStack adoption. Since the survey is opt-in, it is > not comprehensive of all OpenStack use cases, but it gives a pretty good > picture of the evolution and maturity of deployments. > I suggest if possible while publishing this new report, it will be better if we can publish also in Gartner, Forrester etc., if this makes sense. I heard about CERN one among the biggest Openstack deployments in the private cloud domain. > > > 4. Does OpenStack new versions support VDI(Virtual Desktop > Infrastructure) solutions out of the box.? pls give a reference link / url > for the same. > > 5. What is the alternative for VROPS and VRA (including self service > capabilities, Blueprint/Template based deployment) in OpenStack and any > comparative study on the capabilities available. ( Please share inputs if > any to refer. ) > > 6. What is the alternative of VMWARE Network Insights and VMWARE > CodeStream and Vrealize Loginsights in OpenStack ? > > 7. Any major capability available in VMware cloud stack for which > alternative is not available in OpenStack out of the box ? > > 8. Tools to maintain multiple OpenStack Cluster, any concept of > Supervisory cluster available > > 9. Who all are the large organizations who deployed OpenStack as their > alternative over proprietary solutions(eg: VMware cloud and virtualization > products ..) > > 10. Does OpenInfra Foundation also provide direct support For OpenStack > Deployment ? If yes Kindly share the link / url where to contact > /approach for paid support ? in case the clients need paid support ? > > > The OpenInfra Foundation doesn?t directly provide support, but we have an > OpenStack Marketplace (openstack.org/marketplace) where we work with our > members to highlight their OpenStack-powered products. This list is not > comprehensive, but it?s a great directory of OpenStack-powered products. > > > 11. Regarding security of cloud environments /infrastructure, what are > the USPs of OpenStack cloud security over other competing cloud middleware > providers ? and the vulnerability handling model/mechanism in production ? > > Thanks in advance for your invaluable inputs. > > Each and every input from each member is valuable for me to showcase as > solid proofs to defend and promote the openstack over proprietary market > leaders. > > Krish > > > > Please let me know if you have any questions on any of the above resources > / data points that I shared. > > Allison > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bxzhu_5355 at 163.com Wed Jun 22 05:34:04 2022 From: bxzhu_5355 at 163.com (Boxiang Zhu) Date: Wed, 22 Jun 2022 13:34:04 +0800 (GMT+08:00) Subject: [Dev] [Skyline] Boot from Image directly In-Reply-To: References: Message-ID: <7eb07751.2b9f.18189e74778.Coremail.bxzhu_5355@163.com> Hi, Ozcan Not support now. Now if you deploy openstack with cinder, skyline will create instance(os disks) with cinder(volume). But if you have no cinder, skyline can support to create instances(os disks) directly from images. Thanks, Best Regards Boxiang Zhu ---- Replied Message ---- | From | Oezcan Isik | | Date | 6/21/2022 20:16 | | To | openstack-discuss | | Subject | [Dev] [Skyline] Boot from Image directly | Hi, Will Skyline support boot from image directly during instance creation. Instead of choosing a system disk (volume) every time. Kind Regards, -- Ozcan Isik Software Engineer STAG CERN IT -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Wed Jun 22 05:48:44 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Wed, 22 Jun 2022 07:48:44 +0200 Subject: Neutron/OVN + IPv6 In-Reply-To: References: <544CB4E9-AB62-48D2-96D5-2FFD2A57018F@coote.org> Message-ID: Hi, As far as I know there's ongoing work to support BGP with OVN. I'm not sure if it's production ready (bet it's not), but you can totally try it out: https://opendev.org/x/ovn-bgp-agent ??, 21 ???. 2022 ?., 22:48 Tiago Pires : > Hi Tim, > > From the border routers to the Internet, it is not a problem to summarize > the /64 in /48 for example. > My point is how to know the next-hop neutron router's IP that has a subnet > /64 behind it? When you have many routers, we need a dynamic mechanism for > it. > I think the neutron ML2/OVN gap is to not support IPv6 Prefix delegation > that would be the better solution for it. > And Neutron BGP dynamic with OVN is not supported as I could check in the > documentation. > So, I don't know if there is another way to use IPv6 using the integration > with Neutron and OVN. > > Regards, > > Tiago Pires > > Em ter., 21 de jun. de 2022 ?s 15:35, > escreveu: > >> I think that you?d normally advertise much larger subnets than /64 and >> then subnet those for different domains. Routing in ipv6 should be much >> more straightforward as there should be no NAT, which ought to make it much >> easier to work out what?s talking to what. >> >> On 21 Jun 2022, at 18:21, Tiago Pires wrote: >> >> Hi all, >> >> I am trying to figure out how to enable workloads on overlay to use IPv6 >> addresses and communicate with external networks. >> I have to use Neutron ML2/OVN in my scenario and I have doubts regarding >> how the external networks(Internet) can know how to reach the VMs using >> IPv6 address behind a regular neutron router. >> I was checking this link >> https://cloudbau.github.io/openstack/neutron/networking/2016/05/17/neutron-ipv6.html >> that uses the BGP speaker on neutron in order to advertise the /64 subnets >> to a border router, that option should work fine but Neutron/BGP Speaker is >> not supported with OVN as I could check on the documentation. >> How are you using IPv6 integration (Neutron/OVN) to advertise the /64 >> subnets? >> >> Tiago Pires >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.rohmann at inovex.de Wed Jun 22 08:18:26 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Wed, 22 Jun 2022 10:18:26 +0200 Subject: R: R: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve metrics using ceilometer and gnocchi In-Reply-To: References: Message-ID: <95293911-9cf4-78aa-c8cf-6e6990d02943@inovex.de> Hey Molka, Rafael, all, On 08/03/2022 21:02, Molka Gharbaoui wrote: > I removed manually "ceilometer-low" and "ceilometer-high" from > archive-policy list (both were creating a conflict) however nothing > changes: > If I execute ceilometer-upgrade/ceilometer-polling nothing "new" happens. > When I execute ceilometer-agent-notification, > ceilometer-low/ceilometer-high are readded to the archive-policy list > and the error reappears. > I guess I should not execute those commands manually but I cannot find > a way to push the metrics to the database. I just ran into this issue when playing with Ceilometer + Gnocchi on a devstack running stable/yoga and tried the same things. To went through the cascade of calls happening in the code. First there is ceilometer "ensuring" the archive policy exists https://github.com/openstack/ceilometer/blob/bf263b11181f4e44850e991282766b3bdf4f41e1/ceilometer/publisher/gnocchi.py#L269 This is done via python-gnocchiclient and the create method at https://github.com/gnocchixyz/python-gnocchiclient/blob/7355fb2d7d3311f5962230a565574ce8c76c1caa/gnocchiclient/v1/archive_policy.py#L35. which then sends a POST to the create REST API endpoint of Gnocci at https://github.com/gnocchixyz/gnocchi/blob/f124cf76479720d3b9caf439241da12b2974ac90/gnocchi/rest/api.py#L302. The mentioned "ensures_archives_policies" method then checks for the "ArchivePolicyAlreadyExists" exception (https://github.com/openstack/ceilometer/blob/bf263b11181f4e44850e991282766b3bdf4f41e1/ceilometer/publisher/gnocchi.py#L274) and apparently this is the issue here - this exception is NOT thrown, but simply a 409 Conflict error. Exceptions are created and then enriched via the https://github.com/gnocchixyz/python-gnocchiclient/blob/7355fb2d7d3311f5962230a565574ce8c76c1caa/gnocchiclient/exceptions.py#L217 method and in our case it should be https://github.com/gnocchixyz/python-gnocchiclient/blob/7355fb2d7d3311f5962230a565574ce8c76c1caa/gnocchiclient/exceptions.py#L161 . So I first thought the string must have been changed somewhere somehow for it to not match anymore, but all that code is unchanged for years. But, when looking at a trace between the apache2 and the unix socket exposing the gnocci-api I saw something strange: --- SENT --- .7....proxy-sendchunked..1..APACHE_RUN_USER..ubuntu..APACHE_RUN_GROUP..ubuntu .PATH_INFO../v1/archive_policy/??? .HTTP_HOST .192.168.0.100..HTTP_USER_AGENTW.ceilometer-agent-notification keystoneauth1/4.5.0 python-requests/2.27.1 CPython/3.8.10..HTTP_ACCEPT_ENCODING .gzip, deflate..HTTP_ACCEPT..application/json, */*..HTTP_CONNECTION .keep-alive..CONTENT_TYPE..application/json..HTTP_X_AUTH_TOKEN..gAAAAABiscoSYYjDWLVJzL8V12knrSmvI5aRYFTH4cZyQ0HKPBrheWqbh-6PS6Cf56kNTuTw5hwhpYa6UNEXkA_aIYj2xk6hTchgFaBOSrIHq36opuo46ktJCXcsEBQnus1_IZnWwuy8RBlzso6LGKeo37dzT_zloUYj_XhaMRDlgUifeAgUESQ..CONTENT_LENGTH..245..PATHF./usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin..SERVER_SIGNATUREJ.
Apache/2.4.41 (Ubuntu) Server at 192.168.0.100 Port 80
..SERVER_SOFTWARE..Apache/2.4.41 (Ubuntu)..SERVER_NAME .192.168.0.100..SERVER_ADDR .192.168.0.100..SERVER_PORT..80..REMOTE_ADDR .192.168.0.100 .DOCUMENT_ROOT../opt/stack/horizon/.blackhole/..REQUEST_SCHEME..http..CONTEXT_PREFIX....CONTEXT_DOCUMENT_ROOT../opt/stack/horizon/.blackhole/..SERVER_ADMIN..[no address given]..SCRIPT_FILENAME6.proxy:uwsgi://uwsgi-uds-gnocchi-api/v1/archive_policy/..REMOTE_PORT..42324..GATEWAY_INTERFACE..CGI/1.1..SERVER_PROTOCOL..HTTP/1.1..REQUEST_METHOD..POST..QUERY_STRING....REQUEST_URI../metric/v1/archive_policy/..SCRIPT_NAME../metric{"name":"ceilometer-high-rate","aggregation_methods":["mean","rate:mean"],"back_window":0,"definition":[{"granularity":"1 second","timespan":"1 hour"},{"granularity":"1 minute","timespan":"1 day"},{"granularity":"1 hour","timespan":"365 days"}]} --- SENT --- --- RECEIVED --- HTTP/1.1 409 Conflict Content-Length: 228 Content-Type: text/html; charset=UTF-8 Connection: close ? ? 409 Conflict ? ? ?

409 Conflict

? There was a conflict when trying to complete your request.

Archive policy ceilometer-high-rate already exists ? --- RECEIVED --- When though a content type of "application/json" was sent, the gnocchi-api returned "Content-Type: text/html; charset=UTF-8" and a "human readable" error instead of the expected JSON. And that is why the exception is not properly casted and we end up in this archive_policy creation loop. Why the response is html I was not yet able to find out. Regards Christian -------------- next part -------------- An HTML attachment was scrubbed... URL: From allison at openinfra.dev Wed Jun 22 08:46:27 2022 From: allison at openinfra.dev (Allison Price) Date: Wed, 22 Jun 2022 10:46:27 +0200 Subject: Few Open Questions: Openstack vs Proprietary cloud stacks In-Reply-To: References: <37314381-3D50-4359-80CA-16C462A508DA@openinfra.dev> Message-ID: > On Jun 22, 2022, at 7:18 AM, KK CHN wrote: > > > > On Tue, Jun 21, 2022 at 5:36 PM Allison Price > wrote: > Hi Krish, > > I can answer a few of your questions in line. > >> On Jun 21, 2022, at 8:59 AM, KK CHN > wrote: >> >> List, >> >> Very recently I came across a discussion with a banking corporation who are heavy users of VMware in all their Data Centre operations .. >> >> They asked me a lot of questions in comparison with openstack vs other proprietary vendors (such as VMware. ). >> >> >> As I am promoting OpenStack I need to clarify these questions raised by the organization. >> >> I request your inputs for these queries . >> >> 1. Does OpenStack support Hyper Converged Infrastructure ? >> Or any HCI vendors already with OpenStack solutions? Please share the URL/link of case studies of OpenStack HCI. >> Any documentation/Reference to install openstack as HCI solution with supported H/W OEM details . I can try out ... >> >> 2. Is there any performance benchmark for openstack to compare with other such cloud stacks ( URL/reference links kindly share ) >> >> 3. What is the market share of OpenStack all over the world ? ( If any statistics kindly share.) >> Any survey reports available how many large and medium organizations are using Openstack distributions for private/public cloud use in production ? > > There is not an exact report that shows OpenStack market share, but there are a lot of statistics about the growing footprint of OpenStack-powered clouds that we collect from the OpenStack user survey (openstack.org/user-survey ). > > - There are currently over 25 million compute cores of OpenStack in production > - There are over 180 public cloud data centers running OpenStack > - The market for OpenStack is over $7.7 billion USD (source: 451 Research, Market Monitor, Open Source Software, OpenStack 2019) > > We are planning to publish a new report later this year highlighting the current landscape of OpenStack adoption. Since the survey is opt-in, it is not comprehensive of all OpenStack use cases, but it gives a pretty good picture of the evolution and maturity of deployments. > > I suggest if possible while publishing this new report, it will be better if we can publish also in Gartner, Forrester etc., if this makes sense. We often brief analysts after the survey comes out, but given that they have their own research arms, it?s often not published there and doing a survey / report through them is very costly. > > I heard about CERN one among the biggest Openstack deployments in the private cloud domain. Yes, CERN is one of the largest OpenStack deployments (I think around 380,000 cores). We also have a few users who have over a million cores in production including Walmart, Workday, LINE, China Mobile, and Yahoo (4 million to be exact). Other large scale users include OVH, Bloomberg, Kakao, and T-Systems (among many more). > >> >> 4. Does OpenStack new versions support VDI(Virtual Desktop Infrastructure) solutions out of the box.? pls give a reference link / url for the same. >> >> 5. What is the alternative for VROPS and VRA (including self service capabilities, Blueprint/Template based deployment) in OpenStack and any comparative study on the capabilities available. ( Please share inputs if any to refer. ) >> >> 6. What is the alternative of VMWARE Network Insights and VMWARE CodeStream and Vrealize Loginsights in OpenStack ? >> >> 7. Any major capability available in VMware cloud stack for which alternative is not available in OpenStack out of the box ? >> >> 8. Tools to maintain multiple OpenStack Cluster, any concept of Supervisory cluster available >> >> 9. Who all are the large organizations who deployed OpenStack as their alternative over proprietary solutions(eg: VMware cloud and virtualization products ..) >> >> 10. Does OpenInfra Foundation also provide direct support For OpenStack Deployment ? If yes Kindly share the link / url where to contact /approach for paid support ? in case the clients need paid support ? > > The OpenInfra Foundation doesn?t directly provide support, but we have an OpenStack Marketplace (openstack.org/marketplace ) where we work with our members to highlight their OpenStack-powered products. This list is not comprehensive, but it?s a great directory of OpenStack-powered products. > >> >> 11. Regarding security of cloud environments /infrastructure, what are the USPs of OpenStack cloud security over other competing cloud middleware providers ? and the vulnerability handling model/mechanism in production ? >> >> Thanks in advance for your invaluable inputs. >> >> Each and every input from each member is valuable for me to showcase as solid proofs to defend and promote the openstack over proprietary market leaders. >> >> Krish > > > Please let me know if you have any questions on any of the above resources / data points that I shared. > > Allison > >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amotoki at gmail.com Wed Jun 22 09:03:07 2022 From: amotoki at gmail.com (Akihiro Motoki) Date: Wed, 22 Jun 2022 18:03:07 +0900 Subject: [heat][horizon] heat-dashboard-core maintenance Message-ID: Hi, I added heat-core gerrit group to heat-dashboard-core [1]. Heat team and horizon team agreed to maintain heat-dashboard together a while ago and horizon-core was added to heat-dashboard-core. At that time heat-dashboard had active contributors so head-dashboard-core did not include heat-core team. The active contributors in heat-dashboard-core have gone and Rico is the only person from the heat team, so it is time to add heat-core to heat-dashboard-core. In addition, I would like to drop the following folks from heat-dashboard-core. They were active in heat-dashboard but they had no activities for over three years. If there is no objection, I will drop them next week. - Kazunori Shinohara - Keiichi Hikita - Xinni Ge [1] https://review.opendev.org/admin/groups/8803fcad46b4bce76ed436861474878f36e0a8e4,members Thanks, Akihiro Motoki (amotoki) From amonster369 at gmail.com Wed Jun 22 09:07:58 2022 From: amonster369 at gmail.com (A Monster) Date: Wed, 22 Jun 2022 10:07:58 +0100 Subject: add gpu as flavor option Message-ID: After deploying openstack and configuring gpu passthrough, I want to be able to see gpus as an option directly from the flavor table not added as a metadata, as shown in the attached table image, how can I achieve this? Screenshot from 2022-06-22 10-06-58.png -------------- next part -------------- An HTML attachment was scrubbed... URL: From arxcruz at redhat.com Wed Jun 22 11:20:22 2022 From: arxcruz at redhat.com (Arx Cruz) Date: Wed, 22 Jun 2022 13:20:22 +0200 Subject: [Tripleo] Gate blocker - Standalone 9 failing In-Reply-To: References: Message-ID: Hello, This should be fixed now. The bug was also updated, since it was a duplicate one, the new bug is https://bugs.launchpad.net/tripleo/+bug/1979276 Last message from Yatin the patches that caused the problem were reversed, Kind regards, On Tue, Jun 21, 2022 at 1:26 PM Arx Cruz wrote: > Hello, > > We have another gate blocker on standalone centos 9 job: > > https://bugs.launchpad.net/tripleo/+bug/1979300 > > Kind regards, > > -- > > Arx Cruz > > Software Engineer > > Red Hat EMEA > > arxcruz at redhat.com > @RedHat Red Hat > Red Hat > > > -- Arx Cruz Software Engineer Red Hat EMEA arxcruz at redhat.com @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From harshit at voereir.com Wed Jun 22 07:13:53 2022 From: harshit at voereir.com (Harshit Mishra) Date: Wed, 22 Jun 2022 07:13:53 +0000 Subject: [kolla-ansible][xena] Facing network issue when providing SR-IOV support Message-ID: Hi! I have deployed Openstack Xena using kolla-ansible (v13.0.1) on a multi-node setup (1 controller+network, multiple computes). On this cluster I would like to have support for normal, as well as direct (SR-IOV) vNIC types. I have done all the pre-requirement like configure VF on SR-IOV network interface. In the current deployment, I have created two physnets, one for flat network (called physnet1 on br-ex), and one for SR-IOV (called sriovnet1 on Intel 10G 2P X520 card). I am creating one network of VXLAN type for normal vNIC, and one of VLAN type on sriovnet1 (called sriov_network) for direct vNIC. Mapping of PF with provider in sriov-agent.ini: # cat /etc/kolla/neutron-sriov-agent/sriov_agent.ini [sriov_nic] physical_device_mappings = sriovnet1:enp8s0f1 exclude_devices = [securitygroup] firewall_driver = neutron.agent.firewall.NoopFirewallDriver ---------------------------------------------- ml2_conf.ini on controller node: # cat /etc/kolla/neutron-server/ml2_conf.ini [ml2] type_drivers = flat,vlan,vxlan tenant_network_types = vxlan,vlan,flat mechanism_drivers = openvswitch,l2population,sriovnicswitch extension_drivers = port_security [ml2_type_vlan] network_vlan_ranges = physnet1,sriovnet1:1000:1009 [ml2_type_flat] flat_networks = physnet1,sriovnet1 [ml2_sriov] agent_required = False supported_pci_vendor_devs = 8086:10ed [ml2_type_vxlan] vni_ranges = 1:1000 ---------------------------------------------- # grep -nr enabled_filters /etc/kolla/nova-scheduler/ /etc/kolla/nova-scheduler/nova.conf:13:enabled_filters = ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,SameHostFilter,DifferentHostFilter,AggregateInstanceExtraSpecsFilter,PciPassthroughFilter ---------------------------------------------- # grep passthrough /etc/kolla/nova-compute/nova.conf passthrough_whitelist = [{"physical_network": "sriovnet1", "devname": "enp8s0f1"}] The interfaces on VMs from normal vNICs are able to communicate inter-compute. The same is not working for the direct vNICs belonging to the sriov_network. I have tried multiple config changes, but none seem to work. Please help in solving this issue, or suggest a different way of achieving this if this is wrong or not optimal. Thanks! Regards, Harshit Mishra -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.goncharov at gmail.com Wed Jun 22 08:12:16 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Wed, 22 Jun 2022 10:12:16 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: Message-ID: Thanks Clark and Jeremy for comments > On 21. Jun 2022, at 17:34, Clark Boylan wrote: > > On Tue, Jun 21, 2022, at 8:03 AM, Artem Goncharov wrote: >> Hi all, >> >> During the summit we had some nice conversations and I wanted to >> actually use the chance and try to formalise the discussion to gather >> broad opinions and ideally solutions. >> >> Sidenotes:>>>>> >> - I enjoy opendev >> - I really enjoy gerrit and find it an ultimate code review system >> - I can understand people not able to cope with additional complexity >> and prefer pull request pattern to the change pattern > > I think we should be careful with the assertion that Gerrit is more complex. It is different, but I'm not sure it is more complex. Pushing to Gerrit isn't any more complex than pushing to Github. Github even has the extra step of needing to open a pull request explicitly. But it is different and does require learning a different process. Ok, accepted. Personally I agree and disagree with your statement, cause I went multiple times through explaining ?change? model to both experienced and unexperienced people and is usually painful. In the Outreachy we also have same practice each time and this is for people just harder to understand (I see project on GitHub, why can?t I open PR here like I would do for every other project on GitHub). But that is not relevant here. I will try to avoid statement ?more complex? in future communication. BTW, are we absolutely and unbendable strict on using change methodology vs pull? I mean would there be conceptual disagreement on using PR approach on opendev.org with purely gitea for those ?ecosystem? tools? The process (and UI) feels exactly like GitHub and I guess we could cover lot of concerns. I know we currently do not have Zuul support for gitea API, but I would be willing to contribute to that once this is an option at all. My experience show that when there is huge conceptual barrier it is possible to guide people through by moving them slowly into this direction and showing them advantages while going forward (first move them from GH to opendev keeping PR style, then once people see limitations of PR model show how ?change? model addresses those concerns) - doing small steps. Practically this would mean we need: - introduce auth on opendev.org (integrate with existing accounts?) - be ready for the git storage to grow (due to forks) - extend infra maintenance tools to cope with projects configuration - introduce gitea driver in Zuul > >> - I use PR change pattern for relatively simple projects (and lot of >> other huge projects do it this way) >> - I am not blaming and not going to blame anybody here >> End sidenotes>>>>>> >> >> - Some month ago gophercloud folks came to us and wanted to give source >> code under the foundation. Feedback from us was more or less: you >> either host on opendev and use gerrit or not at all. Members of gopher >> cloud rejected proceeding this way with detailed reasoning. I can fully >> understand reasonings and I also observe similar issues with >> auto-closed GitHub PRs (for SDK or OSC) and the fact that those changes >> in most of the cases never really reach gerrit. Especially in the SDK >> area it would have been very useful to get various SDKs under single >> team to spread the know how and synchronise their feature sets. Maybe >> even use shared credentials to public OpenStack clouds to verify and >> publish their coverage. >> >> - Some software solutions (especially those based on Golang) were >> designed with GitHub in mind and are actually not really working >> outside. For example in order to publish provider for HashiCorp >> Terraform into their registry your product must be hosted on GitHub >> (particularly you must upload build artifacts as release on GH). There >> is sadly no other possibility. > > This isn't the only instance we've run into this. The CNCF's Cloud Native landscape (https://landscape.cncf.io/ ) requires open source projects be hosted on Github (not not proprietary systems....). From my perspective, I think this calls out why it is important for us to continue to push back against these rules. Software is built in a number of places and if we force people to use Github (and other proprietary tools) we move away from part of the mission here. I agree we need to support making landscape flexible. But I am not 100% sure we should currently (due to general lack of contributions) simply refuse those willing to contribute into OpenStack . Terraform is a very prominent example here as well - you must be on GitHub to publish your artefacts. Don?t we want to have official TF provider following our best practices (therefore this is not an infrastructure only question)? As mentioned above - if we address concerns of developers by starting supporting also PR model we can have more arguments to strengthen our position. > >> >> - My company has created HashiCorp secretstore plugin for OpenStack and >> we are willing to donate this under the foundation to have proper >> ownership of the software (not to have our company name in the hosting >> path since it is pretty confusing). From features point of view there >> is still place to evolve and it would be cool to get other interested >> parties to participate. We would be able to OpenDev without conceptual >> issues and are going to request projects for that, but that still has >> one issue: publishing of binary artifacts - not immediately available >> in opendev (afaik). On GitHub we publish it as GH Release. > > OpenDev supports publishing of binary artifacts through a number of mechanisms. OpenStack wheel packages are pushed to pypi, docker images are pushed to Docker Hub and Quay and so on, tarballs.opendev.org hosts other random artifacts as well. It sounds like you specifically need Github Releases though. Can you not trigger releases against mirrored content? I think that should work? You might need to be more specific about what the actual needs are here. Awesome hint. I completely forgot we have tarballs.opendev.org . This is absolutely sufficient for my case, but in general we may explore publishing to GitHub mirror releases while still doing development on gitea. That requires having GH app token though (that is actually where those Vault plugins are being useful - you just fetch the token from vault and it expires afterwards). > >> >> As such the point I am trying to address is that we as a very big >> project should be bit more open to the world of tools around OpenStack >> (I mean explicitly **not** the OpenStack itself) and allow easy >> integration for those who want to extend OpenStack ecosystem. Depending >> on the particular application this may set certain limitations on the >> solution (i.e. must be on GitHub, must use Jira, whatever) which might >> be not compatible out of box with the OpenDev scheme. Do we as a open >> community want to set strict boundaries or can we ease some of those >> for the ecosystem around of OpenStack? >> >> - Would it be possible for the project hosted outside opendev be >> governed by OIF and somehow visually belong to OpenStack? (As a user I >> would definitely choose to use GitHub.com/openstack/project_x rather >> then GitHub.com/some_crappy_nic/project_x) > > OIF does have projects that are not hosted on OpenDev. Whether or not OpenStack wants to have projects hosted outside of OpenDev is probably the major item to sort out here. Precisely this is the point. I am sure we can solve technical issues, but we need to have structural things clarified first - thus this discussion. @foundation/tc - please comment. > >> - Can such project enjoy Zuul integration not being part of opendev git >> hosting? Technically it is surely possible (Ansible modules were tested >> this way), but requires in my eyes precise definition. > > The OpenDev team tried to provide CI integration with Github provided projects (some Kata repos) and found that we weren't able to do it at a level that was maintainable. Github itself poses a number of challenges, but we also found that projects there being even more disconnected were even less likely to be involved in helping themselves which is something that we currently rely on to make this collaboratory successful. All that to say we (the OpenDev team) cannot provide CI resources to Github repos. > > We do provide third party CI for a small number of projects with the expectation that people on the OpenDev hosted project side are able to care and feed for that. Fair point. That is why I wrote that I would be able to support this (due to having maintainable solution working on our side). There should be just willingness and allowance to move into this direction. > > I think this calls out the fundamental disconnect here. We use open source tools because we believe that open source needs and should be built with open source tools. That does require a little bit of investment from the project side to ensure the tooling functions as needed and is maintained. I would rather disagree with such statement: just because you host your code on a proprietary hosting (but make it freely available) does not mean your are violating this mission. I also believe we should be using open source tools (and we do that), but that doesn?t mean everyone else is breaking bad [not meant to be abusive statement]. Generally we have again here the same ?infra? concerns in the discussion (which are already known historically, but anyway thanks for making them clear again), but what about other opinions? Are there any other thoughts or am I the only one interested to get this sorted out? Just to repeat, my current case is really about a very strong desire and, I would say even need, to have gophercloud be very close to OpenStackSDK. Can we at least think about placing them under OpenStack GitHub org without Zuul access to have a better affiliation with OpenStack (gopher folks, hope you do not mind of such step. Or maybe we try to demonstrate here how to make this manageable with Zuul - reach me out if there is interest)? > >> >> P.S. For the case when zuul/github integration (management) is the only >> limiting factor I will be gladly supporting the integration and >> maintenance, since we have created an Ansible collection for managing >> of GitHub organisations/projects/teams/members (in the project-config >> spirit) which we use to manage our deployment (and I know others are >> using it as well). >> >> >> Tell me what you think and let us, please, focus on how to cope with >> different expectations rather which one is right and which is wrong. >> >> Thanks a lot, >> Artem -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Wed Jun 22 12:17:37 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 22 Jun 2022 13:17:37 +0100 Subject: [horizon] add gpu as flavor option In-Reply-To: References: Message-ID: <3a01215edecfb945d09865548f79c7fd6897e380.camel@redhat.com> On Wed, 2022-06-22 at 10:07 +0100, A Monster wrote: > After deploying openstack and configuring gpu passthrough, I want to be > able to see gpus as an option directly from the flavor table not added as a > metadata, as shown in the attached table image, how can I achieve this? > > Screenshot from 2022-06-22 10-06-58.png > that is not somethign you can do today unless you modify horizon. gpu passhtouhg uses the pci aliase via flavor extra spec it not an atirbute on teh flavor itself. so horrizon would have to parse the extra specs and extrac the alias and count and dispaly it. similarly resouces used via resouce: cannot be viewed in horizon From senrique at redhat.com Wed Jun 22 12:25:07 2022 From: senrique at redhat.com (Sofia Enriquez) Date: Wed, 22 Jun 2022 09:25:07 -0300 Subject: [cinder] Bug deputy report for week of 06-22-2022 Message-ID: This is a bug report from 06-15-2022 to 06-22-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Low - https://bugs.launchpad.net/cinder/+bug/1978825 "StorPool: really detach volumes and snapshots after copying to/from an image." Fix proposed to master. Cheers, Sofia -- Sof?a Enriquez she/her Software Engineer Red Hat PnT IRC: @enriquetaso @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From openstack at a.spamming.party Wed Jun 22 12:57:29 2022 From: openstack at a.spamming.party (JP E) Date: Wed, 22 Jun 2022 14:57:29 +0200 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: References: Message-ID: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> Hello, My $0.02. If gophercloud is an sdk project solely for openstack clouds (which it looks like it does), I don't see it so different than the (python) SDK we have in governance in terms of scope. As SDKs should ideally have feature parity, it makes more sense to me to group them under a same openstack team. On top of that, being under the openstack governance will bring a certain consistency (coherence), especially in terms of testability (="This SDK version works well with this code, we've tested them together"). Please note that I don't talk about testing/development tools in this previous sentence. Yet, this doesn't seem to work with the current contributors/contributions (different ways of working to start with). So I think the following questions need a clear answer: 1) Do the openstack governance close the door to "different way of working", or do we change governance to allow official projects to NOT be hosted in OpenDev? (Other way to ask this question: Should we allow freedom for the contributors to chose what they want to use as "forge", as long as they are matching a common "interface" (=PTI)?) 2) If we want to change governance for more inclusivity of non-opendev projects, what do we need to change? (Side note: I only see a reference of openstack testing infrastructure in [1], with a mention "Where appropriate", did I miss something?). 3) Is the gophercloud project ready to adapt to the PTI for go [2]? 4) If no, what are the reasons? Would it make sense to review if the PTI [0][2] is still accurate? Some of those questions are relevant for the TC, hence I updated the title with [tc] tag. Good luck on bringing this forward! Regards, JP Evrard [0]: https://governance.openstack.org/tc/reference/project-testing-interface.html [1]: https://governance.openstack.org/tc/reference/new-projects-requirements.html [2]: https://governance.openstack.org/tc/reference/pti/golang.html From artem.goncharov at gmail.com Wed Jun 22 13:05:06 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Wed, 22 Jun 2022 15:05:06 +0200 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> Message-ID: Thanks JP, this is starting moving in the right direction. > On 22. Jun 2022, at 14:57, JP E wrote: > > Hello, > > My $0.02. > > If gophercloud is an sdk project solely for openstack clouds (which it looks like it does), I don't see it so different than the (python) SDK we have in governance in terms of scope. As SDKs should ideally have feature parity, it makes more sense to me to group them under a same openstack team. > > On top of that, being under the openstack governance will bring a certain consistency (coherence), especially in terms of testability (="This SDK version works well with this code, we've tested them together"). Please note that I don't talk about testing/development tools in this previous sentence. > > Yet, this doesn't seem to work with the current contributors/contributions (different ways of working to start with). > So I think the following questions need a clear answer: > 1) Do the openstack governance close the door to "different way of working", or do we change governance to allow official projects to NOT be hosted in OpenDev? (Other way to ask this question: Should we allow freedom for the contributors to chose what they want to use as "forge", as long as they are matching a common "interface" (=PTI)?) > 2) If we want to change governance for more inclusivity of non-opendev projects, what do we need to change? > (Side note: I only see a reference of openstack testing infrastructure in [1], with a mention "Where appropriate", did I miss something?). > 3) Is the gophercloud project ready to adapt to the PTI for go [2]? > 4) If no, what are the reasons? Would it make sense to review if the PTI [0][2] is still accurate? > > Some of those questions are relevant for the TC, hence I updated the title with [tc] tag. > Good luck on bringing this forward! > > Regards, > JP Evrard > > [0]: https://governance.openstack.org/tc/reference/project-testing-interface.html > [1]: https://governance.openstack.org/tc/reference/new-projects-requirements.html > [2]: https://governance.openstack.org/tc/reference/pti/golang.html > From fungi at yuggoth.org Wed Jun 22 13:12:51 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 22 Jun 2022 13:12:51 +0000 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: Message-ID: <20220622131251.bub6yxd5odnsirav@yuggoth.org> On 2022-06-22 10:12:16 +0200 (+0200), Artem Goncharov wrote: [...] > I went multiple times through explaining ?change? model to both > experienced and unexperienced people and is usually painful. And you also have less trouble explaining the convoluted GitHub "pull request" workflow to someone who has never heard of Git or GitHub? > In the Outreachy we also have same practice each time and this is > for people just harder to understand (I see project on GitHub, why > can?t I open PR here like I would do for every other project on > GitHub). [...] OpenStack does its best in the org and repo descriptions on GitHub to highlight that they're convenience mirrors, and to point to the community's accepted workflows. Does that get overlooked? Is there something better that could be done to make it clear? > BTW, are we absolutely and unbendable strict on using change > methodology vs pull? I mean would there be conceptual disagreement > on using PR approach on opendev.org with > purely gitea for those ?ecosystem? tools? The process (and UI) > feels exactly like GitHub and I guess we could cover lot of > concerns. I know we currently do not have Zuul support for gitea > API, but I would be willing to contribute to that once this is an > option at all. My experience show that when there is huge > conceptual barrier it is possible to guide people through by > moving them slowly into this direction and showing them advantages > while going forward (first move them from GH to opendev keeping PR > style, then once people see limitations of PR model show how > ?change? model addresses those concerns) - doing small steps. [...] I think supporting multiple commit workflows and code review tools within the OpenDev Collaboratory would be even more confusing and add further conceptual barrier (what do you mean to commit to OpenStackSDK I need to use a different set of tools and review process than gophercloud, even though they're both in OpenDev?). But from a technical feasibility standpoint, it's also more than just the lack of a Gitea driver for Zuul: our current Gitea server farm is read-only for technical reasons including lack of support in Gitea for a shared writeable storage backend, the fact that we would need to set up separate authentication/account management (we still need help from more people to make progress on our Keycloak SSO deployment), not to mention it would effectively double the security risk profile. > I agree we need to support making landscape flexible. But I am not > 100% sure we should currently (due to general lack of > contributions) simply refuse those willing to contribute into > OpenStack . Terraform is a very prominent example here as well - > you must be on GitHub to publish your artefacts. Don?t we want to > have official TF provider following our best practices (therefore > this is not an infrastructure only question)? [...] Is a source code mirror on GitHub not sufficiently "on GitHub" from Terraform's perspective? > we may explore publishing to GitHub mirror releases while still > doing development on gitea. That requires having GH app token > though (that is actually where those Vault plugins are being > useful - you just fetch the token from vault and it expires > afterwards). [...] OpenStack's GitHub mirroring relies on a Zuul job with account credentials for the openstack org encoded as a secret in the job definition, and other projects do similarly for mirrors to their respective orgs as well. Performing other API interactions in jobs is similarly possible, so telling GitHub to make a "release" of something there would probably be a trivial addition. > I would rather disagree with such statement: just because you host > your code on a proprietary hosting (but make it freely available) > does not mean your are violating this mission. I also believe we > should be using open source tools (and we do that), but that > doesn?t mean everyone else is breaking bad [not meant to be > abusive statement]. You're spending your valuable time producing open source software, so you must see some value in your choice to make it open source and would rather users picked it over proprietary alternatives. I personally would feel rather hypocritical were I to claim that open source development is a good thing when it's the software I'm creating, but that I don't find the open source alternatives to proprietary code hosting and development tools similarly valuable and would rather choose what's convenient or popular instead. How can I expect the communities producing those tools to make traction against proprietary platforms if I choose to be part of that problem rather than part of the solution? I feel similarly disheartened when I attend an open source software conference and see someone give a presentation extolling the virtues of these practices, while clearly running their slide deck from MacOS or Windows. > Generally we have again here the same ?infra? concerns in the > discussion (which are already known historically, but anyway > thanks for making them clear again), but what about other > opinions? Are there any other thoughts or am I the only one > interested to get this sorted out? If I can try to restate your request, it seems like you want to develop software using some different tool choices than those available in the OpenDev Collaboratory. All the software we run in OpenDev (and all the software we use to develop and maintain the services and deployment in OpenDev) is 100% open source, along with its configuration. If you want to run a different combination of those services for your community while reusing some of OpenDev's solutions, you absolutely can do that and a number of people do so already. > Just to repeat, my current case is really about a very strong > desire and, I would say even need, to have gophercloud be very > close to OpenStackSDK. Can we at least think about placing them > under OpenStack GitHub org without Zuul access to have a better > affiliation with OpenStack (gopher folks, hope you do not mind of > such step. Or maybe we try to demonstrate here how to make this > manageable with Zuul - reach me out if there is interest)? [...] I'm probably still misunderstanding what you mean by "very close to" in this statement. If the most important aspect of this is that gophercloud appears in the openstack org on GitHub, but you don't wish to actually use the development methodology and code review workflow of the OpenStack community, then that doesn't seem close at all. But maybe you mean "close" from a marketing perspective? Mirrors to GitHub are not something OpenDev sysadmins really even get involved with, it's just automation individual projects implement and maintain for themselves, so it's the OpenStack TC's choice as to what projects they'll allow to appear in their GitHub org, and really nothing to do with the OpenDev Collaboratory itself. And one thing I still don't feel like you've covered is your earlier assertion that gophercloud project leadership was told the OpenInfra Foundation refused to legally represent them unless they hosted their development in the OpenDev Collaboratory. As I mentioned in my earlier reply, there are already OpenInfra projects which do all their software development in GitHub and run their own CI systems completely separate from OpenDev, so I really doubt you've been given an accurate account of what was discussed between foundation leadership and gophercloud folks. It would probably be good to seek further clarification there. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From artem.goncharov at gmail.com Wed Jun 22 14:38:24 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Wed, 22 Jun 2022 16:38:24 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <20220622131251.bub6yxd5odnsirav@yuggoth.org> References: <20220622131251.bub6yxd5odnsirav@yuggoth.org> Message-ID: <9F4C2642-A085-4F8D-A93C-9994FE392B71@gmail.com> > On 22. Jun 2022, at 15:12, Jeremy Stanley wrote: > > On 2022-06-22 10:12:16 +0200 (+0200), Artem Goncharov wrote: > [...] >> I went multiple times through explaining ?change? model to both >> experienced and unexperienced people and is usually painful. > > And you also have less trouble explaining the convoluted GitHub > "pull request" workflow to someone who has never heard of Git or > GitHub? Yes, I do. And there are much less people who has never heard of GitHub. > >> In the Outreachy we also have same practice each time and this is >> for people just harder to understand (I see project on GitHub, why >> can?t I open PR here like I would do for every other project on >> GitHub). > [...] > > OpenStack does its best in the org and repo descriptions on GitHub > to highlight that they're convenience mirrors, and to point to the > community's accepted workflows. Does that get overlooked? Is there > something better that could be done to make it clear? I doubt we can improve this. We mention in the project description it is mirror, but that is too easy to overlook. Otherwise every readme should start with bold statement: "do not open PRs here" >> > [...] > > I think supporting multiple commit workflows and code review tools > within the OpenDev Collaboratory would be even more confusing and > add further conceptual barrier (what do you mean to commit to > OpenStackSDK I need to use a different set of tools and review > process than gophercloud, even though they're both in OpenDev?). > > But from a technical feasibility standpoint, it's also more than > just the lack of a Gitea driver for Zuul: our current Gitea server > farm is read-only for technical reasons including lack of support in > Gitea for a shared writeable storage backend, the fact that we would > need to set up separate authentication/account management (we still > need help from more people to make progress on our Keycloak SSO > deployment), not to mention it would effectively double the security > risk profile. Thanks for adding more technical infos here, it was just an idea. > >> I agree we need to support making landscape flexible. But I am not >> 100% sure we should currently (due to general lack of >> contributions) simply refuse those willing to contribute into >> OpenStack . Terraform is a very prominent example here as well - >> you must be on GitHub to publish your artefacts. Don?t we want to >> have official TF provider following our best practices (therefore >> this is not an infrastructure only question)? > [...] > > Is a source code mirror on GitHub not sufficiently "on GitHub" from > Terraform's perspective? Should be sufficient > >> we may explore publishing to GitHub mirror releases while still >> doing development on gitea. That requires having GH app token >> though (that is actually where those Vault plugins are being >> useful - you just fetch the token from vault and it expires >> afterwards). > [...] > > OpenStack's GitHub mirroring relies on a Zuul job with account > credentials for the openstack org encoded as a secret in the job > definition, and other projects do similarly for mirrors to their > respective orgs as well. Performing other API interactions in jobs > is similarly possible, so telling GitHub to make a "release" of > something there would probably be a trivial addition. Absolutely sufficient. Basically they refer on the webhook that gets triggered on pushing new ?release?. But that is just one example of things requiring something different to how we work. > >> I would rather disagree with such statement: just because you host >> your code on a proprietary hosting (but make it freely available) >> does not mean your are violating this mission. I also believe we >> should be using open source tools (and we do that), but that >> doesn?t mean everyone else is breaking bad [not meant to be >> abusive statement]. > > You're spending your valuable time producing open source software, > so you must see some value in your choice to make it open source and > would rather users picked it over proprietary alternatives. I > personally would feel rather hypocritical were I to claim that open > source development is a good thing when it's the software I'm > creating, but that I don't find the open source alternatives to > proprietary code hosting and development tools similarly valuable > and would rather choose what's convenient or popular instead. How > can I expect the communities producing those tools to make traction > against proprietary platforms if I choose to be part of that problem For me the problem here is not whether GitHub is open source or not. Surely there are open source solutions, but that require company to actually run this hosting. You can try to to go opendev to do development there, but here you pay with gerrit. > rather than part of the solution? I feel similarly disheartened when > I attend an open source software conference and see someone give a > presentation extolling the virtues of these practices, while clearly > running their slide deck from MacOS or Windows. My company gives me laptop (and actually there are valid reasons for running some form of Enterprise stuff there). I have honestly no freedom in what I use to run slide decks. I would really love to have a purely Linux laptop, but I can?t. However I do my best to use open source tools to create my slide decks. On the other side you mentioned gitea is lacking support for required piece and that is exactly the case where you eventually can?t chase open source anymore. We can try, but I doubt we are able to reimplement so many things in the purely open source style. I guess not even any car now runs with a purely open source firmware. Would you not use it because of that? > >> Generally we have again here the same ?infra? concerns in the >> discussion (which are already known historically, but anyway >> thanks for making them clear again), but what about other >> opinions? Are there any other thoughts or am I the only one >> interested to get this sorted out? > > If I can try to restate your request, it seems like you want to > develop software using some different tool choices than those > available in the OpenDev Collaboratory. All the software we run in > OpenDev (and all the software we use to develop and maintain the > services and deployment in OpenDev) is 100% open source, along with > its configuration. If you want to run a different combination of > those services for your community while reusing some of OpenDev's > solutions, you absolutely can do that and a number of people do so > already. I know it is possible, but we didn?t proceed this way (most likely due to your statement in http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025663.html) > >> Just to repeat, my current case is really about a very strong >> desire and, I would say even need, to have gophercloud be very >> close to OpenStackSDK. Can we at least think about placing them >> under OpenStack GitHub org without Zuul access to have a better >> affiliation with OpenStack (gopher folks, hope you do not mind of >> such step. Or maybe we try to demonstrate here how to make this >> manageable with Zuul - reach me out if there is interest)? > [...] > > I'm probably still misunderstanding what you mean by "very close to" > in this statement. If the most important aspect of this is that > gophercloud appears in the openstack org on GitHub, but you don't > wish to actually use the development methodology and code review > workflow of the OpenStack community, then that doesn't seem close at > all. But maybe you mean "close" from a marketing perspective? > Mirrors to GitHub are not something OpenDev sysadmins really even > get involved with, it's just automation individual projects implement > and maintain for themselves, so it's the OpenStack TC's choice as to > what projects they'll allow to appear in their GitHub org, and > really nothing to do with the OpenDev Collaboratory itself. > > And one thing I still don't feel like you've covered is your earlier > assertion that gophercloud project leadership was told the OpenInfra > Foundation refused to legally represent them unless they hosted > their development in the OpenDev Collaboratory. As I mentioned in my > earlier reply, there are already OpenInfra projects which do all > their software development in GitHub and run their own CI systems > completely separate from OpenDev, so I really doubt you've been > given an accurate account of what was discussed between foundation > leadership and gophercloud folks. It would probably be good to seek > further clarification there. Maybe right. Afaik (at least what I followed) the discussion got stuck on a statement that the folks must move to opendev to achieve that and they could not agree on that due to fear of loosing contributors). I have only http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025884.html as last message to refer to. My ultimate goal (?very close?) would be to have gophercloud on opendev and be a delivery of SDK team so that we can use same tooling and standards to test them. If being located on opendev is not an option I would love to have it on ANY git hosting with opendev Zuul integration to still be able to use same tooling to test them. If this ANY hosting is GitHub I would prefer to see it under OpenStack organisation from marketing perspective. Same would be valid for terraform provider for OpenStack, especially from marketing point of view, but I do not really know whether its maintainers are having an interest in that at all. At the moment those 2 examples are affiliated with OpenStack, but they feel so much detached from OpenStack development, that it does not really look proper to me. Since we may talk generally about "OpenStack affiliated" products of course there might be some limitations. I learned openlab got problems, so gopher and TF are not even able to run Zuul jobs anymore. So unless somebody offers Zuul they can?t even do proper gating. Artem -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Wed Jun 22 15:32:28 2022 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 22 Jun 2022 17:32:28 +0200 Subject: [largescale-sig] Next meeting: June 22nd, 15utc In-Reply-To: References: Message-ID: Hi everyone, Our SIG meeting was held today. We discussed how our Forum session went, as well as potential candidates for our next OpenInfra Live show. frickler gave an update on the progress in moving the Large Scale Journey content from the wiki to a git repository published under docs.openstack.org. You can read the meeting logs at: https://meetings.opendev.org/meetings/large_scale_sig/2022/large_scale_sig.2022-06-22-15.00.html Our next IRC meeting will be July 6, at 1500utc on #openstack-operators on OFTC. Regards, -- Thierry From cboylan at sapwetik.org Wed Jun 22 15:33:10 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Wed, 22 Jun 2022 08:33:10 -0700 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <9F4C2642-A085-4F8D-A93C-9994FE392B71@gmail.com> References: <20220622131251.bub6yxd5odnsirav@yuggoth.org> <9F4C2642-A085-4F8D-A93C-9994FE392B71@gmail.com> Message-ID: On Wed, Jun 22, 2022, at 7:38 AM, Artem Goncharov wrote: >> On 22. Jun 2022, at 15:12, Jeremy Stanley wrote: >> Snipped to address a particular item inline below. >> I think supporting multiple commit workflows and code review tools >> within the OpenDev Collaboratory would be even more confusing and >> add further conceptual barrier (what do you mean to commit to >> OpenStackSDK I need to use a different set of tools and review >> process than gophercloud, even though they're both in OpenDev?). >> >> But from a technical feasibility standpoint, it's also more than >> just the lack of a Gitea driver for Zuul: our current Gitea server >> farm is read-only for technical reasons including lack of support in >> Gitea for a shared writeable storage backend, the fact that we would >> need to set up separate authentication/account management (we still >> need help from more people to make progress on our Keycloak SSO >> deployment), not to mention it would effectively double the security >> risk profile. > > Thanks for adding more technical infos here, it was just an idea. To clarify we believe Gitea does support all of the functionality necessary to run Gitea in a true cluster fashion. The last remaining piece that was missing was the shared code search index. This was addressed through the addition of elasticsearch (we would use opensearch) support for code indexing. At this point the main technical hurdle is that someone (or some group of people) need to completely re-architect how we run and deploy Gitea. This includes testing that the above solution actually fixes the problem, building new deployment tooling, creating a migration plan, and so on. But even then I agree with Jeremy that supporting multiple tools is a massive burden. It is twice the number of tools I need to help people debug ssh for, twice the number of tools we need documentation for since the workflows vary so completely, twice the number of tools to curate content on should that become necessary, twice the number of tools to fix accounts on, and the list goes on. It will only lead to confusion for users and burn out for the small number of people remaining that actually support these tools anymore. > On the other side you mentioned Gitea is lacking support for required > piece and that is exactly the case where you eventually can?t chase > open source anymore. We can try, but I doubt we are able to re-implement > so many things in the purely open source style. I guess not even any > car now runs with a purely open source firmware. Would you not use it > because of that? > I would actually say this is an argument in favor of open source not against it. We identified a deficiency in the tool, described a use case for why change would be beneficial, worked with upstream, and they addressed it. Unfortunately, in the same period of time we lost help for running the Gitea service and haven't been able to take advantage of the new features that were added. It often feels that the problem isn't so much that open source is flawed, and instead that those few of us willing to make it work regularly have to defend why it is worth improving and maintained instead of giving up. From fungi at yuggoth.org Wed Jun 22 16:13:38 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 22 Jun 2022 16:13:38 +0000 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <9F4C2642-A085-4F8D-A93C-9994FE392B71@gmail.com> References: <20220622131251.bub6yxd5odnsirav@yuggoth.org> <9F4C2642-A085-4F8D-A93C-9994FE392B71@gmail.com> Message-ID: <20220622161338.oeeqmp5ox5dvuse5@yuggoth.org> On 2022-06-22 16:38:24 +0200 (+0200), Artem Goncharov wrote: [...] > For me the problem here is not whether GitHub is open source or > not. Surely there are open source solutions, but that require > company to actually run this hosting. You can try to to go opendev > to do development there, but here you pay with gerrit. [...] OpenDev is not the only community-run code hosting service based on open source software. There are quite a few others to choose from, some of which are based on systems which employ a pull request or merge request workflow. But yes, it's easier to not look for them that's it's not a priority for you to begin with. > My company gives me laptop (and actually there are valid reasons > for running some form of Enterprise stuff there). I have honestly > no freedom in what I use to run slide decks. I would really love > to have a purely Linux laptop, but I can?t. However I do my best > to use open source tools to create my slide decks. But we do have some choice in who we work for. As someone who has pressed my employers to use open source software since the 1990s, and sometimes even defied them in order to avoid compromising my ideals, I do get that it's not easy at times. For some, working on open source software is "just a job" and not an integral part of their life or ideology. That's perfectly okay. In fact, I still find it pretty amazing that it's become so normalized for people to get paid from working on open source projects to the extent that it can be "just a job" rather than an ideology now. > On the other side you mentioned gitea is lacking support for > required piece and that is exactly the case where you eventually > can?t chase open source anymore. We can try, but I doubt we are > able to reimplement so many things in the purely open source > style. I guess not even any car now runs with a purely open source > firmware. Would you not use it because of that? This feels like a defeatist attitude, and one that I thankfully don't share. Yes the places where proprietary software still has market share are many and can seem like daunting mountains to try to scale. I take it as a challenge, prioritizing systems with open source firmware, based on open design boards and open specification chipsets for as many of my hardware purchases as possible. I get involved in those communities, and I make sure anything I build is made freely available under open community licenses. It's a lot of effort, and it's not for everyone sure, but please don't be so quick to dismiss the passions of others to solve this problem even if you think it's futile. > I know it is possible, but we didn?t proceed this way (most likely > due to your statement in > http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025663.html) Nowhere in there did I tell you not to use GitHub and not to run your own CI system. I explained that OpenDev doesn't gate GitHub projects with the Zuul instance it runs, and the reasons why we don't. It might be a reason for you not to use OpenDev, but it shouldn't have a bearing on whether you make different choices for the services you run for your community. > Maybe right. Afaik (at least what I followed) the discussion got > stuck on a statement that the folks must move to opendev to > achieve that and they could not agree on that due to fear of > loosing contributors). I have only > http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025884.html > as last message to refer to. I think you're probably conflating "part of OpenStack" with legal representation by the OpenInfra Foundation. OpenStack is an official OpenInfra project, but it is not the only official OpenInfra project. OpenStack does use OpenDev for its development practices at present, but not all OpenInfra projects do so. > My ultimate goal (?very close?) would be to have gophercloud on > opendev and be a delivery of SDK team so that we can use same > tooling and standards to test them. The fact that OpenStack relies heavily on a Gerrit-based code review process is part of those "tooling and standards." I'm still not grasping how using something other than the services OpenDev provides is still "on opendev" but it's likely you have a very different definition for what that means than I do. > If being located on opendev is not an option I would love to have > it on ANY git hosting with opendev Zuul integration to still be > able to use same tooling to test them. As already mentioned, OpenDev's Zuul does already connect to public GitHub in order to serve as a third-party CI and auxiliary build service for projects which are dependencies of projects in OpenDev's Gerrit. If OpenStackSDK wants to run tests which consume pull requests for gophercloud and report back to GitHub with results, that's quite easy to add. Where we draw the line is that we won't consume Zuul configuration from projects which aren't hosted in our Gerrit, and we won't gate projects hosted outside our Gerrit. > If this ANY hosting is GitHub I would prefer to see it under > OpenStack organisation from marketing perspective. Same would be > valid for terraform provider for OpenStack, especially from > marketing point of view, but I do not really know whether its > maintainers are having an interest in that at all. This is entirely up to the OpenStack TC, so I have no opinion on it one way or the other. Part of the formation of the OpenDev Collaboratory was removing the direct GitHub integration we were running in service of OpenStack, and handing over control of that organization to people in the OpenStack community who see some value in having a presence on GitHub and were willing to maintain new integration with it themselves. > At the moment those 2 examples are affiliated with OpenStack, but > they feel so much detached from OpenStack development, that it > does not really look proper to me. Since we may talk generally > about "OpenStack affiliated" products of course there might be > some limitations. Not following the same code review workflows as OpenStack makes them feel very detached from OpenStack development. Simply moving them into the openstack org on GitHub won't change that. > I learned openlab got problems, so gopher and TF are not even able > to run Zuul jobs anymore. So unless somebody offers Zuul they > can?t even do proper gating. While I can't guarantee they'll be willing, VEXXHOST runs managed Zuul services for other open source communities, so might be willing to entertain a request to do so for gophercloud and Terraform: https://vexxhost.com/solutions/managed-zuul/ Similarly, Software Factory hosts a bunch of open source projects and provides Zuul tenants for them: https://softwarefactory-project.io/ -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From christian.rohmann at inovex.de Wed Jun 22 19:54:28 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Wed, 22 Jun 2022 21:54:28 +0200 Subject: R: R: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve metrics using ceilometer and gnocchi In-Reply-To: <95293911-9cf4-78aa-c8cf-6e6990d02943@inovex.de> References: <95293911-9cf4-78aa-c8cf-6e6990d02943@inovex.de> Message-ID: On 22/06/2022 10:18, Christian Rohmann wrote: > > ?a content type of "application/json" was sent, the gnocchi-api > returned "Content-Type: text/html; charset=UTF-8" and a "human > readable" error instead of the expected JSON. And that is why the > exception is not properly casted and we end up in this archive_policy > creation loop. > > Why the response is html I was not yet able to find out. > I now opened an issue at https://github.com/gnocchixyz/gnocchi/issues/1261, but am wondering if this is maybe related to https://github.com/gnocchixyz/python-gnocchiclient/pull/110 ? Regards Christian From ozzzo at yahoo.com Wed Jun 22 21:10:10 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Wed, 22 Jun 2022 21:10:10 +0000 (UTC) Subject: Swift issues in one cluster In-Reply-To: <845416351.6689926.1655818006920@mail.yahoo.com> References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> Message-ID: <918427872.7544677.1655932210049@mail.yahoo.com> Is this bug fixed in Train? https://bugs.launchpad.net/swift/+bug/1572719 On Tuesday, June 21, 2022, 09:26:47 AM EDT, Albert Braden wrote: All of our endpoints are https: +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | | | keystone | identity | True | internal | https://api-int..:5000 | | | | swift | object-store | True | public | https://swift. .:8080/v1/AUTH_%(tenant_id)s | | | | swift | object-store | True | internal | https://swift..:8080/v1/AUTH_%(tenant_id)s | | | | keystone | identity | True | admin | https://api-int. .:35357 | | | | keystone | identity | True | public | https://api-ext. .:5000 | | | | swift | object-store | True | admin | https://swift. .:8080/v1 | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ I don't think this is causing the issue; all of our clusters are setup the same. We did think it was load at first, and got the 2 heaviest users to stop what they were doing, but that didn't make a difference. Our other QA cluster has similar load and identical hardware. When I look at the network graphs, I see traffic spiking up to 1G, but these are 25G interfaces, and none of the resources on the boxes are exhausted. CPU is 97% idle; memory is 30% used, disk is not full. It doesn't look like the problem is load-related. We see the haproxy connections stacking up even when load is low. What else could be causing this? On Friday, June 17, 2022, 11:12:36 PM EDT, Pete Zaitcev wrote: On Fri, 17 Jun 2022 17:33:27 +0000 (UTC) Albert Braden wrote: > $ openstack container list > Unable to establish connection to https://swift..:8080/v1/AUTH_: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') Right away I have a question: why in the world are you connecting to 8080 with HTTPS? > (from Splunk): > Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) > Payload: swift-proxy-server: STDERR: BlockingIOError > Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) > Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) This looks quite fishy to me, because the os.read is in swift/common/utils.py and it's responsible for the mutex. > When we look at network connections, we see haproxy stacking up (many lines of this): >? > # netstat -ntup | sort -b -k2 -n -r | head -n +100 > tcp? 5976932? ? ? 0 127.0.0.1:60738? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5976446? ? ? 0 127.0.0.1:58480? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973217? ? ? 0 127.0.0.1:33244? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973120? ? ? 0 127.0.0.1:51836? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5971968? ? ? 0 127.0.0.1:58516? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? >? ... > > If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue? Indeed if so many requests are established, you're in trouble. The best fix, I think, is to find the customer who's doing it and punish them. Otherwise, quotas and the ratelimiting middleware are your friends. There's also a possibility that your cluster is underperforming, although usually that results in 500 results first. But then again, at times users would "compensate" for issues by just launching way more requests, in effect DoS-ing the cluster even worse. -- Pete -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Wed Jun 22 21:30:47 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 22 Jun 2022 16:30:47 -0500 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> Message-ID: <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> ---- On Wed, 22 Jun 2022 07:57:29 -0500 JP E wrote ---- > Hello, > > My $0.02. > > If gophercloud is an sdk project solely for openstack clouds (which it looks like it does), I don't see it so different than the (python) SDK we have in governance in terms of scope. As SDKs should ideally have feature parity, it makes more sense to me to group them under a same openstack team. > > On top of that, being under the openstack governance will bring a certain consistency (coherence), especially in terms of testability (="This SDK version works well with this code, we've tested them together"). Please note that I don't talk about testing/development tools in this previous sentence. > > Yet, this doesn't seem to work with the current contributors/contributions (different ways of working to start with). Thanks for bringing it and asking the question, I think all are very valid questions and this is something we should be preparing the answer/policy as there might be more such project requests in future. > So I think the following questions need a clear answer: > 1) Do the openstack governance close the door to "different way of working", or do we change governance to allow official projects to NOT be hosted in OpenDev? (Other way to ask this question: Should we allow freedom for the contributors to chose what they want to use as "forge", as long as they are matching a common "interface" (=PTI)?) No, we do not close the door to "different way of working". We are open to ideas/project as long as it aligns with the OpenStack mission. > 2) If we want to change governance for more inclusivity of non-opendev projects, what do we need to change? > (Side note: I only see a reference of openstack testing infrastructure in [1], with a mention "Where appropriate", did I miss something?). AFAIK, there is no governance requirement from OpenStack to have the project be hosted on OpenDev, nor from OpenInfra Foundation/Bylaws (that is why not all Open Infra projects are on OpenDev). So I do not think we need any change in the governance because we do not even have such restrictions. Yes, there are always best efforts or guidelines to use the Open Source tooling in Open Source development but in reality that cannot be true for many reasons including, technical challenges, different country/company IT policies or so. Not all Open Source tooling can be accessed from everywhere. One good example of this is video call tooling. TC, PTG, Board, and Foundation use/rely on the non-open source tool (zoom, google meet etc). I am not starting the discussion of using the Open Source tool in the video meeting or why all these teams are not using it which is a separate discussion but I am trying to say using 100% Open Source tools in Open source development is a little far from now. We need to adjust ourselves here and there. > 3) Is the gophercloud project ready to adapt to the PTI for go [2]? > 4) If no, what are the reasons? Would it make sense to review if the PTI [0][2] is still accurate? > > Some of those questions are relevant for the TC, hence I updated the title with [tc] tag. The question here is not about whether OpenStack Governance allows it or not (because we do not disallow it at least) instead, the question should be how feasible (technical challenges not process challenges) it will be to host non-opendev projects in OpenStack which is what is being discussed in the thread in other replies. This is the first case (if I remember correctly ) where we are asked about OpenStack's new project on non-opendev tooling so we need to see the possibility. I think a few of the questions back to the new project wanted to be non-opendev (gophercloud team) can be: - If using a different tool than Gerrit, there is more possibility that you will not get help from other existing developers using Gerrit. Many of us keep helping every project on common things like fixing CI/CD, common changes, goal changes etc. If you are using very different tooling for code management then it will be difficult to get that help. But if you are using some common very known tooling like GitHub then it should be ok. - You might have some challenges for CI/CD or I will say need to have your own set of tooling to test/debug the code, release the code or so. What tooling you will be using or making sure you satisfy all the requirements for OpenStack PTI. - There might be some other common services like requirement management, and stable releases maintenance or so you would not get and end up doing it in your own way. Or help and extend these services' scope. Basically, you might not get all the common help you get from the OpenStack supporting team or OpenDev tooling and if that is all ok then I do not think we should have any issue hosting you in OpenStack. Instead of such future requests, we should have more clarity in OpenStack governance document about what you need to do if not OpenDev hosted project. -gmann > Good luck on bringing this forward! > > Regards, > JP Evrard > > [0]: https://governance.openstack.org/tc/reference/project-testing-interface.html > [1]: https://governance.openstack.org/tc/reference/new-projects-requirements.html > [2]: https://governance.openstack.org/tc/reference/pti/golang.html > > From fungi at yuggoth.org Wed Jun 22 23:36:34 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 22 Jun 2022 23:36:34 +0000 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> Message-ID: <20220622233634.smszvmpdx6fwualp@yuggoth.org> On 2022-06-22 16:30:47 -0500 (-0500), Ghanshyam Mann wrote: [...] > AFAIK, there is no governance requirement from OpenStack to have > the project be hosted on OpenDev, nor from OpenInfra > Foundation/Bylaws (that is why not all Open Infra projects are on > OpenDev). So I do not think we need any change in the governance > because we do not even have such restrictions. [...] If that's the consensus of the TC, then I recommend updating https://opendev.org/openstack/governance/src/branch/master/reference/new-projects-requirements.rst (particularly the parts where it says "The project uses public code reviews on the OpenStack infrastructure" and "The project has core reviewers and adopts a test-driven gate in the OpenStack infrastructure for changes"). -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From katonalala at gmail.com Thu Jun 23 07:23:21 2022 From: katonalala at gmail.com (Lajos Katona) Date: Thu, 23 Jun 2022 09:23:21 +0200 Subject: Neutron/OVN + IPv6 In-Reply-To: References: <544CB4E9-AB62-48D2-96D5-2FFD2A57018F@coote.org> Message-ID: Hi, For OVN BGP there was a presentation in Berlin: https://youtu.be/eKH14UN856o Lajos (lajoskatona) Dmitriy Rabotyagov ezt ?rta (id?pont: 2022. j?n. 22., Sze, 7:55): > Hi, > > As far as I know there's ongoing work to support BGP with OVN. I'm not > sure if it's production ready (bet it's not), but you can totally try it > out: > https://opendev.org/x/ovn-bgp-agent > > ??, 21 ???. 2022 ?., 22:48 Tiago Pires : > >> Hi Tim, >> >> From the border routers to the Internet, it is not a problem to summarize >> the /64 in /48 for example. >> My point is how to know the next-hop neutron router's IP that has a >> subnet /64 behind it? When you have many routers, we need a dynamic >> mechanism for it. >> I think the neutron ML2/OVN gap is to not support IPv6 Prefix delegation >> that would be the better solution for it. >> And Neutron BGP dynamic with OVN is not supported as I could check in the >> documentation. >> So, I don't know if there is another way to use IPv6 using the >> integration with Neutron and OVN. >> >> Regards, >> >> Tiago Pires >> >> Em ter., 21 de jun. de 2022 ?s 15:35, >> escreveu: >> >>> I think that you?d normally advertise much larger subnets than /64 and >>> then subnet those for different domains. Routing in ipv6 should be much >>> more straightforward as there should be no NAT, which ought to make it much >>> easier to work out what?s talking to what. >>> >>> On 21 Jun 2022, at 18:21, Tiago Pires wrote: >>> >>> Hi all, >>> >>> I am trying to figure out how to enable workloads on overlay to use IPv6 >>> addresses and communicate with external networks. >>> I have to use Neutron ML2/OVN in my scenario and I have doubts regarding >>> how the external networks(Internet) can know how to reach the VMs using >>> IPv6 address behind a regular neutron router. >>> I was checking this link >>> https://cloudbau.github.io/openstack/neutron/networking/2016/05/17/neutron-ipv6.html >>> that uses the BGP speaker on neutron in order to advertise the /64 subnets >>> to a border router, that option should work fine but Neutron/BGP Speaker is >>> not supported with OVN as I could check on the documentation. >>> How are you using IPv6 integration (Neutron/OVN) to advertise the /64 >>> subnets? >>> >>> Tiago Pires >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Thu Jun 23 07:51:42 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 23 Jun 2022 09:51:42 +0200 Subject: Regarding Floating IP is existing Setup In-Reply-To: References: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> Message-ID: <1816246.GmBfMWsNUm@p1> Hi, Dnia wtorek, 21 czerwca 2022 13:55:51 CEST Adivya Singh pisze: > hi Eugen, > > The current setup is 3 controller nodes, The Load is not much on each > controller and the number of DHCP agent is always set to 2 as per the > standard in the neutron.conf, The L3 agent seems to be stables as other > router namespace works fine under it, Only few router Namespace get > affected under the agent. Is it that problem happens for new floating IPs or for the FIPs which were working fine and then suddenly stopped working? If the latter, was there any action which triggered the issue to happen? Is there e.g. only one FIP broken in the router or maybe when it happens, then all FIPs which uses same router are broken? Can You also try to analyze with e.g. tcpdump where traffic is dropped exactly? You can check http://kaplonski.pl/blog/neutron-where-is-my-packet-2/ for some more detailed description how traffic should go from the external network to Your instance. > > Most of the template having issue , Have all instance having FLoating IP, a > Stack with a single floating IP have chance of issue very less > > Regards > Adivya Singh > > On Tue, Jun 21, 2022 at 1:18 PM Eugen Block wrote: > > > Hi, > > > > this sounds very familiar to me, I had to deal with something similar > > a couple of times in a heavily used cluster with 2 control nodes. What > > does your setup look like, is it a HA setup? I would start checking > > the DHCP and L3 agents. After increasing dhcp_agents_per_network to 2 > > in neutron.conf and restarting the services this didn't occur again > > (yet). This would impact floating IPs as well, sometimes I had to > > disable and enable the affected router(s). If you only have one > > control node a different approach is necessary. Do you see a high load > > on the control node? > > > > > > Zitat von Adivya Singh : > > > > > hi Team, > > > > > > We got a issue in Xena release, where we set the environment in Ubuntu > > > Platform, But later we get some issues in Floating IP not reachable. > > > > > > In a Network node, not all router namespace are Impacted and only few of > > > them get affected, So we can not comment Network node has a issue. > > > > > > The L3 agent where the Router is tied up, Worked just fine, as other > > > routers work Fine. > > > > > > and the one having issue in Floating IP, if i unassigned and assigned it > > > starts working most of the time. > > > > > > Any thoughts on this > > > > > > Regards > > > Adivya Singh > > > > > > > > > > > -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From thierry at openstack.org Thu Jun 23 09:01:52 2022 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 23 Jun 2022 11:01:52 +0200 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: <20220622233634.smszvmpdx6fwualp@yuggoth.org> References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> <20220622233634.smszvmpdx6fwualp@yuggoth.org> Message-ID: Jeremy Stanley wrote: > On 2022-06-22 16:30:47 -0500 (-0500), Ghanshyam Mann wrote: > [...] >> AFAIK, there is no governance requirement from OpenStack to have >> the project be hosted on OpenDev, nor from OpenInfra >> Foundation/Bylaws (that is why not all Open Infra projects are on >> OpenDev). So I do not think we need any change in the governance >> because we do not even have such restrictions. > [...] > > If that's the consensus of the TC, then I recommend updating > https://opendev.org/openstack/governance/src/branch/master/reference/new-projects-requirements.rst > (particularly the parts where it says "The project uses public code > reviews on the OpenStack infrastructure" and "The project has core > reviewers and adopts a test-driven gate in the OpenStack > infrastructure for changes"). Another thing to take into account is release management: I don't think the release team will support releases that bridge across different toolchains... one is plenty enough work for our small team. That's not a "no" since we already have official openstack components that are released externally (openstack-puppet modules for example). The line we've been holding is that the center of the map[1] (OpenStack proper) is part of the "openstack" release, which has to be coordinated by the release team, and therefore has to all be on OpenDev. The other boxes (operations tooling, lifecycle management, operations tooling, client tools, integration enablers) don't necessarily have to. Those can be released independently (and off-cycle), and I guess could potentially be developed elsewhere, if the TC is comfortable with that. [1] https://www.openstack.org/openstack-map -- Thierry Carrez (ttx) From eblock at nde.ag Thu Jun 23 10:01:22 2022 From: eblock at nde.ag (Eugen Block) Date: Thu, 23 Jun 2022 10:01:22 +0000 Subject: 504 Gateway time out error while loading OpenStack Projects In-Reply-To: References: <20220615115709.Horde.b9lsBD6CdiKLFNBDa3Jy8AD@webmail.nde.ag> Message-ID: <20220623100122.Horde.FFYobEXQUWyYqWRkX7KEVRx@webmail.nde.ag> Please keep the responses in the list so others can see your responses, too, and chime in with their comments. How is your dashboard configured regarding cache? This is our config (/srv/www/openstack-dashboard/openstack_dashboard/local/local_settings.d/_100_local_settings.py): CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'LOCATION': [ ':11211', ':11211', ] } } Our memcached config has both localhost and its management IP: MEMCACHED_PARAMS="-U 0 -m 64 -l 127.0.0.1, -p 11211 -c 4096" The reason for that is we noticed performance improvements when we replaced the list of control nodes for all the services and point to localhost instead: # before #memcached_servers = controller01:11211,controller02:11211 # after memcached_servers = localhost:11211 Zitat von Adivya Singh : > Hello, > > My setting in memcache looks like this, because it is default taken by > ansible > > -d > -v > -U 0 > -m 1024 > -p 11211 > -u memcache > -l 172.29.x.X > -c 4096 > -t 4 > > So should i change the time out perimeter to something else > > Regards > Adivya Singh > > On Wed, Jun 15, 2022 at 8:53 PM Maysa De Macedo Souza > wrote: > >> Hello, >> >> Maybe increasing the value of a setting like >> memcache_pool_conn_get_timeout can help? >> >> Cheers, >> Maysa. >> >> On Wed, Jun 15, 2022 at 2:05 PM Eugen Block wrote: >> >>> Hi, >>> >>> I remember having a similar issue a few years back in an older >>> openstack release, probably Ocata. We had difficulties loading *all* >>> instances in the admin tab in Horizon dashboard. It turned out to be a >>> memcached misconfiguration. I don't recall the details, unfortunately. >>> But I'd start with memcached. >>> >>> >>> Zitat von Adivya Singh : >>> >>> > hi Team, >>> > >>> > We have Xena release installed on Ubuntu Openstack, Lately we are >>> getting >>> > "504 Gateway Timeout error ", As the we moved from 1 node to 3 node Open >>> > Stack >>> > i do not see any error in haproxy , neither there is a time out when i >>> ping >>> > external-VIP-IP configured. >>> > >>> > Apache 2 are also loaded, tried to delete the Horizon Container and >>> created >>> > it again, but no errors, Still the same errors. >>> > >>> > Any guesses on that >>> > >>> > Regards >>> > Adivya Singh >>> >>> >>> >>> >>> From stephenfin at redhat.com Thu Jun 23 10:28:31 2022 From: stephenfin at redhat.com (Stephen Finucane) Date: Thu, 23 Jun 2022 11:28:31 +0100 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: Message-ID: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> I suspect we're trying to achieve two opposing goals here, at least as far as the tooling side of things goes. On one hand, we'd like to rely on the tooling that opendev provides - namely CI that handles cross-project dependencies (Zuul) and a powerful code review platform (Gerrit) - but we also don't want to alienate existing contributors to Gophercloud who have a negative opinion of Gerrit and don't wish to use it. We could build a system similar to what the SQLAlchemy folks have done (see e.g. [1]) but that requires time and energy and I wouldn't expect the OpenDev folks to do this, meaning we'd have to build and maintain it ourselves. Otherwise, without this kind of integration, the Gerrit/Zuul and GitHub systems couldn't really co-exist. As such, it seems unlikely that we're going to be able to have our cake and eat it too, and we'll have to pick one and deal with the consequences. Regarding the legal side of things, it _sounds_ like there's no big issue with regard to moving the project under the general OpenInfra umbrella but it'll be trickier to move it under the OpenStack project unless the tooling is also aligned. I don't know what the former would gain us and I suspect the latter won't gain us much either unless we align with tooling. Cheers, Stephen [1] https://github.com/sqlalchemy/sqlalchemy/pull/7964 From ralonsoh at redhat.com Thu Jun 23 10:50:07 2022 From: ralonsoh at redhat.com (Rodolfo Alonso Hernandez) Date: Thu, 23 Jun 2022 12:50:07 +0200 Subject: [kolla-ansible][xena] Facing network issue when providing SR-IOV support In-Reply-To: References: Message-ID: Hello Harshit: You need first to fix your underlying network configuration. If the interfaces created on "sriovnet1" can't talk to each other this is because your network is blocking this traffic. Check your TOR switches to enable/allow this VLAN range. Regards. On Thu, Jun 23, 2022 at 9:42 AM Harshit Mishra wrote: > Hello Rodolfo, > > Thanks for your support. > > If the VMs are spawned without any issue, that could be a problem in the > underlying network deployment. When sending traffic from a VM with an > SR-IOV port, try to capture the traffic in the physical function. Please > check the physical network connected to the SR-IOV network cards allow VLAN > traffic in 1000:1009 (according to your configuration). > > Network I have configured of VLAN type in a given range. > > openstack network show 4cacd198-29c3-47b3-bf77-39f1d01c9a22 -c > provider:network_type -c provider:physical_network -c > provider:segmentation_id > +---------------------------+-----------+ > | Field | Value | > +---------------------------+-----------+ > | provider:network_type | vlan | > | provider:physical_network | sriovnet1 | > | provider:segmentation_id | 1002 | > +---------------------------+-----------+ > > > Note: I forget to add that Intra-communication between the VM's working > fine deployed on same HOST but inter-communication not working. > One more thing IP address is not assigning to interfaces of VM. We have to > configure static IP address manually. > > Please find below incoming traffic flow on PF. > > > # tcpdump -i enp8s0f1 > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on enp8s0f1, link-type EN10MB (Ethernet), capture size 262144 > bytes > 12:39:28.584000 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, > Request from fa:16:3e:88:f5:3e (oui Unknown), length 285 > 12:39:39.183491 IP6 fe80::a236:9fff:fe24:803a > ip6-allrouters: ICMP6, > router solicitation, length 16 > 12:39:49.660269 IP6 fe80::a236:9fff:fe53:e598.5240 > ff02::15a.5240: UDP, > length 221 > 12:39:49.660897 IP6 fe80::a236:9fff:fe53:e59a.5240 > ff02::15a.5240: UDP, > length 221 > 12:39:54.105381 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, > Request from a0:36:9f:d8:0d:a8 (oui Unknown), length 293 > 12:39:54.897051 LLDP, length 46 > 12:39:56.281671 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, > Request from 3c:fd:fe:cd:06:71 (oui Unknown), length 293 > 12:40:06.286310 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, > Request from a0:36:9f:d8:0d:aa (oui Unknown), length 293 > 12:40:15.039416 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, > Request from 3c:fd:fe:cd:06:70 (oui Unknown), length 293 > 12:40:19.675902 IP6 fe80::a236:9fff:fe53:e598.5240 > ff02::15a.5240: UDP, > length 221 > 12:40:19.676390 IP6 fe80::a236:9fff:fe53:e59a.5240 > ff02::15a.5240: UDP, > length 221 > 12:40:25.708972 LLDP, length 46 > 12:40:29.288242 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, > Request from fa:16:3e:c4:3f:4f (oui Unknown), length 285 > 12:40:33.008653 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, > Request from fa:16:3e:88:f5:3e (oui Unknown), length 285 > > ? > > Another question: do the VMs with SR-IOV port receive the DHCP reply? If > so, at least you know the compute to controller communication is working > (assuming the DHCP agent is on the controller). > > No, VMs not receiving DHCP reply, you can refer above logs. Yes, DHCP > agent is on controller. > > Kindly let me know anything else is required. > > Thanks! > > Regards, > Harshit Mishra > > ------------------------------ > *From:* Rodolfo Alonso Hernandez > *Sent:* Wednesday, June 22, 2022 6:46 PM > *To:* Harshit Mishra > *Subject:* Re: [kolla-ansible][xena] Facing network issue when providing > SR-IOV support > > Hello Harshit: > > If the VMs are spawned without any issue, that could be a problem in the > underlying network deployment. When sending traffic from a VM with an > SR-IOV port, try to capture the traffic in the physical function. Please > check the physical network connected to the SR-IOV network cards allow VLAN > traffic in 1000:1009 (according to your configuration). > > Another question: do the VMs with SR-IOV port receive the DHCP reply? If > so, at least you know the compute to controller communication is working > (assuming the DHCP agent is on the controller). > > Regards. > > On Wed, Jun 22, 2022 at 2:17 PM Harshit Mishra > wrote: > > Hi! > > > I have deployed Openstack Xena using kolla-ansible (v13.0.1) on a > multi-node setup (1 controller+network, multiple computes). > > > > On this cluster I would like to have support for normal, as well as direct > (SR-IOV) vNIC types. > > > > I have done all the pre-requirement like configure VF on SR-IOV network > interface. > > > > In the current deployment, I have created two physnets, one for flat > network (called physnet1 on br-ex), and one for SR-IOV (called sriovnet1 on > Intel 10G 2P X520 card). I am creating one network of VXLAN type for normal > vNIC, and one of VLAN type on sriovnet1 (called sriov_network) for direct > vNIC. > > > > Mapping of PF with provider in sriov-agent.ini: > > > > # cat /etc/kolla/neutron-sriov-agent/sriov_agent.ini > > [sriov_nic] > > physical_device_mappings = sriovnet1:enp8s0f1 > > exclude_devices = > > > > [securitygroup] > > firewall_driver = neutron.agent.firewall.NoopFirewallDriver > > > > ---------------------------------------------- > > ml2_conf.ini on controller node: > > > > # cat /etc/kolla/neutron-server/ml2_conf.ini > > [ml2] > > type_drivers = flat,vlan,vxlan > > tenant_network_types = vxlan,vlan,flat > > mechanism_drivers = openvswitch,l2population,sriovnicswitch > > extension_drivers = port_security > > > > [ml2_type_vlan] > > network_vlan_ranges = physnet1,sriovnet1:1000:1009 > > > > [ml2_type_flat] > > flat_networks = physnet1,sriovnet1 > > > > [ml2_sriov] > > agent_required = False > > supported_pci_vendor_devs = 8086:10ed > > > > [ml2_type_vxlan] > > vni_ranges = 1:1000 > > > > ---------------------------------------------- > > > > # grep -nr enabled_filters /etc/kolla/nova-scheduler/ > > /etc/kolla/nova-scheduler/nova.conf:13:enabled_filters = > ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,SameHostFilter,DifferentHostFilter,AggregateInstanceExtraSpecsFilter,PciPassthroughFilter > > > > ---------------------------------------------- > > > > # grep passthrough /etc/kolla/nova-compute/nova.conf > > passthrough_whitelist = [{"physical_network": "sriovnet1", "devname": > "enp8s0f1"}] > > > > > > The interfaces on VMs from normal vNICs are able to communicate > inter-compute. The same is not working for the direct vNICs belonging to > the sriov_network. I have tried multiple config changes, but none seem to > work. > > > > Please help in solving this issue, or suggest a different way of achieving > this if this is wrong or not optimal. > > > > Thanks! > > Regards, > Harshit Mishra > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Jun 23 11:14:12 2022 From: smooney at redhat.com (Sean Mooney) Date: Thu, 23 Jun 2022 12:14:12 +0100 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> Message-ID: On Thu, 2022-06-23 at 11:28 +0100, Stephen Finucane wrote: > I suspect we're trying to achieve two opposing goals here, at least as far as > the tooling side of things goes. On one hand, we'd like to rely on the tooling > that opendev provides - namely CI that handles cross-project dependencies (Zuul) > and a powerful code review platform (Gerrit) - but we also don't want to > alienate existing contributors to Gophercloud who have a negative opinion of > Gerrit and don't wish to use it. We could build a system similar to what the > SQLAlchemy folks have done (see e.g. [1]) but that requires time and energy and > I wouldn't expect the OpenDev folks to do this, meaning we'd have to build and > maintain it ourselves. Otherwise, without this kind of integration, the > Gerrit/Zuul and GitHub systems couldn't really co-exist. As such, it seems > unlikely that we're going to be able to have our cake and eat it too, and we'll > have to pick one and deal with the consequences. zuul has the ablity to monitor github repos for pull request including depend on suport. however based on a breif chat yesterday that is not considerd robust and is not somethign the infra tream can really commit to maintaining with there curent staffing. While there is not a grovernance resolution that i could find that state this it was my understandign that if you were to be an offical project within the openstack/openinfra governacne then gerrit for code review was requried and the offcal repo had to be hosted in the openinfrac repos. the github mirrors are just that mirros not the offical soruce. i think it would be harmful to the comunity to biforcate the code review workflow by usign differnt tooling per project honestly. i know many on the openstack side stongly dislike teh github workflow when we have had to use it instead of gerrit and i also unserdstn tha those who are used to the github workflow simialr dislike gerrit. i am not going to go into whihc is better or not but i obviosly am storngly biased towards gerrit having used github and gitlab for code review in the past but from a simpel standpoint of standardising our work flow i thinke we likely shoudl update the project testing interface and new project requirements? to specify the use of gerrit offically going forward. https://github.com/openstack/governance/blob/master/reference/project-testing-interface.rst https://github.com/openstack/governance/blob/master/reference/new-projects-requirements.rst i will note that the new project guid does say """ Releases of OpenStack deliverables are handled by the OpenStack Release Management team through the openstack/releases repository. Official projects are expected to relinquish direct tagging (and branch creation) rights in their Gerrit ACLs once their release jobs are functional.""" so it already has an implict assumtion that all new project will use gerrit it just does not spell that out explcitly. > > Regarding the legal side of things, it _sounds_ like there's no big issue with > regard to moving the project under the general OpenInfra umbrella but it'll be > trickier to move it under the OpenStack project unless the tooling is also > aligned. I don't know what the former would gain us and I suspect the latter > won't gain us much either unless we align with tooling. i woudl say based on the current resoltion that it cant become an offical openstack project without conformign to the new project requriemetn and ptg. i assume Gophercloud is written in go ? so this would be the pti doc for it https://github.com/openstack/governance/blob/master/reference/pti/golang.rst by the way being hosted by openinfra and being an offical openstack project under the governace of the tc and foundation are two differnt things. if Gophercloud just want to use the openinfra ci ectra on ther github repo or be added to x namespace which are unoffcial project that are hosted by openinfra outside fo openstack offical governacec the pti does not apply. i would see the adpoption of the pti and code review workflow as a blocker to includtionin openstack personally but im not a TC/foundation memeber. > > Cheers, > Stephen > > [1] https://github.com/sqlalchemy/sqlalchemy/pull/7964 > > From rlandy at redhat.com Thu Jun 23 11:23:37 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Thu, 23 Jun 2022 07:23:37 -0400 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: +1 On Tue, Jun 21, 2022 at 3:28 PM Sergii Golovatiuk wrote: > +1 > > ??, 8 ???. 2022 ?. ? 21:40, James Slagle : > >> >> I'm formally proposing that we add Brendan Shephard and Takashi Kajinami >> to the core reviewers team for TripleO. Takashi is already considered core >> on puppet-tripleo, and I'm proposing we expand that to all of TripleO, and >> likewise for Brendan. >> >> They both have been contributing and reviewing for a little while now, >> and I feel it's time to add them to the team. Please +1 or let me know any >> concerns before Friday June 15th. Thanks! >> >> -- >> -- James Slagle >> -- >> > > > -- > Sergii Golovatiuk > > Senior Software Developer > > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlandy at redhat.com Thu Jun 23 11:24:32 2022 From: rlandy at redhat.com (Ronelle Landy) Date: Thu, 23 Jun 2022 07:24:32 -0400 Subject: [TripleO] Gate blocker - tempest test error - 'Failed to attach network adapter device' In-Reply-To: References: Message-ID: On Fri, Jun 17, 2022 at 7:13 AM Ronelle Landy wrote: > > > On Thu, Jun 16, 2022 at 10:57 AM Ronelle Landy wrote: > >> Hello All, >> >> We have a new gate blocker on tripleo-ci-centos-9-standalone, >> scenario-010 and possibly other jobs. The jobs fail tempest test: >> tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic >> >> Details of the failure are in: >> https://bugs.launchpad.net/tripleo/+bug/1978969. >> >> We will reply to this thread as soon as we have more information and/or a >> fix. >> > > A workaround has been merged to address the > tripleo-ci-centos-9-standalone failure - > https://review.opendev.org/c/openstack/tripleo-quickstart/+/846194. > > We are following another gate blocker now: > > https://bugs.launchpad.net/tripleo/+bug/1978997 > > A workaround there is under test. > We have cleared the gate now - thank you > > >> Please hold rechecks for now, >> >> Thank you, >> TripleO CI >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From beagles at redhat.com Thu Jun 23 12:31:09 2022 From: beagles at redhat.com (Brent Eagles) Date: Thu, 23 Jun 2022 10:01:09 -0230 Subject: [TripleO] Proposing Brendan Shephard and Takashi Kajinami In-Reply-To: References: Message-ID: On Wed, Jun 08, 2022 at 03:35:55PM -0400, James Slagle wrote: > I'm formally proposing that we add Brendan Shephard and Takashi Kajinami to > the core reviewers team for TripleO. Takashi is already considered core on > puppet-tripleo, and I'm proposing we expand that to all of TripleO, and > likewise for Brendan. > > They both have been contributing and reviewing for a little while now, and > I feel it's time to add them to the team. Please +1 or let me know any > concerns before Friday June 15th. Thanks! > > -- > -- James Slagle > -- +1 -- Brent Eagles Principal Software Engineer Red Hat Inc. From ozzzo at yahoo.com Thu Jun 23 13:42:53 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Thu, 23 Jun 2022 13:42:53 +0000 (UTC) Subject: [Swift] [kolla] Swift issues in one cluster In-Reply-To: <918427872.7544677.1655932210049@mail.yahoo.com> References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> <918427872.7544677.1655932210049@mail.yahoo.com> Message-ID: <440638798.277721.1655991773368@mail.yahoo.com> Can anyone help with this Swift issue? It looks like we are being hit with this bug (1) but this bug doesn't seem to mention where/whether it was ever fixed. This (2) appears to be a fix, and it appears to have been merged, but it doesn't mention the bug, and it's not obvious to me what version it affects. Is anyone else encountering this problem? It appears that customers in this one cluster may be doing something to cause it; we're still trying to track down specifically what they are doing, that they aren't doing in the other clusters. We are running kolla-ansible Train on RHEL7. 1. https://bugs.launchpad.net/swift/+bug/1572719 2. https://opendev.org/openstack/swift/commit/0e81ffd1e1481a73146fce17f61f2ab9e01eb684 On Wednesday, June 22, 2022, 05:10:10 PM EDT, Albert Braden wrote: Is this bug fixed in Train? https://bugs.launchpad.net/swift/+bug/1572719 On Tuesday, June 21, 2022, 09:26:47 AM EDT, Albert Braden wrote: All of our endpoints are https: +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | | | keystone | identity | True | internal | https://api-int..:5000 | | | | swift | object-store | True | public | https://swift. .:8080/v1/AUTH_%(tenant_id)s | | | | swift | object-store | True | internal | https://swift..:8080/v1/AUTH_%(tenant_id)s | | | | keystone | identity | True | admin | https://api-int. .:35357 | | | | keystone | identity | True | public | https://api-ext. .:5000 | | | | swift | object-store | True | admin | https://swift. .:8080/v1 | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ I don't think this is causing the issue; all of our clusters are setup the same. We did think it was load at first, and got the 2 heaviest users to stop what they were doing, but that didn't make a difference. Our other QA cluster has similar load and identical hardware. When I look at the network graphs, I see traffic spiking up to 1G, but these are 25G interfaces, and none of the resources on the boxes are exhausted. CPU is 97% idle; memory is 30% used, disk is not full. It doesn't look like the problem is load-related. We see the haproxy connections stacking up even when load is low. What else could be causing this? On Friday, June 17, 2022, 11:12:36 PM EDT, Pete Zaitcev wrote: On Fri, 17 Jun 2022 17:33:27 +0000 (UTC) Albert Braden wrote: > $ openstack container list > Unable to establish connection to https://swift..:8080/v1/AUTH_: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') Right away I have a question: why in the world are you connecting to 8080 with HTTPS? > (from Splunk): > Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) > Payload: swift-proxy-server: STDERR: BlockingIOError > Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) > Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) This looks quite fishy to me, because the os.read is in swift/common/utils.py and it's responsible for the mutex. > When we look at network connections, we see haproxy stacking up (many lines of this): >? > # netstat -ntup | sort -b -k2 -n -r | head -n +100 > tcp? 5976932? ? ? 0 127.0.0.1:60738? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5976446? ? ? 0 127.0.0.1:58480? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973217? ? ? 0 127.0.0.1:33244? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973120? ? ? 0 127.0.0.1:51836? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5971968? ? ? 0 127.0.0.1:58516? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? >? ... > > If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue? Indeed if so many requests are established, you're in trouble. The best fix, I think, is to find the customer who's doing it and punish them. Otherwise, quotas and the ratelimiting middleware are your friends. There's also a possibility that your cluster is underperforming, although usually that results in 500 results first. But then again, at times users would "compensate" for issues by just launching way more requests, in effect DoS-ing the cluster even worse. -- Pete -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Thu Jun 23 14:30:27 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 23 Jun 2022 09:30:27 -0500 Subject: [all][tc] Technical Committee next weekly meeting on 23 June 2022 at 1500 UTC In-Reply-To: <18182bccff4.dd7e8c1e155311.2291980582584492323@ghanshyammann.com> References: <18182bccff4.dd7e8c1e155311.2291980582584492323@ghanshyammann.com> Message-ID: <18190f8b3f4.bc34233b367191.6693791369512461499@ghanshyammann.com> Hello Everyone, Below is the agenda for Today's TC IRC meeting schedule at 1500 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting == Agenda for tomorrow's TC meeting == * Roll call * Follow up on past action items * Gate health check ** Fixing Zuul config error in OpenStack *** https://etherpad.opendev.org/p/zuul-config-error-openstack ** (rosmaita) update on cinder-backup memory bloat issue * Checks on Zed cycle tracker ** https://etherpad.opendev.org/p/tc-zed-tracker * New ELK service dashboard: e-r service ** https://opensearch.logs.openstack.org/_dashboards/app/discover?security_tenant=global ** https://review.opendev.org/c/openstack/governance-sigs/+/835838 * RBAC feedback in ops meetup ** https://etherpad.opendev.org/p/rbac-zed-ptg#L171 * Create the Environmental Sustainability SIG ** https://review.opendev.org/c/openstack/governance-sigs/+/845336 * Open Reviews ** https://review.opendev.org/q/projects:openstack/governance+is:open -gmann ---- On Mon, 20 Jun 2022 15:10:21 -0500 Ghanshyam Mann wrote ---- > Hello Everyone, > > The technical Committee's next weekly meeting is scheduled for 23 June 2022, at 1500 UTC. > > If you would like to add topics for discussion, please add them to the below wiki page by > Wednesday, 22 June at 2100 UTC. > > https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting > > -gmann > > From gmann at ghanshyammann.com Thu Jun 23 15:02:14 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 23 Jun 2022 10:02:14 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> Message-ID: <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> ---- On Thu, 23 Jun 2022 06:14:12 -0500 Sean Mooney wrote ---- > On Thu, 2022-06-23 at 11:28 +0100, Stephen Finucane wrote: > > I suspect we're trying to achieve two opposing goals here, at least as far as > > the tooling side of things goes. On one hand, we'd like to rely on the tooling > > that opendev provides - namely CI that handles cross-project dependencies (Zuul) > > and a powerful code review platform (Gerrit) - but we also don't want to > > alienate existing contributors to Gophercloud who have a negative opinion of > > Gerrit and don't wish to use it. We could build a system similar to what the > > SQLAlchemy folks have done (see e.g. [1]) but that requires time and energy and > > I wouldn't expect the OpenDev folks to do this, meaning we'd have to build and > > maintain it ourselves. Otherwise, without this kind of integration, the > > Gerrit/Zuul and GitHub systems couldn't really co-exist. As such, it seems > > unlikely that we're going to be able to have our cake and eat it too, and we'll > > have to pick one and deal with the consequences. > > zuul has the ablity to monitor github repos for pull request including depend on suport. > however based on a breif chat yesterday that is not considerd robust and is not somethign > the infra tream can really commit to maintaining with there curent staffing. > > While there is not a grovernance resolution that i could find that state this it was > my understandign that if you were to be an offical project within the openstack/openinfra > governacne then gerrit for code review was requried and the offcal repo had to be hosted > in the openinfrac repos. the github mirrors are just that mirros not the offical soruce. > > i think it would be harmful to the comunity to biforcate the code review workflow by > usign differnt tooling per project honestly. i know many on the openstack side stongly dislike > teh github workflow when we have had to use it instead of gerrit and i also unserdstn tha those > who are used to the github workflow simialr dislike gerrit. > > i am not going to go into whihc is better or not but i obviosly am storngly biased towards gerrit having > used github and gitlab for code review in the past but from a simpel standpoint of standardising our work > flow i thinke we likely shoudl update the project testing interface and new project requirements > to specify the use of gerrit offically going forward. > > https://github.com/openstack/governance/blob/master/reference/project-testing-interface.rst > https://github.com/openstack/governance/blob/master/reference/new-projects-requirements.rst > > i will note that the new project guid does say > > """ > Releases of OpenStack deliverables are handled by the OpenStack Release Management team through the openstack/releases repository. Official projects > are expected to relinquish direct tagging (and branch creation) rights in their Gerrit ACLs once their release jobs are functional.""" > > so it already has an implict assumtion that all new project will use gerrit it just does not spell that out explcitly. > > > > > Regarding the legal side of things, it _sounds_ like there's no big issue with > > regard to moving the project under the general OpenInfra umbrella but it'll be > > trickier to move it under the OpenStack project unless the tooling is also > > aligned. I don't know what the former would gain us and I suspect the latter > > won't gain us much either unless we align with tooling. > i woudl say based on the current resoltion that it cant become an offical openstack project > without conformign to the new project requriemetn and ptg. > > i assume Gophercloud is written in go ? so this would be the pti doc for it > https://github.com/openstack/governance/blob/master/reference/pti/golang.rst > > by the way being hosted by openinfra and being an offical openstack project > under the governace of the tc and foundation are two differnt things. > > if Gophercloud just want to use the openinfra ci ectra on ther github repo or be added to x namespace which > are unoffcial project that are hosted by openinfra outside fo openstack offical governacec the pti does not apply. > i would see the adpoption of the pti and code review workflow as a blocker to includtionin openstack personally > but im not a TC/foundation memeber. Yes, both are separate things and I think we are mixing both or at least if we have such impression or governance is not so clear about it then we should fix it. I replied in another reply about governance point of view and IMO yes we should allow such new projects hosted on new tooling or so but they need to make sure all the help on CI/CD, release etc are taken care by them self or they help opendev team to support such things. If either of them cannot be done and they do not fulfill the PTI or any other new project requirement criteria then they cannot be in OpenStack. But overall, using different tooling should not be any barrier to be in OpenStack community. At some extend, we allowed it for Skyline too (at least some way/process different there and we have accepted them as OpenStack project and given time to work on things to do in OpenStack way). [1] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/029198.html -gmann > > > > > Cheers, > > Stephen > > > > [1] https://github.com/sqlalchemy/sqlalchemy/pull/7964 > > > > > > > From katonalala at gmail.com Thu Jun 23 15:22:18 2022 From: katonalala at gmail.com (Lajos Katona) Date: Thu, 23 Jun 2022 17:22:18 +0200 Subject: [neutron] Drivers meeting agenda - 24.06.2022. Message-ID: Hi Neutron Drivers, The agenda for tomorrow's drivers meeting is at [1]. * [RFE] Improve performance of bulk port create/delete with networking-generic-switch (#linkhttps:// bugs.launchpad.net/neutron/+bug/1976270) * Neutron RBAC not sharing subnet (#link https://bugs.launchpad.net/neutron/+bug/1975603) * [ovn]Floating IP adds distributed attributes (#link https://bugs.launchpad.net/neutron/+bug/1978039) * [RFE] Adding the "rekey" parameter to the api for strongswan like "dpd_action" (#link https://bugs.launchpad.net/neutron/+bug/1979044) [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda See you at the meeting tomorrow. Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Thu Jun 23 15:30:24 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 23 Jun 2022 15:30:24 +0000 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> Message-ID: <20220623153023.fwnstkz5hetite6q@yuggoth.org> On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: [...] > Yes, both are separate things and I think we are mixing both or at > least if we have such impression or governance is not so clear > about it then we should fix it. I replied in another reply about > governance point of view and IMO yes we should allow such new > projects hosted on new tooling or so but they need to make sure > all the help on CI/CD, release etc are taken care by them self or > they help opendev team to support such things. If either of them > cannot be done and they do not fulfill the PTI or any other new > project requirement criteria then they cannot be in OpenStack. [...] "Governance" (the new project requirements document) right now clearly states that new projects need to perform their code review and gating tests on the "OpenStack Infrastructure" (the former name for the OpenDev Collaboratory because that document hasn't been updated to reflect the new name). You'll at a minimum need a vote of the TC to remove those restrictions, so all this assumes that the rest of the TC agrees with you that doing code review in GitHub with a separate GitHub-connected CI system is allowable for new official OpenStack project teams and deliverables. This is not "governance point of view" it's *your* point of view, so please be clear that the decision is one the TC as a whole will need to make. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From ozzzo at yahoo.com Thu Jun 23 16:40:29 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Thu, 23 Jun 2022 16:40:29 +0000 (UTC) Subject: [Swift] [kolla] Swift issues in one cluster In-Reply-To: References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> <918427872.7544677.1655932210049@mail.yahoo.com> <440638798.277721.1655991773368@mail.yahoo.com> Message-ID: <1210414976.7924712.1656002429999@mail.yahoo.com> We're running 2.23.3 but it appears that we are experiencing the bug (1). We tracked the problem down to a client who recently started using a java library to read large files from Swift. When he moves his activity to the other QA cluster, the problem follows. Am I guessing correctly that the bug was never fixed, and that (2) fixes a different problem? On Thursday, June 23, 2022, 10:43:44 AM EDT, Clay Gerrard wrote: https://review.opendev.org/c/openstack/swift/+/575254 has been included in every released swift tag since 2.21.0 I believe Train included a swift version of at least 2.22?https://releases.openstack.org/train/#swift Nvidia doesn't use haproxy in front of our swift proxies, and we don't see BlockingIOError in tracebacks - the traceback might go away if you upgrade to the latest swift (2.29) and/or eventlet and/or python 3.8ish On Thu, Jun 23, 2022 at 8:49 AM Albert Braden wrote: Can anyone help with this Swift issue? It looks like we are being hit with this bug (1) but this bug doesn't seem to mention where/whether it was ever fixed. This (2) appears to be a fix, and it appears to have been merged, but it doesn't mention the bug, and it's not obvious to me what version it affects. Is anyone else encountering this problem? It appears that customers in this one cluster may be doing something to cause it; we're still trying to track down specifically what they are doing, that they aren't doing in the other clusters. We are running kolla-ansible Train on RHEL7. 1. https://bugs.launchpad.net/swift/+bug/1572719 2. https://opendev.org/openstack/swift/commit/0e81ffd1e1481a73146fce17f61f2ab9e01eb684 On Wednesday, June 22, 2022, 05:10:10 PM EDT, Albert Braden wrote: Is this bug fixed in Train? https://bugs.launchpad.net/swift/+bug/1572719 On Tuesday, June 21, 2022, 09:26:47 AM EDT, Albert Braden wrote: All of our endpoints are https: +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | | | keystone | identity | True | internal | https://api-int..:5000 | | | | swift | object-store | True | public | https://swift. .:8080/v1/AUTH_%(tenant_id)s | | | | swift | object-store | True | internal | https://swift..:8080/v1/AUTH_%(tenant_id)s | | | | keystone | identity | True | admin | https://api-int. .:35357 | | | | keystone | identity | True | public | https://api-ext. .:5000 | | | | swift | object-store | True | admin | https://swift. .:8080/v1 | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ I don't think this is causing the issue; all of our clusters are setup the same. We did think it was load at first, and got the 2 heaviest users to stop what they were doing, but that didn't make a difference. Our other QA cluster has similar load and identical hardware. When I look at the network graphs, I see traffic spiking up to 1G, but these are 25G interfaces, and none of the resources on the boxes are exhausted. CPU is 97% idle; memory is 30% used, disk is not full. It doesn't look like the problem is load-related. We see the haproxy connections stacking up even when load is low. What else could be causing this? On Friday, June 17, 2022, 11:12:36 PM EDT, Pete Zaitcev wrote: On Fri, 17 Jun 2022 17:33:27 +0000 (UTC) Albert Braden wrote: > $ openstack container list > Unable to establish connection to https://swift..:8080/v1/AUTH_: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') Right away I have a question: why in the world are you connecting to 8080 with HTTPS? > (from Splunk): > Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) > Payload: swift-proxy-server: STDERR: BlockingIOError > Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) > Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) This looks quite fishy to me, because the os.read is in swift/common/utils.py and it's responsible for the mutex. > When we look at network connections, we see haproxy stacking up (many lines of this): >? > # netstat -ntup | sort -b -k2 -n -r | head -n +100 > tcp? 5976932? ? ? 0 127.0.0.1:60738? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5976446? ? ? 0 127.0.0.1:58480? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973217? ? ? 0 127.0.0.1:33244? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973120? ? ? 0 127.0.0.1:51836? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5971968? ? ? 0 127.0.0.1:58516? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? >? ... > > If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue? Indeed if so many requests are established, you're in trouble. The best fix, I think, is to find the customer who's doing it and punish them. Otherwise, quotas and the ratelimiting middleware are your friends. There's also a possibility that your cluster is underperforming, although usually that results in 500 results first. But then again, at times users would "compensate" for issues by just launching way more requests, in effect DoS-ing the cluster even worse. -- Pete -- Clay Gerrard210 788 9431 -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Thu Jun 23 17:00:58 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 23 Jun 2022 12:00:58 -0500 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> <20220622233634.smszvmpdx6fwualp@yuggoth.org> Message-ID: <1819182829a.f4f70d37376766.6935407412644022771@ghanshyammann.com> ---- On Thu, 23 Jun 2022 04:01:52 -0500 Thierry Carrez wrote ---- > Jeremy Stanley wrote: > > On 2022-06-22 16:30:47 -0500 (-0500), Ghanshyam Mann wrote: > > [...] > >> AFAIK, there is no governance requirement from OpenStack to have > >> the project be hosted on OpenDev, nor from OpenInfra > >> Foundation/Bylaws (that is why not all Open Infra projects are on > >> OpenDev). So I do not think we need any change in the governance > >> because we do not even have such restrictions. > > [...] > > > > If that's the consensus of the TC, then I recommend updating > > https://opendev.org/openstack/governance/src/branch/master/reference/new-projects-requirements.rst > > (particularly the parts where it says "The project uses public code > > reviews on the OpenStack infrastructure" and "The project has core > > reviewers and adopts a test-driven gate in the OpenStack > > infrastructure for changes"). Yes, that is what I mean that we have not explicitly said it has to be on OpenDev tooling as must requirement but yes we need to update the governance documentation also to have a clear view if "no opendev tooling then what are governance requirement" > > Another thing to take into account is release management: I don't think > the release team will support releases that bridge across different > toolchains... one is plenty enough work for our small team. > > That's not a "no" since we already have official openstack components > that are released externally (openstack-puppet modules for example). The > line we've been holding is that the center of the map[1] (OpenStack > proper) is part of the "openstack" release, which has to be coordinated > by the release team, and therefore has to all be on OpenDev. The other > boxes (operations tooling, lifecycle management, operations tooling, > client tools, integration enablers) don't necessarily have to. Those can > be released independently (and off-cycle), and I guess could potentially > be developed elsewhere, if the TC is comfortable with that. True, I mentioned in my reply that if project is not using the OpenDev or OpenStack standardrs tool, programing lang then they end up doing lot of things like CI/CD, release, dependency management either by their own or help those team to support the new tooling. If we get agreement on this then this is what we need to update in governance new project requirement as explicitly so that they do not have false expectation that they will get release, CI/CD, deps management support from existing team and existing tooling. -gmann > > [1] https://www.openstack.org/openstack-map > > -- > Thierry Carrez (ttx) > > From gmann at ghanshyammann.com Thu Jun 23 17:13:50 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 23 Jun 2022 12:13:50 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <20220623153023.fwnstkz5hetite6q@yuggoth.org> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> Message-ID: <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley wrote ---- > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > [...] > > Yes, both are separate things and I think we are mixing both or at > > least if we have such impression or governance is not so clear > > about it then we should fix it. I replied in another reply about > > governance point of view and IMO yes we should allow such new > > projects hosted on new tooling or so but they need to make sure > > all the help on CI/CD, release etc are taken care by them self or > > they help opendev team to support such things. If either of them > > cannot be done and they do not fulfill the PTI or any other new > > project requirement criteria then they cannot be in OpenStack. > [...] > > "Governance" (the new project requirements document) right now > clearly states that new projects need to perform their code review > and gating tests on the "OpenStack Infrastructure" (the former name > for the OpenDev Collaboratory because that document hasn't been > updated to reflect the new name). You'll at a minimum need a vote of > the TC to remove those restrictions, so all this assumes that the > rest of the TC agrees with you that doing code review in GitHub with > a separate GitHub-connected CI system is allowable for new official > OpenStack project teams and deliverables. > > This is not "governance point of view" it's *your* point of view, so > please be clear that the decision is one the TC as a whole will need > to make. I think there is some misunderstanding here. I have never said anywhere that this is "TC agreed view" off-course this is my opinion as a community member as well as TC member. Any community member or TC members can provide their opinion but that should not be considered as "TC agreed plan" until that is explicitly mentioned in email or TC pass the resolution. We can have different views from TC members or chair but any of that should not be considered as "TC agreement" unless mentioned. I think this is how every email discussion is. I have this in my list to give a clear picture from TC as an agreed plan: Step1: Continue the discussion in ML (here) Step2: After having a good amount of feedback here and we still not resolved the things, I will add this topic to TC meeting and get the TC consensus. Step3: Propose Governance resolution or documentation update Step4: Update the same in ML as "TC agreed plan". -gmann > -- > Jeremy Stanley > From dsmigiel at redhat.com Thu Jun 23 17:17:48 2022 From: dsmigiel at redhat.com (Dariusz Smigiel) Date: Thu, 23 Jun 2022 10:17:48 -0700 Subject: [ER] Revive elastic recheck Message-ID: Hey Team! I started cherry picking changes from the rdo branch back to the master branch for Elastic Recheck. [1] I'm planning to continue doing so, to revive the master branch and continue development in the main branch. Some of the changes are incompatible between releases. We're gonna clear them case by case to enable it in the future. Thanks, Darius [1]: https://review.opendev.org/c/opendev/elastic-recheck/+/847259 From fungi at yuggoth.org Thu Jun 23 17:46:57 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 23 Jun 2022 17:46:57 +0000 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: <1819182829a.f4f70d37376766.6935407412644022771@ghanshyammann.com> References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> <20220622233634.smszvmpdx6fwualp@yuggoth.org> <1819182829a.f4f70d37376766.6935407412644022771@ghanshyammann.com> Message-ID: <20220623174656.uqd5o4qd4tmgxo52@yuggoth.org> On 2022-06-23 12:00:58 -0500 (-0500), Ghanshyam Mann wrote: > Jeremy Stanley wrote: [...] > > > If that's the consensus of the TC, then I recommend updating > > > https://opendev.org/openstack/governance/src/branch/master/reference/new-projects-requirements.rst > > > (particularly the parts where it says "The project uses public code > > > reviews on the OpenStack infrastructure" and "The project has core > > > reviewers and adopts a test-driven gate in the OpenStack > > > infrastructure for changes"). > > Yes, that is what I mean that we have not explicitly said it has > to be on OpenDev tooling as must requirement but yes we need to > update the governance documentation also to have a clear view if > "no opendev tooling then what are governance requirement" [...] I'm not sure who "we" is in this context, but the TC has "explicitly said it has to be on OpenDev tooling as must requirement" through the passages I quoted above (the referenced document absolutely states this, and was ratified by the TC). Removing that restriction is a substantive change to OpenStack's governing documents, not a mere clarification. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Thu Jun 23 17:55:01 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 23 Jun 2022 17:55:01 +0000 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> Message-ID: <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley wrote ---- > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > [...] > > > Yes, both are separate things and I think we are mixing both or at > > > least if we have such impression or governance is not so clear > > > about it then we should fix it. I replied in another reply about > > > governance point of view and IMO yes we should allow such new > > > projects hosted on new tooling or so but they need to make sure > > > all the help on CI/CD, release etc are taken care by them self or > > > they help opendev team to support such things. If either of them > > > cannot be done and they do not fulfill the PTI or any other new > > > project requirement criteria then they cannot be in OpenStack. > > [...] > > > > "Governance" (the new project requirements document) right now > > clearly states that new projects need to perform their code review > > and gating tests on the "OpenStack Infrastructure" (the former name > > for the OpenDev Collaboratory because that document hasn't been > > updated to reflect the new name). You'll at a minimum need a vote of > > the TC to remove those restrictions, so all this assumes that the > > rest of the TC agrees with you that doing code review in GitHub with > > a separate GitHub-connected CI system is allowable for new official > > OpenStack project teams and deliverables. > > > > This is not "governance point of view" it's *your* point of view, so > > please be clear that the decision is one the TC as a whole will need > > to make. > > I think there is some misunderstanding here. I have never said > anywhere that this is "TC agreed view" off-course this is my > opinion as a community member as well as TC member. > > Any community member or TC members can provide their opinion but > that should not be considered as "TC agreed plan" until that is > explicitly mentioned in email or TC pass the resolution. We can > have different views from TC members or chair but any of that > should not be considered as "TC agreement" unless mentioned. I > think this is how every email discussion is. You referred to it above as the "governance point of view" so I just wanted to make certain you don't actually believe the governing documents are unclear on this particular point, and understand that OpenStack absolutely will need TC consensus on lifting a longstanding restriction in order to allow an official deliverable to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev Collaboratory). > I have this in my list to give a clear picture from TC as an > agreed plan: > > Step1: Continue the discussion in ML (here) > Step2: After having a good amount of feedback here and we still > not resolved the things, I will add this topic to TC > meeting and get the TC consensus. > Step3: Propose Governance resolution or documentation update > Step4: Update the same in ML as "TC agreed plan". Thanks, this looks like a reasonable way forward. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From gmann at ghanshyammann.com Thu Jun 23 18:04:24 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 23 Jun 2022 13:04:24 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> Message-ID: <18191bc9594.12346e6c7378742.3937740254313119611@ghanshyammann.com> ---- On Thu, 23 Jun 2022 12:55:01 -0500 Jeremy Stanley wrote ---- > On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley wrote ---- > > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > > [...] > > > > Yes, both are separate things and I think we are mixing both or at > > > > least if we have such impression or governance is not so clear > > > > about it then we should fix it. I replied in another reply about > > > > governance point of view and IMO yes we should allow such new > > > > projects hosted on new tooling or so but they need to make sure > > > > all the help on CI/CD, release etc are taken care by them self or > > > > they help opendev team to support such things. If either of them > > > > cannot be done and they do not fulfill the PTI or any other new > > > > project requirement criteria then they cannot be in OpenStack. > > > [...] > > > > > > "Governance" (the new project requirements document) right now > > > clearly states that new projects need to perform their code review > > > and gating tests on the "OpenStack Infrastructure" (the former name > > > for the OpenDev Collaboratory because that document hasn't been > > > updated to reflect the new name). You'll at a minimum need a vote of > > > the TC to remove those restrictions, so all this assumes that the > > > rest of the TC agrees with you that doing code review in GitHub with > > > a separate GitHub-connected CI system is allowable for new official > > > OpenStack project teams and deliverables. > > > > > > This is not "governance point of view" it's *your* point of view, so > > > please be clear that the decision is one the TC as a whole will need > > > to make. > > > > I think there is some misunderstanding here. I have never said > > anywhere that this is "TC agreed view" off-course this is my > > opinion as a community member as well as TC member. > > > > Any community member or TC members can provide their opinion but > > that should not be considered as "TC agreed plan" until that is > > explicitly mentioned in email or TC pass the resolution. We can > > have different views from TC members or chair but any of that > > should not be considered as "TC agreement" unless mentioned. I > > think this is how every email discussion is. > > You referred to it above as the "governance point of view" so I just > wanted to make certain you don't actually believe the governing > documents are unclear on this particular point, and understand that > OpenStack absolutely will need TC consensus on lifting a > longstanding restriction in order to allow an official deliverable > to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev > Collaboratory). Sure. Sorry it that created the confusion but let's go with the plan I mentioned below. At least we should explicitly update "OpenStack Infrastructure" to be OpenDev or we mean this as anything else now. It should have been updated at the time of when OpenDev was created but while updating it we should re-iterate the requirement itself. -gmann > > > I have this in my list to give a clear picture from TC as an > > agreed plan: > > > > Step1: Continue the discussion in ML (here) > > Step2: After having a good amount of feedback here and we still > > not resolved the things, I will add this topic to TC > > meeting and get the TC consensus. > > Step3: Propose Governance resolution or documentation update > > Step4: Update the same in ML as "TC agreed plan". > > Thanks, this looks like a reasonable way forward. > -- > Jeremy Stanley > From yasufum.o at gmail.com Thu Jun 23 18:29:58 2022 From: yasufum.o at gmail.com (Yasufumi Ogawa) Date: Fri, 24 Jun 2022 03:29:58 +0900 Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team In-Reply-To: References: Message-ID: <86ab7ae4-8cf2-379d-e10e-06327f55430d@gmail.com> Hi heat-translator cores, We tacker team would like to keep contributing to maintain heat-translator, so appreciate if you agree to the proposal kindly. Cheers, Yasufumi On 2022/06/09 14:14, ueha.ayumu at fujitsu.com wrote: > Hi Rico, > > This is just a friendly reminder that we are waiting for your action. > Or please let us know if there are any other necessary processes. > Thanks. > > Best Regards, > Ueha > > -----Original Message----- > From: ueha.ayumu at fujitsu.com > Sent: Thursday, June 2, 2022 5:39 PM > To: openstack-discuss at lists.openstack.org; 'Rico Lin' > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team > > Hi Rico, > > This proposal is great helpful for us! > There seems to be no particular objection, so could you add it to heat-translator-core if there is no problem? > Thanks. > > Best Regards, > Ueha > > -----Original Message----- > From: ueha.ayumu at fujitsu.com > Sent: Wednesday, May 25, 2022 1:40 PM > To: openstack-discuss at lists.openstack.org > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team > > +1 > > Best Regards, > Ueha > > -----Original Message----- > From: Yoshito Ito > Sent: Wednesday, May 25, 2022 12:31 PM > To: openstack-discuss at lists.openstack.org > Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team > > Hi heat-translator team! > > I would like to propose Hiromu Asahina (h-asahina) as a member of the heat-translator core team. I'm now not able to actively contribute to heat-translator, and he can lead the team instead of me. > > He is mainly contributing to Tacker by fixing issues, refactoring existing codes, and a lot of helpful reviews, with his experience as a research engineer in NTT. Tacker highly depends on heat-translator in its main functionality, so I believe he can manage it with his knowledge of Tacker. > > Please vote for his nomination by answering this mail till June 1st. > > Best Regards, > > Yoshito Ito > > From dsmigiel at redhat.com Thu Jun 23 19:47:15 2022 From: dsmigiel at redhat.com (Dariusz Smigiel) Date: Thu, 23 Jun 2022 12:47:15 -0700 Subject: [ER] Revive elastic recheck In-Reply-To: References: Message-ID: Update on the merge process. Thanks to fungi, who pointed me towards [1], I prepared patch [2] to merge all changes from the rdo branch back to the master branch. Initial patch [3] is right now abandoned. Thanks, Darius [1]: https://docs.opendev.org/opendev/infra-manual/latest/drivers.html#merge-feature-branch-into-master [2]: https://review.opendev.org/c/opendev/elastic-recheck/+/847405/ [3]: https://review.opendev.org/c/opendev/elastic-recheck/+/847259 On Thu, Jun 23, 2022 at 10:17 AM Dariusz Smigiel wrote: > > Hey Team! > > I started cherry picking changes from the rdo branch back to the > master branch for Elastic Recheck. [1] > I'm planning to continue doing so, to revive the master branch and > continue development in the main branch. > > Some of the changes are incompatible between releases. > We're gonna clear them case by case to enable it in the future. > > Thanks, > Darius > > [1]: https://review.opendev.org/c/opendev/elastic-recheck/+/847259 From kennelson11 at gmail.com Thu Jun 23 20:02:57 2022 From: kennelson11 at gmail.com (Kendall Nelson) Date: Thu, 23 Jun 2022 15:02:57 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> Message-ID: On Thu, Jun 23, 2022 at 12:55 PM Jeremy Stanley wrote: > On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley < > fungi at yuggoth.org> wrote ---- > > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > > [...] > > > > Yes, both are separate things and I think we are mixing both or at > > > > least if we have such impression or governance is not so clear > > > > about it then we should fix it. I replied in another reply about > > > > governance point of view and IMO yes we should allow such new > > > > projects hosted on new tooling or so but they need to make sure > > > > all the help on CI/CD, release etc are taken care by them self or > > > > they help opendev team to support such things. If either of them > > > > cannot be done and they do not fulfill the PTI or any other new > > > > project requirement criteria then they cannot be in OpenStack. > > > [...] > > > > > > "Governance" (the new project requirements document) right now > > > clearly states that new projects need to perform their code review > > > and gating tests on the "OpenStack Infrastructure" (the former name > > > for the OpenDev Collaboratory because that document hasn't been > > > updated to reflect the new name). You'll at a minimum need a vote of > > > the TC to remove those restrictions, so all this assumes that the > > > rest of the TC agrees with you that doing code review in GitHub with > > > a separate GitHub-connected CI system is allowable for new official > > > OpenStack project teams and deliverables. > > > > > > This is not "governance point of view" it's *your* point of view, so > > > please be clear that the decision is one the TC as a whole will need > > > to make. > > > > I think there is some misunderstanding here. I have never said > > anywhere that this is "TC agreed view" off-course this is my > > opinion as a community member as well as TC member. > > > > Any community member or TC members can provide their opinion but > > that should not be considered as "TC agreed plan" until that is > > explicitly mentioned in email or TC pass the resolution. We can > > have different views from TC members or chair but any of that > > should not be considered as "TC agreement" unless mentioned. I > > think this is how every email discussion is. > > You referred to it above as the "governance point of view" so I just > wanted to make certain you don't actually believe the governing > documents are unclear on this particular point, and understand that > OpenStack absolutely will need TC consensus on lifting a > longstanding restriction in order to allow an official deliverable > to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev > Collaboratory). > As a TC member, I do not think that any project that is a part of OpenStack should be hosted outside OpenDev. To be a part of OpenStack, you have to be hosted in OpenDev. I think that splitting to use Github + OpenDev will create too much complexity for new contributors in addition to everything else already mentioned about CI and release management further up in this thread. > > I have this in my list to give a clear picture from TC as an > > agreed plan: > > > > Step1: Continue the discussion in ML (here) > > Step2: After having a good amount of feedback here and we still > > not resolved the things, I will add this topic to TC > > meeting and get the TC consensus. > > Step3: Propose Governance resolution or documentation update > > Step4: Update the same in ML as "TC agreed plan". > > Thanks, this looks like a reasonable way forward. > Documents should definitely get updated! +2! -- > Jeremy Stanley > -Kendall Nelson -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricolin at ricolky.com Thu Jun 23 20:03:25 2022 From: ricolin at ricolky.com (Rico Lin) Date: Fri, 24 Jun 2022 04:03:25 +0800 Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team In-Reply-To: <86ab7ae4-8cf2-379d-e10e-06327f55430d@gmail.com> References: <86ab7ae4-8cf2-379d-e10e-06327f55430d@gmail.com> Message-ID: Hi team Sorry for the late reply, just added Hiromu to the heat-translator-core. And welcome Hiromu. *Rico Lin* On Fri, Jun 24, 2022 at 2:30 AM Yasufumi Ogawa wrote: > Hi heat-translator cores, > > We tacker team would like to keep contributing to maintain > heat-translator, so appreciate if you agree to the proposal kindly. > > Cheers, > Yasufumi > > On 2022/06/09 14:14, ueha.ayumu at fujitsu.com wrote: > > Hi Rico, > > > > This is just a friendly reminder that we are waiting for your action. > > Or please let us know if there are any other necessary processes. > > Thanks. > > > > Best Regards, > > Ueha > > > > -----Original Message----- > > From: ueha.ayumu at fujitsu.com > > Sent: Thursday, June 2, 2022 5:39 PM > > To: openstack-discuss at lists.openstack.org; 'Rico Lin' < > ricolin at ricolky.com> > > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina > (h-asahina) for heat-translator core team > > > > Hi Rico, > > > > This proposal is great helpful for us! > > There seems to be no particular objection, so could you add it to > heat-translator-core if there is no problem? > > Thanks. > > > > Best Regards, > > Ueha > > > > -----Original Message----- > > From: ueha.ayumu at fujitsu.com > > Sent: Wednesday, May 25, 2022 1:40 PM > > To: openstack-discuss at lists.openstack.org > > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina > (h-asahina) for heat-translator core team > > > > +1 > > > > Best Regards, > > Ueha > > > > -----Original Message----- > > From: Yoshito Ito > > Sent: Wednesday, May 25, 2022 12:31 PM > > To: openstack-discuss at lists.openstack.org > > Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) > for heat-translator core team > > > > Hi heat-translator team! > > > > I would like to propose Hiromu Asahina (h-asahina) as a member of the > heat-translator core team. I'm now not able to actively contribute to > heat-translator, and he can lead the team instead of me. > > > > He is mainly contributing to Tacker by fixing issues, refactoring > existing codes, and a lot of helpful reviews, with his experience as a > research engineer in NTT. Tacker highly depends on heat-translator in its > main functionality, so I believe he can manage it with his knowledge of > Tacker. > > > > Please vote for his nomination by answering this mail till June 1st. > > > > Best Regards, > > > > Yoshito Ito > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tiagohp at gmail.com Thu Jun 23 20:27:14 2022 From: tiagohp at gmail.com (Tiago Pires) Date: Thu, 23 Jun 2022 17:27:14 -0300 Subject: ovn-controller/OVS stranger behaviour Message-ID: Hi all, I'm trying to understand a stranger's behaviour regarding to ovn-controller/OVS. In my setup I have OVN 21.09/ OVS 2.16 and Ubuntu Xena and sometimes when a new VM is created, this VM can reach other VMs in east-west traffic (even in differents Chassis) but it can't reach an external network (e.g. Internet) through Chassi Gateway. I ran the following trace: # ovs-appctl ofproto/trace br-int in_port="93",icmp,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_ttl=64 And I got this output: Final flow: recirc_id=0xc157b1,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 Megaflow: recirc_id=0xc157b1,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src= 192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ttl=64,nw_frag=no Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:00:00:00:00)),set(ipv4(ttl=63)),userspace(pid=3451843211,controller(reason=1,dont_send=1,continuation=0,recirc_id=12670898,rule_cookie=0x3e26215e,controller_id=0,max_len=65535)) It seems the Datapath is querying the controller and I did not understand the reason. So, I did an ovn-controller recompute (ovn-appctl -t ovn-controller recompute) on the Chassi where the VM is placed to check if it could change the behaviour and I could trace the packet with success and the VM started to communicate with the Internet normally: Final flow: recirc_id=0x2,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 Megaflow: recirc_id=0x2,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,tun_id=0/0xffffff,tun_metadata0=NP,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src= 192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ecn=0,nw_ttl=64,nw_frag=no Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(tunnel(tun_id=0x2a,dst=10.X6.X3.133,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x30002}),flags(df|csum|key))),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:5e:00:04:00)),set(ipv4(ttl=63)),2 The Datapath action is using the tunnel with the Chassi Gateway. It happens always with new VMs but sometimes. After running the recompute on the Chassi, I created additional VMs and this issue did not happen. In my Chassi I have enable these parameters also: ovn-monitor-all="true" ovn-openflow-probe-interval="0" ovn-remote-probe-interval="180000" I did some troubleshooting and I'm seeing this error (ovs-vswitchd) always when a VM is created in a Chassi: 2022-06-23T11:47:08.385Z|07907|bridge|WARN|could not open network device tap8a43df0c-fd (No such device) 2022-06-23T11:47:09.282Z|07908|bridge|INFO|bridge br-int: added interface tap8a43df0c-fd on port 51 2022-06-23T11:47:09.645Z|07909|bridge|INFO|bridge br-int: added interface tap3200bf1c-20 on port 52 2022-06-23T11:47:19.329Z|07911|connmgr|INFO|br-int<->unix#1468: 430 flow_mods in the 7 s starting 10 s ago (410 adds, 20 deletes) On this commit http://patchwork.ozlabs.org/project/ovn/patch/1608197000-637-1-git-send-email-dceara at redhat.com/ it solved something similar to my issue. It seems the ovs-vswitchd is missing some flows and when I run the recompute it fixes it. So, in order to avoid this issue I'm testing at this moment to run the recompute through libvirt hook when a VM gets "started" status. Do you know this behaviour could be bug related? Regards, Tiago Pires Do you know this behaviour could be bug related? -------------- next part -------------- An HTML attachment was scrubbed... URL: From alsotoes at gmail.com Thu Jun 23 23:13:49 2022 From: alsotoes at gmail.com (Alvaro Soto) Date: Thu, 23 Jun 2022 18:13:49 -0500 Subject: [event] OpenInfradays Mexico 2022 (Virtual) Message-ID: You're all invited to participate in the CFP for OID-MX22 https://openinfradays.mx Let me know if you have any questions. --- Alvaro Soto Escobar Note: My work hours may not be your work hours. Please do not feel the need to respond during a time that is not convenient for you. ---------------------------------------------------------- Great people talk about ideas, ordinary people talk about things, small people talk... about other people. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Fri Jun 24 00:42:49 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 23 Jun 2022 19:42:49 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> Message-ID: <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> ---- On Thu, 23 Jun 2022 15:02:57 -0500 Kendall Nelson wrote ---- > > > On Thu, Jun 23, 2022 at 12:55 PM Jeremy Stanley wrote: > On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley wrote ---- > > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > > [...] > > > > Yes, both are separate things and I think we are mixing both or at > > > > least if we have such impression or governance is not so clear > > > > about it then we should fix it. I replied in another reply about > > > > governance point of view and IMO yes we should allow such new > > > > projects hosted on new tooling or so but they need to make sure > > > > all the help on CI/CD, release etc are taken care by them self or > > > > they help opendev team to support such things. If either of them > > > > cannot be done and they do not fulfill the PTI or any other new > > > > project requirement criteria then they cannot be in OpenStack. > > > [...] > > > > > > "Governance" (the new project requirements document) right now > > > clearly states that new projects need to perform their code review > > > and gating tests on the "OpenStack Infrastructure" (the former name > > > for the OpenDev Collaboratory because that document hasn't been > > > updated to reflect the new name). You'll at a minimum need a vote of > > > the TC to remove those restrictions, so all this assumes that the > > > rest of the TC agrees with you that doing code review in GitHub with > > > a separate GitHub-connected CI system is allowable for new official > > > OpenStack project teams and deliverables. > > > > > > This is not "governance point of view" it's *your* point of view, so > > > please be clear that the decision is one the TC as a whole will need > > > to make. > > > > I think there is some misunderstanding here. I have never said > > anywhere that this is "TC agreed view" off-course this is my > > opinion as a community member as well as TC member. > > > > Any community member or TC members can provide their opinion but > > that should not be considered as "TC agreed plan" until that is > > explicitly mentioned in email or TC pass the resolution. We can > > have different views from TC members or chair but any of that > > should not be considered as "TC agreement" unless mentioned. I > > think this is how every email discussion is. > > You referred to it above as the "governance point of view" so I just > wanted to make certain you don't actually believe the governing > documents are unclear on this particular point, and understand that > OpenStack absolutely will need TC consensus on lifting a > longstanding restriction in order to allow an official deliverable > to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev > Collaboratory). > > As a TC member, I do not think that any project that is a part of OpenStack should be hosted outside OpenDev. To be a part of OpenStack, you have to be hosted in OpenDev. I think that splitting to use Github + OpenDev will create too much complexity for new contributors in addition to everything else already mentioned about CI and release management further up in this thread. Well, I am not sure how valid the new contributors criteria is nowadays because we hardly get any new contributors in any of the OpenStack project. But here we are denying the interested contributors to contribute their project to OpenStack because of the tooling they want to use in developing their code. I am not sure if we will get other new contributors in gophercloud if it is hosted on opendev. -gmann > > > I have this in my list to give a clear picture from TC as an > > agreed plan: > > > > Step1: Continue the discussion in ML (here) > > Step2: After having a good amount of feedback here and we still > > not resolved the things, I will add this topic to TC > > meeting and get the TC consensus. > > Step3: Propose Governance resolution or documentation update > > Step4: Update the same in ML as "TC agreed plan". > > Thanks, this looks like a reasonable way forward. > > Documents should definitely get updated! +2! > -- > Jeremy Stanley > > -Kendall Nelson From yasufum.o at gmail.com Fri Jun 24 06:57:18 2022 From: yasufum.o at gmail.com (Yasufumi Ogawa) Date: Fri, 24 Jun 2022 15:57:18 +0900 Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team In-Reply-To: References: <86ab7ae4-8cf2-379d-e10e-06327f55430d@gmail.com> Message-ID: <0dbea349-5390-6d6e-ee19-bae1128eea43@gmail.com> Thanks Rico and all! On 2022/06/24 5:03, Rico Lin wrote: > Hi team > > Sorry for the?late reply, just added Hiromu?to the?heat-translator-core. > > And welcome Hiromu. > > *Rico Lin* > > > On Fri, Jun 24, 2022 at 2:30 AM Yasufumi Ogawa > wrote: > > Hi heat-translator cores, > > We tacker team would like to keep contributing to maintain > heat-translator, so appreciate if you agree to the proposal kindly. > > Cheers, > Yasufumi > > On 2022/06/09 14:14, ueha.ayumu at fujitsu.com > wrote: > > Hi Rico, > > > > This is just a friendly reminder that we are waiting for your action. > > Or please let us know if there are any other necessary processes. > > Thanks. > > > > Best Regards, > > Ueha > > > > -----Original Message----- > > From: ueha.ayumu at fujitsu.com > > > > Sent: Thursday, June 2, 2022 5:39 PM > > To: openstack-discuss at lists.openstack.org > ; 'Rico Lin' > > > > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina > (h-asahina) for heat-translator core team > > > > Hi Rico, > > > > This proposal is great helpful for us! > > There seems to be no particular objection, so could you add it to > heat-translator-core if there is no problem? > > Thanks. > > > > Best Regards, > > Ueha > > > > -----Original Message----- > > From: ueha.ayumu at fujitsu.com > > > > Sent: Wednesday, May 25, 2022 1:40 PM > > To: openstack-discuss at lists.openstack.org > > > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina > (h-asahina) for heat-translator core team > > > > +1 > > > > Best Regards, > > Ueha > > > > -----Original Message----- > > From: Yoshito Ito > > > Sent: Wednesday, May 25, 2022 12:31 PM > > To: openstack-discuss at lists.openstack.org > > > Subject: [heat-translator][tacker] Propose Hiromu Asahina > (h-asahina) for heat-translator core team > > > > Hi heat-translator team! > > > > I would like to propose Hiromu Asahina (h-asahina) as a member of > the heat-translator core team. I'm now not able to actively > contribute to heat-translator, and he can lead the team instead of me. > > > > He is mainly contributing to Tacker by fixing issues, refactoring > existing codes, and a lot of helpful reviews, with his experience as > a research engineer in NTT. Tacker highly depends on heat-translator > in its main functionality, so I believe he can manage it with his > knowledge of Tacker. > > > > Please vote for his nomination by answering this mail till June 1st. > > > > Best Regards, > > > > Yoshito Ito > > > > > From thierry at openstack.org Fri Jun 24 08:25:08 2022 From: thierry at openstack.org (Thierry Carrez) Date: Fri, 24 Jun 2022 10:25:08 +0200 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: <20220623174656.uqd5o4qd4tmgxo52@yuggoth.org> References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> <20220622233634.smszvmpdx6fwualp@yuggoth.org> <1819182829a.f4f70d37376766.6935407412644022771@ghanshyammann.com> <20220623174656.uqd5o4qd4tmgxo52@yuggoth.org> Message-ID: <03c44869-18e4-a6ab-139d-95b304ba17ec@openstack.org> Jeremy Stanley wrote: > On 2022-06-23 12:00:58 -0500 (-0500), Ghanshyam Mann wrote: >> Yes, that is what I mean that we have not explicitly said it has >> to be on OpenDev tooling as must requirement but yes we need to >> update the governance documentation also to have a clear view if >> "no opendev tooling then what are governance requirement" > [...] > > I'm not sure who "we" is in this context, but the TC has "explicitly > said it has to be on OpenDev tooling as must requirement" through > the passages I quoted above (the referenced document absolutely > states this, and was ratified by the TC). Removing that restriction > is a substantive change to OpenStack's governing documents, not a > mere clarification. I agree.. Historically, the TC has had a very strong stance that we should not fragment the community onto multiple platforms (for communication or development) or languages (for code or speech). The stated goal was to reinforce that OpenStack is a single project you join (and elect TC members to steward), rather than a loose coalition of several separate projects making independent choices. So allowing different development platforms would definitely be a step in a whole new direction, and a pretty significant change. I'd expect a discussion on allowing different parallel communication platforms to follow next. -- Thierry Carrez (ttx) From lucasagomes at gmail.com Fri Jun 24 09:13:51 2022 From: lucasagomes at gmail.com (Lucas Alvares Gomes) Date: Fri, 24 Jun 2022 10:13:51 +0100 Subject: ovn-controller/OVS stranger behaviour In-Reply-To: References: Message-ID: Hi, On Thu, Jun 23, 2022 at 9:30 PM Tiago Pires wrote: > > Hi all, > > I'm trying to understand a stranger's behaviour regarding to ovn-controller/OVS. > In my setup I have OVN 21.09/ OVS 2.16 and Ubuntu Xena and sometimes when a new VM is created, this VM can reach other VMs in east-west traffic (even in differents Chassis) but it can't reach an external network (e.g. Internet) through Chassi Gateway. > I ran the following trace: > # ovs-appctl ofproto/trace br-int in_port="93",icmp,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_ttl=64 > > And I got this output: > > Final flow: recirc_id=0xc157b1,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 > Megaflow: recirc_id=0xc157b1,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ttl=64,nw_frag=no > Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:00:00:00:00)),set(ipv4(ttl=63)),userspace(pid=3451843211,controller(reason=1,dont_send=1,continuation=0,recirc_id=12670898,rule_cookie=0x3e26215e,controller_id=0,max_len=65535)) > It seems the Datapath is querying the controller and I did not understand the reason. > > So, I did an ovn-controller recompute (ovn-appctl -t ovn-controller recompute) on the Chassi where the VM is placed to check if it could change the behaviour and I could trace the packet with success and the VM started to communicate with the Internet normally: > > Final flow: recirc_id=0x2,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 > Megaflow: recirc_id=0x2,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,tun_id=0/0xffffff,tun_metadata0=NP,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ecn=0,nw_ttl=64,nw_frag=no > Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(tunnel(tun_id=0x2a,dst=10.X6.X3.133,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x30002}),flags(df|csum|key))),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:5e:00:04:00)),set(ipv4(ttl=63)),2 > The Datapath action is using the tunnel with the Chassi Gateway. > This sounds like a bug in the ovn-controller to me. The fact that it worked after a recompute which forces ovn-controller to recalculate all flows tells me that there may be a bug in the "incremental processing" mechanism (a mechanism that calculates the changes based on deltas). > It happens always with new VMs but sometimes. After running the recompute on the Chassi, I created additional VMs and this issue did not happen. > > In my Chassi I have enable these parameters also: > ovn-monitor-all="true" > ovn-openflow-probe-interval="0" > ovn-remote-probe-interval="180000" > > I did some troubleshooting and I'm seeing this error (ovs-vswitchd) always when a VM is created in a Chassi: > 2022-06-23T11:47:08.385Z|07907|bridge|WARN|could not open network device tap8a43df0c-fd (No such device) > 2022-06-23T11:47:09.282Z|07908|bridge|INFO|bridge br-int: added interface tap8a43df0c-fd on port 51 > 2022-06-23T11:47:09.645Z|07909|bridge|INFO|bridge br-int: added interface tap3200bf1c-20 on port 52 > 2022-06-23T11:47:19.329Z|07911|connmgr|INFO|br-int<->unix#1468: 430 flow_mods in the 7 s starting 10 s ago (410 adds, 20 deletes) > Hmm... At a first glance it does not look related to the issue you are experiencing but, core OVN or OVS experts may know better. > On this commit http://patchwork.ozlabs.org/project/ovn/patch/1608197000-637-1-git-send-email-dceara at redhat.com/ it solved something similar to my issue. It seems the ovs-vswitchd is missing some flows and when I run the recompute it fixes it. Right yeah, we've seen a few bugs related to the incremental processing mechanism in the past. Things are much more stable nowadays but you may be hitting a new one. > So, in order to avoid this issue I'm testing at this moment to run the recompute through libvirt hook when a VM gets "started" status. > > Do you know this behaviour could be bug related? > > Regards, > > Tiago Pires > > Do you know this behaviour could be bug related? From lucasagomes at gmail.com Fri Jun 24 09:17:29 2022 From: lucasagomes at gmail.com (Lucas Alvares Gomes) Date: Fri, 24 Jun 2022 10:17:29 +0100 Subject: ovn-controller/OVS stranger behaviour In-Reply-To: References: Message-ID: Hi, I was going to forward this ML to the ovs-discuss ML (the ML that the core OVN folks watches) but I see that you already posted it there and already got some suggestions. I think we should continue the discussion over there, from an OpenStack perspective I don't think there's much we can do here because clearly ML2/OVN is updating the OVN NB DB accordingly but ovn-controller is not picking up the changes and installing the appropriate flows. On Fri, Jun 24, 2022 at 10:13 AM Lucas Alvares Gomes wrote: > > Hi, > > On Thu, Jun 23, 2022 at 9:30 PM Tiago Pires wrote: > > > > Hi all, > > > > I'm trying to understand a stranger's behaviour regarding to ovn-controller/OVS. > > In my setup I have OVN 21.09/ OVS 2.16 and Ubuntu Xena and sometimes when a new VM is created, this VM can reach other VMs in east-west traffic (even in differents Chassis) but it can't reach an external network (e.g. Internet) through Chassi Gateway. > > I ran the following trace: > > # ovs-appctl ofproto/trace br-int in_port="93",icmp,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_ttl=64 > > > > And I got this output: > > > > Final flow: recirc_id=0xc157b1,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 > > Megaflow: recirc_id=0xc157b1,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ttl=64,nw_frag=no > > Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:00:00:00:00)),set(ipv4(ttl=63)),userspace(pid=3451843211,controller(reason=1,dont_send=1,continuation=0,recirc_id=12670898,rule_cookie=0x3e26215e,controller_id=0,max_len=65535)) > > It seems the Datapath is querying the controller and I did not understand the reason. > > > > So, I did an ovn-controller recompute (ovn-appctl -t ovn-controller recompute) on the Chassi where the VM is placed to check if it could change the behaviour and I could trace the packet with success and the VM started to communicate with the Internet normally: > > > > Final flow: recirc_id=0x2,eth,icmp,reg0=0x300,reg11=0xd,reg12=0x10,reg13=0xf,reg14=0x3,reg15=0x2,metadata=0x29,in_port=93,vlan_tci=0x0000,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.140,nw_dst=8.8.8.8,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=0,icmp_code=0 > > Megaflow: recirc_id=0x2,ct_state=+new-est-rel-rpl-inv+trk,ct_label=0/0x1,eth,icmp,tun_id=0/0xffffff,tun_metadata0=NP,in_port=93,dl_src=fa:16:3e:26:34:ef,dl_dst=fa:16:3e:65:68:6e,nw_src=192.168.40.128/26,nw_dst=8.0.0.0/7,nw_ecn=0,nw_ttl=64,nw_frag=no > > Datapath actions: ct(commit,zone=15,label=0/0x1,nat(src)),set(tunnel(tun_id=0x2a,dst=10.X6.X3.133,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x30002}),flags(df|csum|key))),set(eth(src=fa:16:3e:ec:7f:dd,dst=00:00:5e:00:04:00)),set(ipv4(ttl=63)),2 > > The Datapath action is using the tunnel with the Chassi Gateway. > > > > This sounds like a bug in the ovn-controller to me. The fact that it > worked after a recompute which forces ovn-controller to recalculate > all flows tells me that there may be a bug in the "incremental > processing" mechanism (a mechanism that calculates the changes based > on deltas). > > > It happens always with new VMs but sometimes. After running the recompute on the Chassi, I created additional VMs and this issue did not happen. > > > > In my Chassi I have enable these parameters also: > > ovn-monitor-all="true" > > ovn-openflow-probe-interval="0" > > ovn-remote-probe-interval="180000" > > > > I did some troubleshooting and I'm seeing this error (ovs-vswitchd) always when a VM is created in a Chassi: > > 2022-06-23T11:47:08.385Z|07907|bridge|WARN|could not open network device tap8a43df0c-fd (No such device) > > 2022-06-23T11:47:09.282Z|07908|bridge|INFO|bridge br-int: added interface tap8a43df0c-fd on port 51 > > 2022-06-23T11:47:09.645Z|07909|bridge|INFO|bridge br-int: added interface tap3200bf1c-20 on port 52 > > 2022-06-23T11:47:19.329Z|07911|connmgr|INFO|br-int<->unix#1468: 430 flow_mods in the 7 s starting 10 s ago (410 adds, 20 deletes) > > > > Hmm... At a first glance it does not look related to the issue you are > experiencing but, core OVN or OVS experts may know better. > > > On this commit http://patchwork.ozlabs.org/project/ovn/patch/1608197000-637-1-git-send-email-dceara at redhat.com/ it solved something similar to my issue. It seems the ovs-vswitchd is missing some flows and when I run the recompute it fixes it. > > Right yeah, we've seen a few bugs related to the incremental > processing mechanism in the past. Things are much more stable nowadays > but you may be hitting a new one. > > > > So, in order to avoid this issue I'm testing at this moment to run the recompute through libvirt hook when a VM gets "started" status. > > > > Do you know this behaviour could be bug related? > > > > Regards, > > > > Tiago Pires > > > > Do you know this behaviour could be bug related? From zigo at debian.org Fri Jun 24 12:47:03 2022 From: zigo at debian.org (Thomas Goirand) Date: Fri, 24 Jun 2022 14:47:03 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: Message-ID: <21fa4a8a-0422-8dda-2eee-cf3536e77a4a@debian.org> On 6/21/22 17:34, Clark Boylan wrote: > This isn't the only instance we've run into this. The CNCF's Cloud Native > landscape (https://landscape.cncf.io/) requires open source projects be > hosted on Github (not not proprietary systems....). From my perspective, > I think this calls out why it is important for us to continue to push > back against these rules. Software is built in a number of places and if > we force people to use Github (and other proprietary tools) we move away > from part of the mission here. I very much agree here, especially considering that Github is a non-free platform. If it was hosted let's say on *any* Gitlab instance, I would think differently. > I think this calls out the fundamental disconnect here. We use open > source tools because we believe that open source needs and should be > built with open source tools. That does require a little bit of > investment from the project side to ensure the tooling functions > as needed and is maintained. Yeah, definitively! Thanks a lot to the infra team for insisting on these subjects. OpenStack wouldn't be where it is without you guys. Cheers, Thomas Goirand (zigo) From zigo at debian.org Fri Jun 24 12:56:59 2022 From: zigo at debian.org (Thomas Goirand) Date: Fri, 24 Jun 2022 14:56:59 +0200 Subject: [all] Failure to build (FTBFS) with Sphinx 5.0 and docutils 0.18 Message-ID: <6f68aab4-8a3e-93dd-68cc-99a0a9ba0313@debian.org> Hi, A number of bugs were filed against the Debian packages I maintain. Here's the list of bugs: https://bugs.debian.org/1013410 Nova https://bugs.debian.org/1013406 os-api-ref https://bugs.debian.org/1013402 openstacksdk https://bugs.debian.org/1013394 dogpile.cache https://bugs.debian.org/1013393 repoze.who https://bugs.debian.org/1013382 cloudkitty I'd appreciate a bit of help from the OpenStack project here, to fix these before Sphinx 5 and Docutils 0.18 are uploaded to Debian Unstable. If you do have a fix, a quick link in these bug reports would be lovely. Cheers, Thomas Goirand (zigo) From zigo at debian.org Fri Jun 24 13:00:34 2022 From: zigo at debian.org (Thomas Goirand) Date: Fri, 24 Jun 2022 15:00:34 +0200 Subject: [all][horizon][django 4.0] Failure to build (FTBFS) with Django 4.0 Message-ID: <0bae4082-0960-cdc3-8895-c98ef9c6732c@debian.org> Hi, A number of RC bugs were filed against my packages in Debian Unstable: https://bugs.debian.org/1013488 Horizon https://bugs.debian.org/1013475 django-pyscss https://bugs.debian.org/1013515 ironic-ui https://bugs.debian.org/1013605 django-formtools https://bugs.debian.org/1013603 enmerkar https://bugs.debian.org/1013506 ara If you have a fix, and can post the link to the patch in the Debian bug tracker, that'd be very much appreciated and help these packages not being auto-removed from Debian testing. Otherwise, you can always ping me on IRC. Cheers, Thomas Goirand (zigo) From fungi at yuggoth.org Fri Jun 24 14:16:21 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 24 Jun 2022 14:16:21 +0000 Subject: [all] Failure to build (FTBFS) with Sphinx 5.0 and docutils 0.18 In-Reply-To: <6f68aab4-8a3e-93dd-68cc-99a0a9ba0313@debian.org> References: <6f68aab4-8a3e-93dd-68cc-99a0a9ba0313@debian.org> Message-ID: <20220624141620.w3qnnkfiwkgx7z7l@yuggoth.org> On 2022-06-24 14:56:59 +0200 (+0200), Thomas Goirand wrote: [...] > I'd appreciate a bit of help from the OpenStack project here, to fix these > before Sphinx 5 and Docutils 0.18 are uploaded to Debian Unstable. > > If you do have a fix, a quick link in these bug reports would be lovely. Yes, it looks like https://review.opendev.org/846848 (autoproposed) is attempting to update our constraints for Sphinx from 4.5.0 to 5.0.2 (though keeping docutils 0.17.1 at the moment, presumably due to a transitive version cap somewhere in the dependency set) and is failing its cross-osc-tox-docs canary build with the following: Warning, treated as error: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'. Unfortunately that's not an easy condition to scan our entire codebase for (captions with more or fewer than one '%s' in them), so it's hard to know how widespread the problem is. Looking at the bug you linked for the nova package build, it seems like that's a got unit a test failure unrelated to Sphinx or docutils. Are you sure it's related? For the os-api-ref bug, that seems to be an API response mismatch in its tests. I can't say for sure without looking deeper that the tests don't use Sphinx or docutils to generate the HTML involved, but it's not clear to me that's the cause at least. The openstacksdk bug similarly appears to be for a unit test failure, something to do with tempfile handling I think, not documentation building. The cloudkitty bug does look like a problem with the documentation source being incompatible with Sphinx involving the wsmeext extension, so different than we're seeing on the requirements tests at least. The dogpile.cache and repoze.who bugs also look Sphinx-related, though the OpenStack community doesn't develop those projects so someone will probably need to reach out to them wherever they collaborate officially. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Fri Jun 24 14:21:54 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 24 Jun 2022 14:21:54 +0000 Subject: [all][horizon][django 4.0] Failure to build (FTBFS) with Django 4.0 In-Reply-To: <0bae4082-0960-cdc3-8895-c98ef9c6732c@debian.org> References: <0bae4082-0960-cdc3-8895-c98ef9c6732c@debian.org> Message-ID: <20220624142154.m4y7fwcjho7nhbla@yuggoth.org> On 2022-06-24 15:00:34 +0200 (+0200), Thomas Goirand wrote: > A number of RC bugs were filed against my packages in Debian Unstable: > > https://bugs.debian.org/1013488 Horizon > https://bugs.debian.org/1013475 django-pyscss > https://bugs.debian.org/1013515 ironic-ui > https://bugs.debian.org/1013605 django-formtools > https://bugs.debian.org/1013603 enmerkar > https://bugs.debian.org/1013506 ara > > If you have a fix, and can post the link to the patch in the Debian bug > tracker, that'd be very much appreciated and help these packages not being > auto-removed from Debian testing. Otherwise, you can always ping me on IRC. It looks like global requirements has been getting conservative Django version cap increases. Most recently, https://review.opendev.org/813405 raised it from <3.0 to <3.3 in October. If OpenStack is going to start testing with 4.x versions, a similar change will need to be proposed so we can see what the fallout might be upstream. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From ozzzo at yahoo.com Fri Jun 24 15:24:34 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Fri, 24 Jun 2022 15:24:34 +0000 (UTC) Subject: [Swift] [kolla] Swift issues in one cluster In-Reply-To: <1210414976.7924712.1656002429999@mail.yahoo.com> References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> <918427872.7544677.1655932210049@mail.yahoo.com> <440638798.277721.1655991773368@mail.yahoo.com> <1210414976.7924712.1656002429999@mail.yahoo.com> Message-ID: <1368115593.8368767.1656084274624@mail.yahoo.com> Reading in [1] I see this: "Having another look at that issue, it sounds like slow client shouldn't be handled by OpenStack services but rather with a load balancer, especially if the service is Internet facing" I don't understand what is being recommended here. We have 60 Swift servers, and customer traffic goes directly to those servers. It seems like a load-balancer would be a performance-reducing bottleneck. Our clusters are not internet-facing, and I haven't seen internet-facing Swift clusters at any of my employers. What is the solution for this problem? We have worked around it by getting our customer to stop using his Java app to download large files for now, but we would like to find a long-term solution. Is there any hope of getting this bug fixed? On Thursday, June 23, 2022, 12:49:29 PM EDT, Albert Braden wrote: We're running 2.23.3 but it appears that we are experiencing the bug (1). We tracked the problem down to a client who recently started using a java library to read large files from Swift. When he moves his activity to the other QA cluster, the problem follows. Am I guessing correctly that the bug was never fixed, and that (2) fixes a different problem? On Thursday, June 23, 2022, 10:43:44 AM EDT, Clay Gerrard wrote: https://review.opendev.org/c/openstack/swift/+/575254 has been included in every released swift tag since 2.21.0 I believe Train included a swift version of at least 2.22?https://releases.openstack.org/train/#swift Nvidia doesn't use haproxy in front of our swift proxies, and we don't see BlockingIOError in tracebacks - the traceback might go away if you upgrade to the latest swift (2.29) and/or eventlet and/or python 3.8ish On Thu, Jun 23, 2022 at 8:49 AM Albert Braden wrote: Can anyone help with this Swift issue? It looks like we are being hit with this bug (1) but this bug doesn't seem to mention where/whether it was ever fixed. This (2) appears to be a fix, and it appears to have been merged, but it doesn't mention the bug, and it's not obvious to me what version it affects. Is anyone else encountering this problem? It appears that customers in this one cluster may be doing something to cause it; we're still trying to track down specifically what they are doing, that they aren't doing in the other clusters. We are running kolla-ansible Train on RHEL7. 1. https://bugs.launchpad.net/swift/+bug/1572719 2. https://opendev.org/openstack/swift/commit/0e81ffd1e1481a73146fce17f61f2ab9e01eb684 On Wednesday, June 22, 2022, 05:10:10 PM EDT, Albert Braden wrote: Is this bug fixed in Train? https://bugs.launchpad.net/swift/+bug/1572719 On Tuesday, June 21, 2022, 09:26:47 AM EDT, Albert Braden wrote: All of our endpoints are https: +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ | | | keystone | identity | True | internal | https://api-int..:5000 | | | | swift | object-store | True | public | https://swift. .:8080/v1/AUTH_%(tenant_id)s | | | | swift | object-store | True | internal | https://swift..:8080/v1/AUTH_%(tenant_id)s | | | | keystone | identity | True | admin | https://api-int. .:35357 | | | | keystone | identity | True | public | https://api-ext. .:5000 | | | | swift | object-store | True | admin | https://swift. .:8080/v1 | +----------------------------------+--------+--------------+--------------+---------+-----------+---------------------------------------------------------+ I don't think this is causing the issue; all of our clusters are setup the same. We did think it was load at first, and got the 2 heaviest users to stop what they were doing, but that didn't make a difference. Our other QA cluster has similar load and identical hardware. When I look at the network graphs, I see traffic spiking up to 1G, but these are 25G interfaces, and none of the resources on the boxes are exhausted. CPU is 97% idle; memory is 30% used, disk is not full. It doesn't look like the problem is load-related. We see the haproxy connections stacking up even when load is low. What else could be causing this? On Friday, June 17, 2022, 11:12:36 PM EDT, Pete Zaitcev wrote: On Fri, 17 Jun 2022 17:33:27 +0000 (UTC) Albert Braden wrote: > $ openstack container list > Unable to establish connection to https://swift..:8080/v1/AUTH_: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer') Right away I have a question: why in the world are you connecting to 8080 with HTTPS? > (from Splunk): > Payload: swift-proxy-server: STDERR: File "/usr/lib64/python3.6/socket.py", line 604, in write#012 return self._sock.send(b) > Payload: swift-proxy-server: STDERR: BlockingIOError > Payload: swift-proxy-server: STDERR: os.read(self.rfd, 1) > Payload: swift-proxy-server: STDERR: File "/usr/lib/python3.6/site-packages/eventlet/wsgi.py", line 818, in process_request#012 proto.__init__(conn_state, self) This looks quite fishy to me, because the os.read is in swift/common/utils.py and it's responsible for the mutex. > When we look at network connections, we see haproxy stacking up (many lines of this): >? > # netstat -ntup | sort -b -k2 -n -r | head -n +100 > tcp? 5976932? ? ? 0 127.0.0.1:60738? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5976446? ? ? 0 127.0.0.1:58480? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973217? ? ? 0 127.0.0.1:33244? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5973120? ? ? 0 127.0.0.1:51836? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? > tcp? 5971968? ? ? 0 127.0.0.1:58516? ? ? ? 127.0.0.1:8080? ? ? ? ? ESTABLISHED 13045/haproxy? ? ? >? ... > > If we restart the swift_haproxy and swift_proxy_server containers then the problem goes away, and comes back over a few minutes. Where should we be looking for the root cause of this issue? Indeed if so many requests are established, you're in trouble. The best fix, I think, is to find the customer who's doing it and punish them. Otherwise, quotas and the ratelimiting middleware are your friends. There's also a possibility that your cluster is underperforming, although usually that results in 500 results first. But then again, at times users would "compensate" for issues by just launching way more requests, in effect DoS-ing the cluster even worse. -- Pete -- Clay Gerrard210 788 9431 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdratlif at globalnoc.iu.edu Fri Jun 24 15:44:23 2022 From: jdratlif at globalnoc.iu.edu (jdratlif at globalnoc.iu.edu) Date: Fri, 24 Jun 2022 11:44:23 -0400 Subject: [sdk] renaming a volume with the SDK Message-ID: <063f01d887e1$47222100$d5666300$@globalnoc.iu.edu> I'm trying to set a property on a volume, specifically the name. But when I try to do this, it tells me the Session object has no default_microversion. I have all my auth in environment variables. Using TOKEN authentication. What am I doing wrong here? import openstack from openstack.block_storage.v3._proxy import Proxy as BlockStorageService from openstack.block_storage.v3.volume import Volume # enable openstack debug logging openstack.enable_logging(debug=True) conn = openstack.connect() cinder: BlockStorageService = conn.block_storage volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") volume.name = "test_rename" volume.commit(conn.session) Thanks, --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6687 bytes Desc: not available URL: From clay.gerrard at gmail.com Fri Jun 24 15:54:36 2022 From: clay.gerrard at gmail.com (Clay Gerrard) Date: Fri, 24 Jun 2022 10:54:36 -0500 Subject: [Swift] [kolla] Swift issues in one cluster In-Reply-To: <1368115593.8368767.1656084274624@mail.yahoo.com> References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> <918427872.7544677.1655932210049@mail.yahoo.com> <440638798.277721.1655991773368@mail.yahoo.com> <1210414976.7924712.1656002429999@mail.yahoo.com> <1368115593.8368767.1656084274624@mail.yahoo.com> Message-ID: On Fri, Jun 24, 2022 at 10:29 AM Albert Braden wrote: > > "Having another look at that issue, it sounds like slow client shouldn't be handled by OpenStack services but rather with a load balancer, especially if the service is Internet facing" > > I don't understand what is being recommended here. I think they were suggesting using a http proxy application - maybe haproxy - will have more options to protect network resources from misbehaving clients than the swift proxy application. Like kicking off keep-alive connections after a while, or slow clients that hang up resources. > We have 60 Swift servers, and customer traffic goes directly to those servers. It seems like a load-balancer would be a performance-reducing bottleneck. That's cool, do you use round robin dns or something? > Is there any hope of getting this bug fixed? If we can reproduce the problem you're seeing there's some chance we could offer a solution through just a code change, but it's going to be difficult if repro requires haproxy in the pipeline. If there is a problem w/o haproxy, it might have more to do with eventlet.wsgi or python's base http server than swift... can you affirm the issue when clients talk directly to the python/eventlet/swift application? -- Clay Gerrard From johnsomor at gmail.com Fri Jun 24 16:14:18 2022 From: johnsomor at gmail.com (Michael Johnson) Date: Fri, 24 Jun 2022 09:14:18 -0700 Subject: [all] Failure to build (FTBFS) with Sphinx 5.0 and docutils 0.18 In-Reply-To: <20220624141620.w3qnnkfiwkgx7z7l@yuggoth.org> References: <6f68aab4-8a3e-93dd-68cc-99a0a9ba0313@debian.org> <20220624141620.w3qnnkfiwkgx7z7l@yuggoth.org> Message-ID: I took a quick look at os-api-ref. The tests are failing with docutils 0.18. They have changed how column widths are defined for tables and the tests have static HTML assertions that are failing[1]. >From the docutils release notes[2] I found that there is now a table-style directive "colwidths-grid" that in theory would restore the colgroup output[3]. [1] https://opendev.org/openstack/os-api-ref/src/branch/master/os_api_ref/tests/test_microversions.py#L60 [2] https://docutils.sourceforge.io/RELEASE-NOTES.html#release-0-18-2021-10-26 [3] https://docutils.sourceforge.io/docs/user/config.html#toc-entry-82 The challenge I have is I'm not very familiar with docutils, so I'm not really sure how to apply that style setting. Maybe this will provide enough hints someone more familiar with docutils can quickly fix this. Michael On Fri, Jun 24, 2022 at 7:28 AM Jeremy Stanley wrote: > > On 2022-06-24 14:56:59 +0200 (+0200), Thomas Goirand wrote: > [...] > > I'd appreciate a bit of help from the OpenStack project here, to fix these > > before Sphinx 5 and Docutils 0.18 are uploaded to Debian Unstable. > > > > If you do have a fix, a quick link in these bug reports would be lovely. > > Yes, it looks like https://review.opendev.org/846848 (autoproposed) > is attempting to update our constraints for Sphinx from 4.5.0 to > 5.0.2 (though keeping docutils 0.17.1 at the moment, presumably due > to a transitive version cap somewhere in the dependency set) and is > failing its cross-osc-tox-docs canary build with the following: > > Warning, treated as error: extlinks: Sphinx-6.0 will require a > caption string to contain exactly one '%s' and all other '%' > need to be escaped as '%%'. > > Unfortunately that's not an easy condition to scan our entire > codebase for (captions with more or fewer than one '%s' in them), so > it's hard to know how widespread the problem is. > > Looking at the bug you linked for the nova package build, it seems > like that's a got unit a test failure unrelated to Sphinx or > docutils. Are you sure it's related? > > For the os-api-ref bug, that seems to be an API response mismatch in > its tests. I can't say for sure without looking deeper that the > tests don't use Sphinx or docutils to generate the HTML involved, > but it's not clear to me that's the cause at least. > > The openstacksdk bug similarly appears to be for a unit test > failure, something to do with tempfile handling I think, not > documentation building. > > The cloudkitty bug does look like a problem with the documentation > source being incompatible with Sphinx involving the wsmeext > extension, so different than we're seeing on the requirements tests > at least. > > The dogpile.cache and repoze.who bugs also look Sphinx-related, > though the OpenStack community doesn't develop those projects so > someone will probably need to reach out to them wherever they > collaborate officially. > -- > Jeremy Stanley From kennelson11 at gmail.com Fri Jun 24 16:35:13 2022 From: kennelson11 at gmail.com (Kendall Nelson) Date: Fri, 24 Jun 2022 11:35:13 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <21fa4a8a-0422-8dda-2eee-cf3536e77a4a@debian.org> References: <21fa4a8a-0422-8dda-2eee-cf3536e77a4a@debian.org> Message-ID: On Fri, Jun 24, 2022 at 7:47 AM Thomas Goirand wrote: > On 6/21/22 17:34, Clark Boylan wrote: > > This isn't the only instance we've run into this. The CNCF's Cloud Native > > landscape (https://landscape.cncf.io/) requires open source projects be > > hosted on Github (not not proprietary systems....). From my perspective, > > I think this calls out why it is important for us to continue to push > > back against these rules. Software is built in a number of places and if > > we force people to use Github (and other proprietary tools) we move away > > from part of the mission here. > > I very much agree here, especially considering that Github is a non-free > platform. If it was hosted let's say on *any* Gitlab instance, I would > think differently. > > > I think this calls out the fundamental disconnect here. We use open > > source tools because we believe that open source needs and should be > > built with open source tools. That does require a little bit of > > investment from the project side to ensure the tooling functions > > as needed and is maintained. > > Yeah, definitively! > > Thanks a lot to the infra team for insisting on these subjects. > OpenStack wouldn't be where it is without you guys. > +2 +W We wouldn't be as successful as a community without the tools you have worked so hard on to keep us moving forward! > > Cheers, > > Thomas Goirand (zigo) > > -Kendall Nelson -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Fri Jun 24 16:43:41 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Fri, 24 Jun 2022 22:13:41 +0530 Subject: Regarding Floating IP is existing Setup In-Reply-To: <1816246.GmBfMWsNUm@p1> References: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> <1816246.GmBfMWsNUm@p1> Message-ID: Hi, Thanks for the advice and the link, What i saw when i do testing using tcpdump was "ARP" was not working, and it is not able to associate the FLoating IP with the MAC address of the interface in the VM, When i do the associate and disassociate the VM , it works fine But the Router NameSpace got changed. Regards Adivya Singh On Thu, Jun 23, 2022 at 1:22 PM Slawek Kaplonski wrote: > Hi, > > Dnia wtorek, 21 czerwca 2022 13:55:51 CEST Adivya Singh pisze: > > hi Eugen, > > > > The current setup is 3 controller nodes, The Load is not much on each > > controller and the number of DHCP agent is always set to 2 as per the > > standard in the neutron.conf, The L3 agent seems to be stables as other > > router namespace works fine under it, Only few router Namespace get > > affected under the agent. > > Is it that problem happens for new floating IPs or for the FIPs which were > working fine and then suddenly stopped working? If the latter, was there > any action which triggered the issue to happen? > Is there e.g. only one FIP broken in the router or maybe when it happens, > then all FIPs which uses same router are broken? > > Can You also try to analyze with e.g. tcpdump where traffic is dropped > exactly? You can check > http://kaplonski.pl/blog/neutron-where-is-my-packet-2/ for some more > detailed description how traffic should go from the external network to > Your instance. > > > > > Most of the template having issue , Have all instance having FLoating > IP, a > > Stack with a single floating IP have chance of issue very less > > > > Regards > > Adivya Singh > > > > On Tue, Jun 21, 2022 at 1:18 PM Eugen Block wrote: > > > > > Hi, > > > > > > this sounds very familiar to me, I had to deal with something similar > > > a couple of times in a heavily used cluster with 2 control nodes. What > > > does your setup look like, is it a HA setup? I would start checking > > > the DHCP and L3 agents. After increasing dhcp_agents_per_network to 2 > > > in neutron.conf and restarting the services this didn't occur again > > > (yet). This would impact floating IPs as well, sometimes I had to > > > disable and enable the affected router(s). If you only have one > > > control node a different approach is necessary. Do you see a high load > > > on the control node? > > > > > > > > > Zitat von Adivya Singh : > > > > > > > hi Team, > > > > > > > > We got a issue in Xena release, where we set the environment in > Ubuntu > > > > Platform, But later we get some issues in Floating IP not reachable. > > > > > > > > In a Network node, not all router namespace are Impacted and only > few of > > > > them get affected, So we can not comment Network node has a issue. > > > > > > > > The L3 agent where the Router is tied up, Worked just fine, as other > > > > routers work Fine. > > > > > > > > and the one having issue in Floating IP, if i unassigned and > assigned it > > > > starts working most of the time. > > > > > > > > Any thoughts on this > > > > > > > > Regards > > > > Adivya Singh > > > > > > > > > > > > > > > > > > > > -- > Slawek Kaplonski > Principal Software Engineer > Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Fri Jun 24 16:44:27 2022 From: kennelson11 at gmail.com (Kendall Nelson) Date: Fri, 24 Jun 2022 11:44:27 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> Message-ID: On Thu, Jun 23, 2022 at 7:43 PM Ghanshyam Mann wrote: > ---- On Thu, 23 Jun 2022 15:02:57 -0500 Kendall Nelson < > kennelson11 at gmail.com> wrote ---- > > > > > > On Thu, Jun 23, 2022 at 12:55 PM Jeremy Stanley > wrote: > > On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > > > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley < > fungi at yuggoth.org> wrote ---- > > > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > > > [...] > > > > > Yes, both are separate things and I think we are mixing both or > at > > > > > least if we have such impression or governance is not so clear > > > > > about it then we should fix it. I replied in another reply about > > > > > governance point of view and IMO yes we should allow such new > > > > > projects hosted on new tooling or so but they need to make sure > > > > > all the help on CI/CD, release etc are taken care by them self or > > > > > they help opendev team to support such things. If either of them > > > > > cannot be done and they do not fulfill the PTI or any other new > > > > > project requirement criteria then they cannot be in OpenStack. > > > > [...] > > > > > > > > "Governance" (the new project requirements document) right now > > > > clearly states that new projects need to perform their code review > > > > and gating tests on the "OpenStack Infrastructure" (the former name > > > > for the OpenDev Collaboratory because that document hasn't been > > > > updated to reflect the new name). You'll at a minimum need a vote > of > > > > the TC to remove those restrictions, so all this assumes that the > > > > rest of the TC agrees with you that doing code review in GitHub > with > > > > a separate GitHub-connected CI system is allowable for new official > > > > OpenStack project teams and deliverables. > > > > > > > > This is not "governance point of view" it's *your* point of view, > so > > > > please be clear that the decision is one the TC as a whole will > need > > > > to make. > > > > > > I think there is some misunderstanding here. I have never said > > > anywhere that this is "TC agreed view" off-course this is my > > > opinion as a community member as well as TC member. > > > > > > Any community member or TC members can provide their opinion but > > > that should not be considered as "TC agreed plan" until that is > > > explicitly mentioned in email or TC pass the resolution. We can > > > have different views from TC members or chair but any of that > > > should not be considered as "TC agreement" unless mentioned. I > > > think this is how every email discussion is. > > > > You referred to it above as the "governance point of view" so I just > > wanted to make certain you don't actually believe the governing > > documents are unclear on this particular point, and understand that > > OpenStack absolutely will need TC consensus on lifting a > > longstanding restriction in order to allow an official deliverable > > to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev > > Collaboratory). > > > > As a TC member, I do not think that any project that is a part of > OpenStack should be hosted outside OpenDev. To be a part of OpenStack, you > have to be hosted in OpenDev. I think that splitting to use Github + > OpenDev will create too much complexity for new contributors in addition to > everything else already mentioned about CI and release management further > up in this thread. > > Well, I am not sure how valid the new contributors criteria is nowadays > because we hardly get any new contributors in > any of the OpenStack project. But here we are denying the interested > contributors to contribute their project to OpenStack > because of the tooling they want to use in developing their code. I am not > sure if we will get other new contributors in > gophercloud if it is hosted on opendev. > I feel like that is kind of a defeatist attitude- that we shouldn't keep new contributors to OpenStack in mind. There are new people coming in every round of outreachy, every semester at NDSU, BU, Northwestern... I am in contact with 3 more universities since the Berlin summit looking to form similar relationships with us. That's not nothing? If the process of getting started becomes more complex, I think we will struggle even more to get new people involved and build them into the next generation of contributors to step up when the current folks have to/ want to move on. Many projects already have a very short bench and I can only see fracturing our development processes as making it harder to build up the number of contributors we have to support our teams and those that have been in leadership roles so long they are looking to step down soon. > > -gmann > > > > > > I have this in my list to give a clear picture from TC as an > > > agreed plan: > > > > > > Step1: Continue the discussion in ML (here) > > > Step2: After having a good amount of feedback here and we still > > > not resolved the things, I will add this topic to TC > > > meeting and get the TC consensus. > > > Step3: Propose Governance resolution or documentation update > > > Step4: Update the same in ML as "TC agreed plan". > > > > Thanks, this looks like a reasonable way forward. > > > > Documents should definitely get updated! +2! > > -- > > Jeremy Stanley > > > > -Kendall Nelson > -Kendall Nelson -------------- next part -------------- An HTML attachment was scrubbed... URL: From ozzzo at yahoo.com Fri Jun 24 16:46:40 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Fri, 24 Jun 2022 16:46:40 +0000 (UTC) Subject: [Swift] [kolla] Swift issues in one cluster In-Reply-To: References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> <918427872.7544677.1655932210049@mail.yahoo.com> <440638798.277721.1655991773368@mail.yahoo.com> <1210414976.7924712.1656002429999@mail.yahoo.com> <1368115593.8368767.1656084274624@mail.yahoo.com> Message-ID: <1926732307.8415136.1656089200951@mail.yahoo.com> AFAIK we are running vanilla Swift. Clients usually connect to the Swift endpoint by running "swift" or "openstack container" commands, and Swift uses exabgp to route the traffic. I'm not exactly sure what the Java library is doing but I can dig more into that if it would help. I see swift_proxy_server and swift_haproxy containers running on every Swift node. Is that not the normal configuration? According to [1] it's pretty easy to reproduce: "You can reproduce this by issuing a GET request for a few hundred MB file and never consuming the response, but keep the client socket open. Swift will log a 499 but the socket does not always close." On Friday, June 24, 2022, 12:02:05 PM EDT, Clay Gerrard wrote: On Fri, Jun 24, 2022 at 10:29 AM Albert Braden wrote: > > "Having another look at that issue, it sounds like slow client shouldn't be handled by OpenStack services but rather with a load balancer, especially if the service is Internet facing" > > I don't understand what is being recommended here. I think they were suggesting using a http proxy application - maybe haproxy - will have more options to protect network resources from misbehaving clients than the swift proxy application.? Like kicking off keep-alive connections after a while, or slow clients that hang up resources. > We have 60 Swift servers, and customer traffic goes directly to those servers. It seems like a load-balancer would be a performance-reducing bottleneck. That's cool, do you use round robin dns or something? > Is there any hope of getting this bug fixed? If we can reproduce the problem you're seeing there's some chance we could offer a solution through just a code change, but it's going to be difficult if repro requires haproxy in the pipeline.? If there is a problem w/o haproxy, it might have more to do with eventlet.wsgi or python's base http server than swift... can you affirm the issue when clients talk directly to the python/eventlet/swift application? -- Clay Gerrard -------------- next part -------------- An HTML attachment was scrubbed... URL: From clay.gerrard at gmail.com Fri Jun 24 17:07:02 2022 From: clay.gerrard at gmail.com (Clay Gerrard) Date: Fri, 24 Jun 2022 12:07:02 -0500 Subject: [Swift] [kolla] Swift issues in one cluster In-Reply-To: <1926732307.8415136.1656089200951@mail.yahoo.com> References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> <918427872.7544677.1655932210049@mail.yahoo.com> <440638798.277721.1655991773368@mail.yahoo.com> <1210414976.7924712.1656002429999@mail.yahoo.com> <1368115593.8368767.1656084274624@mail.yahoo.com> <1926732307.8415136.1656089200951@mail.yahoo.com> Message-ID: On Fri, Jun 24, 2022 at 11:46 AM Albert Braden wrote: > > "You can reproduce this by issuing a GET request for a few hundred MB file and never consuming the response, but keep the client socket open. Swift will log a 499 but the socket does not always close." Was that the behavior that you were seeing? Swift logs 499, but the socket stays open? Eventlet 0.22.0 says eventlet.wsgi will timeout idle clients now: https://eventlet.net/doc/changelog.html#id23 ... so maybe that bug as written is no longer valid - but it could still be related to what you're seeing. Wasn't the original traceback a BlockingIOError - that seems different from a hung client socket from an idle client. From artem.goncharov at gmail.com Fri Jun 24 18:01:09 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Fri, 24 Jun 2022 20:01:09 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> Message-ID: > > I feel like that is kind of a defeatist attitude- that we shouldn't keep new contributors to OpenStack in mind. There are new people > coming in every round of outreachy, every semester at NDSU, BU, Northwestern... I am in contact with 3 more universities since > the Berlin summit looking to form similar relationships with us. That's not nothing? That?s not nothing, but those are not long-term contributors requiring huge mentoring effort from us. Therefore this is not something I would take into account for this particular discussion. This is more about customers and operators of OpenStack based clouds > > If the process of getting started becomes more complex, I think we will struggle even more to get new people involved and build > them into the next generation of contributors to step up when the current folks have to/ want to move on. Many projects already > have a very short bench and I can only see fracturing our development processes as making it harder to build up the number of > contributors we have to support our teams and those that have been in leadership roles so long they are looking to step down soon. > This way block those for whom our process is too complex from using easier approach. Or initially I also raised the point that sometimes it may be just not applicable due to particular regulations I barely remember any of the mentioned students that we able to go through getting accounts and gerrit managed within few days. And from my experience unless someone experienced is sitting and helping out it is far away from intuitive. The problem actually start from the point that it is not intuitive how to find the beginning. Even now for the sake of experiment it took me around 3 minutes knowing which particular document I am searching to get to it. Now I am at the way crossing: do I use https://docs.openstack.org/doc-contrib-guide/quickstart/first-timers.html , https://wiki.openstack.org/wiki/How_To_Contribute , https://docs.opendev.org/opendev/infra-manual/latest/developers.html#development-workflow or https://docs.openstack.org/contributors/users/index.html ? I bet we can find more docs. Depending on your entry point you will end up on different documents with different doc bugs. Then once you see those documents you may want to run away just from their complexity and amount of required steps. I feel this is really overwhelming how much you need to go through before you start at least understanding what you really need to do. This is likely not the case for any of us here, because we know how it works. But that IS the case for newcomers. I feel it is not complex to cope with gerrit, but rather getting all things set up is what make people frightened. Please do not treat my statement as some form of blaming what infra folks are doing or anything else. We have huge amount of work done, tons of docs and already this is better than not having them. But most of our docs are created by developers who are known not to be good at writing good documentation. Those other enterprise companies are spending millions of __your_currency_here__ to polish user experience and make sure newcomer does not end up on broken process because this is what is generating their money income. Back to the point: - can we have official OpenStack projects outside opendev? [yes/no] - can we have non-official OpenStack projects (lets call them affiliated) not hosted on opendev under *some* OpenStack governance? [yes/no] - can we have OpenStack manage *git* organisation (i.e. GitHub organisation) as well as artifacts publishing for those outside of opendev as some form of limited governance? This may or may not include some more regulations regarding PTI, code-review, etc. Purpose is to improve marketing side of the delivery and be able to revive it once current maintainers depart [yes/no] - can we provide Zuul gating for those official or affiliated projects outside of opendev (with strict regulations what and how)? [yes/no] - can we _think_ on allowing people use alternative public auth providers (i.e. GitHub, ...) accounts with OpenStack to lower the entry barrier? [yes/no] The questions above are not technical at the moment, but rather legal. It is required to clarify law before we even can start thinking who and how. -Artem -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdratlif at globalnoc.iu.edu Fri Jun 24 18:13:30 2022 From: jdratlif at globalnoc.iu.edu (jdratlif at globalnoc.iu.edu) Date: Fri, 24 Jun 2022 14:13:30 -0400 Subject: [sdk] renaming a volume with the SDK In-Reply-To: <063f01d887e1$47222100$d5666300$@globalnoc.iu.edu> References: <063f01d887e1$47222100$d5666300$@globalnoc.iu.edu> Message-ID: <080201d887f6$1c212b30$54638190$@globalnoc.iu.edu> from keystoneauth1.adapter import Adapter adapter = Adapter(conn.session, service_type="block-storage") Adding this and passing adapter instead of conn.session to the commit call seems to work. --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University From: jdratlif at globalnoc.iu.edu Sent: Friday, June 24, 2022 11:44 AM To: openstack-discuss at lists.openstack.org Cc: Ratliff, John Subject: [sdk] renaming a volume with the SDK I'm trying to set a property on a volume, specifically the name. But when I try to do this, it tells me the Session object has no default_microversion. I have all my auth in environment variables. Using TOKEN authentication. What am I doing wrong here? import openstack from openstack.block_storage.v3._proxy import Proxy as BlockStorageService from openstack.block_storage.v3.volume import Volume # enable openstack debug logging openstack.enable_logging(debug=True) conn = openstack.connect() cinder: BlockStorageService = conn.block_storage volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") volume.name = "test_rename" volume.commit(conn.session) Thanks, --- John Ratliff Systems Automation Engineer GlobalNOC @ Indiana University -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 6687 bytes Desc: not available URL: From fungi at yuggoth.org Fri Jun 24 18:19:14 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 24 Jun 2022 18:19:14 +0000 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> Message-ID: <20220624181838.ngnxizuqrvd66oyd@yuggoth.org> On 2022-06-24 20:01:09 +0200 (+0200), Artem Goncharov wrote: [...] > - can we have official OpenStack projects outside opendev? > [yes/no] > - can we have non-official OpenStack projects (lets call them > affiliated) not hosted on opendev under *some* OpenStack > governance? [yes/no] > - can we have OpenStack manage *git* organisation (i.e. GitHub > organisation) as well as artifacts publishing for those outside > of opendev as some form of limited governance? This may or may > not include some more regulations regarding PTI, code-review, > etc. Purpose is to improve marketing side of the delivery and be > able to revive it once current maintainers depart [yes/no] Those are up to the TC members to decide, so I'll leave that to them. > - can we provide Zuul gating for those official or affiliated > projects outside of opendev (with strict regulations what and > how)? [yes/no] If that's a question for the OpenDev sysadmins, we've got a pretty firm collective "no" on it at this point based on prior discussions. We're happy for projects to set up advisory testing for dependencies hosted on GitHub (think "third-party CI" type feedback on those projects' pull requests) and do things like provide builds of ARM/AArch64 wheels for pyca/cryptography in order to make OpenStack's ARM-based jobs much faster, but configuring our Zuul deployment to provide gating services for projects hosted outside the Gerrit we operate isn't happening in my opinion. > - can we _think_ on allowing people use alternative public auth > providers (i.e. GitHub, ...) accounts with OpenStack to lower the > entry barrier? [yes/no] [...] That's one of the goals for https://docs.opendev.org/opendev/infra-specs/latest/specs/central-auth.html so if you have time or know someone who has time to help make further progress on it, please let us know. We have a Keycloak PoC already up so we can demo authenticating Zuul admin API calls (at https://keycloak.opendev.org/ currently), but the larger effort is migration planning and integrating the existing Launchpad OpenID as a source so we can let existing users continue to authenticate, at least temporarily. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From elod.illes at est.tech Fri Jun 24 18:28:30 2022 From: elod.illes at est.tech (=?UTF-8?B?RWzFkWQgSWxsw6lz?=) Date: Fri, 24 Jun 2022 20:28:30 +0200 Subject: [all][stable] Propose to EOL Pike series Message-ID: Hi, It seems that time has come to (mass)transition Pike to End of Life due to lack of maintainers (and backports + reviews; see the list of recent changes [1]). Some core projects have made this step already. Others are still open but not really maintained, which can be seen by the number of daily periodic stable job failures on pike branch, which is around 40 for several months now. (By the way, periodic stable job failures number is constantly high [2], due to some projects which seem to be abandoned completely (murano, *-powervm), but that is another story). Please let me know what do you think, or indicate if any of the projects wants to keep its pike branch open / in Extended Maintenance. [1] https://review.opendev.org/q/branch:stable/pike [2] http://lists.openstack.org/pipermail/openstack-stable-maint/2022-May/thread.html Thanks, El?d From gmann at ghanshyammann.com Fri Jun 24 18:39:12 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 24 Jun 2022 13:39:12 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> Message-ID: <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> ---- On Fri, 24 Jun 2022 11:44:27 -0500 Kendall Nelson wrote --- > > > On Thu, Jun 23, 2022 at 7:43 PM Ghanshyam Mann wrote: > ---- On Thu, 23 Jun 2022 15:02:57 -0500 Kendall Nelson wrote ---- > > > > > > On Thu, Jun 23, 2022 at 12:55 PM Jeremy Stanley wrote: > > On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > > > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley wrote ---- > > > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > > > [...] > > > > > Yes, both are separate things and I think we are mixing both or at > > > > > least if we have such impression or governance is not so clear > > > > > about it then we should fix it. I replied in another reply about > > > > > governance point of view and IMO yes we should allow such new > > > > > projects hosted on new tooling or so but they need to make sure > > > > > all the help on CI/CD, release etc are taken care by them self or > > > > > they help opendev team to support such things. If either of them > > > > > cannot be done and they do not fulfill the PTI or any other new > > > > > project requirement criteria then they cannot be in OpenStack. > > > > [...] > > > > > > > > "Governance" (the new project requirements document) right now > > > > clearly states that new projects need to perform their code review > > > > and gating tests on the "OpenStack Infrastructure" (the former name > > > > for the OpenDev Collaboratory because that document hasn't been > > > > updated to reflect the new name). You'll at a minimum need a vote of > > > > the TC to remove those restrictions, so all this assumes that the > > > > rest of the TC agrees with you that doing code review in GitHub with > > > > a separate GitHub-connected CI system is allowable for new official > > > > OpenStack project teams and deliverables. > > > > > > > > This is not "governance point of view" it's *your* point of view, so > > > > please be clear that the decision is one the TC as a whole will need > > > > to make. > > > > > > I think there is some misunderstanding here. I have never said > > > anywhere that this is "TC agreed view" off-course this is my > > > opinion as a community member as well as TC member. > > > > > > Any community member or TC members can provide their opinion but > > > that should not be considered as "TC agreed plan" until that is > > > explicitly mentioned in email or TC pass the resolution. We can > > > have different views from TC members or chair but any of that > > > should not be considered as "TC agreement" unless mentioned. I > > > think this is how every email discussion is. > > > > You referred to it above as the "governance point of view" so I just > > wanted to make certain you don't actually believe the governing > > documents are unclear on this particular point, and understand that > > OpenStack absolutely will need TC consensus on lifting a > > longstanding restriction in order to allow an official deliverable > > to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev > > Collaboratory). > > > > As a TC member, I do not think that any project that is a part of OpenStack should be hosted outside OpenDev. To be a part of OpenStack, you have to be hosted in OpenDev. I think that splitting to use Github + OpenDev will create too much complexity for new contributors in addition to everything else already mentioned about CI and release management further up in this thread. > > Well, I am not sure how valid the new contributors criteria is nowadays because we hardly get any new contributors in > any of the OpenStack project. But here we are denying the interested contributors to contribute their project to OpenStack > because of the tooling they want to use in developing their code. I am not sure if we will get other new contributors in > gophercloud if it is hosted on opendev. > > I feel like that is kind of a defeatist attitude- that we shouldn't keep new contributors to OpenStack in mind. There are new peoplecoming in every round of outreachy, every semester at NDSU, BU, Northwestern... I am in contact with 3 more universities since the Berlin summit looking to form similar relationships with us. That's not nothing? > If the process of getting started becomes more complex, I think we will struggle even more to get new people involved and buildthem into the next generation of contributors to step up when the current folks have to/ want to move on. Many projects already > have a very short bench and I can only see fracturing our development processes as making it harder to build up the number ofcontributors we have to support our teams and those that have been in leadership roles so long they are looking to step down soon. No, it's not a defeatist attitude but instead a practical attitude and move forward to progress the things rather than stuck on currently-not-working stuff. Being practical based on facts and not being optimistic is not defeat. I think Artem also explained it what I meant by 'new contributor' in this context. But let me elaborate on my statement. There are two different sets of new contributors 1. short-term: outreach, intern or so 2. Long-term. I am talking about the 2nd one here as 1st one is anyways going after a short time and it is the same for them to learn Gerrit or GitHub. Long-term new contributors are someone we need to think that our process/tooling are ok for them to sustain as long term or it is too complex/split to work. For example: "Hey, my company is asking me to be a new long-term maintainer in OpenStack but I deny it because OpenStack uses more than one set of tooling to develop the code or run the tests". In this example, before saying more than one tooling will be difficult for new contributors we should answer two very practical questions: 1. How many of such new contributors do we have in OpenStack 2. If there is, do they need to learn two processes? I think they will be contributing to a single project following a single process/tool. If we get some super contributor to all projects then it is a different thing. And practical data is "we do not get new contributors helping us with help needed things", Below points can justify it: 1. no help on our help needed (upstream investment opportunity) list "because of fewer contributors". 2. Many projects get retired in every cycle "because of fewer contributors". 3. OpenStack Infra (OpenDev) is shrinking, many services are shutting down "because of fewer contributors". 4. OpenStack Community is not able to finish user-facing features like RBAC, OSC on time "because of fewer contributors". 5. Most of our existing contributors are overloaded and slowly burnout "because of fewer contributors". OpenDev maintainer overloaded is a good example. 6. OpenStack supporting teams like QA, infra, requirement, stable branch, and releases do not have enough maintainers that they require. I am saying what strategy or program will work to fix these but in the current situation, we have to accept that these are problems because we do not get new contributors. If we cannot fix them then we have to adjust ourself as per current situation even change the things which previously agreed or worked. I am starting another discussion here to fix this new contributor problem in this thread but I am saying that considering the new contributors criteria in this email thread context and not changing the things is not relevant, -gmann > > -gmann > > > > > > I have this in my list to give a clear picture from TC as an > > > agreed plan: > > > > > > Step1: Continue the discussion in ML (here) > > > Step2: After having a good amount of feedback here and we still > > > not resolved the things, I will add this topic to TC > > > meeting and get the TC consensus. > > > Step3: Propose Governance resolution or documentation update > > > Step4: Update the same in ML as "TC agreed plan". > > > > Thanks, this looks like a reasonable way forward. > > > > Documents should definitely get updated! +2! > > -- > > Jeremy Stanley > > > > -Kendall Nelson > > -Kendall Nelson From elod.illes at est.tech Fri Jun 24 18:39:19 2022 From: elod.illes at est.tech (=?UTF-8?B?RWzFkWQgSWxsw6lz?=) Date: Fri, 24 Jun 2022 20:39:19 +0200 Subject: [oslo] Propose to EOL stable/queens, stable/rocky on all the oslo scope In-Reply-To: <3ff9fe9f-e75f-62a9-1da5-8cdb140427a8@est.tech> References: <3055264.zr5fvq113q@p1> <25b21881-bd0b-f763-9bb5-a66340108455@nemebean.com> <3ff9fe9f-e75f-62a9-1da5-8cdb140427a8@est.tech> Message-ID: <9136d244-0054-e902-30f5-673c834265d8@est.tech> Hi Oslo team, Is there anything that hinders the transition to EOL? This thread is seemingly got stuck somewhere. ...the periodic stable job failure count is still quite high, mostly because oslo's old stable branches are still failing :-/ Since those gates are broken for several months now and no patches were able to land, I think moving oslo's rocky, queens and pike branches to End of Life is unfortunately quite reasonable. (Also note, that I've just sent a mail about mass-EOL Pike series [1]) [1] http://lists.openstack.org/pipermail/openstack-discuss/2022-June/029250.html Thanks, El?d On 2021. 10. 14. 15:42, El?d Ill?s wrote: > What Ben wrote is correct. > > One comment for the topic: oslo projects have Pike open (and broken) > as well, so together with stablre/rocky and stable/queens stable/pike > branches should be also marked as End of Life if no maintainers > stepping up for these branches. > > Thanks, > > El?d > > > On 2021. 10. 04. 22:59, Ben Nemec wrote: >> >> >> On 10/4/21 2:00 PM, Slawek Kaplonski wrote: >>> Hi, >>> >>> On poniedzia?ek, 4 pa?dziernika 2021 20:46:29 CEST feilong wrote: >>>> Hi Herve, >>>> >>>> Please correct me, does that mean we have to also EOL stable/queens >>>> and >>>> stable/rocky for most of the other projects technically? Or it >>>> should be >>>> OK? Thanks. >>> >>> I don't think we have to. I think it's not that common that we are >>> using new >>> versions of oslo libs in those stable branches so IMHO if all works >>> fine for >>> some project and it has maintainers, it still can be in EM phase. >>> Or is my understanding wrong here? >> >> The Oslo libs released for those versions will continue to work, so >> you're right that it wouldn't be necessary to EOL all of the >> consumers of Oslo. >> >> The danger would be if a critical bug were found in one of those old >> releases and a fix needed to be released. However, at this point the >> likelihood of finding such a serious bug seems pretty low, and in >> some cases it may be possible to use a newer Oslo release with an >> older service. >> >>> >>>> >>>> On 5/10/21 5:09 am, Herve Beraud wrote: >>>>> Hi, >>>>> >>>>> On our last meeting of the oslo team we discussed the problem with >>>>> broken stable >>>>> branches (rocky and older) in oslo's projects [1]. >>>>> >>>>> Indeed, almost all these branches are broken. El?d Ill?s kindly >>>>> generated a list of periodic-stable errors on Oslo's stable >>>>> branches [2]. >>>>> >>>>> Given the lack of active maintainers on Oslo and given the current >>>>> status of the CI in those branches, I propose to make them End Of >>>>> Life. >>>>> >>>>> I will wait until the end of month for anyone who would like to maybe >>>>> step up >>>>> as maintainer of those branches and who would at least try to fix CI >>>>> of them. >>>>> >>>>> If no one will volunteer for that, I'll EOLing those branches for all >>>>> the projects under the oslo umbrella. >>>>> >>>>> Let us know your thoughts. >>>>> >>>>> Thank you for your attention. >>>>> >>>>> [1] >>>>> https://meetings.opendev.org/meetings/oslo/2021/oslo. >>> 2021-10-04-15.00.log.tx >>>>> t >>>>> [2] >>>>> http://lists.openstack.org/pipermail/openstack-discuss/2021-July/ >>> 023939.html >>> >>> >> > > From gmann at ghanshyammann.com Fri Jun 24 18:48:34 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 24 Jun 2022 13:48:34 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> Message-ID: <181970b6026.d2d008bb444269.3149216954267005995@ghanshyammann.com> ---- On Fri, 24 Jun 2022 13:39:12 -0500 Ghanshyam Mann wrote --- > ---- On Fri, 24 Jun 2022 11:44:27 -0500 Kendall Nelson wrote --- > > > > > > On Thu, Jun 23, 2022 at 7:43 PM Ghanshyam Mann wrote: > > ---- On Thu, 23 Jun 2022 15:02:57 -0500 Kendall Nelson wrote ---- > > > > > > > > > On Thu, Jun 23, 2022 at 12:55 PM Jeremy Stanley wrote: > > > On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > > > > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley wrote ---- > > > > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > > > > [...] > > > > > > Yes, both are separate things and I think we are mixing both or at > > > > > > least if we have such impression or governance is not so clear > > > > > > about it then we should fix it. I replied in another reply about > > > > > > governance point of view and IMO yes we should allow such new > > > > > > projects hosted on new tooling or so but they need to make sure > > > > > > all the help on CI/CD, release etc are taken care by them self or > > > > > > they help opendev team to support such things. If either of them > > > > > > cannot be done and they do not fulfill the PTI or any other new > > > > > > project requirement criteria then they cannot be in OpenStack. > > > > > [...] > > > > > > > > > > "Governance" (the new project requirements document) right now > > > > > clearly states that new projects need to perform their code review > > > > > and gating tests on the "OpenStack Infrastructure" (the former name > > > > > for the OpenDev Collaboratory because that document hasn't been > > > > > updated to reflect the new name). You'll at a minimum need a vote of > > > > > the TC to remove those restrictions, so all this assumes that the > > > > > rest of the TC agrees with you that doing code review in GitHub with > > > > > a separate GitHub-connected CI system is allowable for new official > > > > > OpenStack project teams and deliverables. > > > > > > > > > > This is not "governance point of view" it's *your* point of view, so > > > > > please be clear that the decision is one the TC as a whole will need > > > > > to make. > > > > > > > > I think there is some misunderstanding here. I have never said > > > > anywhere that this is "TC agreed view" off-course this is my > > > > opinion as a community member as well as TC member. > > > > > > > > Any community member or TC members can provide their opinion but > > > > that should not be considered as "TC agreed plan" until that is > > > > explicitly mentioned in email or TC pass the resolution. We can > > > > have different views from TC members or chair but any of that > > > > should not be considered as "TC agreement" unless mentioned. I > > > > think this is how every email discussion is. > > > > > > You referred to it above as the "governance point of view" so I just > > > wanted to make certain you don't actually believe the governing > > > documents are unclear on this particular point, and understand that > > > OpenStack absolutely will need TC consensus on lifting a > > > longstanding restriction in order to allow an official deliverable > > > to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev > > > Collaboratory). > > > > > > As a TC member, I do not think that any project that is a part of OpenStack should be hosted outside OpenDev. To be a part of OpenStack, you have to be hosted in OpenDev. I think that splitting to use Github + OpenDev will create too much complexity for new contributors in addition to everything else already mentioned about CI and release management further up in this thread. > > > > Well, I am not sure how valid the new contributors criteria is nowadays because we hardly get any new contributors in > > any of the OpenStack project. But here we are denying the interested contributors to contribute their project to OpenStack > > because of the tooling they want to use in developing their code. I am not sure if we will get other new contributors in > > gophercloud if it is hosted on opendev. > > > > I feel like that is kind of a defeatist attitude- that we shouldn't keep new contributors to OpenStack in mind. There are new peoplecoming in every round of outreachy, every semester at NDSU, BU, Northwestern... I am in contact with 3 more universities since the Berlin summit looking to form similar relationships with us. That's not nothing? > > If the process of getting started becomes more complex, I think we will struggle even more to get new people involved and buildthem into the next generation of contributors to step up when the current folks have to/ want to move on. Many projects already > > have a very short bench and I can only see fracturing our development processes as making it harder to build up the number ofcontributors we have to support our teams and those that have been in leadership roles so long they are looking to step down soon. > > No, it's not a defeatist attitude but instead a practical attitude and move forward to progress the things rather > than stuck on currently-not-working stuff. Being practical based on facts and not being optimistic is not defeat. > > I think Artem also explained it what I meant by 'new contributor' in this context. But let me elaborate on my statement. > There are two different sets of new contributors 1. short-term: outreach, intern or so 2. Long-term. I am talking about the 2nd > one here as 1st one is anyways going after a short time and it is the same for them to learn Gerrit or GitHub. > Long-term new contributors are someone we need to think that our process/tooling are ok for them to sustain as > long term or it is too complex/split to work. For example: "Hey, my company is asking me to be a new long-term > maintainer in OpenStack but I deny it because OpenStack uses more than one set of tooling to develop the code or run > the tests". In this example, before saying more than one tooling will be difficult for new contributors we should answer > two very practical questions: > > 1. How many of such new contributors do we have in OpenStack > 2. If there is, do they need to learn two processes? I think they will be contributing to a single project following a single > process/tool. If we get some super contributor to all projects then it is a different thing. > > And practical data is "we do not get new contributors helping us with help needed things", Below points can justify it: > > 1. no help on our help needed (upstream investment opportunity) list "because of fewer contributors". > 2. Many projects get retired in every cycle "because of fewer contributors". > 3. OpenStack Infra (OpenDev) is shrinking, many services are shutting down "because of fewer contributors". > 4. OpenStack Community is not able to finish user-facing features like RBAC, OSC on time "because of fewer contributors". > 5. Most of our existing contributors are overloaded and slowly burnout "because of fewer contributors". OpenDev maintainer > overloaded is a good example. > 6. OpenStack supporting teams like QA, infra, requirement, stable branch, and releases do not have enough maintainers > that they require. > > I am saying what strategy or program will work to fix these but in the current situation, we have to accept that these are > problems because we do not get new contributors. If we cannot fix them then we have to adjust ourself as per current > situation even change the things which previously agreed or worked. > > I am starting another discussion here to fix this new contributor problem in this thread but I am saying that considering the > new contributors criteria in this email thread context and not changing the things is not relevant, * I am *not* starting another discussion here to fix..... -gmann > > -gmann > > > > > -gmann > > > > > > > > > I have this in my list to give a clear picture from TC as an > > > > agreed plan: > > > > > > > > Step1: Continue the discussion in ML (here) > > > > Step2: After having a good amount of feedback here and we still > > > > not resolved the things, I will add this topic to TC > > > > meeting and get the TC consensus. > > > > Step3: Propose Governance resolution or documentation update > > > > Step4: Update the same in ML as "TC agreed plan". > > > > > > Thanks, this looks like a reasonable way forward. > > > > > > Documents should definitely get updated! +2! > > > -- > > > Jeremy Stanley > > > > > > -Kendall Nelson > > > > -Kendall Nelson > > From gmann at ghanshyammann.com Fri Jun 24 19:30:55 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 24 Jun 2022 14:30:55 -0500 Subject: [all][foundation][ecosystem][tc] External projects under the foundation hat In-Reply-To: <03c44869-18e4-a6ab-139d-95b304ba17ec@openstack.org> References: <2c111cd9-619c-4eab-a389-42b4f8c46907@www.fastmail.com> <1818d532d6c.119c40617310633.4788333975737676820@ghanshyammann.com> <20220622233634.smszvmpdx6fwualp@yuggoth.org> <1819182829a.f4f70d37376766.6935407412644022771@ghanshyammann.com> <20220623174656.uqd5o4qd4tmgxo52@yuggoth.org> <03c44869-18e4-a6ab-139d-95b304ba17ec@openstack.org> Message-ID: <181973228dc.fc30f2cf444961.7259826772732303573@ghanshyammann.com> ---- On Fri, 24 Jun 2022 03:25:08 -0500 Thierry Carrez wrote --- > Jeremy Stanley wrote: > > On 2022-06-23 12:00:58 -0500 (-0500), Ghanshyam Mann wrote: > >> Yes, that is what I mean that we have not explicitly said it has > >> to be on OpenDev tooling as must requirement but yes we need to > >> update the governance documentation also to have a clear view if > >> "no opendev tooling then what are governance requirement" > > [...] > > > > I'm not sure who "we" is in this context, but the TC has "explicitly > > said it has to be on OpenDev tooling as must requirement" through > > the passages I quoted above (the referenced document absolutely > > states this, and was ratified by the TC). Removing that restriction > > is a substantive change to OpenStack's governing documents, not a > > mere clarification. Well, I will not say those statements mentioned on the 'new project application' page are different from "project has to be hosted on OpenDev" but at the same time, they are not exactly the same. There is a difference between the "OpenStack Infrastructure" mentioned in that document since there was no OpenDev and now when OpenDev is a separate governing body than OpenStack. At least it is not just renaming the word "Open Infrastructure" to "OpenDev" directly but we should discuss if that can be something from OpenStack governance like "TACT" SIG or OpenDev directly. At least discuss, agree and explicitly state that so that we do not have any confusion or have the same question (like this thread) in future. or Even if in future new people maintaining OpenStack and not aware that Open Infrastructure became OpenDev may have more confusion on these two term. > > I agree.. Historically, the TC has had a very strong stance that we > should not fragment the community onto multiple platforms (for > communication or development) or languages (for code or speech). The > stated goal was to reinforce that OpenStack is a single project you join > (and elect TC members to steward), rather than a loose coalition of > several separate projects making independent choices. > > So allowing different development platforms would definitely be a step > in a whole new direction, and a pretty significant change. I'd expect a > discussion on allowing different parallel communication platforms to > follow next. True, I do not deny that fact but I am saying there are changes in the situation and a good user-facing OpenStack tool is asking if they can be part of OpenStack or not but on the different tools, they want to maintain their code. The question is accept that request or deny it, If deny from an OpenStack perspective then can we have that as a separate project under OpenInfra where there is no strict requirement of being hosted on OpenDev. Why we need to think about it because it is a beneficial tooling for OpenStack users. -gmann > > -- > Thierry Carrez (ttx) > > From cboylan at sapwetik.org Fri Jun 24 20:09:38 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Fri, 24 Jun 2022 13:09:38 -0700 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> Message-ID: <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> On Fri, Jun 24, 2022, at 11:39 AM, Ghanshyam Mann wrote: > snip > > No, it's not a defeatist attitude but instead a practical attitude and > move forward to progress the things rather > than stuck on currently-not-working stuff. Being practical based on > facts and not being optimistic is not defeat. > > I think Artem also explained it what I meant by 'new contributor' in > this context. But let me elaborate on my statement. > There are two different sets of new contributors 1. short-term: > outreach, intern or so 2. Long-term. I am talking about the 2nd > one here as 1st one is anyways going after a short time and it is the > same for them to learn Gerrit or GitHub. > Long-term new contributors are someone we need to think that our > process/tooling are ok for them to sustain as > long term or it is too complex/split to work. For example: "Hey, my > company is asking me to be a new long-term > maintainer in OpenStack but I deny it because OpenStack uses more than > one set of tooling to develop the code or run > the tests". In this example, before saying more than one tooling will > be difficult for new contributors we should answer > two very practical questions: > > 1. How many of such new contributors do we have in OpenStack > 2. If there is, do they need to learn two processes? I think they will > be contributing to a single project following a single > process/tool. If we get some super contributor to all projects then it > is a different thing. > > And practical data is "we do not get new contributors helping us with > help needed things", Below points can justify it: > > 1. no help on our help needed (upstream investment opportunity) list > "because of fewer contributors". > 2. Many projects get retired in every cycle "because of fewer > contributors". > 3. OpenStack Infra (OpenDev) is shrinking, many services are shutting > down "because of fewer contributors". > 4. OpenStack Community is not able to finish user-facing features like > RBAC, OSC on time "because of fewer contributors". > 5. Most of our existing contributors are overloaded and slowly burnout > "because of fewer contributors". OpenDev maintainer > overloaded is a good example. > 6. OpenStack supporting teams like QA, infra, requirement, stable > branch, and releases do not have enough maintainers > that they require. > > I am saying what strategy or program will work to fix these but in the > current situation, we have to accept that these are > problems because we do not get new contributors. If we cannot fix them > then we have to adjust ourself as per current > situation even change the things which previously agreed or worked. > > I am starting another discussion here to fix this new contributor > problem in this thread but I am saying that considering the > new contributors criteria in this email thread context and not changing > the things is not relevant, > While you didn't want to start a new conversation on this topic, I do think it is worthwhile to consider why we think this might help anything if that is the argument being made to justify these potential actions. I think there are two main angles to look at this. The first is does this reduce the burden on us? Potentially if you replaced self hosted services with services hosted by others that could be the case. This isn't risk free (think what happened with transifex or docker hub), but the risk may be worth the return. As mentioned elsewhere in the thread if we want to continue to uphold our community values it would be better to consider hosted open source options instead. You also have to accept the limitations of whatever the new platform is, because if you start doing a bunch of custom work around it you're back where you started and haven't reduced much burden. The other is whether or not we're likely to infuse new blood by making a change. This is where things get a lot more murky for me. I don't think it is universally true that choosing to use github is going to magically create new contributors. You only have to look at gnocchi to see a historical example of this. Our projects need maintainers. Maintenance of software requires some non trivial involvement. You might get a few more drive by contributions, but that will only help the projects grow long term if you can convert some subset to maintainers. This is difficult. This is difficult regardless of the platform you choose to develop on. I think that if the goal is attracting new contributors we should be explicit about why the actions we are taking are believed to help with that not just simply assert that it is the case. Also, adding a new project like gophercloud won't automatically add contributors to openstack. Contributors to gophercloud will continue to be contributors to gophercloud. In fact you point out above it is unlikely they will contribute to other projects. For this reason I think it is also worthwhile to think about what benefits of such a change exist. Does openstack benefit in any way by adding gophercloud to openstack but not changing anything about their contribution methods? Does gophercloud benefit in any way? To me this is essentially the status quo only with new additional governance overhead. From gmann at ghanshyammann.com Fri Jun 24 20:24:34 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 24 Jun 2022 15:24:34 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> Message-ID: <181976345ea.120c9f346445765.3461952296572449418@ghanshyammann.com> ---- On Fri, 24 Jun 2022 15:09:38 -0500 Clark Boylan wrote --- > On Fri, Jun 24, 2022, at 11:39 AM, Ghanshyam Mann wrote: > > > > snip > > > > > No, it's not a defeatist attitude but instead a practical attitude and > > move forward to progress the things rather > > than stuck on currently-not-working stuff. Being practical based on > > facts and not being optimistic is not defeat. > > > > I think Artem also explained it what I meant by 'new contributor' in > > this context. But let me elaborate on my statement. > > There are two different sets of new contributors 1. short-term: > > outreach, intern or so 2. Long-term. I am talking about the 2nd > > one here as 1st one is anyways going after a short time and it is the > > same for them to learn Gerrit or GitHub. > > Long-term new contributors are someone we need to think that our > > process/tooling are ok for them to sustain as > > long term or it is too complex/split to work. For example: "Hey, my > > company is asking me to be a new long-term > > maintainer in OpenStack but I deny it because OpenStack uses more than > > one set of tooling to develop the code or run > > the tests". In this example, before saying more than one tooling will > > be difficult for new contributors we should answer > > two very practical questions: > > > > 1. How many of such new contributors do we have in OpenStack > > 2. If there is, do they need to learn two processes? I think they will > > be contributing to a single project following a single > > process/tool. If we get some super contributor to all projects then it > > is a different thing. > > > > And practical data is "we do not get new contributors helping us with > > help needed things", Below points can justify it: > > > > 1. no help on our help needed (upstream investment opportunity) list > > "because of fewer contributors". > > 2. Many projects get retired in every cycle "because of fewer > > contributors". > > 3. OpenStack Infra (OpenDev) is shrinking, many services are shutting > > down "because of fewer contributors". > > 4. OpenStack Community is not able to finish user-facing features like > > RBAC, OSC on time "because of fewer contributors". > > 5. Most of our existing contributors are overloaded and slowly burnout > > "because of fewer contributors". OpenDev maintainer > > overloaded is a good example. > > 6. OpenStack supporting teams like QA, infra, requirement, stable > > branch, and releases do not have enough maintainers > > that they require. > > > > I am saying what strategy or program will work to fix these but in the > > current situation, we have to accept that these are > > problems because we do not get new contributors. If we cannot fix them > > then we have to adjust ourself as per current > > situation even change the things which previously agreed or worked. > > > > I am starting another discussion here to fix this new contributor > > problem in this thread but I am saying that considering the > > new contributors criteria in this email thread context and not changing > > the things is not relevant, > > > > While you didn't want to start a new conversation on this topic, I do think it is worthwhile to consider why we think this might help anything if that is the argument being made to justify these potential actions. > > I think there are two main angles to look at this. > > The first is does this reduce the burden on us? Potentially if you replaced self hosted services with services hosted by others that could be the case. This isn't risk free (think what happened with transifex or docker hub), but the risk may be worth the return. As mentioned elsewhere in the thread if we want to continue to uphold our community values it would be better to consider hosted open source options instead. You also have to accept the limitations of whatever the new platform is, because if you start doing a bunch of custom work around it you're back where you started and haven't reduced much burden. > > The other is whether or not we're likely to infuse new blood by making a change. This is where things get a lot more murky for me. I don't think it is universally true that choosing to use github is going to magically create new contributors. You only have to look at gnocchi to see a historical example of this. Our projects need maintainers. Maintenance of software requires some non trivial involvement. You might get a few more drive by contributions, but that will only help the projects grow long term if you can convert some subset to maintainers. This is difficult. This is difficult regardless of the platform you choose to develop on. > > I think that if the goal is attracting new contributors we should be explicit about why the actions we are taking are believed to help with that not just simply assert that it is the case. > > Also, adding a new project like gophercloud won't automatically add contributors to openstack. Contributors to gophercloud will continue to be contributors to gophercloud. In fact you point out above it is unlikely they will contribute to other projects. For this reason I think it is also worthwhile to think about what benefits of such a change exist. Does openstack benefit in any way by adding gophercloud to openstack but not changing anything about their contribution methods? Does gophercloud benefit in any way? To me this is essentially the status quo only with new additional governance overhead. Exactly, that is what I was trying to make this point. Having 'new contributors' are very fewer chances nowadays even if we stick to single tooling of more than one. gophercloud in OpenStack or outside, on OpenDev or Github, separate governance project or within any of the OpenInfra projects, none of these factors will make any difference in getting or not getting the "new contributors" in OpenStack That is why I am saying let's not have these 'new contributors' point to decide the gophercloud cloud request to be in different tooling. -gmann > > From artem.goncharov at gmail.com Fri Jun 24 20:36:11 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Fri, 24 Jun 2022 22:36:11 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> Message-ID: > > > > Also, adding a new project like gophercloud won't automatically add > contributors to openstack. Contributors to gophercloud will continue to be > contributors to gophercloud. In fact you point out above it is unlikely > they will contribute to other projects. For this reason I think it is also > worthwhile to think about what benefits of such a change exist. Does > openstack benefit in any way by adding gophercloud to openstack but not > changing anything about their contribution methods? Does gophercloud > benefit in any way? To me this is essentially the status quo only with new > additional governance overhead. > Ugh, if gophercloud is not helping OpenStack we can forget about letting users run k8 on Openstack, Openshift on OpenStack, we can say Ansible and OSC are not helping OpenStack, because this is the same vector of tooling: it's not OpenStack services, but ecosystem around OpenStack. We even had a keynote on that during summit. There are users ready to submit small fixes to the tooling they use daily. But nobody of them is willing to go through our current contribution requirements, they just abandon this and stay dissatisfied. Users are not even able to do easy bug reporting since also for that they need to learn new and very uncomfortable tools. This is not user friendly from my pov. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Fri Jun 24 20:43:41 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Fri, 24 Jun 2022 13:43:41 -0700 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> Message-ID: <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> On Fri, Jun 24, 2022, at 1:36 PM, Artem Goncharov wrote: >> >> >> Also, adding a new project like gophercloud won't automatically add contributors to openstack. Contributors to gophercloud will continue to be contributors to gophercloud. In fact you point out above it is unlikely they will contribute to other projects. For this reason I think it is also worthwhile to think about what benefits of such a change exist. Does openstack benefit in any way by adding gophercloud to openstack but not changing anything about their contribution methods? Does gophercloud benefit in any way? To me this is essentially the status quo only with new additional governance overhead. > > > Ugh, if gophercloud is not helping OpenStack we can forget about > letting users run k8 on Openstack, Openshift on OpenStack, we can say > Ansible and OSC are not helping OpenStack, because this is the same > vector of tooling: it's not OpenStack services, but ecosystem around > OpenStack. We even had a keynote on that during summit. > > There are users ready to submit small fixes to the tooling they use > daily. But nobody of them is willing to go through our current > contribution requirements, they just abandon this and stay > dissatisfied. Users are not even able to do easy bug reporting since > also for that they need to learn new and very uncomfortable tools. This > is not user friendly from my pov. The point I was trying to make was more that nothing stops that contribution from happening today on Github if that is where those individuals prefer to be. What changes if the repo moves under the Github openstack organization and we formally add the project to openstack governance? There are some changes for sure, but most aren't going to be visible/important to those individuals. I'm trying to get us to be explicit about the benefits and cross check them against our goals. I'm not necessarily saying it is a bad idea. It may be that those people want to vote in TC elections and have a greater say in the direction of openstack as a whole. If that is the case let's say that and make sure that benefits like that align with any goals of such a move. I'm not saying that contributions like that don't benefit the ecosystem. I'm specifically trying to understand what benefits to this specific change there are to both parties. It isn't clear to me if part of the move is explicitly ignoring the way openstack has chosen to do things. From gmann at ghanshyammann.com Fri Jun 24 21:58:01 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 24 Jun 2022 16:58:01 -0500 Subject: [all][tc] What's happening in Technical Committee: summary 24 June 2022: Reading: 5 min Message-ID: <18197b8d26a.12163cfb5446812.6672996879110683424@ghanshyammann.com> Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on 23 June. Most of the meeting discussions are summarized in this email. Meeting full logs are available @https://meetings.opendev.org/meetings/tc/2022/tc.2022-06-23-15.00.log.html * Next TC weekly meeting will be on 30 July Thursday at 15:00 UTC, feel free to add the topic on the agenda[1] by 29 July. 2. What we completed this week: ========================= * Defined "Emerging technology and inactive projects" framework[2]. Details can be found here[4]. Thanks to Slawek for working on this framework. A brief about this framework: 1. When a new project applies to be an official OpenStack project and they are not fully ready/doing things as OpenStack way. Still, they can be an official OpenStack with the agreement of filling the gap for OpenStack way. This framework will help such projects to have them early in the OpenStack community but also with the timeline of filling the gaps. (Skyline is a good example of such a project). 2. Inactive projects: If any project become inactive and no maintainer merges the things then this framework will help to detect such projects before the final release of work and give time to recover before we decide to retire them. This will avoid last-minute issues during the final release for example: what we had in the Yoga release[3] * Clarified "ATC" and "AC" terms in TC charter as per the bylaws perspective[5] * Added cinder ibm storwize charm repo[6] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[7], Two are completed and others items are in-progress. Open Reviews ----------------- * Four open reviews for ongoing activities[8]. Create the Environmental Sustainability SIG --------------------------------------------------- There is a proposal for 'Environmental Sustainability ' SIG[9]. Because of its wider scope at the OpenInfra projects level (not just OpenStack) we are discussing if it is a good idea to be an OpenStack SIG or something at the OpenInfra level like a working group under Board or so. New ELK service dashboard: e-r service ----------------------------------------------- Updates from dpawlik: "Changes in ci log processing: created account on docker hub for ci log processing project (credentials are in AWS secret service); enabled pushing logscraper and log gearman container images into the docker hub." Ananya working on integrating the Elastic recheck with Upstream opensearch[10] Dasm working on merging 'origin/rdo' into the merge branch[11] Consistent and Secure Default RBAC ------------------------------------------- We discussed the operator feedback in the policy popup meeting on Tuesday 21 June[12] where operators from KDDI joined and wrote their feedback in etherpad[13]. After considering the feedback and all the challenges we have with 'scope', we decided to drop the 'scope' implementation and work on implementing the 'project persona'. I have proposed the goal document updates[14] 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[15]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[16]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[17][18]. Project updates ------------------- * Add Cinder Dell EMC PowerStore charm[19] * Retire openstack-helm-deployments[20] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[21]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [22] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/839880 [3] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027864.html [4] https://governance.openstack.org/tc/reference/emerging-technology-and-inactive-projects.html [5] https://review.opendev.org/c/openstack/governance/+/844882 [6] https://review.opendev.org/c/openstack/governance/+/846900 [7] https://etherpad.opendev.org/p/tc-zed-tracker [8] https://review.opendev.org/q/projects:openstack/governance+status:open [9] https://review.opendev.org/c/openstack/governance-sigs/+/845336 [10] https://review.opendev.org/c/opendev/elastic-recheck/+/845777 [11] https://review.opendev.org/c/opendev/elastic-recheck/+/847405/ [12] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 [13] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 [14] https://review.opendev.org/c/openstack/governance/+/847418 [15] https://review.opendev.org/c/openstack/governance/+/836888 [16] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.html [17] https://etherpad.opendev.org/p/zuul-config-error-openstack [18] http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028603.html [19] https://review.opendev.org/c/openstack/governance/+/846890 [20] https://review.opendev.org/c/openstack/governance/+/847413 [21] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [22] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann From tkajinam at redhat.com Sat Jun 25 08:05:44 2022 From: tkajinam at redhat.com (Takashi Kajinami) Date: Sat, 25 Jun 2022 17:05:44 +0900 Subject: [vmware-nsx] Missing releases and project status In-Reply-To: References: Message-ID: Because I've not heard any feedback about my previous email, I proposed deprecating puppet-neutron's support for NSX-T plugin. [1] https://review.opendev.org/c/openstack/puppet-neutron/+/847628 On Wed, May 18, 2022 at 1:12 PM Takashi Kajinami wrote: > Hello, > > I recently noticed that the vmware-nsx repo doesn't have the > stable/wallaby branch > and the stable/yoga branch. Also, no release has been created since > victoria release. > > There is a bug fix recently merged directly to stable/xena, and this was > backported > to ussuri and train with victoria skipped. > https://review.opendev.org/c/x/vmware-nsx/+/836034 > I'm quite confused with this because it looks like master and victoria are > no longer maintained. > > May I ask for some clarification about the current status of the project, > especially regarding the following points. > - Is there any plan to create stable/wallaby and stable/yoga ? > - Is there any plan to create a release for stable/wallaby and later ? > - May I know the maintenance status of each branch ? > - What is the future plan of that repo? Will you have stable/zed release ? > > I'm asking this because the plugin is supported by puppet-neutron but > we are thinking of deprecating/removing the implementation if the plugin > will be unmaintained. > > I'd appreciate your input on this. > > Thank you, > Takashi > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.morin at gmail.com Sat Jun 25 12:04:06 2022 From: arnaud.morin at gmail.com (Arnaud) Date: Sat, 25 Jun 2022 14:04:06 +0200 Subject: [neutron] Drivers meeting agenda - 24.06.2022. In-Reply-To: References: Message-ID: <02555A50-DB26-438F-AD0D-EA1BF3E107D8@gmail.com> Hey team, I wasn't able to be on the meeting but I red over the logs. Thanks for taking care of the bugs ;). About the rbac not sharing subnet, I think you agreed on the fact that this can be solved because those networks are seen differently in the neutron code than public shared networks? Cheers, Arnaud Le 23 juin 2022 17:22:18 GMT+02:00, Lajos Katona a ?crit?: >Hi Neutron Drivers, > >The agenda for tomorrow's drivers meeting is at [1]. >* [RFE] Improve performance of bulk port create/delete with >networking-generic-switch (#linkhttps:// >bugs.launchpad.net/neutron/+bug/1976270) >* Neutron RBAC not sharing subnet (#link >https://bugs.launchpad.net/neutron/+bug/1975603) >* [ovn]Floating IP adds distributed attributes (#link >https://bugs.launchpad.net/neutron/+bug/1978039) >* [RFE] Adding the "rekey" parameter to the api for strongswan like >"dpd_action" (#link https://bugs.launchpad.net/neutron/+bug/1979044) > >[1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda > >See you at the meeting tomorrow. >Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From cory at hawkvelt.id.au Sat Jun 25 13:35:46 2022 From: cory at hawkvelt.id.au (Cory Hawkvelt) Date: Sat, 25 Jun 2022 23:05:46 +0930 Subject: [nova] NVLink passthrough Message-ID: Hey team, Is anyone working with NVLink in their clouds? How are you handling passing through the right set of GPU's per NVLink 'group' I have servers with 4 sets of 2 way NVLinks(8 cards in pairs of 2) and I'm able to passthrough the PCI devices to the VM no problem but there is no guarantee that the NVLink pair get passed through together and if we end up with 1 GPU from one pair and 1 GPU from another pair then we run into all sorts of issues(As you'd expect) So I'm looking to understand how Nova can be NVLink aware I guess but I'm struggling to find any conversation ro material on the topic but I assume it's been done before? I did find this talk [1] which mentioned this problem but they write some sort of hack to sit in between nova and qemu, while quite a clever solution it seems like there must be a better way to do this in 2022? [1] - https://www.openstack.org/videos/summits/vancouver-2018/can-we-boost-more-hpc-performance-integrate-ibm-power-servers-with-gpus-to-openstack-environment Cheers, Cory -------------- next part -------------- An HTML attachment was scrubbed... URL: From nsitlani03 at gmail.com Mon Jun 27 05:19:20 2022 From: nsitlani03 at gmail.com (Namrata Sitlani) Date: Mon, 27 Jun 2022 10:49:20 +0530 Subject: [magnum] [xena] [IPv6] Review Patch for dual-stack IPv4/6 work in Magnum-managed Kubernetes 1.21 Message-ID: Hello All, We run release Xena, and we have deployed Kubernetes version 1.21.10 on Magnum with Fedora CoreOS version 34 and we are trying to have IPv6 support for that Kubernetes cluster, as Kubernetes 1.21 and later versions have dual-stack(IPv4/IPv6) support is enabled by default. To achieve that, we also need IPv6 support in magnum, which is not currently supported but there has been a patch for "Support IPv6 dual subnet" in a pending state for almost a year. Following is the patch link: https://review.opendev.org/c/openstack/magnum/+/802235 We are interested in the use case for dual-stack support in magnum. We would highly appreciate it if someone could help us land that patch. Kindly let us know if we can provide any additional information or help for the same. Thanks, Namrata Sitlani -------------- next part -------------- An HTML attachment was scrubbed... URL: From hiromu.asahina.az at hco.ntt.co.jp Mon Jun 27 05:59:52 2022 From: hiromu.asahina.az at hco.ntt.co.jp (Hiromu Asahina) Date: Mon, 27 Jun 2022 14:59:52 +0900 Subject: [heat-translator][tacker] Propose Hiromu Asahina (h-asahina) for heat-translator core team In-Reply-To: <1d4bd8d0-c77e-19e1-a2b7-5c08e69bb01d@gmail.com> References: <0dbea349-5390-6d6e-ee19-bae1128eea43@gmail.com> <1d4bd8d0-c77e-19e1-a2b7-5c08e69bb01d@gmail.com> Message-ID: Thank you for your warm feedback. I'll do my best :) By the way, me and Ueha-san who just joined the heat-translator core before me are not listed on heat-translator drivers [1]. I guess it means we don't have the permissions to control the heat-translator launchpad. Is it possible to add us to that list? Best Regards, Hiromu (h-asahina) [1] https://launchpad.net/~heat-translator-drivers/+members On 2022/06/27 13:42, Yasufumi Ogawa wrote: > > > > -------- Forwarded Message -------- > Subject: Re: [heat-translator][tacker] Propose Hiromu Asahina > (h-asahina) for heat-translator core team > Date: Fri, 24 Jun 2022 15:57:18 +0900 > From: Yasufumi Ogawa > To: Rico Lin > CC: ueha.ayumu at fujitsu.com , > openstack-discuss at lists.openstack.org > , HADDLETON, Robert W (Bob) > , Ghanshyam Mann > > Thanks Rico and all! > > On 2022/06/24 5:03, Rico Lin wrote: >> Hi team >> >> Sorry for the?late reply, just added Hiromu?to the?heat-translator-core. >> >> And welcome Hiromu. >> >> *Rico Lin* >> >> >> On Fri, Jun 24, 2022 at 2:30 AM Yasufumi Ogawa > > wrote: >> >> ??? Hi heat-translator cores, >> >> ??? We tacker team would like to keep contributing to maintain >> ??? heat-translator, so appreciate if you agree to the proposal kindly. >> >> ??? Cheers, >> ??? Yasufumi >> >> ??? On 2022/06/09 14:14, ueha.ayumu at fujitsu.com >> ??? wrote: >> ???? > Hi Rico, >> ???? > >> ???? > This is just a friendly reminder that we are waiting for your >> action. >> ???? > Or please let us know if there are any other necessary processes. >> ???? > Thanks. >> ???? > >> ???? > Best Regards, >> ???? > Ueha >> ???? > >> ???? > -----Original Message----- >> ???? > From: ueha.ayumu at fujitsu.com >> ??? > >> ???? > Sent: Thursday, June 2, 2022 5:39 PM >> ???? > To: openstack-discuss at lists.openstack.org >> ??? ; 'Rico Lin' >> ??? > >> ???? > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina >> ??? (h-asahina) for heat-translator core team >> ???? > >> ???? > Hi Rico, >> ???? > >> ???? > This proposal is great helpful for us! >> ???? > There seems to be no particular objection, so could you add it to >> ??? heat-translator-core if there is no problem? >> ???? > Thanks. >> ???? > >> ???? > Best Regards, >> ???? > Ueha >> ???? > >> ???? > -----Original Message----- >> ???? > From: ueha.ayumu at fujitsu.com >> ??? > >> ???? > Sent: Wednesday, May 25, 2022 1:40 PM >> ???? > To: openstack-discuss at lists.openstack.org >> ??? >> ???? > Subject: RE: [heat-translator][tacker] Propose Hiromu Asahina >> ??? (h-asahina) for heat-translator core team >> ???? > >> ???? > +1 >> ???? > >> ???? > Best Regards, >> ???? > Ueha >> ???? > >> ???? > -----Original Message----- >> ???? > From: Yoshito Ito > ??? > >> ???? > Sent: Wednesday, May 25, 2022 12:31 PM >> ???? > To: openstack-discuss at lists.openstack.org >> ??? >> ???? > Subject: [heat-translator][tacker] Propose Hiromu Asahina >> ??? (h-asahina) for heat-translator core team >> ???? > >> ???? > Hi heat-translator team! >> ???? > >> ???? > I would like to propose Hiromu Asahina (h-asahina) as a member of >> ??? the heat-translator core team. I'm now not able to actively >> ??? contribute to heat-translator, and he can lead the team instead >> of me. >> ???? > >> ???? > He is mainly contributing to Tacker by fixing issues, refactoring >> ??? existing codes, and a lot of helpful reviews, with his experience as >> ??? a research engineer in NTT. Tacker highly depends on heat-translator >> ??? in its main functionality, so I believe he can manage it with his >> ??? knowledge of Tacker. >> ???? > >> ???? > Please vote for his nomination by answering this mail till >> June 1st. >> ???? > >> ???? > Best Regards, >> ???? > >> ???? > Yoshito Ito >> ???? > >> ???? > >> From jean-francois.taltavull at elca.ch Mon Jun 27 06:55:07 2022 From: jean-francois.taltavull at elca.ch (=?utf-8?B?VGFsdGF2dWxsIEplYW4tRnJhbsOnb2lz?=) Date: Mon, 27 Jun 2022 06:55:07 +0000 Subject: [Ceph Rados Gateway] 403 when using S3 client In-Reply-To: <54e6b45a4930418c95f0626ec78c3d1a@elca.ch> References: <8b05dea37bd2412ab15d54ed806b2658@elca.ch> <612721648572028@mail.yandex.ru> <96f86768b73d456ba1937a9177c3831f@elca.ch> <64208f0fd7344c0bab323b66c503f73a@elca.ch> <54e6b45a4930418c95f0626ec78c3d1a@elca.ch> Message-ID: <767298131a584b9f9595fcaaa95c02ce@elca.ch> Hi Dmitriy, In other words, is S3 auth V4 signature handled by default when Rados GW is deployed with OSA or is there a role variable that needs to be set? Kind regards, Jean-Francois From: Taltavull Jean-Fran?ois Sent: jeudi, 16 juin 2022 17:46 To: 'Jonathan Rosser' ; 'Dmitriy Rabotyagov' ; 'openstack-discuss at lists.openstack.org' Subject: RE: [Ceph Rados Gateway] 403 when using S3 client Hi Dmitriy, hi Jonathan, I finally managed to interact with RGW S3 API with ?s3cmd? client, but only if I add the option ?--signature-v2? to the command line. If I don?t, I get the message ?ERROR: S3 error: 403 (AccessDenied)?. The RGW is configured to use keystone as the users authority and it looks like the S3 auth requests including a version 4 signature were not supported. Is there a RGW or a Keystone configuration variable to enable S3 V4 signature ? Deployment characteristics: - OSA 23.2.0 - OpenStack Wallaby - Ceph and RGW Octopus Kind regards, Jean-Francois From: Taltavull Jean-Fran?ois Sent: mercredi, 30 mars 2022 11:01 To: 'Jonathan Rosser' >; openstack-discuss at lists.openstack.org Subject: RE: [Ceph Rados Gateway] 403 when using S3 client Hi Jonathan, The keystone URL is correct. HAProxy has been configured to handle this kind or URL. And everything works fine with the openstack client. From: Jonathan Rosser > Sent: mercredi, 30 mars 2022 10:44 To: openstack-discuss at lists.openstack.org Subject: Re: [Ceph Rados Gateway] 403 when using S3 client EXTERNAL MESSAGE - This email comes from outside ELCA companies. Hi Jean-Francois. I have the following difference to your config: rgw keystone url = http://xx.xx.xx.xx:5000 The normal OSA loadbalancer setup would have the keystone service on port 5000. Jonathan. On 30/03/2022 09:24, Taltavull Jean-Fran?ois wrote: Hi Dmitriy, I just tried with s3cmd but I still get a 403. Here is the rgw section of ceph.conf: rgw_keystone_url = http://xxxxx.xxxx.xxx/identity rgw_keystone_api_version = 3 rgw_keystone_admin_user = radosgw rgw_keystone_admin_password = xxxxxxxxxxxxxxxxxxxxxxxxx rgw_keystone_admin_project = service rgw_keystone_admin_domain = default rgw_keystone_accepted_roles = member, _member_, admin, swiftoperator rgw_keystone_accepted_admin_roles = ResellerAdmin rgw_keystone_implicit_tenants = true rgw_swift_account_in_url = true rgw_swift_versioning_enabled = true rgw_enable_apis = swift,s3 rgw_s3_auth_use_keystone = true From: Dmitriy Rabotyagov Sent: mardi, 29 mars 2022 18:49 To: openstack-discuss Subject: Re: [Ceph Rados Gateway] 403 when using S3 client EXTERNAL MESSAGE - This email comes from outside ELCA companies. - ??? Hi Jean-Francois. It's quite hard to understand what exactly could went wrong based on the information you've provided. Highly likely it's related to the RGW configuration itself and it's integration with keystone to be specific. Would be helpful if you could provide your ceph.conf regarding rgw configuration. I'm also not 100% sure if awscli does work with RGW... At least I always used s3cmd or rclone to interact with RGW S3 API. 29.03.2022, 16:36, "Taltavull Jean-Fran?ois" >: Hi All, I get an http 403 error code when I try to get the bucket list with Ubuntu (Focal) S3 client (awscli). S3 api has been activated in radosgw config file and EC2 credentials have been created and put in S3 client config file. Otherwise, everything is working fine with OpenStack client. My deployment: - OSA 23.2.0 - OpenStack Wallaby - Ceph and Rados GW Octopus Has any of you already experienced this kind of behaviour ? Many thanks, Jean-Francois -- Kind Regards, Dmitriy Rabotyagov -------------- next part -------------- An HTML attachment was scrubbed... URL: From artem.goncharov at gmail.com Mon Jun 27 09:39:42 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Mon, 27 Jun 2022 11:39:42 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> Message-ID: <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> >> > > The point I was trying to make was more that nothing stops that contribution from happening today on Github if that is where those individuals prefer to be. What changes if the repo moves under the Github openstack organization and we formally add the project to openstack governance? There are some changes for sure, but most aren't going to be visible/important to those individuals. > > I'm trying to get us to be explicit about the benefits and cross check them against our goals. I'm not necessarily saying it is a bad idea. It may be that those people want to vote in TC elections and have a greater say in the direction of openstack as a whole. If that is the case let's say that and make sure that benefits like that align with any goals of such a move. > > I'm not saying that contributions like that don't benefit the ecosystem. I'm specifically trying to understand what benefits to this specific change there are to both parties. It isn't clear to me if part of the move is explicitly ignoring the way openstack has chosen to do things. > Let me give here couple of examples: - When I am doing some bigger changes in SDK/OSC/Ansible it is logical to have a devstack running locally to be able to try it out and somehow ensure tests will work without burning CI in endless patchsets due to typos and other type of bugs. Chances that I will be able just to start devstack on my laptop and it immediately works are terribly low. I am not going to go into details why, this is not relevant here. Due to this in most cases, unless I do something much bigger and ready to invest eventually 1-2 hour to get devstack running I skip this and go into the blind flight relying only on CI. And as mentioned earlier producing sometimes lot of patchsets. - My contributions to Zuul are actually facing very similar challenge. I am able to reverse engineer how to be able to run tests locally, but honestly this is not something I really want to do, especially in a fast evolving software. Due to that for fixing one bug I am spending 20 patchsets and weeks of time (everybody here know perfectly how much time we usually wait for CI results and endless rechecks). With that I honestly think twice, do I have enough ?free time? and commitement to try fix the bug properly or I have a workaround I can apply in my installation - I have seen from other teams, but now talking explicitly about myself: using storyboard is so inconvenient that I just stopped using it (neither for reporting myself, nor looking what others report to my projects). I even see often people reusing some external issue trackers just to make it somehow usable (I am waiting so desperately for us finally enabling gitea issues) - Very often when somebody is asking me about particular place in code they (and this is often valid for the referred super contributors) share link to GitHub and not opendev. What I am trying to say with that is that there are convenience factors we should not neglect. Once developer feels not comfortable with the contribution workflow he/she will not do this at all, unless he/she is a long-time committed contributor. Users willing to report issue in OSC/SDK/Ansible area often give up on that after seeing what they need to do in order to at least report the issue. I give up on tracking what users report to the projects I am responsible for in an official way. The ones willing to contribute a small fix give up after seeing their PR is auto closed. This is not about: we are bad. This is about if developer doesn?t feel comfortable in the work we will not win him for us. This is something I learn through pain during last 10 years or so. I am very glad to hear that proper SSO is moving closer. Also awesome to hear we may be close to get issues feature of gitea. Those are the things that should make daily life a tick easier and we may really improve this. Artem From zigo at debian.org Mon Jun 27 11:06:06 2022 From: zigo at debian.org (Thomas Goirand) Date: Mon, 27 Jun 2022 13:06:06 +0200 Subject: [all][horizon][django 4.0] Failure to build (FTBFS) with Django 4.0 In-Reply-To: <20220624142154.m4y7fwcjho7nhbla@yuggoth.org> References: <0bae4082-0960-cdc3-8895-c98ef9c6732c@debian.org> <20220624142154.m4y7fwcjho7nhbla@yuggoth.org> Message-ID: <3bd62161-5d73-d24b-c79d-62f310a5cbcb@debian.org> On 6/24/22 16:21, Jeremy Stanley wrote: > On 2022-06-24 15:00:34 +0200 (+0200), Thomas Goirand wrote: >> A number of RC bugs were filed against my packages in Debian Unstable: >> >> https://bugs.debian.org/1013488 Horizon >> https://bugs.debian.org/1013475 django-pyscss >> https://bugs.debian.org/1013515 ironic-ui >> https://bugs.debian.org/1013605 django-formtools >> https://bugs.debian.org/1013603 enmerkar >> https://bugs.debian.org/1013506 ara >> >> If you have a fix, and can post the link to the patch in the Debian bug >> tracker, that'd be very much appreciated and help these packages not being >> auto-removed from Debian testing. Otherwise, you can always ping me on IRC. > > It looks like global requirements has been getting conservative > Django version cap increases. Most recently, > https://review.opendev.org/813405 raised it from <3.0 to <3.3 in > October. If OpenStack is going to start testing with 4.x versions, a > similar change will need to be proposed so we can see what the > fallout might be upstream. Yeah, the usual stuff... Also as usual: django 4.x upload has been careless and that already broke stuff in Debian Unstable, so I need a fix. Cheers, Thomas Goirand (zigo) From zigo at debian.org Mon Jun 27 11:08:28 2022 From: zigo at debian.org (Thomas Goirand) Date: Mon, 27 Jun 2022 13:08:28 +0200 Subject: [all] Failure to build (FTBFS) with Sphinx 5.0 and docutils 0.18 In-Reply-To: <20220624141620.w3qnnkfiwkgx7z7l@yuggoth.org> References: <6f68aab4-8a3e-93dd-68cc-99a0a9ba0313@debian.org> <20220624141620.w3qnnkfiwkgx7z7l@yuggoth.org> Message-ID: On 6/24/22 16:16, Jeremy Stanley wrote: > The dogpile.cache and repoze.who bugs also look Sphinx-related, > though the OpenStack community doesn't develop those projects so > someone will probably need to reach out to them wherever they > collaborate officially. Kind of... If I'm not mistaking, Dogpile.cache is maintained by Mike Bayer, who's employed by Red Hat to work on OpenStack compat for SQLAlchemy. I'll ask him directly and see if he can help. Cheers, Thomas Goirand (zigo) From ozzzo at yahoo.com Mon Jun 27 12:17:13 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Mon, 27 Jun 2022 12:17:13 +0000 (UTC) Subject: [Swift] [kolla] Swift issues in one cluster In-Reply-To: References: <483887237.5247998.1655487207819.ref@mail.yahoo.com> <483887237.5247998.1655487207819@mail.yahoo.com> <20220617220556.0be3fb91@niphredil.zaitcev.lan> <845416351.6689926.1655818006920@mail.yahoo.com> <918427872.7544677.1655932210049@mail.yahoo.com> <440638798.277721.1655991773368@mail.yahoo.com> <1210414976.7924712.1656002429999@mail.yahoo.com> <1368115593.8368767.1656084274624@mail.yahoo.com> <1926732307.8415136.1656089200951@mail.yahoo.com> Message-ID: <2097098406.9224484.1656332233119@mail.yahoo.com> We were seeing the ChunkWriteTimeout all along but I wasn't focusing on it at first. We're working on getting the customer setup in our lab environment so that he can duplicate the issue there. On Friday, June 24, 2022, 01:07:04 PM EDT, Clay Gerrard wrote: On Fri, Jun 24, 2022 at 11:46 AM Albert Braden wrote: > > "You can reproduce this by issuing a GET request for a few hundred MB file and never consuming the response, but keep the client socket open. Swift will log a 499 but the socket does not always close." Was that the behavior that you were seeing?? Swift logs 499, but the socket stays open? Eventlet 0.22.0 says eventlet.wsgi will timeout idle clients now: https://eventlet.net/doc/changelog.html#id23 ... so maybe that bug as written is no longer valid - but it could still be related to what you're seeing.? Wasn't the original traceback a BlockingIOError - that seems different from a hung client socket from an idle client. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bence.romsics at gmail.com Mon Jun 27 12:27:40 2022 From: bence.romsics at gmail.com (Bence Romsics) Date: Mon, 27 Jun 2022 14:27:40 +0200 Subject: [neutron] bug deputy report for week of 2022-06-20 Message-ID: Hi Neutron Team, Here comes the deputy report for last week: High: * https://bugs.launchpad.net/neutron/+bug/1979047 Interface attach fails with libvirt.libvirtError: internal error: unable to execute QEMU command 'netdev_add': File descriptor named '(null)' has not been found gate failure caused by libvirt bug in centos stream 9, affected tests blacklisted until centos fix arrives: https://review.opendev.org/c/openstack/neutron/+/846905 * https://bugs.launchpad.net/neutron/+bug/1979661 [stable/train] member_batch_update breaks contract with octavia-api interface fix merged: https://review.opendev.org/c/openstack/networking-ovn/+/847255 Medium: * https://bugs.launchpad.net/neutron/+bug/1979643 ``get_ports`` query done in ``check_baremetal_ports_dhcp_options`` cannot filter by VNIC type fix waiting for workflow+1: https://review.opendev.org/c/openstack/neutron/+/847346 Still in discussion with reporter: * https://bugs.launchpad.net/neutron/+bug/1979528 neutron dnsmasq keeps pushing the default gateway * https://bugs.launchpad.net/neutron/+bug/1979089 l3ha router delete race condition if state changes to master Cheers, Bence (rubasov) From rdhasman at redhat.com Mon Jun 27 12:47:24 2022 From: rdhasman at redhat.com (Rajat Dhasmana) Date: Mon, 27 Jun 2022 18:17:24 +0530 Subject: [cinder] Spec Freeze in effect Message-ID: Hello Argonauts, As per the scheduled deadline[1], the cinder spec freeze is in effect. If you would like to continue working on your spec to get it merged and implemented in the Zed development cycle, please request a spec freeze exception as a new email to the Openstack Discuss mailing list[2] describing the reason for not making it into the deadline and hence it will be granted the spec freeze exception. The deadline for proposing the spec freeze exception is by 1st July, 2022 and the specs granted spec freeze exception should be merged by 8th July, 2022. Following are the specs that are targeted for the Zed cycle: 1) Spec to introduce additional task status field Patch: https://review.opendev.org/c/openstack/cinder-specs/+/818551 Needs an update from the author. 2) WIP: Add new volume transaction tracking Patch: https://review.opendev.org/c/openstack/cinder-specs/+/845176 Since this is a WIP, I'm not sure the intention is to get it into Zed specifically. 3) volume list query optimization Patch: https://review.opendev.org/c/openstack/cinder-specs/+/726070 Has a +2 but still lacks review from the main cinder core team. 4) Update original resource's az Patch: https://review.opendev.org/c/openstack/cinder-specs/+/778437 Hasn't been updated since 31st March so looks less likely to make it into Zed. 5) New Quota System Patch: https://review.opendev.org/c/openstack/cinder-specs/+/819693 Needs to be retargeted for Zed and updated as per PTG discussion. Also would like to request the cinder core team to take a look at the open specs. [1] https://releases.openstack.org/zed/schedule.html#z-cinder-spec-freeze [2] openstack-discuss at lists.openstack.org Thanks and regards Rajat Dhasmana -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Mon Jun 27 12:51:12 2022 From: smooney at redhat.com (Sean Mooney) Date: Mon, 27 Jun 2022 13:51:12 +0100 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> Message-ID: On Mon, 2022-06-27 at 11:39 +0200, Artem Goncharov wrote: > > > > > > > The point I was trying to make was more that nothing stops that contribution from happening today on Github if that is where those individuals > > prefer to be. What changes if the repo moves under the Github openstack organization and we formally add the project to openstack governance? > > There are some changes for sure, but most aren't going to be visible/important to those individuals. > > > > I'm trying to get us to be explicit about the benefits and cross check them against our goals. I'm not necessarily saying it is a bad idea. It may > > be that those people want to vote in TC elections and have a greater say in the direction of openstack as a whole. If that is the case let's say > > that and make sure that benefits like that align with any goals of such a move. > > > > I'm not saying that contributions like that don't benefit the ecosystem. I'm specifically trying to understand what benefits to this specific > > change there are to both parties. It isn't clear to me if part of the move is explicitly ignoring the way openstack has chosen to do things. > > > > Let me give here couple of examples: > > - When I am doing some bigger changes in SDK/OSC/Ansible it is logical to have a devstack running locally to be able to try it out and somehow > ensure tests will work without burning CI in endless patchsets due to typos and other type of bugs. Chances that I will be able just to start > devstack on my laptop and it immediately works are terribly low. I am not going to go into details why, this is not relevant here. Due to this in > most cases, unless I do something much bigger and ready to invest eventually 1-2 hour to get devstack running I skip this and go into the blind > flight relying only on CI. And as mentioned earlier producing sometimes lot of patchsets. > - My contributions to Zuul are actually facing very similar challenge. I am able to reverse engineer how to be able to run tests locally, but > honestly this is not something I really want to do, especially in a fast evolving software. Due to that for fixing one bug I am spending 20 > patchsets and weeks of time (everybody here know perfectly how much time we usually wait for CI results and endless rechecks). With that I honestly > think twice, do I have enough ?free time? and commitement to try fix the bug properly or I have a workaround I can apply in my installation > - I have seen from other teams, but now talking explicitly about myself: using storyboard is so inconvenient that I just stopped using it (neither > for reporting myself, nor looking what others report to my projects). I even see often people reusing some external issue trackers just to make it > somehow usable (I am waiting so desperately for us finally enabling gitea issues) i dont like storyboard but i have no issue with using launch pad as it currently stands. it does what we need of it for bug/feature tracking . i dont really object to evaluating gitia issue if they were avaiable but i dont think there is a stong need to change form launch pad in general. its certenly preferable to the mix of bugzilla,jria and trello i also have to deal with but i woudl hope that if we were to use gitia issues we woudl consolidate and not have 3 solution upstream. keepign track of where we keep track of things upstream and downstream is already a full time job so adding more options is an increased burden so i woudl hope we could consolidate. > - Very often when somebody is asking me about particular place in code they (and this is often valid for the referred super contributors) share link > to GitHub and not opendev. > > What I am trying to say with that is that there are convenience factors we should not neglect. Once developer feels not comfortable with the > contribution workflow he/she will not do this at all, unless he/she is a long-time committed contributor. Users willing to report issue in > OSC/SDK/Ansible area often give up on that after seeing what they need to do in order to at least report the issue. I give up on tracking what users > report to the projects I am responsible for in an official way. The ones willing to contribute a small fix give up after seeing their PR is auto > closed. > This is not about: we are bad. This is about if developer doesn?t feel comfortable in the work we will not win him for us. This is something I learn > through pain during last 10 years or so. ther eare some convince factors to github and many inconvenices., it has a vastly inferior code review model that if i was force to use would push me out of the openstack comunity long term. im sure there are others too that feel stongly that moving to a github pull request based workflow woudl be a large regerssion and make them less inclined to continue working on openstack. > > I am very glad to hear that proper SSO is moving closer. Also awesome to hear we may be close to get issues feature of gitea. Those are the things > that should make daily life a tick easier and we may really improve this. > > Artem > From smooney at redhat.com Mon Jun 27 12:56:12 2022 From: smooney at redhat.com (Sean Mooney) Date: Mon, 27 Jun 2022 13:56:12 +0100 Subject: [nova] NVLink passthrough In-Reply-To: References: Message-ID: <75183909a357132622f82b3a6573a3ded9238ca6.camel@redhat.com> On Sat, 2022-06-25 at 23:05 +0930, Cory Hawkvelt wrote: > Hey team, > > Is anyone working with NVLink in their clouds? How are you handling passing > through the right set of GPU's per NVLink 'group' in terms of nova vGPU support we really only supprot passing a singel vGPU to the guest. for pci_passthough we dont currently have nay kind of grouping by nvlink group or pf ectra so that is not somethign that is really supproted today. > > I have servers with 4 sets of 2 way NVLinks(8 cards in pairs of 2) and I'm > able to passthrough the PCI devices to the VM no problem but there is no > guarantee that the NVLink pair get passed through together and if we end up > with 1 GPU from one pair and 1 GPU from another pair then we run into all > sorts of issues(As you'd expect) > > So I'm looking to understand how Nova can be NVLink aware I guess but I'm > struggling to find any conversation ro material on the topic but I assume > it's been done before? today it is not but this could actully be supproted indrectly with teh pci in placment work that is in flight in zed. i say indirectly as that wont actully allow you to model the nvlink groups cleanly but https://specs.openstack.org/openstack/nova-specs/specs/zed/approved/pci-device-tracking-in-placement.html provides a way to list device with traits or cutsom resocue classes. that would allow you to tag the device in teh whitelist with the nvlink group device_spec = { "address: "" "resource_class":"CUSTOM_NVIDIA_A100", "traits": "CUSTOM_NVLINK_GROUP_1" } the pci alias will then be able to request the gpu using the resouce class and trait so you coudl create an alias per group that unfortunetly means we need to have a flaovr per alias to use all the groups so you would need 4 flaovor > > I did find this talk [1] which mentioned this problem but they write some > sort of hack to sit in between nova and qemu, while quite a clever solution > it seems like there must be a better way to do this in 2022? you are the first person im aware of to actully ask for nvlink affinity upstream so no unfortunetly no one as really looked at a better way. if we can discover the groups once we complte modling pci device in plcmane we coudl model the group stuctor there and perhaps extend the alias to allow that to be requested cleanly. basically we woudl model the groups in the resouce provider tree and use the same subtree request parmater to get the pairs of devicecs. this woudl be a similar mechanium that we woudl use for pf affinity or anti affintiy. but no unfortunetly even with master there is not really anything you can do today to enable you use case in nova. the other related feature that comes up with gpu passthough is we do not supprot multifunction device passthough today. e.g. on nvidia gpus the audio and graphics endpoint are two differnt pci subfunciton on a single device. the windows driver expect that and wont work if you pashtough both as they are passed though to the guest as two seperate devices. > > [1] - > https://www.openstack.org/videos/summits/vancouver-2018/can-we-boost-more-hpc-performance-integrate-ibm-power-servers-with-gpus-to-openstack-environment > > Cheers, > Cory From iurygregory at gmail.com Mon Jun 27 13:08:33 2022 From: iurygregory at gmail.com (Iury Gregory) Date: Mon, 27 Jun 2022 10:08:33 -0300 Subject: [ironic] Thoughts on closing some branches Message-ID: Hello Ironicers o/ I would like to gather some feedback before moving forward with the oficial request to close some branches. 1) Closing some bugfix branches I would like to check if we are ok with closing all other bugfix branches except for: *ironic:* bugfix/20.2 | bugfix/19.0 | bugfix/18.1 *ironic-python-agent:* bugfix/8.6 | bugfix/8.3 | bugfix/8.1 *ironic-inspector:* bugfix/10.12 | bugfix/10.9 | bugfix/10.7 These are the branches I know people use, but it's good to double check before removing all other bugfix/*. The reason to close is because it became a bit more complicated to maintain CI working properly in those branches, so if we don't have people using and looking at backports and making CI work I think we can close them. 2) Closing all stable branches pre-train. Thanks! -- *Att[]'s* *Iury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic PTL * *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From Russell.Stather at ignitiongroup.co.za Mon Jun 27 13:12:53 2022 From: Russell.Stather at ignitiongroup.co.za (Russell Stather) Date: Mon, 27 Jun 2022 13:12:53 +0000 Subject: How is this overallocation possible? Message-ID: See image below. How can I be using 38 vcpus on a compute with 24 total? Is this an error in horizon? [cid:988b50f9-b9c8-4ccc-9240-90f2d1ff3df6] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 41377 bytes Desc: image.png URL: From artem.goncharov at gmail.com Mon Jun 27 13:30:56 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Mon, 27 Jun 2022 15:30:56 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> Message-ID: > > ther eare some convince factors to github and many inconvenices., > it has a vastly inferior code review model that if i was force to use would push me out of the openstack comunity long term. > im sure there are others too that feel stongly that moving to a github pull request based workflow woudl be a large regerssion > and make them less inclined to continue working on openstack. The thread is being very explicit about external projects and not the OpenStack itself. -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Mon Jun 27 13:44:39 2022 From: smooney at redhat.com (Sean Mooney) Date: Mon, 27 Jun 2022 14:44:39 +0100 Subject: How is this overallocation possible? In-Reply-To: References: Message-ID: <4d05f846782bcb68afc906616bedb6f8186ac3d6.camel@redhat.com> On Mon, 2022-06-27 at 13:12 +0000, Russell Stather wrote: > See image below. How can I be using 38 vcpus on a compute with 24 total? Is this an error in horizon? no by defualt nova is configured for a 16:1 cpu allocation ratio. https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.initial_cpu_allocation_ratio we normmaly dont recommend seting that over about 4 i have a patch i need to rebase ot change that to 4 https://review.opendev.org/c/openstack/nova/+/830829 so no its not a horizon bug. > > [cid:988b50f9-b9c8-4ccc-9240-90f2d1ff3df6] From smooney at redhat.com Mon Jun 27 13:55:58 2022 From: smooney at redhat.com (Sean Mooney) Date: Mon, 27 Jun 2022 14:55:58 +0100 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> Message-ID: <01581dd3e606105771403abdda0fac7221b21bfa.camel@redhat.com> On Mon, 2022-06-27 at 15:30 +0200, Artem Goncharov wrote: > > > > ther eare some convince factors to github and many inconvenices., > > it has a vastly inferior code review model that if i was force to use would push me out of the openstack comunity long term. > > im sure there are others too that feel stongly that moving to a github pull request based workflow woudl be a large regerssion > > and make them less inclined to continue working on openstack. > > The thread is being very explicit about external projects and not the OpenStack itself. yep but that is unhelpful. if any external project that work with openstack want to become part of openstack under the foundatiosn governace it is nolonger external. so if gophercloud was to become part of openstack it would not be external and if it wanted to you github pull requests for it workflow it woudl be deviating form the other openstack projects. external project that are not part of openstack governacne can use any tooling they like. if we start allowing arbiatry internal and external project to use gerrit or github workflows of worse both concurrently we will start getting request to supprot that for other proejct like nova neutron ectra. i woudl see that as damaging to the exsting colaberator based and something to be avoided if we can. im not really sure what gophercloud want to achive by being part of openstack without adopting the openstack ways of doing things that they cant acive by bing a nice go sdk for openstack on there own with the well wishes and or support of the openstack comunity. the 4 opens are a core part of the culture of openstack simiarly the ways of workign with irc/gerrit/zuul/ptgs are also a part of the openstack way. i am wondering why gophercloud want to actully becoem an offial proejct if they dont want to adopt the open developement workflow (note i did not say model) that openstack uses? > From katonalala at gmail.com Mon Jun 27 13:57:20 2022 From: katonalala at gmail.com (Lajos Katona) Date: Mon, 27 Jun 2022 15:57:20 +0200 Subject: [neutron] Drivers meeting agenda - 24.06.2022. In-Reply-To: <02555A50-DB26-438F-AD0D-EA1BF3E107D8@gmail.com> References: <02555A50-DB26-438F-AD0D-EA1BF3E107D8@gmail.com> Message-ID: Hi, No problem, I think we covered this topic and had a good discussion on the subnet RBAC, topic. the agreement was that issue is really something that can be solved and we go with the review of the patch, which is not perfect currently, but this is why we have review process: https://review.opendev.org/c/openstack/neutron/+/843871/1/neutron/db/l3_db.py#861 Lajos Arnaud ezt ?rta (id?pont: 2022. j?n. 25., Szo, 14:04): > Hey team, > I wasn't able to be on the meeting but I red over the logs. > Thanks for taking care of the bugs ;). > > About the rbac not sharing subnet, I think you agreed on the fact that > this can be solved because those networks are seen differently in the > neutron code than public shared networks? > > Cheers, > Arnaud > > Le 23 juin 2022 17:22:18 GMT+02:00, Lajos Katona a > ?crit : >> >> Hi Neutron Drivers, >> >> The agenda for tomorrow's drivers meeting is at [1]. >> * [RFE] Improve performance of bulk port create/delete with >> networking-generic-switch (#linkhttps:// >> bugs.launchpad.net/neutron/+bug/1976270) >> * Neutron RBAC not sharing subnet (#link >> https://bugs.launchpad.net/neutron/+bug/1975603) >> * [ovn]Floating IP adds distributed attributes (#link >> https://bugs.launchpad.net/neutron/+bug/1978039) >> * [RFE] Adding the "rekey" parameter to the api for strongswan like >> "dpd_action" (#link https://bugs.launchpad.net/neutron/+bug/1979044) >> >> [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda >> >> See you at the meeting tomorrow. >> Lajos Katona (lajoskatona) >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Mon Jun 27 14:04:43 2022 From: katonalala at gmail.com (Lajos Katona) Date: Mon, 27 Jun 2022 16:04:43 +0200 Subject: [Neutron] team meeting - 28.06.2022. - cancelled Message-ID: Hi, As I said in the last Team meeting (see [1]) I am OoO this Tuesday, so can't drive the coming meeting. [1]: https://meetings.opendev.org/meetings/networking/2022/networking.2022-06-21-14.01.log.html#l-24 Best wishes, and see you online. Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Mon Jun 27 13:50:36 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Mon, 27 Jun 2022 15:50:36 +0200 Subject: How is this overallocation possible? In-Reply-To: References: Message-ID: If I recall correctly, default CPU allocation ratio is 16. Which means you can multiply CPU cores by 16 and it will be fine. Please check https://docs.openstack.org/nova/yoga/admin/scheduling.html#allocation-ratios for more details on that. Horizon just don't know what ratio is set so shows just CPU cores of compute. ??, 27 ???. 2022 ?., 15:31 Russell Stather < Russell.Stather at ignitiongroup.co.za>: > See image below. How can I be using 38 vcpus on a compute with 24 total? > Is this an error in horizon? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 41377 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 41377 bytes Desc: not available URL: From smooney at redhat.com Mon Jun 27 14:24:23 2022 From: smooney at redhat.com (Sean Mooney) Date: Mon, 27 Jun 2022 15:24:23 +0100 Subject: How is this overallocation possible? In-Reply-To: References: Message-ID: On Mon, 2022-06-27 at 15:50 +0200, Dmitriy Rabotyagov wrote: > If I recall correctly, default CPU allocation ratio is 16. Which means you > can multiply CPU cores by 16 and it will be fine. > Please check > https://docs.openstack.org/nova/yoga/admin/scheduling.html#allocation-ratios > for more details on that. > > Horizon just don't know what ratio is set so shows just CPU cores of > compute. ya there has been a long runnign feature request for horizon to deprecate this interface and replace it with one based on placement resouce providers going forward. by long running i mean like 3+ year before placment was split out of nova. so i really dont know if that is even on the horizon projects radar to go fix anymore but hopfully that will get fixed eventually. the allocation ratios resveation amoutns and other info is aviabel in placment it also woudl allow horizon to see other resouce like vgpus, mdevs, pinned vs unpinned cpus and in the futrue pci devices. its a rather large feature but horizon should more or less consider the hypervior api deprecated. we have not deprecatedd all parts of the api but in general we would prefer if horizon did not use it and instead uses placment. https://specs.openstack.org/openstack/nova-specs/specs/wallaby/implemented/modernize-os-hypervisors-api.html covers the most recent change to the hypervior api in wallaby but the desire to move away form using it is much older then that. > > ??, 27 ???. 2022 ?., 15:31 Russell Stather < > Russell.Stather at ignitiongroup.co.za>: > > > See image below. How can I be using 38 vcpus on a compute with 24 total? > > Is this an error in horizon? > > > > > > From radoslaw.piliszek at gmail.com Mon Jun 27 15:21:08 2022 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Mon, 27 Jun 2022 17:21:08 +0200 Subject: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: On Wed, 8 Jun 2022 at 01:19, Andy Botting wrote: > > Hi Rados?aw, Hi Andy, Sorry for the late reply, been busy vacationing and then dealing with COVID-19. > > First of all, wow, that looks very interesting and in fact very much > > what I'm looking for. As I mentioned in the original message, the > > things this solution lacks are not something blocking for me. > > Regarding the approach to Guacamole, I know that it's preferable to > > have guacamole extension (that provides the dynamic inventory) > > developed rather than meddle with the internal database but I guess it > > is a good start. > > An even better approach would be something like the Guacozy project > (https://guacozy.readthedocs.io) I am not convinced. The project looks dead by now. [1] It offers a different UI which may appeal to certain users but I think sticking to vanilla Guacamole should do us right... For the time being at least. ;-) > They were able to use the Guacmole JavaScript libraries directly to > embed the HTML5 desktop within a React? app. I think this is a much > better approach, and I'd love to be able to do something similar in > the future. Would make the integration that much nicer. Well, as an example of embedding in the UI - sure. But it does not invalidate the need to modify Guacamole's database or write an extension to it so that it has the necessary creds. > > > > Any "quickstart setting up" would be awesome to have at this stage. As > > this is a Django app, I think I should be able to figure out the bits > > and bolts to get it up and running in some shape but obviously it will > > impede wider adoption. > > Yeah I agree. I'm in the process of documenting it, so I'll aim to get > a quickstart guide together. > > I have a private repo with code to set up a development environment > which uses Heat and Ansible - this might be the quickest way to get > started. I'm happy to share this with you privately if you like. I'm interested. Please share it. > > On the note of adoption, if I find it usable, I can provide support > > for it in Kolla [1] and help grow the project's adoption this way. > > Kolla could be useful. We're already using containers for this project > now, and I have a helm chart for deploying to k8s. > https://github.com/NeCTAR-RC/bumblebee-helm Nice! The catch is obviously that some orgs frown upon K8s because they lack the necessary know-how. Kolla by design avoids the use of K8s. OpenStack components are not cloud-native anyway so benefits of using K8s are diminished (yet it makes sense to use K8s if there is enough experience with it as it makes certain ops more streamlined and simpler this way). > Also, an important part is making sure the images are set up correctly > with XRDP, etc. Our images are built using Packer, and the config for > them can be found at https://github.com/NeCTAR-RC/bumblebee-images Ack, thanks for sharing. > > Also, since this is OpenStack-centric, maybe you could consider > > migrating to OpenDev at some point to collaborate with interested > > parties using a common system? > > Just food for thought at the moment. > > I think it would be more appropriate to start a new project. I think > our codebase has too many assumptions about the underlying cloud. > > We inherited the code from another project too, so it's got twice the cruft. I see. Well, that's good to know at least. > > Writing to let you know I have also found the following related paper: [1] > > and reached out to its authors in the hope to enable further > > collaboration to happen. > > The paper is not open access so I have only obtained it for myself and > > am unsure if licensing permits me to share, thus I also asked the > > authors to share their copy (that they have copyrights to). > > I have obviously let them know of the existence of this thread. ;-) > > Let's stay tuned. > > > > [1] https://link.springer.com/chapter/10.1007/978-3-030-99191-3_12 > > This looks interesting. A collaboration would be good if there is > enough interest in the community. I am looking forward to the collaboration happening. This could really liven up the OpenStack VDI. [1] https://github.com/paidem/guacozy/ -yoctozepto From arnaud.morin at gmail.com Mon Jun 27 15:56:21 2022 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Mon, 27 Jun 2022 15:56:21 +0000 Subject: [large-scale][cinder] backend_native_threads_pool_size option with rbd backend Message-ID: Hey all, Is there any recommendation on the number of threads to use when using RBD backend (option backend_native_threads_pool_size)? The doc is saying that 20 is the default but it should be increased, specially for the RBD driver, but up to which value? Is there anyone tuning this parameter in their openstack deployments? If yes, maybe we can add some recommendations on openstack large-scale doc about it? Cheers, Arnaud. From papathanail at uom.edu.gr Mon Jun 27 16:11:43 2022 From: papathanail at uom.edu.gr (GEORGIOS PAPATHANAIL) Date: Mon, 27 Jun 2022 19:11:43 +0300 Subject: Openstack instance is unreachable Message-ID: I am facing the following issue with my new Openstack installation. The installation is a little bit weird and to elaborate more, I have a controller and a compute node running as VMs in XenServer. The compute node has nested virtualization enabled and uses qemu in order to provision VMs. I have a provider flat network that has a public range of IPs and VMs use this network. I am able to create instances and access them via console, but I can't ping them or ssh even if they have public IPs. I am not very familiar with Xen Server, is there any configuration that is needed (bridges, promisc mode, etc)? -- *George Papathanail* *Associate Researcher* *Department of Applied Informatics* *University of Macedonia* -------------- next part -------------- An HTML attachment was scrubbed... URL: From JRACE at augusta.edu Mon Jun 27 17:09:08 2022 From: JRACE at augusta.edu (Race, Jonathan) Date: Mon, 27 Jun 2022 17:09:08 +0000 Subject: [EXTERNAL] Re: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? In-Reply-To: References: Message-ID: We leverage guacamole within our codebase. Granted we do not have it tied explicitly into the various function of OpenStack. We use it currently as a service that resides on our controller and only have it tied into our galera cluster for db. We then leverage a separate tool for creating the users and connections to the various environments. We also published a pypi package 'guacamole-api-wrapper' for Interacting with guacamole's api. https://gitlab.com/gacybercenter/gacyberrange/projects/kinetic https://gitlab.com/gacybercenter/gacyberrange/projects/guacamole-api-wrapper https://pypi.org/project/guacamole-api-wrapper/ Jonathan Race Cyber Range Engineer | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane?|?Augusta, Georgia?|?30901 (c) 762-716-7733 www.gacyberrange.org -----Original Message----- From: Rados?aw Piliszek Sent: Monday, June 27, 2022 11:21 AM To: Andy Botting Cc: openstack-discuss Subject: [EXTERNAL] Re: [vdi][daas][ops] What are your solutions to VDI/DaaS on OpenStack? CAUTION: EXTERNAL SENDER This email originated from an external source. Please exercise caution before opening attachments, clicking links, replying, or providing information to the sender. If you believe it to be fraudulent, contact the AU Cybersecurity Hotline at 72-CYBER (2-9237 / 706-722-9237) or 72CYBER at augusta.edu. ________________________________ On Wed, 8 Jun 2022 at 01:19, Andy Botting wrote: > > Hi Rados?aw, Hi Andy, Sorry for the late reply, been busy vacationing and then dealing with COVID-19. > > First of all, wow, that looks very interesting and in fact very much > > what I'm looking for. As I mentioned in the original message, the > > things this solution lacks are not something blocking for me. > > Regarding the approach to Guacamole, I know that it's preferable to > > have guacamole extension (that provides the dynamic inventory) > > developed rather than meddle with the internal database but I guess > > it is a good start. > > An even better approach would be something like the Guacozy project > (https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgua > cozy.readthedocs.io%2F&data=05%7C01%7Cjrace%40augusta.edu%7C0bae57 > 2ce5504767b36b08da5850be84%7C8783ac6bd05b4292b483e65f1fdfee91%7C0%7C0% > 7C637919401065166186%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQI > joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=86 > N0GNnBVmPvZBgef3y4ZN2bxbRV4d0tKGuqmBxh6XA%3D&reserved=0) I am not convinced. The project looks dead by now. [1] It offers a different UI which may appeal to certain users but I think sticking to vanilla Guacamole should do us right... For the time being at least. ;-) > They were able to use the Guacmole JavaScript libraries directly to > embed the HTML5 desktop within a React? app. I think this is a much > better approach, and I'd love to be able to do something similar in > the future. Would make the integration that much nicer. Well, as an example of embedding in the UI - sure. But it does not invalidate the need to modify Guacamole's database or write an extension to it so that it has the necessary creds. > > > > Any "quickstart setting up" would be awesome to have at this stage. > > As this is a Django app, I think I should be able to figure out the > > bits and bolts to get it up and running in some shape but obviously > > it will impede wider adoption. > > Yeah I agree. I'm in the process of documenting it, so I'll aim to get > a quickstart guide together. > > I have a private repo with code to set up a development environment > which uses Heat and Ansible - this might be the quickest way to get > started. I'm happy to share this with you privately if you like. I'm interested. Please share it. > > On the note of adoption, if I find it usable, I can provide support > > for it in Kolla [1] and help grow the project's adoption this way. > > Kolla could be useful. We're already using containers for this project > now, and I have a helm chart for deploying to k8s. > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > ub.com%2FNeCTAR-RC%2Fbumblebee-helm&data=05%7C01%7Cjrace%40augusta > .edu%7C0bae572ce5504767b36b08da5850be84%7C8783ac6bd05b4292b483e65f1fdf > ee91%7C0%7C0%7C637919401065166186%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w > LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C > &sdata=ylD9k2jVOE9O3SPnT%2BCVmsDgUsJnJgXa0D%2Fu9v2lnqQ%3D&rese > rved=0 Nice! The catch is obviously that some orgs frown upon K8s because they lack the necessary know-how. Kolla by design avoids the use of K8s. OpenStack components are not cloud-native anyway so benefits of using K8s are diminished (yet it makes sense to use K8s if there is enough experience with it as it makes certain ops more streamlined and simpler this way). > Also, an important part is making sure the images are set up correctly > with XRDP, etc. Our images are built using Packer, and the config for > them can be found at > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > ub.com%2FNeCTAR-RC%2Fbumblebee-images&data=05%7C01%7Cjrace%40augus > ta.edu%7C0bae572ce5504767b36b08da5850be84%7C8783ac6bd05b4292b483e65f1f > dfee91%7C0%7C0%7C637919401065166186%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC > 4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C% > 7C&sdata=Zuyd0TDTxEKwojCN67pZe5iyH5jcQx4UHzS14eKLNiA%3D&reserv > ed=0 Ack, thanks for sharing. > > Also, since this is OpenStack-centric, maybe you could consider > > migrating to OpenDev at some point to collaborate with interested > > parties using a common system? > > Just food for thought at the moment. > > I think it would be more appropriate to start a new project. I think > our codebase has too many assumptions about the underlying cloud. > > We inherited the code from another project too, so it's got twice the cruft. I see. Well, that's good to know at least. > > Writing to let you know I have also found the following related > > paper: [1] and reached out to its authors in the hope to enable > > further collaboration to happen. > > The paper is not open access so I have only obtained it for myself > > and am unsure if licensing permits me to share, thus I also asked > > the authors to share their copy (that they have copyrights to). > > I have obviously let them know of the existence of this thread. ;-) > > Let's stay tuned. > > > > [1] > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli > > nk.springer.com%2Fchapter%2F10.1007%2F978-3-030-99191-3_12&data= > > 05%7C01%7Cjrace%40augusta.edu%7C0bae572ce5504767b36b08da5850be84%7C8 > > 783ac6bd05b4292b483e65f1fdfee91%7C0%7C0%7C637919401065166186%7CUnkno > > wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWw > > iLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MIKr%2BsAl4D3V5cdASEJZ26h2 > > a6mKxugaRi1rdS7MSzc%3D&reserved=0 > > This looks interesting. A collaboration would be good if there is > enough interest in the community. I am looking forward to the collaboration happening. This could really liven up the OpenStack VDI. [1] https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpaidem%2Fguacozy%2F&data=05%7C01%7Cjrace%40augusta.edu%7C0bae572ce5504767b36b08da5850be84%7C8783ac6bd05b4292b483e65f1fdfee91%7C0%7C0%7C637919401065166186%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Wr0XGRY86LmYCgYM5TVo3oLso9mB7BOJiYTkk7PaYeY%3D&reserved=0 -yoctozepto From alsotoes at gmail.com Mon Jun 27 18:18:44 2022 From: alsotoes at gmail.com (Alvaro Soto) Date: Mon, 27 Jun 2022 13:18:44 -0500 Subject: is gyan project dead? Message-ID: The last commit was like 4 years old [1] and it seems like all links are broken except for this one. [2] there is any other project that took the functionality? [1] - https://opendev.org/x/gyan [2] - https://wiki.openstack.org/wiki/Gyan Cheers! -- Alvaro Soto Escobar *Note: My work hours may not be your work hours. Please do not feel the need to respond during a time that is not convenient for you.* ---------------------------------------------------------- Great people talk about ideas, ordinary people talk about things, small people talk... about other people. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Russell.Stather at ignitiongroup.co.za Mon Jun 27 18:46:08 2022 From: Russell.Stather at ignitiongroup.co.za (Russell Stather) Date: Mon, 27 Jun 2022 18:46:08 +0000 Subject: How is this overallocation possible? In-Reply-To: <4d05f846782bcb68afc906616bedb6f8186ac3d6.camel@redhat.com> References: <4d05f846782bcb68afc906616bedb6f8186ac3d6.camel@redhat.com> Message-ID: Thank you understood. ________________________________ From: Sean Mooney Sent: 27 June 2022 15:44 To: Russell Stather ; openstack-discuss at lists.openstack.org Subject: Re: How is this overallocation possible? On Mon, 2022-06-27 at 13:12 +0000, Russell Stather wrote: > See image below. How can I be using 38 vcpus on a compute with 24 total? Is this an error in horizon? no by defualt nova is configured for a 16:1 cpu allocation ratio. https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.initial_cpu_allocation_ratio we normmaly dont recommend seting that over about 4 i have a patch i need to rebase ot change that to 4 https://review.opendev.org/c/openstack/nova/+/830829 so no its not a horizon bug. > > [cid:988b50f9-b9c8-4ccc-9240-90f2d1ff3df6] -------------- next part -------------- An HTML attachment was scrubbed... URL: From Russell.Stather at ignitiongroup.co.za Mon Jun 27 18:46:24 2022 From: Russell.Stather at ignitiongroup.co.za (Russell Stather) Date: Mon, 27 Jun 2022 18:46:24 +0000 Subject: How is this overallocation possible? In-Reply-To: References: Message-ID: Thank you , got it. ________________________________ From: Dmitriy Rabotyagov Sent: 27 June 2022 15:50 Cc: openstack-discuss Subject: Re: How is this overallocation possible? If I recall correctly, default CPU allocation ratio is 16. Which means you can multiply CPU cores by 16 and it will be fine. Please check https://docs.openstack.org/nova/yoga/admin/scheduling.html#allocation-ratios for more details on that. Horizon just don't know what ratio is set so shows just CPU cores of compute. ??, 27 ???. 2022 ?., 15:31 Russell Stather >: See image below. How can I be using 38 vcpus on a compute with 24 total? Is this an error in horizon? [cid:988b50f9-b9c8-4ccc-9240-90f2d1ff3df6] -------------- next part -------------- An HTML attachment was scrubbed... URL: From franck.vedel at univ-grenoble-alpes.fr Mon Jun 27 18:51:06 2022 From: franck.vedel at univ-grenoble-alpes.fr (Franck VEDEL) Date: Mon, 27 Jun 2022 20:51:06 +0200 Subject: [kolla-ansible][wallaby][nova][spice] Message-ID: <7D07BF3B-CDD1-49E8-9947-0F0FC45440D3@univ-grenoble-alpes.fr> hello I have two problems and I ask you for a little help before doing stupid things. If I correctly diagnosed the first problem, which is to replace novnc by spice in my installation, it is not possible. I'm using Centos, I'm on the Wallaby version, and the last tag for the Docker image is Train. So my first question is: can I somehow replace novnc with something else, spice or another? Second question: I would like to switch from Wallaby to Xena (unless we can switch from Wallaby to Yoga... is it possible?) . After a year of operation, this is the first time I have had to do an update with an openstack in production (student instances have been deleted, end of the academic year) but not used for a few days. Is the following procedure correct: git clone --branch stable/xena https://opendev.org/openstack/kolla-ansible mv kolla-ansible kolla-ansible-xena (to keep multiple versions) pip install ./kolla-ansible-xena kolla-ansible -i multinode bootstrap-servers kolla-ansible -i multinode prechecks kolla-ansible -i multinode deploy What to do with globals.yml and the /etc/kolla/config overload? no changes ? Thank you in advance for your help, for your answers, and for an update procedure if you have one. Sincerely Franck From ozzzo at yahoo.com Mon Jun 27 18:55:09 2022 From: ozzzo at yahoo.com (Albert Braden) Date: Mon, 27 Jun 2022 18:55:09 +0000 (UTC) Subject: [kolla] [nova] Duplicated compute service IDs in Openstack client References: <62407475.9479920.1656356109448.ref@mail.yahoo.com> Message-ID: <62407475.9479920.1656356109448@mail.yahoo.com> We're running kolla-ansible Train. We were trying to delete some compute services today and we got an error: $ openstack compute service delete 54 Failed to delete compute service with ID '54': Service id 54 refers to multiple services. (HTTP 400) (Request-ID: req-3bc400a4-367e-41dc-85f8-fa9011cc96cc) 1 of 1 compute services failed to delete. When we used "openstack compute service list" we saw that ID 54 was assigned to a nova-compute service on a hypervisor, and to a nova-scheduler service on a controller. We worked around it by using "nova service-list" to get the UUID-style ID and then "nova service-delete" After that I sorted by ID with " --sort-column ID" and found that some active services have duplicate IDs: | 33 | nova-compute |-compute504. | AZ22 | enabled | up | 2022-06-27T18:22:47.000000 | | 33 | nova-scheduler | -ctrl1. | internal | enabled | up | 2022-06-27T18:22:43.000000 | Is this a bug in the Openstack client? From gmann at ghanshyammann.com Mon Jun 27 19:17:47 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 27 Jun 2022 14:17:47 -0500 Subject: is gyan project dead? In-Reply-To: References: Message-ID: <181a6993686.dac24219132387.8640956625503082214@ghanshyammann.com> ---- On Mon, 27 Jun 2022 13:18:44 -0500 Alvaro Soto wrote --- > The last commit was like 4 years old [1] and it seems like all links are broken except for this one. [2] there is any other project that took the functionality? > [1] - https://opendev.org/x/gyan[2] - https://wiki.openstack.org/wiki/Gyan I think yes, I can see the major of the contribution from bharath who is not active in upstream nowadays. Also, this is not an official OpenStack project and not sure if there was any plan for that. -gmann > > Cheers!-- > > Alvaro Soto Escobar > > Note: My work hours may not be your work hours. Please do not feel the need to respond during a time that is not convenient for you. > ---------------------------------------------------------- > Great people talk about ideas, > ordinary people talk about things, > small people talk... about other people. From gmann at ghanshyammann.com Mon Jun 27 19:53:49 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 27 Jun 2022 14:53:49 -0500 Subject: [all][tc] Technical Committee next weekly meeting on 30 June 2022 at 1500 UTC Message-ID: <181a6ba323f.104aa718f133256.6630300845350952590@ghanshyammann.com> Hello Everyone, The technical Committee's next weekly meeting is scheduled for 30 June 2022, at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, 29 June at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann From midhunlaln66 at gmail.com Tue Jun 28 06:12:18 2022 From: midhunlaln66 at gmail.com (Midhunlal Nb) Date: Tue, 28 Jun 2022 11:42:18 +0530 Subject: cinder-volume showing down Message-ID: Hi team, I am using the openstack rocky version and everything was working fine,but today i am getting some errors while creating new instances and instances not creating.I checked services and it is showing down --------+ | Binary | Host | Zone | Status | State | Updated At | +------------------+--------------+------+---------+-------+-------------------- --------+ | cinder-scheduler | controller | nova | enabled | up | 2022-06-28T05:18:38 .000000 | | cinder-volume | storage1 at lvm | nova | enabled | down | 2022-06-28T05:02:20 .000000 | ---> The cinder log showing below error 2022-06-28 10:37:51.149 1629 ERROR cinder.service [-] Manager for service cinder-volume storage1 at lvm is reporting problems, not sending heartbeat. Service will appear "down". Please help Thanks & Regards Midhunlal N B -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.aminian.server at gmail.com Tue Jun 28 06:51:14 2022 From: p.aminian.server at gmail.com (Parsa Aminian) Date: Tue, 28 Jun 2022 11:21:14 +0430 Subject: token Message-ID: hello Is it possible to generate a token for users in openstack ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdhasman at redhat.com Tue Jun 28 07:13:51 2022 From: rdhasman at redhat.com (Rajat Dhasmana) Date: Tue, 28 Jun 2022 12:43:51 +0530 Subject: [cinder] This week's meeting will be in video+IRC Message-ID: Hello Argonauts, This week's meeting will be held in video + IRC mode with details as follows: Date: 29th June, 2022 Time: 1400 UTC Meeting link: https://bluejeans.com/556681290 IRC Channel: #openstack-meeting-alt Make sure you're connected to both the bluejeans meeting and IRC since we do roll call and also summarize the discussion points on IRC. Thanks and regards Rajat Dhasmana -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdhasman at redhat.com Tue Jun 28 07:19:11 2022 From: rdhasman at redhat.com (Rajat Dhasmana) Date: Tue, 28 Jun 2022 12:49:11 +0530 Subject: cinder-volume showing down In-Reply-To: References: Message-ID: Hi Midhunlal, On Tue, Jun 28, 2022 at 11:54 AM Midhunlal Nb wrote: > Hi team, > I am using the openstack rocky version and everything was working fine,but > today i am getting some errors while creating new instances and instances > not creating.I checked services and it is showing down > > --------+ > | Binary | Host | Zone | Status | State | Updated At > | > +------------------+--------------+------+---------+-------+-------------------- > --------+ > | cinder-scheduler | controller | nova | enabled | up | > 2022-06-28T05:18:38 .000000 | > | cinder-volume | storage1 at lvm | nova | enabled | down | > 2022-06-28T05:02:20 .000000 | > > ---> The cinder log showing below error > 2022-06-28 10:37:51.149 1629 ERROR cinder.service [-] Manager for service > cinder-volume storage1 at lvm is reporting problems, not sending heartbeat. > Service will appear "down". > Please help > > The cinder-volume service could be down for a variety of reasons depending on the backend. This error is generic for every backend and doesn't tell us anything about why the backend is down. Since it is lvm, I would suggest you to check if proper physical volumes (pvdisplay) and volume groups (vgs) are created. > > Thanks & Regards > Midhunlal N B > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Tue Jun 28 07:22:41 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 28 Jun 2022 09:22:41 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> Message-ID: <3069979.mvXUDI8C0e@p1> Hi, Dnia czwartek, 23 czerwca 2022 22:02:57 CEST Kendall Nelson pisze: > On Thu, Jun 23, 2022 at 12:55 PM Jeremy Stanley wrote: > > > On 2022-06-23 12:13:50 -0500 (-0500), Ghanshyam Mann wrote: > > > ---- On Thu, 23 Jun 2022 10:30:24 -0500 Jeremy Stanley < > > fungi at yuggoth.org> wrote ---- > > > > On 2022-06-23 10:02:14 -0500 (-0500), Ghanshyam Mann wrote: > > > > [...] > > > > > Yes, both are separate things and I think we are mixing both or at > > > > > least if we have such impression or governance is not so clear > > > > > about it then we should fix it. I replied in another reply about > > > > > governance point of view and IMO yes we should allow such new > > > > > projects hosted on new tooling or so but they need to make sure > > > > > all the help on CI/CD, release etc are taken care by them self or > > > > > they help opendev team to support such things. If either of them > > > > > cannot be done and they do not fulfill the PTI or any other new > > > > > project requirement criteria then they cannot be in OpenStack. > > > > [...] > > > > > > > > "Governance" (the new project requirements document) right now > > > > clearly states that new projects need to perform their code review > > > > and gating tests on the "OpenStack Infrastructure" (the former name > > > > for the OpenDev Collaboratory because that document hasn't been > > > > updated to reflect the new name). You'll at a minimum need a vote of > > > > the TC to remove those restrictions, so all this assumes that the > > > > rest of the TC agrees with you that doing code review in GitHub with > > > > a separate GitHub-connected CI system is allowable for new official > > > > OpenStack project teams and deliverables. > > > > > > > > This is not "governance point of view" it's *your* point of view, so > > > > please be clear that the decision is one the TC as a whole will need > > > > to make. > > > > > > I think there is some misunderstanding here. I have never said > > > anywhere that this is "TC agreed view" off-course this is my > > > opinion as a community member as well as TC member. > > > > > > Any community member or TC members can provide their opinion but > > > that should not be considered as "TC agreed plan" until that is > > > explicitly mentioned in email or TC pass the resolution. We can > > > have different views from TC members or chair but any of that > > > should not be considered as "TC agreement" unless mentioned. I > > > think this is how every email discussion is. > > > > You referred to it above as the "governance point of view" so I just > > wanted to make certain you don't actually believe the governing > > documents are unclear on this particular point, and understand that > > OpenStack absolutely will need TC consensus on lifting a > > longstanding restriction in order to allow an official deliverable > > to be hosted outside the "OpenStack Infrastructure" (a.k.a. OpenDev > > Collaboratory). > > > > As a TC member, I do not think that any project that is a part of OpenStack > should be hosted outside OpenDev. To be a part of OpenStack, you have to > be hosted in OpenDev. I think that splitting to use Github + OpenDev will > create too much complexity for new contributors in addition to everything > else > already mentioned about CI and release management further up in this > thread. I second that. It reminds me the Storyboard and Launchpad thing, where some projects are using one and others the other one bug tracker so new people don't even exactly know where they should report bug against particular project. If we will have some projects hosted on opendev and using Gerrit and others hosted on Github only and using Github's workflow, it will be IMO similar mess. > > > > > I have this in my list to give a clear picture from TC as an > > > agreed plan: > > > > > > Step1: Continue the discussion in ML (here) > > > Step2: After having a good amount of feedback here and we still > > > not resolved the things, I will add this topic to TC > > > meeting and get the TC consensus. > > > Step3: Propose Governance resolution or documentation update > > > Step4: Update the same in ML as "TC agreed plan". > > > > Thanks, this looks like a reasonable way forward. > > > > Documents should definitely get updated! +2! > > -- > > Jeremy Stanley > > > > -Kendall Nelson > -- Slawek Kaplonski Principal Software Engineer Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From felix.huettner at mail.schwarz Tue Jun 28 07:25:07 2022 From: felix.huettner at mail.schwarz (=?utf-8?B?RmVsaXggSMO8dHRuZXI=?=) Date: Tue, 28 Jun 2022 07:25:07 +0000 Subject: cinder-volume showing down In-Reply-To: References: Message-ID: Hi Midhunlal, You will most of the time find the actual error earlier in the log. The line you see here is just the regular update that the service is still down. If I need to troubleshoot such issues I always restart cinder-volume and take a look at the logs it produces during startup. Most of the time you can find a clue there. Best Regards Felix H?ttner From: Midhunlal Nb Sent: Tuesday, June 28, 2022 8:12 AM To: openstack-discuss Subject: cinder-volume showing down Hi team, I am using the openstack rocky version and everything was working fine,but today i am getting some errors while creating new instances and instances not creating.I checked services and it is showing down --------+ | Binary | Host | Zone | Status | State | Updated At | +------------------+--------------+------+---------+-------+-------------------- --------+ | cinder-scheduler | controller | nova | enabled | up | 2022-06-28T05:18:38 .000000 | | cinder-volume | storage1 at lvm | nova | enabled | down | 2022-06-28T05:02:20 .000000 | ---> The cinder log showing below error 2022-06-28 10:37:51.149 1629 ERROR cinder.service [-] Manager for service cinder-volume storage1 at lvm is reporting problems, not sending heartbeat. Service will appear "down". Please help Thanks & Regards Midhunlal N B Diese E Mail enth?lt m?glicherweise vertrauliche Inhalte und ist nur f?r die Verwertung durch den vorgesehenen Empf?nger bestimmt. Sollten Sie nicht der vorgesehene Empf?nger sein, setzen Sie den Absender bitte unverz?glich in Kenntnis und l?schen diese E Mail. Hinweise zum Datenschutz finden Sie hier. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christian.rohmann at inovex.de Tue Jun 28 07:48:54 2022 From: christian.rohmann at inovex.de (Christian Rohmann) Date: Tue, 28 Jun 2022 09:48:54 +0200 Subject: Nova not updating to new size of an extended in-use / attached cinder volume (Ceph RBD) to guest In-Reply-To: <93533ed1417037539e651ba50e7b721afb1e4bdd.camel@redhat.com> References: <37cf15be-48d6-fdbc-a003-7117bda10dbc@inovex.de> <20210215111834.7nw3bdqsccoik2ss@lyarwood-laptop.usersys.redhat.com> <23b84a12-91a7-b739-d88d-bbc630bd9d5f@inovex.de> <5f0d1404f3bb1774918288912a98195f1d48f361.camel@redhat.com> <66b67bb4d7494601a87436bdc1d7b00b@binero.com> <48679cb7-e7c1-4d19-2831-720bfa9ca5af@inovex.de> <529b5d86-fc37-dc2d-7dd2-aabffdb0d945@inovex.de> <93533ed1417037539e651ba50e7b721afb1e4bdd.camel@redhat.com> Message-ID: <49f28d1c-5930-40cf-d340-f0b8b17e08f7@inovex.de> Hey Sean, On 06/05/2021 18:29, Sean Mooney wrote: > that woudl make sense give the externa event api is admin only and only inteed to be use by services > so the fix would be for cidner to use an admin credtial not the user one to send the event to nova. Thanks, yes and that can just be achieved by configuring one which is then used for such calls. But instead of a fully privileged "admin" user there rather should exist a proper RBAC role to only allow one service (cinder in this case) to do what it required to function (e.g. send events to Nova) and not just "everything for every other service". This first of all violates the least privilege principle, but in an ecosystem that made up of individual projects of varying security qualities and which are highly distributed it's just a bad idea to give every component and their dog the keys to the kindom. There was a forum on exactly that issue at the Summit and how that is one aspect of the RBAC , see the etherpad: https://etherpad.opendev.org/p/deprivilization-of-service-accounts Regards Christian From pierre at stackhpc.com Tue Jun 28 08:03:46 2022 From: pierre at stackhpc.com (Pierre Riteau) Date: Tue, 28 Jun 2022 10:03:46 +0200 Subject: [kolla] [nova] Duplicated compute service IDs in Openstack client In-Reply-To: <62407475.9479920.1656356109448@mail.yahoo.com> References: <62407475.9479920.1656356109448.ref@mail.yahoo.com> <62407475.9479920.1656356109448@mail.yahoo.com> Message-ID: Hello Albert, You are seeing conflicting IDs because these services belong to different Nova cells and are in different databases: your scheduler services are in cell0 while your compute hosts are probably in a default cell. This is why, starting from Nova API 2.53 [1], compute services are identified by UUID instead of database IDs. This is not really a bug with the OpenStack client, but a known issue that it uses the 2.1 API by default, with all its limitations. You can specify a newer API version like this: openstack --os-compute-api-version 2.53 compute service list openstack --os-compute-api-version 2.53 compute service delete [1] https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-pike On Mon, 27 Jun 2022 at 21:02, Albert Braden wrote: > We're running kolla-ansible Train. We were trying to delete some compute > services today and we got an error: > > $ openstack compute service delete 54 > Failed to delete compute service with ID '54': Service id 54 refers to > multiple services. (HTTP 400) (Request-ID: > req-3bc400a4-367e-41dc-85f8-fa9011cc96cc) > 1 of 1 compute services failed to delete. > > When we used "openstack compute service list" we saw that ID 54 was > assigned to a nova-compute service on a hypervisor, and to a nova-scheduler > service on a controller. We worked around it by using "nova service-list" > to get the UUID-style ID and then "nova service-delete" > > After that I sorted by ID with " --sort-column ID" and found that some > active services have duplicate IDs: > > | 33 | nova-compute |-compute504. | AZ22 | enabled | up | > 2022-06-27T18:22:47.000000 | > | 33 | nova-scheduler | -ctrl1. | internal | enabled | up > | 2022-06-27T18:22:43.000000 | > > Is this a bug in the Openstack client? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From elfosardo at gmail.com Tue Jun 28 08:27:42 2022 From: elfosardo at gmail.com (Riccardo Pittau) Date: Tue, 28 Jun 2022 10:27:42 +0200 Subject: [ironic] Thoughts on closing some branches In-Reply-To: References: Message-ID: All good for both points, thanks Iury! Ciao, Riccardo On Mon, Jun 27, 2022 at 3:19 PM Iury Gregory wrote: > Hello Ironicers o/ > > I would like to gather some feedback before moving forward with the > oficial request to close some branches. > > 1) Closing some bugfix branches > I would like to check if we are ok with closing all other bugfix branches > except for: > *ironic:* bugfix/20.2 | bugfix/19.0 | bugfix/18.1 > *ironic-python-agent:* bugfix/8.6 | bugfix/8.3 | bugfix/8.1 > *ironic-inspector:* bugfix/10.12 | bugfix/10.9 | bugfix/10.7 > These are the branches I know people use, but it's good to double check > before removing all other bugfix/*. > > The reason to close is because it became a bit more complicated to > maintain CI working properly in those branches, so if we don't have people > using and looking at backports and making CI work I think we can close them. > > 2) Closing all stable branches pre-train. > > Thanks! > > -- > *Att[]'s* > > *Iury Gregory Melo Ferreira * > *MSc in Computer Science at UFCG* > *Ironic PTL * > *Senior Software Engineer at Red Hat Brazil* > *Social*: https://www.linkedin.com/in/iurygregory > *E-mail: iurygregory at gmail.com * > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Tue Jun 28 08:42:10 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 28 Jun 2022 09:42:10 +0100 Subject: [kolla] [nova] Duplicated compute service IDs in Openstack client In-Reply-To: References: <62407475.9479920.1656356109448.ref@mail.yahoo.com> <62407475.9479920.1656356109448@mail.yahoo.com> Message-ID: <58005c7979d66e0334d0fa370e4c512b1fbf3e40.camel@redhat.com> On Tue, 2022-06-28 at 10:03 +0200, Pierre Riteau wrote: > Hello Albert, > > You are seeing conflicting IDs because these services belong to different > Nova cells and are in different databases: your scheduler services are in > cell0 while your compute hosts are probably in a default cell. > This is why, starting from Nova API 2.53 [1], compute services are > identified by UUID instead of database IDs. > > This is not really a bug with the OpenStack client, but a known issue that > it uses the 2.1 API by default, with all its limitations. You can specify a > newer API version like this: > > openstack --os-compute-api-version 2.53 compute service list > > openstack --os-compute-api-version 2.53 compute service delete > > [1] > https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-pike yep exactly its not a bug in nova or the client its not even really a know issue its how it was intended to work. the old nova clint help test used to call out the duplicates for multi cell envs usage: nova service-delete Delete the service by integer ID. If deleting a nova-compute service, be sure to stop the actual nova- compute process on the physical host before deleting the service with this command. Failing to do so can lead to the running service re-creating orphaned compute_nodes table records in the database. (Supported by API versions '2.0' - '2.latest') Positional arguments: ID of service as an integer. Note that this may not uniquely identify a service in a multi-cell deployment. the openstack client (which you shoudl use) does not call that out but its equally true. openstack --os-compute-api-version 2.1 help service delete usage: openstack service delete [-h] [ ...] Delete service(s) positional arguments: Service(s) to delete (type, name or ID) optional arguments: -h, --help show this help message and exit you should prefer the uuid and its requried in a cellv2 deployument in most casess. > > On Mon, 27 Jun 2022 at 21:02, Albert Braden wrote: > > > We're running kolla-ansible Train. We were trying to delete some compute > > services today and we got an error: > > > > $ openstack compute service delete 54 > > Failed to delete compute service with ID '54': Service id 54 refers to > > multiple services. (HTTP 400) (Request-ID: > > req-3bc400a4-367e-41dc-85f8-fa9011cc96cc) > > 1 of 1 compute services failed to delete. > > > > When we used "openstack compute service list" we saw that ID 54 was > > assigned to a nova-compute service on a hypervisor, and to a nova-scheduler > > service on a controller. We worked around it by using "nova service-list" > > to get the UUID-style ID and then "nova service-delete" > > > > After that I sorted by ID with " --sort-column ID" and found that some > > active services have duplicate IDs: > > > > > 33 | nova-compute |-compute504. | AZ22 | enabled | up | > > 2022-06-27T18:22:47.000000 | > > > 33 | nova-scheduler | -ctrl1. | internal | enabled | up > > > 2022-06-27T18:22:43.000000 | > > > > Is this a bug in the Openstack client? > > > > From Arne.Wiebalck at cern.ch Tue Jun 28 08:44:45 2022 From: Arne.Wiebalck at cern.ch (Arne.Wiebalck at cern.ch) Date: Tue, 28 Jun 2022 10:44:45 +0200 Subject: [ironic] Thoughts on closing some branches In-Reply-To: References: Message-ID: <607653579.24852.1656405885985@cernmail.cern.ch> Sounds good to me as well. Thanks, Iury! > On 06/28/2022 10:27 AM Riccardo Pittau wrote: > > > > > All good for both points, thanks Iury! > > > > Ciao, > Riccardo > > > On Mon, Jun 27, 2022 at 3:19 PM Iury Gregory > wrote: > > > > Hello Ironicers o/ > > > > > > I would like to gather some feedback before moving forward with the oficial request to close some branches. > > > > > > 1) Closing some bugfix branches > > I would like to check if we are ok with closing all other bugfix branches except for: > > ironic: bugfix/20.2 | bugfix/19.0 | bugfix/18.1 > > ironic-python-agent: bugfix/8.6 | bugfix/8.3 | bugfix/8.1 > > > > ironic-inspector: bugfix/10.12 | bugfix/10.9 | bugfix/10.7 > > > > > > > > These are the branches I know people use, but it's good to double check before removing all other bugfix/*. > > > > > > The reason to close is because it became a bit more complicated to maintain CI working properly in those branches, so if we don't have people using and looking at backports and making CI work I think we can close them. > > > > > > > > 2) Closing all stable branches pre-train. > > > > > > Thanks! > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Att[]'s > > Iury Gregory Melo Ferreira > > MSc in Computer Science at UFCG > > > > Ironic PTL > > Senior Software Engineer at Red Hat Brazil > > Social: https://www.linkedin.com/in/iurygregory > > E-mail: iurygregory at gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Tue Jun 28 09:01:23 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 28 Jun 2022 10:01:23 +0100 Subject: Nova not updating to new size of an extended in-use / attached cinder volume (Ceph RBD) to guest In-Reply-To: <49f28d1c-5930-40cf-d340-f0b8b17e08f7@inovex.de> References: <37cf15be-48d6-fdbc-a003-7117bda10dbc@inovex.de> <20210215111834.7nw3bdqsccoik2ss@lyarwood-laptop.usersys.redhat.com> <23b84a12-91a7-b739-d88d-bbc630bd9d5f@inovex.de> <5f0d1404f3bb1774918288912a98195f1d48f361.camel@redhat.com> <66b67bb4d7494601a87436bdc1d7b00b@binero.com> <48679cb7-e7c1-4d19-2831-720bfa9ca5af@inovex.de> <529b5d86-fc37-dc2d-7dd2-aabffdb0d945@inovex.de> <93533ed1417037539e651ba50e7b721afb1e4bdd.camel@redhat.com> <49f28d1c-5930-40cf-d340-f0b8b17e08f7@inovex.de> Message-ID: On Tue, 2022-06-28 at 09:48 +0200, Christian Rohmann wrote: > Hey Sean, > > > On 06/05/2021 18:29, Sean Mooney wrote: > > that woudl make sense give the externa event api is admin only and only inteed to be use by services > > so the fix would be for cidner to use an admin credtial not the user one to send the event to nova. > > Thanks, yes and that can just be achieved by configuring one which is > then used for such calls. > > But instead of a fully privileged "admin" user there rather should exist > a proper RBAC role to only allow one service (cinder in this case) to do > what it required to function (e.g. send events to Nova) and not just > "everything for every other service". This first of all violates the > least privilege principle, but in an ecosystem that made up of > individual projects of varying security qualities and which are highly > distributed it's just a bad idea to give every component and their dog > the keys to the kindom. that is what the service role is inteded to be as i mentioned before. the admin user was what was agreed to use before. the service user will not have admin permisisons it will only be used for service to service comunication for api operations that need higher permision then a normal user but not full adming the external event api is one such interface that even normal operators are not intented to ever call its purly a service to service api. just being facnk we are entirly aware that this violates the principal of least privlaage but you are missign the context that indivigual servies are not ment to creat arbitary roles. we are ment to use standard roles and the service role which will be intoduced in phase 2 of the rbac cross project goal https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#phase-2 is the standard role we agreed to use for that usecase. in prior iterations of the secure rbac work prior to the reset in direction in yoga we had the concetp of system scoped tokens being scoped to a specifc system so system=all vs system=compute vs system=networking that was dropped as far as i can tell when we reset the direction today we can create role and assign them to group or users directly currently we have no way to model that a user shoudl have this role only on a speciifc service endpoint. i think to adress your use case we need that abltiy to say the neutron user has the service role on the nova endpoint but not on any other endpoint. that will enable your reduction of pivaldages but it not currently planned in our 3 phased adoption fo secure rbac at this time. > > There was a forum on exactly that issue at the Summit and how that is > one aspect of the RBAC , see the etherpad: > https://etherpad.opendev.org/p/deprivilization-of-service-accounts yes im aware but jsut to be frack repating this without taking on bored the feedback i had previously given that 1 this si a know gap that we are intentially not adressing in the scope fo the current project goal 2 that the service role is planned to be create to adress marking an endpoint as admin when it does not need to be is not helpful. i was not in berlin and dont want to discurage you from expressing your opipion on the list and engagian with the comunity to improve openstack securtiy. the way to do that productivly is to get inovled with the secure rbac work and to either work with others to develop the keystoen feature required to alow ues to assocate role grant to service endpoints or to come up with another solution that can then be added too the cross project goal. unless we do a hard pivort away form standardised roles approch and adopt oath/api-key style permissions model used by github or games like eve onlien for decades where you cate a key tha thas sepcific permision on speicifc api endpoiing tna then use that api key in to make requests i dotn see another way to adress this cleanly via the api. https://docs.github.com/en/rest/overview/permissions-required-for-github-apps to achive something like ^ in an openstack context we would need to develop oslo midelway or similar that coudl read the policy.yaml file used by each service and expose the roles and access requriement for all api endpoint within the serve at a well know url. e.g. nova.cloud.com/policy_rules then keystone would have to aggreate that and allow you to create an applciation credital or similar that was delagated a subset or permissions on the indivigual serce endpoint. you would then configure the service to use the applciation credideial instead. (note applications credential are the closest thign we have to an api key or githubs application based permissions model bug they are ment to have roles deleaged to them not per api end point permssiosn so its not a direct 1:1 mapping) that would be a very differnt workflow form what we have today. if you are willing to sign up to do some of the work then im sure you can help drive the direction of this since you seam interested. in any case without creating per project roles today which is not somethign we wanted to do previosly we need a feature in keystone to do better then just the service role. but with the service roles no service will need admin if we implemeent thigns correctly. > > > Regards > > > Christian > > From midhunlaln66 at gmail.com Tue Jun 28 09:42:09 2022 From: midhunlaln66 at gmail.com (Midhunlal Nb) Date: Tue, 28 Jun 2022 15:12:09 +0530 Subject: cinder-volume showing down In-Reply-To: References: Message-ID: Hi, When I restart cinder-volume then it came up after few minutes again going down On Tue, Jun 28, 2022, 12:55 PM Felix H?ttner wrote: > Hi Midhunlal, > > > > You will most of the time find the actual error earlier in the log. The > line you see here is just the regular update that the service is still down. > > If I need to troubleshoot such issues I always restart cinder-volume and > take a look at the logs it produces during startup. > Most of the time you can find a clue there. > > > > Best Regards > > *Felix H?ttner* > > > > *From:* Midhunlal Nb > *Sent:* Tuesday, June 28, 2022 8:12 AM > *To:* openstack-discuss > *Subject:* cinder-volume showing down > > > > Hi team, > I am using the openstack rocky version and everything was working fine,but > today i am getting some errors while creating new instances and instances > not creating.I checked services and it is showing down > > --------+ > | Binary | Host | Zone | Status | State | Updated At > | > +------------------+--------------+------+---------+-------+-------------------- > --------+ > | cinder-scheduler | controller | nova | enabled | up | > 2022-06-28T05:18:38 .000000 | > | cinder-volume | storage1 at lvm | nova | enabled | down | > 2022-06-28T05:02:20 .000000 | > > > > ---> The cinder log showing below error > > 2022-06-28 10:37:51.149 1629 ERROR cinder.service [-] Manager for service > cinder-volume storage1 at lvm is reporting problems, not sending heartbeat. > Service will appear "down". > > Please help > > > Thanks & Regards > Midhunlal N B > > Diese E Mail enth?lt m?glicherweise vertrauliche Inhalte und ist nur f?r > die Verwertung durch den vorgesehenen Empf?nger bestimmt. Sollten Sie nicht > der vorgesehene Empf?nger sein, setzen Sie den Absender bitte unverz?glich > in Kenntnis und l?schen diese E Mail. Hinweise zum Datenschutz finden Sie > hier . > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eblock at nde.ag Tue Jun 28 10:07:40 2022 From: eblock at nde.ag (Eugen Block) Date: Tue, 28 Jun 2022 10:07:40 +0000 Subject: cinder-volume showing down In-Reply-To: References: Message-ID: <20220628100740.Horde.FcUpqU0oqTzrsvNk3aYvViC@webmail.nde.ag> If you shared more details (e.g. add some cinder logs) others might be able to help understand the issue. Have you checked disk space on the cinder-volume host? Could the LVM be full? Check syslog (and paste relevant information here) for any further hints, otherwise it's just wild guessing. Zitat von Midhunlal Nb : > Hi, > When I restart cinder-volume then it came up after few minutes again going > down > > On Tue, Jun 28, 2022, 12:55 PM Felix H?ttner > wrote: > >> Hi Midhunlal, >> >> >> >> You will most of the time find the actual error earlier in the log. The >> line you see here is just the regular update that the service is still down. >> >> If I need to troubleshoot such issues I always restart cinder-volume and >> take a look at the logs it produces during startup. >> Most of the time you can find a clue there. >> >> >> >> Best Regards >> >> *Felix H?ttner* >> >> >> >> *From:* Midhunlal Nb >> *Sent:* Tuesday, June 28, 2022 8:12 AM >> *To:* openstack-discuss >> *Subject:* cinder-volume showing down >> >> >> >> Hi team, >> I am using the openstack rocky version and everything was working fine,but >> today i am getting some errors while creating new instances and instances >> not creating.I checked services and it is showing down >> >> --------+ >> | Binary | Host | Zone | Status | State | Updated At >> | >> +------------------+--------------+------+---------+-------+-------------------- >> --------+ >> | cinder-scheduler | controller | nova | enabled | up | >> 2022-06-28T05:18:38 .000000 | >> | cinder-volume | storage1 at lvm | nova | enabled | down | >> 2022-06-28T05:02:20 .000000 | >> >> >> >> ---> The cinder log showing below error >> >> 2022-06-28 10:37:51.149 1629 ERROR cinder.service [-] Manager for service >> cinder-volume storage1 at lvm is reporting problems, not sending heartbeat. >> Service will appear "down". >> >> Please help >> >> >> Thanks & Regards >> Midhunlal N B >> >> Diese E Mail enth?lt m?glicherweise vertrauliche Inhalte und ist nur f?r >> die Verwertung durch den vorgesehenen Empf?nger bestimmt. Sollten Sie nicht >> der vorgesehene Empf?nger sein, setzen Sie den Absender bitte unverz?glich >> in Kenntnis und l?schen diese E Mail. Hinweise zum Datenschutz finden Sie >> hier . >> From eblock at nde.ag Tue Jun 28 10:57:23 2022 From: eblock at nde.ag (Eugen Block) Date: Tue, 28 Jun 2022 10:57:23 +0000 Subject: Openstack instance is unreachable In-Reply-To: Message-ID: <20220628105723.Horde.VEeg85NrLzpcrDTFq6t70GG@webmail.nde.ag> Hi, this is one of the most common issues users post report in this list. Did you try to find a solution? Do your instances in the flat network get an IP address? If not, try to launch instances with "--config-drive true". If the instances do get an IP, do your security-group rules allow access? Zitat von GEORGIOS PAPATHANAIL : > I am facing the following issue with my new Openstack installation. The > installation is a little bit weird and to elaborate more, I have a > controller and a compute node running as VMs in XenServer. > > The compute node has nested virtualization enabled and uses qemu in order > to provision VMs. I have a provider flat network that has a public range of > IPs and VMs use this network. > > I am able to create instances and access them via console, but I can't ping > them or ssh even if they have public IPs. > > I am not very familiar with Xen Server, is there any configuration that is > needed (bridges, promisc mode, etc)? > > -- > *George Papathanail* > *Associate Researcher* > *Department of Applied Informatics* > *University of Macedonia* From papathanail at uom.edu.gr Tue Jun 28 11:08:50 2022 From: papathanail at uom.edu.gr (GEORGIOS PAPATHANAIL) Date: Tue, 28 Jun 2022 14:08:50 +0300 Subject: Openstack instance is unreachable In-Reply-To: <20220628105723.Horde.VEeg85NrLzpcrDTFq6t70GG@webmail.nde.ag> References: <20220628105723.Horde.VEeg85NrLzpcrDTFq6t70GG@webmail.nde.ag> Message-ID: Hi, My instances are on a flat network. They get public IP address. The security group is configured correctly. I did not launch an instance with ?config-drive-true. ???? ??? 28 ???? 2022 ???? 14:04 ? ??????? Eugen Block ??????: > Hi, > > this is one of the most common issues users post report in this list. > Did you try to find a solution? Do your instances in the flat network > get an IP address? If not, try to launch instances with > "--config-drive true". If the instances do get an IP, do your > security-group rules allow access? > > > Zitat von GEORGIOS PAPATHANAIL : > > > I am facing the following issue with my new Openstack installation. The > > installation is a little bit weird and to elaborate more, I have a > > controller and a compute node running as VMs in XenServer. > > > > The compute node has nested virtualization enabled and uses qemu in order > > to provision VMs. I have a provider flat network that has a public range > of > > IPs and VMs use this network. > > > > I am able to create instances and access them via console, but I can't > ping > > them or ssh even if they have public IPs. > > > > I am not very familiar with Xen Server, is there any configuration that > is > > needed (bridges, promisc mode, etc)? > > > > -- > > *George Papathanail* > > *Associate Researcher* > > *Department of Applied Informatics* > > *University of Macedonia* > > > > > -- *George Papathanail* *Associate Researcher* *Department of Applied Informatics* *University of Macedonia* -------------- next part -------------- An HTML attachment was scrubbed... URL: From adivya1.singh at gmail.com Tue Jun 28 12:42:51 2022 From: adivya1.singh at gmail.com (Adivya Singh) Date: Tue, 28 Jun 2022 18:12:51 +0530 Subject: Regarding Floating IP is existing Setup In-Reply-To: References: <20220621074234.Horde.MUtmd-n0LSJ3fRW9xbZ4EVk@webmail.nde.ag> <1816246.GmBfMWsNUm@p1> Message-ID: hi Slawek, it happens with a given router namespace at a time Regards Adivya Singh On Fri, Jun 24, 2022 at 10:13 PM Adivya Singh wrote: > Hi, > > Thanks for the advice and the link, > > What i saw when i do testing using tcpdump was "ARP" was not working, and > it is not able to associate the FLoating IP with the MAC address of the > interface in the VM, When i do the associate and disassociate the VM , it > works fine > > But the Router NameSpace got changed. > > Regards > Adivya Singh > > On Thu, Jun 23, 2022 at 1:22 PM Slawek Kaplonski > wrote: > >> Hi, >> >> Dnia wtorek, 21 czerwca 2022 13:55:51 CEST Adivya Singh pisze: >> > hi Eugen, >> > >> > The current setup is 3 controller nodes, The Load is not much on each >> > controller and the number of DHCP agent is always set to 2 as per the >> > standard in the neutron.conf, The L3 agent seems to be stables as other >> > router namespace works fine under it, Only few router Namespace get >> > affected under the agent. >> >> Is it that problem happens for new floating IPs or for the FIPs which >> were working fine and then suddenly stopped working? If the latter, was >> there any action which triggered the issue to happen? >> Is there e.g. only one FIP broken in the router or maybe when it happens, >> then all FIPs which uses same router are broken? >> >> Can You also try to analyze with e.g. tcpdump where traffic is dropped >> exactly? You can check >> http://kaplonski.pl/blog/neutron-where-is-my-packet-2/ for some more >> detailed description how traffic should go from the external network to >> Your instance. >> >> > >> > Most of the template having issue , Have all instance having FLoating >> IP, a >> > Stack with a single floating IP have chance of issue very less >> > >> > Regards >> > Adivya Singh >> > >> > On Tue, Jun 21, 2022 at 1:18 PM Eugen Block wrote: >> > >> > > Hi, >> > > >> > > this sounds very familiar to me, I had to deal with something similar >> > > a couple of times in a heavily used cluster with 2 control nodes. What >> > > does your setup look like, is it a HA setup? I would start checking >> > > the DHCP and L3 agents. After increasing dhcp_agents_per_network to 2 >> > > in neutron.conf and restarting the services this didn't occur again >> > > (yet). This would impact floating IPs as well, sometimes I had to >> > > disable and enable the affected router(s). If you only have one >> > > control node a different approach is necessary. Do you see a high load >> > > on the control node? >> > > >> > > >> > > Zitat von Adivya Singh : >> > > >> > > > hi Team, >> > > > >> > > > We got a issue in Xena release, where we set the environment in >> Ubuntu >> > > > Platform, But later we get some issues in Floating IP not reachable. >> > > > >> > > > In a Network node, not all router namespace are Impacted and only >> few of >> > > > them get affected, So we can not comment Network node has a issue. >> > > > >> > > > The L3 agent where the Router is tied up, Worked just fine, as other >> > > > routers work Fine. >> > > > >> > > > and the one having issue in Floating IP, if i unassigned and >> assigned it >> > > > starts working most of the time. >> > > > >> > > > Any thoughts on this >> > > > >> > > > Regards >> > > > Adivya Singh >> > > >> > > >> > > >> > > >> > > >> > >> >> >> -- >> Slawek Kaplonski >> Principal Software Engineer >> Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hclareth7 at gmail.com Mon Jun 27 18:54:59 2022 From: hclareth7 at gmail.com (Hernando Ariza Perez) Date: Mon, 27 Jun 2022 13:54:59 -0500 Subject: Issues With trove guest agent Message-ID: Dear Trove community, My name is Hernando, I?m writing this email because I spend a lot of time trying to make trove yoga version works, the service functionality seems that is working, however the guest agent not, I built an image following the build process in the documentation ( https://docs.openstack.org/trove/ussuri/admin/building_guest_images.html ), also I used the image that you have in http://tarballs.openstack.org/trove/images/ So right now, when I create a data store instance, the instances is active normally and I can reach it, I go inside it, and I saw that the guest agent service doesn?t pull the datastore docker container image. I put the guest agent config file manually in the image, because trove task manager never put it via cloud unit, it only put the guest info conf. So this case is weird because I?m didn?t get any log error, that?s why I?m here. Well could you please provide me some guides about this process? Maybe an example of the new trove config files, I?ll really appreciate this help. Thanks for read, Regards Hernando Clareth -------------- next part -------------- An HTML attachment was scrubbed... URL: From swogatpradhan22 at gmail.com Tue Jun 28 08:39:57 2022 From: swogatpradhan22 at gmail.com (Swogat Pradhan) Date: Tue, 28 Jun 2022 14:09:57 +0530 Subject: Openstack keystone LDAP integration | openstack user list --domain domain.com | Internal server error (HTTP 500) Message-ID: Description of problem: I am trying to integrate AD server in keystone and facing 'Internal server error' domain configuration: [stack at hkg2director ~]$ cat workplace/keystone_domain_specific_ldap_backend.yaml # This is an example template on how to configure keystone domain specific LDAP # backends. This will configure a domain called tripleoldap will the attributes # specified. parameter_defaults: KeystoneLDAPDomainEnable: true KeystoneLDAPBackendConfigs: domain.com: url: ldap://172.25.161.211 user: cn=Openstack,ou=Admins,dc=domain,dc=com password: password suffix: dc=domain,dc=com user_tree_dn: ou=APAC,dc=domain,dc=com user_filter: "(|(memberOf=cn=openstackadmin,ou=Groups,dc=domain,dc=com)(memberOf=cn=openstackeditor,ou=Groups,dc=domain,dc=com)(memberOf=cn=openstackviewer,ou=Groups,dc=domain,dc=com)" user_objectclass: person user_id_attribute: cn group_tree_dn: ou=Groups,dc=domain,dc=com group_objectclass: Groups group_id_attribute: cn When i issue the command: $ openstack user list --domain domain.com Output: Internal server error (HTTP 500) Keystone_wsgi_error.log: [Tue Jun 28 06:46:49.112848 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] mod_wsgi (pid=45): Exception occurred processing WSGI script '/var/www/cgi-bin/keystone/keystone'. [Tue Jun 28 06:46:49.121797 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] Traceback (most recent call last): [Tue Jun 28 06:46:49.122202 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 2464, in __call__ [Tue Jun 28 06:46:49.122218 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.wsgi_app(environ, start_response) [Tue Jun 28 06:46:49.122231 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/werkzeug/middleware/proxy_fix.py", line 187, in __call__ [Tue Jun 28 06:46:49.122238 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.app(environ, start_response) [Tue Jun 28 06:46:49.122248 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ [Tue Jun 28 06:46:49.122254 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) [Tue Jun 28 06:46:49.122264 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func [Tue Jun 28 06:46:49.122270 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.func(req, *args, **kwargs) [Tue Jun 28 06:46:49.122284 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/oslo_middleware/base.py", line 124, in __call__ [Tue Jun 28 06:46:49.122294 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] response = req.get_response(self.application) [Tue Jun 28 06:46:49.122304 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send [Tue Jun 28 06:46:49.122310 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] application, catch_exc_info=False) [Tue Jun 28 06:46:49.122320 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in call_application [Tue Jun 28 06:46:49.122326 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] app_iter = application(self.environ, start_response) [Tue Jun 28 06:46:49.122337 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 143, in __call__ [Tue Jun 28 06:46:49.122344 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return resp(environ, start_response) [Tue Jun 28 06:46:49.122354 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ [Tue Jun 28 06:46:49.122364 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) [Tue Jun 28 06:46:49.122374 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func [Tue Jun 28 06:46:49.122382 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.func(req, *args, **kwargs) [Tue Jun 28 06:46:49.122392 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/oslo_middleware/base.py", line 124, in __call__ [Tue Jun 28 06:46:49.122400 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] response = req.get_response(self.application) [Tue Jun 28 06:46:49.122413 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send [Tue Jun 28 06:46:49.122421 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] application, catch_exc_info=False) [Tue Jun 28 06:46:49.122432 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in call_application [Tue Jun 28 06:46:49.122439 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] app_iter = application(self.environ, start_response) [Tue Jun 28 06:46:49.122463 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ [Tue Jun 28 06:46:49.122470 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) [Tue Jun 28 06:46:49.122481 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func [Tue Jun 28 06:46:49.122490 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.func(req, *args, **kwargs) [Tue Jun 28 06:46:49.122500 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/osprofiler/web.py", line 112, in __call__ [Tue Jun 28 06:46:49.122507 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return request.get_response(self.application) [Tue Jun 28 06:46:49.122517 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send [Tue Jun 28 06:46:49.122525 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] application, catch_exc_info=False) [Tue Jun 28 06:46:49.122535 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in call_application [Tue Jun 28 06:46:49.122542 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] app_iter = application(self.environ, start_response) [Tue Jun 28 06:46:49.122552 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ [Tue Jun 28 06:46:49.122562 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) [Tue Jun 28 06:46:49.122572 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func [Tue Jun 28 06:46:49.122579 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.func(req, *args, **kwargs) [Tue Jun 28 06:46:49.122589 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/oslo_middleware/request_id.py", line 58, in __call__ [Tue Jun 28 06:46:49.122596 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] response = req.get_response(self.application) [Tue Jun 28 06:46:49.122605 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send [Tue Jun 28 06:46:49.122612 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] application, catch_exc_info=False) [Tue Jun 28 06:46:49.122622 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in call_application [Tue Jun 28 06:46:49.122630 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] app_iter = application(self.environ, start_response) [Tue Jun 28 06:46:49.122670 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/url_normalize.py", line 38, in __call__ [Tue Jun 28 06:46:49.122696 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.app(environ, start_response) [Tue Jun 28 06:46:49.122729 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ [Tue Jun 28 06:46:49.122743 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) [Tue Jun 28 06:46:49.122753 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func [Tue Jun 28 06:46:49.122761 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.func(req, *args, **kwargs) [Tue Jun 28 06:46:49.122772 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystonemiddleware/auth_token/__init__.py", line 341, in __call__ [Tue Jun 28 06:46:49.122786 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] response = req.get_response(self._app) [Tue Jun 28 06:46:49.122800 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send [Tue Jun 28 06:46:49.122807 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] application, catch_exc_info=False) [Tue Jun 28 06:46:49.122817 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in call_application [Tue Jun 28 06:46:49.122824 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] app_iter = application(self.environ, start_response) [Tue Jun 28 06:46:49.122835 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/werkzeug/middleware/dispatcher.py", line 78, in __call__ [Tue Jun 28 06:46:49.122845 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return app(environ, start_response) [Tue Jun 28 06:46:49.122856 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 2450, in wsgi_app [Tue Jun 28 06:46:49.122863 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] response = self.handle_exception(e) [Tue Jun 28 06:46:49.122874 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in error_router [Tue Jun 28 06:46:49.122883 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return original_handler(e) [Tue Jun 28 06:46:49.122893 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in error_router [Tue Jun 28 06:46:49.122900 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return original_handler(e) [Tue Jun 28 06:46:49.122910 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in error_router [Tue Jun 28 06:46:49.122921 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return original_handler(e) [Tue Jun 28 06:46:49.122932 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] [Previous line repeated 27 more times] [Tue Jun 28 06:46:49.122943 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1867, in handle_exception [Tue Jun 28 06:46:49.122952 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] reraise(exc_type, exc_value, tb) [Tue Jun 28 06:46:49.122964 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise [Tue Jun 28 06:46:49.122971 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] raise value.with_traceback(tb) [Tue Jun 28 06:46:49.122981 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app [Tue Jun 28 06:46:49.122988 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] response = self.full_dispatch_request() [Tue Jun 28 06:46:49.122998 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request [Tue Jun 28 06:46:49.123007 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] rv = self.handle_user_exception(e) [Tue Jun 28 06:46:49.123018 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in error_router [Tue Jun 28 06:46:49.123025 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return original_handler(e) [Tue Jun 28 06:46:49.123035 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in error_router [Tue Jun 28 06:46:49.123044 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return original_handler(e) [Tue Jun 28 06:46:49.123059 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in error_router [Tue Jun 28 06:46:49.123066 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return original_handler(e) [Tue Jun 28 06:46:49.123077 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] [Previous line repeated 27 more times] [Tue Jun 28 06:46:49.123089 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception [Tue Jun 28 06:46:49.123097 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] reraise(exc_type, exc_value, tb) [Tue Jun 28 06:46:49.123107 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise [Tue Jun 28 06:46:49.123118 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] raise value.with_traceback(tb) [Tue Jun 28 06:46:49.123129 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request [Tue Jun 28 06:46:49.123137 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] rv = self.dispatch_request() [Tue Jun 28 06:46:49.123147 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request [Tue Jun 28 06:46:49.123154 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.view_functions[rule.endpoint](**req.view_args) [Tue Jun 28 06:46:49.123165 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 468, in wrapper [Tue Jun 28 06:46:49.123175 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] resp = resource(*args, **kwargs) [Tue Jun 28 06:46:49.123186 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask/views.py", line 89, in view [Tue Jun 28 06:46:49.123193 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.dispatch_request(*args, **kwargs) [Tue Jun 28 06:46:49.123204 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 583, in dispatch_request [Tue Jun 28 06:46:49.123211 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] resp = meth(*args, **kwargs) [Tue Jun 28 06:46:49.123222 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/api/users.py", line 183, in get [Tue Jun 28 06:46:49.123232 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self._list_users() [Tue Jun 28 06:46:49.123245 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/api/users.py", line 215, in _list_users [Tue Jun 28 06:46:49.123252 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] domain_scope=domain, hints=hints) [Tue Jun 28 06:46:49.123263 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 115, in wrapped [Tue Jun 28 06:46:49.123273 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] __ret_val = __f(*args, **kwargs) [Tue Jun 28 06:46:49.123282 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 414, in wrapper [Tue Jun 28 06:46:49.123289 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return f(self, *args, **kwargs) [Tue Jun 28 06:46:49.123299 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 424, in wrapper [Tue Jun 28 06:46:49.123308 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return f(self, *args, **kwargs) [Tue Jun 28 06:46:49.123327 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 1108, in list_users [Tue Jun 28 06:46:49.123337 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] ref_list = self._handle_shadow_and_local_users(driver, hints) [Tue Jun 28 06:46:49.123351 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 1091, in _handle_shadow_and_local_users [Tue Jun 28 06:46:49.123358 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return driver.list_users(hints) + fed_res [Tue Jun 28 06:46:49.123368 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", line 85, in list_users [Tue Jun 28 06:46:49.123376 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.user.get_all_filtered(hints) [Tue Jun 28 06:46:49.123387 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", line 328, in get_all_filtered [Tue Jun 28 06:46:49.123394 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] for user in self.get_all(query, hints)] [Tue Jun 28 06:46:49.123406 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", line 320, in get_all [Tue Jun 28 06:46:49.123413 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] hints=hints) [Tue Jun 28 06:46:49.123425 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 1949, in get_all [Tue Jun 28 06:46:49.123432 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return super(EnabledEmuMixIn, self).get_all(ldap_filter, hints) [Tue Jun 28 06:46:49.123443 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 1637, in get_all [Tue Jun 28 06:46:49.123453 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] for x in self._ldap_get_all(hints, ldap_filter)] [Tue Jun 28 06:46:49.123464 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/common/driver_hints.py", line 42, in wrapper [Tue Jun 28 06:46:49.123472 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return f(self, hints, *args, **kwargs) [Tue Jun 28 06:46:49.123482 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 1590, in _ldap_get_all [Tue Jun 28 06:46:49.123489 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] attrs) [Tue Jun 28 06:46:49.123500 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 986, in search_s [Tue Jun 28 06:46:49.123507 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] attrlist, attrsonly) [Tue Jun 28 06:46:49.123517 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 679, in wrapper [Tue Jun 28 06:46:49.123524 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return func(self, conn, *args, **kwargs) [Tue Jun 28 06:46:49.123535 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 814, in search_s [Tue Jun 28 06:46:49.123542 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] attrsonly) [Tue Jun 28 06:46:49.123552 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 870, in search_s [Tue Jun 28 06:46:49.123559 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout) [Tue Jun 28 06:46:49.123578 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 1286, in search_ext_s [Tue Jun 28 06:46:49.123586 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return self._apply_method_s(SimpleLDAPObject.search_ext_s,*args,**kwargs) [Tue Jun 28 06:46:49.123596 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 1224, in _apply_method_s [Tue Jun 28 06:46:49.123603 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] return func(self,*args,**kwargs) [Tue Jun 28 06:46:49.123613 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 863, in search_ext_s [Tue Jun 28 06:46:49.123621 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] msgid = self.search_ext(base,scope,filterstr,attrlist,attrsonly,serverctrls,clientctrls,timeout,sizelimit) [Tue Jun 28 06:46:49.123631 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 859, in search_ext [Tue Jun 28 06:46:49.123650 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] timeout,sizelimit, [Tue Jun 28 06:46:49.123664 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 340, in _ldap_call [Tue Jun 28 06:46:49.123672 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] reraise(exc_type, exc_value, exc_traceback) [Tue Jun 28 06:46:49.123690 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/compat.py", line 46, in reraise [Tue Jun 28 06:46:49.123701 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] raise exc_value [Tue Jun 28 06:46:49.123713 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 324, in _ldap_call [Tue Jun 28 06:46:49.123720 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] result = func(*args,**kwargs) [Tue Jun 28 06:46:49.123754 2022] [wsgi:error] [pid 45] [remote 172.25.201.201:58080] ldap.FILTER_ERROR: {'result': -7, 'desc': 'Bad search filter', 'ctrls': []} Version-Release number of selected component (if applicable): How reproducible: Configure domain in keystone. Steps to Reproduce: 1. setup 3 groups in ldap 2. create a user 3. configure ldap in keystone Actual results: When i issue the command: $ openstack user list --domain domain.com Output: Internal server error (HTTP 500) Expected results: When i issue the command: $ openstack user list --domain domain.com Output: should display users in the groups -------------- next part -------------- An HTML attachment was scrubbed... URL: From emilien at redhat.com Tue Jun 28 13:12:22 2022 From: emilien at redhat.com (Emilien Macchi) Date: Tue, 28 Jun 2022 09:12:22 -0400 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <01581dd3e606105771403abdda0fac7221b21bfa.camel@redhat.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> <01581dd3e606105771403abdda0fac7221b21bfa.camel@redhat.com> Message-ID: Sorry for the late reply, I'm still catching up e-mail backlog and plan to dig more in this thread at some point. I just wanted to answer Sean's question very simply. See inline below: On Mon, Jun 27, 2022 at 9:59 AM Sean Mooney wrote: > On Mon, 2022-06-27 at 15:30 +0200, Artem Goncharov wrote: > > > > > > ther eare some convince factors to github and many inconvenices., > > > it has a vastly inferior code review model that if i was force to use > would push me out of the openstack comunity long term. > > > im sure there are others too that feel stongly that moving to a github > pull request based workflow woudl be a large regerssion > > > and make them less inclined to continue working on openstack. > > > > The thread is being very explicit about external projects and not the > OpenStack itself. > yep but that is unhelpful. > if any external project that work with openstack want to become part of > openstack under the foundatiosn governace it is > nolonger external. > > so if gophercloud was to become part of openstack it would not be external > and if it wanted to you github pull requests > for it workflow it woudl be deviating form the other openstack projects. > > external project that are not part of openstack governacne can use any > tooling they like. > > if we start allowing arbiatry internal and external project to use gerrit > or github workflows of worse both concurrently > we will start getting request to supprot that for other proejct like nova > neutron ectra. i woudl see that as damaging > to the exsting colaberator based and something to be avoided if we can. > > im not really sure what gophercloud want to achive by being part of > openstack without adopting the openstack > ways of doing things that they cant acive by bing a nice go sdk for > openstack on there own with the well wishes > and or support of the openstack comunity. > > the 4 opens are a core part of the culture of openstack > simiarly the ways of workign with irc/gerrit/zuul/ptgs are also a part of > the openstack way. > > i am wondering why gophercloud want to actully becoem an offial proejct if > they dont want to adopt the open developement workflow (note i did not say > model) that openstack uses? > I'm a Gophercloud maintainer and can provide some context. Some of us at Red Hat inherited the project ( https://github.com/gophercloud/gophercloud/issues/2246) at the end of last year. The first thing we did was to check if the project could fit under the opendev umbrella as it seemed like the natural place to us. The discussion was run in the open: https://github.com/gophercloud/gophercloud/issues/2257 and http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025660.html The main reasons were: * Gain more sustainability, contributors around the community and more diversity in maintainers * More stable CI (not relevant anymore since we moved to Github Actions, and we do not rely on openlab anymore) * CI integration in other projects * Better governance When we asked the Gophercloud contributors about using gerrit, the feedback wasn't positive (details in #2257) so at this point we decided to not proceed further at the time. Due to the nature of the project, a lot of our pull-requests are "drive-by contributions" (e.g. to add new fields to the API) by new contributors; which ought to be considered if we were going to Gerrit. That being said, if we get more contributions from the OpenStack community, this would certainly help to justify the move under opendev. -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From gthiemonge at redhat.com Tue Jun 28 13:27:23 2022 From: gthiemonge at redhat.com (Gregory Thiemonge) Date: Tue, 28 Jun 2022 15:27:23 +0200 Subject: [Octavia] Proposing Tom Weininger as core reviewer Message-ID: Hi Folks, I would like to propose Tom Weininger as a core reviewer for the Octavia project. Since he joined the project, Tom has been a major contributor to Octavia, and he is an excellent reviewer. Please send your feedback in this thread, if there is no objection until next week, we will add him to the list of core reviewers. Thanks Gregory -------------- next part -------------- An HTML attachment was scrubbed... URL: From akamyshnikova at mirantis.com Tue Jun 28 13:38:00 2022 From: akamyshnikova at mirantis.com (Anna Taraday) Date: Tue, 28 Jun 2022 17:38:00 +0400 Subject: [Octavia] Proposing Tom Weininger as core reviewer In-Reply-To: References: Message-ID: +1 for Tom Thank you for your hard work! On Tue, Jun 28, 2022 at 5:35 PM Gregory Thiemonge wrote: > Hi Folks, > > I would like to propose Tom Weininger as a core reviewer for the Octavia > project. > Since he joined the project, Tom has been a major contributor to Octavia, > and he is an excellent reviewer. > > Please send your feedback in this thread, if there is no objection until > next week, we will add him to the list of core reviewers. > > Thanks > Gregory > -- Ann Taraday Senior Software Engineer ataraday at mirantis.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Tue Jun 28 13:44:22 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 28 Jun 2022 14:44:22 +0100 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> <01581dd3e606105771403abdda0fac7221b21bfa.camel@redhat.com> Message-ID: On Tue, 2022-06-28 at 09:12 -0400, Emilien Macchi wrote: > Sorry for the late reply, I'm still catching up e-mail backlog and plan to > dig more in this thread at some point. I just wanted to answer Sean's > question very simply. See inline below: > > On Mon, Jun 27, 2022 at 9:59 AM Sean Mooney wrote: > > > On Mon, 2022-06-27 at 15:30 +0200, Artem Goncharov wrote: > > > > > > > > ther eare some convince factors to github and many inconvenices., > > > > it has a vastly inferior code review model that if i was force to use > > would push me out of the openstack comunity long term. > > > > im sure there are others too that feel stongly that moving to a github > > pull request based workflow woudl be a large regerssion > > > > and make them less inclined to continue working on openstack. > > > > > > The thread is being very explicit about external projects and not the > > OpenStack itself. > > yep but that is unhelpful. > > if any external project that work with openstack want to become part of > > openstack under the foundatiosn governace it is > > nolonger external. > > > > so if gophercloud was to become part of openstack it would not be external > > and if it wanted to you github pull requests > > for it workflow it woudl be deviating form the other openstack projects. > > > > external project that are not part of openstack governacne can use any > > tooling they like. > > > > if we start allowing arbiatry internal and external project to use gerrit > > or github workflows of worse both concurrently > > we will start getting request to supprot that for other proejct like nova > > neutron ectra. i woudl see that as damaging > > to the exsting colaberator based and something to be avoided if we can. > > > > im not really sure what gophercloud want to achive by being part of > > openstack without adopting the openstack > > ways of doing things that they cant acive by bing a nice go sdk for > > openstack on there own with the well wishes > > and or support of the openstack comunity. > > > > the 4 opens are a core part of the culture of openstack > > simiarly the ways of workign with irc/gerrit/zuul/ptgs are also a part of > > the openstack way. > > > > i am wondering why gophercloud want to actully becoem an offial proejct if > > they dont want to adopt the open developement workflow (note i did not say > > model) that openstack uses? > > > > I'm a Gophercloud maintainer and can provide some context. Some of us at > Red Hat inherited the project ( > https://github.com/gophercloud/gophercloud/issues/2246) at the end of last > year. The first thing we did was to check if the project could fit under the > opendev umbrella as it seemed like the natural place to us. The discussion > was run in the open: https://github.com/gophercloud/gophercloud/issues/2257 > and > http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025660.html > The main reasons were: > * Gain more sustainability, contributors around the community and more > diversity in maintainers > * More stable CI (not relevant anymore since we moved to Github Actions, > and we do not rely on openlab anymore) > * CI integration in other projects > * Better governance Thanks for that context. with that in mind im not sure it makes sense to proceed with movign it under openstack governance. i agree that it woudl be nice form a governance perspective to include it under the sdk team but if the opendev ways or workign dont work for the existing contibutor base im not sure it makes sense for ether comuntiy i do agree these ^ are good reasons to consider this change but im not conviced it woudl be good for openstack to add github hosting and review workflow. > > When we asked the Gophercloud contributors about using gerrit, the feedback > wasn't positive (details in #2257) so at this point we decided to not > proceed further at the time. > Due to the nature of the project, a lot of our pull-requests are "drive-by > contributions" (e.g. to add new fields to the API) by new contributors; > which ought to be considered if we were going to Gerrit. > > That being said, if we get more contributions from the OpenStack community, > this would certainly help to justify the move under opendev. From smooney at redhat.com Tue Jun 28 14:05:59 2022 From: smooney at redhat.com (Sean Mooney) Date: Tue, 28 Jun 2022 15:05:59 +0100 Subject: Openstack instance is unreachable In-Reply-To: References: <20220628105723.Horde.VEeg85NrLzpcrDTFq6t70GG@webmail.nde.ag> Message-ID: On Tue, 2022-06-28 at 14:08 +0300, GEORGIOS PAPATHANAIL wrote: > Hi, > > My instances are on a flat network. > They get public IP address. > The security group is configured correctly. > > I did not launch an instance with ?config-drive-true. if they are booting on an external flat newtork with public ips then you need to configre neutron to serve metadta by the dhcp agent if you are usign ml2/ovs since there is no neutron router to impelement the metadta proxy in that configurtion. if you are using ml2/ovn the docs for configuring this are here https://docs.openstack.org/neutron/latest/contributor/internals/ovn/metadata_api.html for ml2/ovs you will need to use isolated metadata? https://docs.openstack.org/neutron/latest/admin/archives/config-agents.html#dhcp-agent-setup-ovs-plug-in you might find https://docs.openstack.org/operations-guide/ops-network-troubleshooting.html and https://www.rdoproject.org/networking/networking-in-too-much-detail/ useful too for general overview of how things work. > > > ???? ??? 28 ???? 2022 ???? 14:04 ? ??????? Eugen Block > ??????: > > > Hi, > > > > this is one of the most common issues users post report in this list. > > Did you try to find a solution? Do your instances in the flat network > > get an IP address? If not, try to launch instances with > > "--config-drive true". If the instances do get an IP, do your > > security-group rules allow access? > > > > > > Zitat von GEORGIOS PAPATHANAIL : > > > > > I am facing the following issue with my new Openstack installation. The > > > installation is a little bit weird and to elaborate more, I have a > > > controller and a compute node running as VMs in XenServer. > > > > > > The compute node has nested virtualization enabled and uses qemu in order > > > to provision VMs. I have a provider flat network that has a public range > > of > > > IPs and VMs use this network. > > > > > > I am able to create instances and access them via console, but I can't > > ping > > > them or ssh even if they have public IPs. > > > > > > I am not very familiar with Xen Server, is there any configuration that > > is > > > needed (bridges, promisc mode, etc)? > > > > > > -- > > > *George Papathanail* > > > *Associate Researcher* > > > *Department of Applied Informatics* > > > *University of Macedonia* > > > > > > > > > > -- > *George Papathanail* > *Associate Researcher* > *Department of Applied Informatics* > *University of Macedonia* From flux.adam at gmail.com Tue Jun 28 14:24:52 2022 From: flux.adam at gmail.com (Adam Harwell) Date: Tue, 28 Jun 2022 07:24:52 -0700 Subject: [Octavia] Proposing Tom Weininger as core reviewer In-Reply-To: References: Message-ID: +1 from me as well! On Tue, Jun 28, 2022 at 6:42 AM Anna Taraday wrote: > +1 for Tom > > Thank you for your hard work! > > On Tue, Jun 28, 2022 at 5:35 PM Gregory Thiemonge > wrote: > >> Hi Folks, >> >> I would like to propose Tom Weininger as a core reviewer for the Octavia >> project. >> Since he joined the project, Tom has been a major contributor to Octavia, >> and he is an excellent reviewer. >> >> Please send your feedback in this thread, if there is no objection until >> next week, we will add him to the list of core reviewers. >> >> Thanks >> Gregory >> > > > -- > > Ann Taraday > > Senior Software Engineer > ataraday at mirantis.com > > -- Thanks, --Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From satish.txt at gmail.com Tue Jun 28 14:14:58 2022 From: satish.txt at gmail.com (Satish Patel) Date: Tue, 28 Jun 2022 10:14:58 -0400 Subject: Openstack keystone LDAP integration | openstack user list --domain domain.com | Internal server error (HTTP 500) In-Reply-To: References: Message-ID: Swogat, We don't have AD but we are successfully running FreeIPA based LDAP integration with keystone and i blog that out here if you want to cross check some config parts https://satishdotpatel.github.io/openstack-ldap-integration/ On Tue, Jun 28, 2022 at 8:54 AM Swogat Pradhan wrote: > Description of problem: > I am trying to integrate AD server in keystone and facing 'Internal server > error' > domain configuration: > [stack at hkg2director ~]$ cat > workplace/keystone_domain_specific_ldap_backend.yaml > # This is an example template on how to configure keystone domain specific > LDAP > # backends. This will configure a domain called tripleoldap will the > attributes > # specified. > parameter_defaults: > KeystoneLDAPDomainEnable: true > KeystoneLDAPBackendConfigs: > domain.com: > url: ldap://172.25.161.211 > user: cn=Openstack,ou=Admins,dc=domain,dc=com > password: password > suffix: dc=domain,dc=com > user_tree_dn: ou=APAC,dc=domain,dc=com > user_filter: > "(|(memberOf=cn=openstackadmin,ou=Groups,dc=domain,dc=com)(memberOf=cn=openstackeditor,ou=Groups,dc=domain,dc=com)(memberOf=cn=openstackviewer,ou=Groups,dc=domain,dc=com)" > user_objectclass: person > user_id_attribute: cn > > group_tree_dn: ou=Groups,dc=domain,dc=com > group_objectclass: Groups > group_id_attribute: cn > > When i issue the command: > $ openstack user list --domain domain.com > Output: Internal server error (HTTP 500) > > Keystone_wsgi_error.log: > [Tue Jun 28 06:46:49.112848 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] mod_wsgi (pid=45): Exception occurred processing > WSGI script '/var/www/cgi-bin/keystone/keystone'. > [Tue Jun 28 06:46:49.121797 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] Traceback (most recent call last): > [Tue Jun 28 06:46:49.122202 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 2464, in __call__ > [Tue Jun 28 06:46:49.122218 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.wsgi_app(environ, start_response) > [Tue Jun 28 06:46:49.122231 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/werkzeug/middleware/proxy_fix.py", line > 187, in __call__ > [Tue Jun 28 06:46:49.122238 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.app(environ, start_response) > [Tue Jun 28 06:46:49.122248 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ > [Tue Jun 28 06:46:49.122254 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) > [Tue Jun 28 06:46:49.122264 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func > [Tue Jun 28 06:46:49.122270 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.func(req, *args, **kwargs) > [Tue Jun 28 06:46:49.122284 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/oslo_middleware/base.py", line 124, in > __call__ > [Tue Jun 28 06:46:49.122294 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] response = req.get_response(self.application) > [Tue Jun 28 06:46:49.122304 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send > [Tue Jun 28 06:46:49.122310 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] application, catch_exc_info=False) > [Tue Jun 28 06:46:49.122320 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in > call_application > [Tue Jun 28 06:46:49.122326 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] app_iter = application(self.environ, start_response) > [Tue Jun 28 06:46:49.122337 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 143, in __call__ > [Tue Jun 28 06:46:49.122344 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return resp(environ, start_response) > [Tue Jun 28 06:46:49.122354 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ > [Tue Jun 28 06:46:49.122364 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) > [Tue Jun 28 06:46:49.122374 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func > [Tue Jun 28 06:46:49.122382 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.func(req, *args, **kwargs) > [Tue Jun 28 06:46:49.122392 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/oslo_middleware/base.py", line 124, in > __call__ > [Tue Jun 28 06:46:49.122400 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] response = req.get_response(self.application) > [Tue Jun 28 06:46:49.122413 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send > [Tue Jun 28 06:46:49.122421 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] application, catch_exc_info=False) > [Tue Jun 28 06:46:49.122432 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in > call_application > [Tue Jun 28 06:46:49.122439 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] app_iter = application(self.environ, start_response) > [Tue Jun 28 06:46:49.122463 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ > [Tue Jun 28 06:46:49.122470 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) > [Tue Jun 28 06:46:49.122481 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func > [Tue Jun 28 06:46:49.122490 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.func(req, *args, **kwargs) > [Tue Jun 28 06:46:49.122500 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/osprofiler/web.py", line 112, in __call__ > [Tue Jun 28 06:46:49.122507 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return request.get_response(self.application) > [Tue Jun 28 06:46:49.122517 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send > [Tue Jun 28 06:46:49.122525 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] application, catch_exc_info=False) > [Tue Jun 28 06:46:49.122535 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in > call_application > [Tue Jun 28 06:46:49.122542 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] app_iter = application(self.environ, start_response) > [Tue Jun 28 06:46:49.122552 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ > [Tue Jun 28 06:46:49.122562 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) > [Tue Jun 28 06:46:49.122572 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func > [Tue Jun 28 06:46:49.122579 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.func(req, *args, **kwargs) > [Tue Jun 28 06:46:49.122589 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/oslo_middleware/request_id.py", line 58, > in __call__ > [Tue Jun 28 06:46:49.122596 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] response = req.get_response(self.application) > [Tue Jun 28 06:46:49.122605 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send > [Tue Jun 28 06:46:49.122612 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] application, catch_exc_info=False) > [Tue Jun 28 06:46:49.122622 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in > call_application > [Tue Jun 28 06:46:49.122630 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] app_iter = application(self.environ, start_response) > [Tue Jun 28 06:46:49.122670 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/server/flask/request_processing/middleware/url_normalize.py", > line 38, in __call__ > [Tue Jun 28 06:46:49.122696 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.app(environ, start_response) > [Tue Jun 28 06:46:49.122729 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 129, in __call__ > [Tue Jun 28 06:46:49.122743 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] resp = self.call_func(req, *args, **kw) > [Tue Jun 28 06:46:49.122753 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/dec.py", line 193, in call_func > [Tue Jun 28 06:46:49.122761 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.func(req, *args, **kwargs) > [Tue Jun 28 06:46:49.122772 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystonemiddleware/auth_token/__init__.py", > line 341, in __call__ > [Tue Jun 28 06:46:49.122786 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] response = req.get_response(self._app) > [Tue Jun 28 06:46:49.122800 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1314, in send > [Tue Jun 28 06:46:49.122807 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] application, catch_exc_info=False) > [Tue Jun 28 06:46:49.122817 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/webob/request.py", line 1278, in > call_application > [Tue Jun 28 06:46:49.122824 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] app_iter = application(self.environ, start_response) > [Tue Jun 28 06:46:49.122835 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/werkzeug/middleware/dispatcher.py", line > 78, in __call__ > [Tue Jun 28 06:46:49.122845 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return app(environ, start_response) > [Tue Jun 28 06:46:49.122856 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 2450, in wsgi_app > [Tue Jun 28 06:46:49.122863 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] response = self.handle_exception(e) > [Tue Jun 28 06:46:49.122874 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in > error_router > [Tue Jun 28 06:46:49.122883 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return original_handler(e) > [Tue Jun 28 06:46:49.122893 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in > error_router > [Tue Jun 28 06:46:49.122900 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return original_handler(e) > [Tue Jun 28 06:46:49.122910 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in > error_router > [Tue Jun 28 06:46:49.122921 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return original_handler(e) > [Tue Jun 28 06:46:49.122932 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] [Previous line repeated 27 more times] > [Tue Jun 28 06:46:49.122943 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 1867, in > handle_exception > [Tue Jun 28 06:46:49.122952 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] reraise(exc_type, exc_value, tb) > [Tue Jun 28 06:46:49.122964 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise > [Tue Jun 28 06:46:49.122971 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] raise value.with_traceback(tb) > [Tue Jun 28 06:46:49.122981 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app > [Tue Jun 28 06:46:49.122988 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] response = self.full_dispatch_request() > [Tue Jun 28 06:46:49.122998 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 1952, in > full_dispatch_request > [Tue Jun 28 06:46:49.123007 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] rv = self.handle_user_exception(e) > [Tue Jun 28 06:46:49.123018 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in > error_router > [Tue Jun 28 06:46:49.123025 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return original_handler(e) > [Tue Jun 28 06:46:49.123035 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in > error_router > [Tue Jun 28 06:46:49.123044 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return original_handler(e) > [Tue Jun 28 06:46:49.123059 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 272, in > error_router > [Tue Jun 28 06:46:49.123066 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return original_handler(e) > [Tue Jun 28 06:46:49.123077 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] [Previous line repeated 27 more times] > [Tue Jun 28 06:46:49.123089 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 1821, in > handle_user_exception > [Tue Jun 28 06:46:49.123097 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] reraise(exc_type, exc_value, tb) > [Tue Jun 28 06:46:49.123107 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise > [Tue Jun 28 06:46:49.123118 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] raise value.with_traceback(tb) > [Tue Jun 28 06:46:49.123129 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 1950, in > full_dispatch_request > [Tue Jun 28 06:46:49.123137 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] rv = self.dispatch_request() > [Tue Jun 28 06:46:49.123147 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/app.py", line 1936, in > dispatch_request > [Tue Jun 28 06:46:49.123154 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return > self.view_functions[rule.endpoint](**req.view_args) > [Tue Jun 28 06:46:49.123165 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 468, in > wrapper > [Tue Jun 28 06:46:49.123175 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] resp = resource(*args, **kwargs) > [Tue Jun 28 06:46:49.123186 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask/views.py", line 89, in view > [Tue Jun 28 06:46:49.123193 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.dispatch_request(*args, **kwargs) > [Tue Jun 28 06:46:49.123204 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 583, in > dispatch_request > [Tue Jun 28 06:46:49.123211 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] resp = meth(*args, **kwargs) > [Tue Jun 28 06:46:49.123222 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/api/users.py", line 183, in get > [Tue Jun 28 06:46:49.123232 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self._list_users() > [Tue Jun 28 06:46:49.123245 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/api/users.py", line 215, in > _list_users > [Tue Jun 28 06:46:49.123252 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] domain_scope=domain, hints=hints) > [Tue Jun 28 06:46:49.123263 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 115, in > wrapped > [Tue Jun 28 06:46:49.123273 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] __ret_val = __f(*args, **kwargs) > [Tue Jun 28 06:46:49.123282 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 414, in > wrapper > [Tue Jun 28 06:46:49.123289 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return f(self, *args, **kwargs) > [Tue Jun 28 06:46:49.123299 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 424, in > wrapper > [Tue Jun 28 06:46:49.123308 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return f(self, *args, **kwargs) > [Tue Jun 28 06:46:49.123327 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 1108, in > list_users > [Tue Jun 28 06:46:49.123337 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] ref_list = > self._handle_shadow_and_local_users(driver, hints) > [Tue Jun 28 06:46:49.123351 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 1091, in > _handle_shadow_and_local_users > [Tue Jun 28 06:46:49.123358 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return driver.list_users(hints) + fed_res > [Tue Jun 28 06:46:49.123368 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", > line 85, in list_users > [Tue Jun 28 06:46:49.123376 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return self.user.get_all_filtered(hints) > [Tue Jun 28 06:46:49.123387 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", > line 328, in get_all_filtered > [Tue Jun 28 06:46:49.123394 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] for user in self.get_all(query, hints)] > [Tue Jun 28 06:46:49.123406 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", > line 320, in get_all > [Tue Jun 28 06:46:49.123413 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] hints=hints) > [Tue Jun 28 06:46:49.123425 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", > line 1949, in get_all > [Tue Jun 28 06:46:49.123432 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return super(EnabledEmuMixIn, > self).get_all(ldap_filter, hints) > [Tue Jun 28 06:46:49.123443 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", > line 1637, in get_all > [Tue Jun 28 06:46:49.123453 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] for x in self._ldap_get_all(hints, ldap_filter)] > [Tue Jun 28 06:46:49.123464 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/common/driver_hints.py", line > 42, in wrapper > [Tue Jun 28 06:46:49.123472 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return f(self, hints, *args, **kwargs) > [Tue Jun 28 06:46:49.123482 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", > line 1590, in _ldap_get_all > [Tue Jun 28 06:46:49.123489 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] attrs) > [Tue Jun 28 06:46:49.123500 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", > line 986, in search_s > [Tue Jun 28 06:46:49.123507 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] attrlist, attrsonly) > [Tue Jun 28 06:46:49.123517 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", > line 679, in wrapper > [Tue Jun 28 06:46:49.123524 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return func(self, conn, *args, **kwargs) > [Tue Jun 28 06:46:49.123535 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", > line 814, in search_s > [Tue Jun 28 06:46:49.123542 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] attrsonly) > [Tue Jun 28 06:46:49.123552 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 870, in > search_s > [Tue Jun 28 06:46:49.123559 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return > self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout) > [Tue Jun 28 06:46:49.123578 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 1286, in > search_ext_s > [Tue Jun 28 06:46:49.123586 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return > self._apply_method_s(SimpleLDAPObject.search_ext_s,*args,**kwargs) > [Tue Jun 28 06:46:49.123596 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 1224, in > _apply_method_s > [Tue Jun 28 06:46:49.123603 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] return func(self,*args,**kwargs) > [Tue Jun 28 06:46:49.123613 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 863, in > search_ext_s > [Tue Jun 28 06:46:49.123621 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] msgid = > self.search_ext(base,scope,filterstr,attrlist,attrsonly,serverctrls,clientctrls,timeout,sizelimit) > [Tue Jun 28 06:46:49.123631 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 859, in > search_ext > [Tue Jun 28 06:46:49.123650 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] timeout,sizelimit, > [Tue Jun 28 06:46:49.123664 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 340, in > _ldap_call > [Tue Jun 28 06:46:49.123672 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] reraise(exc_type, exc_value, exc_traceback) > [Tue Jun 28 06:46:49.123690 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/compat.py", line 46, in reraise > [Tue Jun 28 06:46:49.123701 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] raise exc_value > [Tue Jun 28 06:46:49.123713 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] File > "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 324, in > _ldap_call > [Tue Jun 28 06:46:49.123720 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] result = func(*args,**kwargs) > [Tue Jun 28 06:46:49.123754 2022] [wsgi:error] [pid 45] [remote > 172.25.201.201:58080] ldap.FILTER_ERROR: {'result': -7, 'desc': 'Bad > search filter', 'ctrls': []} > > Version-Release number of selected component (if applicable): > > How reproducible: > Configure domain in keystone. > > Steps to Reproduce: > 1. setup 3 groups in ldap > 2. create a user > 3. configure ldap in keystone > > Actual results: > When i issue the command: > $ openstack user list --domain domain.com > Output: Internal server error (HTTP 500) > > Expected results: > When i issue the command: > $ openstack user list --domain domain.com > Output: should display users in the groups > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dms at danplanet.com Tue Jun 28 14:33:44 2022 From: dms at danplanet.com (Dan Smith) Date: Tue, 28 Jun 2022 07:33:44 -0700 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <3069979.mvXUDI8C0e@p1> (Slawek Kaplonski's message of "Tue, 28 Jun 2022 09:22:41 +0200") References: <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <3069979.mvXUDI8C0e@p1> Message-ID: > I second that. It reminds me the Storyboard and Launchpad thing, where > some projects are using one and others the other one bug tracker so > new people don't even exactly know where they should report bug > against particular project. > If we will have some projects hosted on opendev and using Gerrit and > others hosted on Github only and using Github's workflow, it will be > IMO similar mess. Agree. --Dan From johnsomor at gmail.com Tue Jun 28 17:50:07 2022 From: johnsomor at gmail.com (Michael Johnson) Date: Tue, 28 Jun 2022 10:50:07 -0700 Subject: [Octavia] Proposing Tom Weininger as core reviewer In-Reply-To: References: Message-ID: +1, Tom has been doing great work. Michael On Tue, Jun 28, 2022 at 7:29 AM Adam Harwell wrote: > +1 from me as well! > > On Tue, Jun 28, 2022 at 6:42 AM Anna Taraday > wrote: > >> +1 for Tom >> >> Thank you for your hard work! >> >> On Tue, Jun 28, 2022 at 5:35 PM Gregory Thiemonge >> wrote: >> >>> Hi Folks, >>> >>> I would like to propose Tom Weininger as a core reviewer for the Octavia >>> project. >>> Since he joined the project, Tom has been a major contributor to >>> Octavia, and he is an excellent reviewer. >>> >>> Please send your feedback in this thread, if there is no objection until >>> next week, we will add him to the list of core reviewers. >>> >>> Thanks >>> Gregory >>> >> >> >> -- >> >> Ann Taraday >> >> Senior Software Engineer >> ataraday at mirantis.com >> >> -- > Thanks, > --Adam > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Tue Jun 28 17:58:02 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Tue, 28 Jun 2022 12:58:02 -0500 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> <01581dd3e606105771403abdda0fac7221b21bfa.camel@redhat.com> Message-ID: <181ab768d78.e7d80f24207384.7294791342745223843@ghanshyammann.com> ---- On Tue, 28 Jun 2022 08:44:22 -0500 Sean Mooney wrote --- > On Tue, 2022-06-28 at 09:12 -0400, Emilien Macchi wrote: > > Sorry for the late reply, I'm still catching up e-mail backlog and plan to > > dig more in this thread at some point. I just wanted to answer Sean's > > question very simply. See inline below: > > > > On Mon, Jun 27, 2022 at 9:59 AM Sean Mooney wrote: > > > > > On Mon, 2022-06-27 at 15:30 +0200, Artem Goncharov wrote: > > > > > > > > > > ther eare some convince factors to github and many inconvenices., > > > > > it has a vastly inferior code review model that if i was force to use > > > would push me out of the openstack comunity long term. > > > > > im sure there are others too that feel stongly that moving to a github > > > pull request based workflow woudl be a large regerssion > > > > > and make them less inclined to continue working on openstack. > > > > > > > > The thread is being very explicit about external projects and not the > > > OpenStack itself. > > > yep but that is unhelpful. > > > if any external project that work with openstack want to become part of > > > openstack under the foundatiosn governace it is > > > nolonger external. > > > > > > so if gophercloud was to become part of openstack it would not be external > > > and if it wanted to you github pull requests > > > for it workflow it woudl be deviating form the other openstack projects. > > > > > > external project that are not part of openstack governacne can use any > > > tooling they like. > > > > > > if we start allowing arbiatry internal and external project to use gerrit > > > or github workflows of worse both concurrently > > > we will start getting request to supprot that for other proejct like nova > > > neutron ectra. i woudl see that as damaging > > > to the exsting colaberator based and something to be avoided if we can. > > > > > > im not really sure what gophercloud want to achive by being part of > > > openstack without adopting the openstack > > > ways of doing things that they cant acive by bing a nice go sdk for > > > openstack on there own with the well wishes > > > and or support of the openstack comunity. > > > > > > the 4 opens are a core part of the culture of openstack > > > simiarly the ways of workign with irc/gerrit/zuul/ptgs are also a part of > > > the openstack way. > > > > > > i am wondering why gophercloud want to actully becoem an offial proejct if > > > they dont want to adopt the open developement workflow (note i did not say > > > model) that openstack uses? > > > > > > > I'm a Gophercloud maintainer and can provide some context. Some of us at > > Red Hat inherited the project ( > > https://github.com/gophercloud/gophercloud/issues/2246) at the end of last > > year. The first thing we did was to check if the project could fit under the > > opendev umbrella as it seemed like the natural place to us. The discussion > > was run in the open: https://github.com/gophercloud/gophercloud/issues/2257 > > and > > http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025660.html > > The main reasons were: > > * Gain more sustainability, contributors around the community and more > > diversity in maintainers > > * More stable CI (not relevant anymore since we moved to Github Actions, > > and we do not rely on openlab anymore) > > * CI integration in other projects > > * Better governance > > Thanks for that context. > > with that in mind im not sure it makes sense to proceed with movign it under openstack > governance. > > i agree that it woudl be nice form a governance perspective to include it under the sdk team > but if the opendev ways or workign dont work for the existing contibutor base im not sure > it makes sense for ether comuntiy i do agree these ^ are good reasons to consider this change > but im not conviced it woudl be good for openstack to add github hosting and review workflow. I might agree on all those points of not splitting the tooling but considering the current situation, I do not think we will have any new contributors in Gophercloud or any of the existing contributors will be contributing in Gophercloud so it hardly matters from the contributors' point of view if that is in GitHub or OpenDev. If SDK team is ok with Github then I think we should be ok. Yes, if we have a few existing contributors who want to contribute to Gophercloud then it makes sense to have it in OpenDev otherwise I feel like we are stuck on process which can be more beneficial if we change it. -gmann > > > > When we asked the Gophercloud contributors about using gerrit, the feedback > > wasn't positive (details in #2257) so at this point we decided to not > > proceed further at the time. > > Due to the nature of the project, a lot of our pull-requests are "drive-by > > contributions" (e.g. to add new fields to the API) by new contributors; > > which ought to be considered if we were going to Gerrit. > > > > That being said, if we get more contributions from the OpenStack community, > > this would certainly help to justify the move under opendev. > > > From artem.goncharov at gmail.com Tue Jun 28 19:20:59 2022 From: artem.goncharov at gmail.com (Artem Goncharov) Date: Tue, 28 Jun 2022 21:20:59 +0200 Subject: [all][foundation][ecosystem] External projects under the foundation hat In-Reply-To: <181ab768d78.e7d80f24207384.7294791342745223843@ghanshyammann.com> References: <21000fc87f66a1cf5271f763107de04dea9b9c3d.camel@redhat.com> <1819115cd76.f03616e1369932.9154626309206727468@ghanshyammann.com> <20220623153023.fwnstkz5hetite6q@yuggoth.org> <181918e4a93.d24e8dfe377241.3439187030074622677@ghanshyammann.com> <20220623175501.57ykoyhhqxsqulbx@yuggoth.org> <18193295a36.b7ad02a5384978.4225674611869053011@ghanshyammann.com> <1819702cd57.cf2c2c0c444103.7783104418868634231@ghanshyammann.com> <8bfd4ec4-4091-44cf-91f6-77c5c7a104ff@www.fastmail.com> <8b28ddf5-fb71-4a00-a635-58f682a673d1@www.fastmail.com> <2FD4FA1D-D9C5-4792-993A-2924CF6AA720@gmail.com> <01581dd3e606105771403abdda0fac7221b21bfa.camel@redhat.com> <181ab768d78.e7d80f24207384.7294791342745223843@ghanshyammann.com> Message-ID: On Tue, Jun 28, 2022, 19:58 Ghanshyam Mann wrote: > ---- On Tue, 28 Jun 2022 08:44:22 -0500 Sean Mooney > wrote --- > > On Tue, 2022-06-28 at 09:12 -0400, Emilien Macchi wrote: > > > Sorry for the late reply, I'm still catching up e-mail backlog and > plan to > > > dig more in this thread at some point. I just wanted to answer Sean's > > > question very simply. See inline below: > > > > > > On Mon, Jun 27, 2022 at 9:59 AM Sean Mooney > wrote: > > > > > > > On Mon, 2022-06-27 at 15:30 +0200, Artem Goncharov wrote: > > > > > > > > > > > > ther eare some convince factors to github and many > inconvenices., > > > > > > it has a vastly inferior code review model that if i was force > to use > > > > would push me out of the openstack comunity long term. > > > > > > im sure there are others too that feel stongly that moving to a > github > > > > pull request based workflow woudl be a large regerssion > > > > > > and make them less inclined to continue working on openstack. > > > > > > > > > > The thread is being very explicit about external projects and not > the > > > > OpenStack itself. > > > > yep but that is unhelpful. > > > > if any external project that work with openstack want to become > part of > > > > openstack under the foundatiosn governace it is > > > > nolonger external. > > > > > > > > so if gophercloud was to become part of openstack it would not be > external > > > > and if it wanted to you github pull requests > > > > for it workflow it woudl be deviating form the other openstack > projects. > > > > > > > > external project that are not part of openstack governacne can use > any > > > > tooling they like. > > > > > > > > if we start allowing arbiatry internal and external project to use > gerrit > > > > or github workflows of worse both concurrently > > > > we will start getting request to supprot that for other proejct > like nova > > > > neutron ectra. i woudl see that as damaging > > > > to the exsting colaberator based and something to be avoided if we > can. > > > > > > > > im not really sure what gophercloud want to achive by being part of > > > > openstack without adopting the openstack > > > > ways of doing things that they cant acive by bing a nice go sdk for > > > > openstack on there own with the well wishes > > > > and or support of the openstack comunity. > > > > > > > > the 4 opens are a core part of the culture of openstack > > > > simiarly the ways of workign with irc/gerrit/zuul/ptgs are also a > part of > > > > the openstack way. > > > > > > > > i am wondering why gophercloud want to actully becoem an offial > proejct if > > > > they dont want to adopt the open developement workflow (note i did > not say > > > > model) that openstack uses? > > > > > > > > > > I'm a Gophercloud maintainer and can provide some context. Some of us > at > > > Red Hat inherited the project ( > > > https://github.com/gophercloud/gophercloud/issues/2246) at the end > of last > > > year. The first thing we did was to check if the project could fit > under the > > > opendev umbrella as it seemed like the natural place to us. The > discussion > > > was run in the open: > https://github.com/gophercloud/gophercloud/issues/2257 > > > and > > > > http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025660.html > > > The main reasons were: > > > * Gain more sustainability, contributors around the community and more > > > diversity in maintainers > > > * More stable CI (not relevant anymore since we moved to Github > Actions, > > > and we do not rely on openlab anymore) > > > * CI integration in other projects > > > * Better governance > > > > Thanks for that context. > > > > with that in mind im not sure it makes sense to proceed with movign it > under openstack > > governance. > > > > i agree that it woudl be nice form a governance perspective to include > it under the sdk team > > but if the opendev ways or workign dont work for the existing > contibutor base im not sure > > it makes sense for ether comuntiy i do agree these ^ are good reasons > to consider this change > > but im not conviced it woudl be good for openstack to add github > hosting and review workflow. > > I might agree on all those points of not splitting the tooling but > considering the current situation, > I do not think we will have any new contributors in Gophercloud or any of > the existing contributors > will be contributing in Gophercloud so it hardly matters from the > contributors' point of view if that is > in GitHub or OpenDev. If SDK team is ok with Github then I think we should > be ok. > > Yes, if we have a few existing contributors who want to contribute to > Gophercloud then > it makes sense to have it in OpenDev otherwise I feel like we are stuck on > process which > can be more beneficial if we change it. > > -gmann > > > > > > > When we asked the Gophercloud contributors about using gerrit, the > feedback > > > wasn't positive (details in #2257) so at this point we decided to not > > > proceed further at the time. > > > Due to the nature of the project, a lot of our pull-requests are > "drive-by > > > contributions" (e.g. to add new fields to the API) by new > contributors; > > > which ought to be considered if we were going to Gerrit. > > > > > > That being said, if we get more contributions from the OpenStack > community, > > > this would certainly help to justify the move under opendev. > > Agree here. At some point in time processes might need to change to allow growth or any evolution. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Tue Jun 28 20:15:06 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Tue, 28 Jun 2022 15:15:06 -0500 Subject: [tc][policy][ptl][operator] OpenSatck RBAC goal is to focus on project personas first (postpone the 'scope' implementation) Message-ID: <181abf40999.1291eff02210814.4353892754671729638@ghanshyammann.com> Hello Everyone, We have received a good amount of feedback from the operator. In ops meetup Berlin as well as from KDDI (Japanese telco). I have summarized the feedback in etherpad[1]. From feedback, it is clear that 'scope' enable will break heat so do NFV operators and even other operators are also confused with the 'scope' concept. Most operators want legacy admin to work as it is (able to do everything in deployment). We discussed this feedback in the policy popup meeting[2] and based on feedback and our outstanding issue of 'scope enable break heat create_stack', we decided to postpone the `scope` implementation. That is the way forward to at least implement the project personas which is asked by many operators. Basically the below direction: * Finish delivering project personas This is to introduce the `member` and `reader` roles to operate things within their project. By default, any other project role like `foo` will not be allowed to do anything in the project. * Postpone the `scope` implementation for later Some standalone services like Ironic can still have the `scope` implementation as long as it does not break any cross-service communication. Other services will make sure they work for project scope personas even with enforce_scope enabled. We are not saying 'scope' things are not good or we will never do it but at the same time, I am not sure when we will do it in future. At least moving this giant goal to focus on the project personas first will help us to deliver one good feature (project personas) for operators otherwise we are stuck. Complete details about the reason to postpone the 'scope' implementation and projects persons detail are proposed in community-wide goal, please review there. - https://review.opendev.org/c/openstack/governance/+/847418 [1] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 [2] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 From stephenfin at redhat.com Tue Jun 28 20:22:44 2022 From: stephenfin at redhat.com (Stephen Finucane) Date: Tue, 28 Jun 2022 21:22:44 +0100 Subject: [sdk] renaming a volume with the SDK In-Reply-To: <080201d887f6$1c212b30$54638190$@globalnoc.iu.edu> References: <063f01d887e1$47222100$d5666300$@globalnoc.iu.edu> <080201d887f6$1c212b30$54638190$@globalnoc.iu.edu> Message-ID: <8c8ee4d966f86466047c50cb63a2fc2be2964830.camel@redhat.com> On Fri, 2022-06-24 at 14:13 -0400, jdratlif at globalnoc.iu.edu wrote: > from keystoneauth1.adapter import Adapter > adapter = Adapter(conn.session, service_type="block-storage") > ? > Adding this and passing adapter instead of conn.session to the commit call > seems to work. You can also pass the proxy object itself. For example: cinder: BlockStorageService = conn.block_storage volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") volume.name = "test_rename" volume.commit(cinder) Normally there would be an proxy method for this called 'update_volume' too. For some reason this doesn't exist. If it did, you could simply do: cinder: BlockStorageService = conn.block_storage volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") cinder.update_volume(volume, name="test_rename") But someone needs to add that proxy method first. Stephen > ? > --- > John Ratliff > Systems Automation Engineer > GlobalNOC @ Indiana University > ? > > From: jdratlif at globalnoc.iu.edu > > Sent: Friday, June 24, 2022 11:44 AM > > To: openstack-discuss at lists.openstack.org > > Cc: Ratliff, John > > Subject: [sdk] renaming a volume with the SDK > > ? > > I?m trying to set a property on a volume, specifically the name. But when I > > try to do this, it tells me the Session object has no default_microversion. > > I have all my auth in environment variables. Using TOKEN authentication. > > ? > > What am I doing wrong here? > > ? > > import openstack > > from openstack.block_storage.v3._proxy import Proxy as BlockStorageService > > from openstack.block_storage.v3.volume import Volume > > ? > > # enable openstack debug logging > > openstack.enable_logging(debug=True) > > ? > > conn = openstack.connect() > > ? > > cinder: BlockStorageService = conn.block_storage > > volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") > > volume.name = "test_rename" > > volume.commit(conn.session) > > ? > > ? > > Thanks, > > ? > > --- > > John Ratliff > > Systems Automation Engineer > > GlobalNOC @ Indiana University From smooney at redhat.com Wed Jun 29 10:12:29 2022 From: smooney at redhat.com (Sean Mooney) Date: Wed, 29 Jun 2022 11:12:29 +0100 Subject: [tc][policy][ptl][operator] OpenSatck RBAC goal is to focus on project personas first (postpone the 'scope' implementation) In-Reply-To: <181abf40999.1291eff02210814.4353892754671729638@ghanshyammann.com> References: <181abf40999.1291eff02210814.4353892754671729638@ghanshyammann.com> Message-ID: On Tue, 2022-06-28 at 15:15 -0500, Ghanshyam Mann wrote: > Hello Everyone, > > We have received a good amount of feedback from the operator. In ops meetup Berlin as well > as from KDDI (Japanese telco). I have summarized the feedback in etherpad[1]. From feedback, > it is clear that 'scope' enable will break heat so do NFV operators and even other operators are also > confused with the 'scope' concept. Most operators want legacy admin to work as it is (able > to do everything in deployment). > > We discussed this feedback in the policy popup meeting[2] and based on feedback and our > outstanding issue of 'scope enable break heat create_stack', we decided to postpone the > `scope` implementation. That is the way forward to at least implement the project personas which > is asked by many operators. Basically the below direction: > > * Finish delivering project personas > This is to introduce the `member` and `reader` roles to operate things within their project. > By default, any other project role like `foo` will not be allowed to do anything in > the project. > > * Postpone the `scope` implementation for later > Some standalone services like Ironic can still have the `scope` implementation as > long as it does not break any cross-service communication. Other services will make sure > they work for project scope personas even with enforce_scope enabled. > > We are not saying 'scope' things are not good or we will never do it but at the same time, I am not sure > when we will do it in future. At least moving this giant goal to focus on the project personas first will > help us to deliver one good feature (project personas) for operators otherwise we are stuck. > > Complete details about the reason to postpone the 'scope' implementation and projects persons > detail are proposed in community-wide goal, please review there. > > - https://review.opendev.org/c/openstack/governance/+/847418 ack ill be sure to read that. just on the topic of scope i do think there is a better way forward then useing scope to have a similar capablity but more flexible and user friendly. i mentioned this on another thread but if we had the abiltiy in keystoen to scope roles to service endpoint that woudl allwo us to achive much of the usecase scope was intended for. e.g. grant the neutron user the admin role scoped to just the nova project so it can call the external events api. once we have the service role we would only grant service role scoped to nova instead of admin similarly we coudl do the same for nova so it has the servivce role on neutron to do port bidnign but no elevated permeission on say keystone or horizon. that woudl allow operators to also subdevice there permissiosn so they could grant admin on nova to the cloud operator to define flavors but no admin rights on say cinder which might be manged by a differnt storage admin user. for a netowrk monitoring systems you could for instnace grant it the reader role on a set of project but scope that role to just neutron apis. if we have the ablity to scope roles by keystone service?cataloge entry and layer that with our ablity to assign rules ot user/groups/application credentials on a per project basis it woudl be a very flexiable sytem to reduce the scope of acces granted to applciation. going one step beyond that instead of haveing system scope we can have a system role that will be delegate the ablity to do the things we planned to make system scoped like creating flavors. admin would be the supper set of system, service, member and reader with extra capablity to work aroucess all tenant as we have today if desired. i honestly think that with a system and serivce roles and the ablity to scope roles to service endpoitn we can do everything we wanted to do with system scope is a much simpler way that is more flexible in the long run. > > [1] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 > [2] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 > > From senrique at redhat.com Wed Jun 29 11:00:00 2022 From: senrique at redhat.com (Sofia Enriquez) Date: Wed, 29 Jun 2022 08:00:00 -0300 Subject: [cinder] Bug deputy report for week of 06-29-2022 Message-ID: This is a bug report from 06-22-2022 to 06-29-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium - https://bugs.launchpad.net/cinder/+bug/1979826 "devstack-plugin-nfs-tempest-full CI job failing." Unassigned. - https://storyboard.openstack.org/#!/story/2009287 "Unexpected exception on ?image create --volume.? Fix proposed to master. Low - https://bugs.launchpad.net/cinder/+bug/1979666 "Dell PowerMax - inconsistency in SG names." Fix proposed to master. - https://bugs.launchpad.net/cinder/+bug/1979667 "Dell PowerMax - creating async bootable volume fails." Fix proposed to master. - https://bugs.launchpad.net/cinder/+bug/1979668 "Dell PowerMax - Volume in multiple SG remains visible as manageable." No fix proposed to master yet. Cheers, Sofia -- Sof?a Enriquez she/her Software Engineer Red Hat PnT IRC: @enriquetaso @RedHat Red Hat Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From amonster369 at gmail.com Wed Jun 29 12:07:15 2022 From: amonster369 at gmail.com (A Monster) Date: Wed, 29 Jun 2022 13:07:15 +0100 Subject: What is the best configuration of network interfaces to deploy openstack Message-ID: So I'm deploying openstack xena using kolla ansible, and i've used infiniband network to connect between the serves of my cluster, however I was wondering whether it is advisable to use a single interface for for " *api_interface*" and "*storage_interface*" and " *tunnel_interface*" or maybe using separate interfaces would be better? Furthermore, I want to ask if using *ipoib *for the infiniband network is a better option than a regular ethernet rg45 network. Thank you all. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amonster369 at gmail.com Wed Jun 29 12:12:09 2022 From: amonster369 at gmail.com (A Monster) Date: Wed, 29 Jun 2022 13:12:09 +0100 Subject: Glance folder to store images Message-ID: I noticied in the glance configuration file after deploying openstack that filesystem_store_datadir=/var/lib/glance/images/ however, upon inspection, I found that images weren't stored in that folder, instead all images where located in this folder */var/lib/docker/volumes/glance/_data/images/.* So I'm thinking if actually filesystem_store_datadir is not used to store images but maybe something else. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Danny.Webb at thehutgroup.com Wed Jun 29 12:47:42 2022 From: Danny.Webb at thehutgroup.com (Danny Webb) Date: Wed, 29 Jun 2022 12:47:42 +0000 Subject: Glance folder to store images In-Reply-To: References: Message-ID: Look at where the volume is mounted in the container with docker inspect glance_api. It's likely mapped to that location in the container from docker volume. ________________________________ From: A Monster Sent: 29 June 2022 13:12 To: openstack-discuss Subject: Glance folder to store images CAUTION: This email originates from outside THG ________________________________ I noticied in the glance configuration file after deploying openstack that filesystem_store_datadir=/var/lib/glance/images/ however, upon inspection, I found that images weren't stored in that folder, instead all images where located in this folder /var/lib/docker/volumes/glance/_data/images/. So I'm thinking if actually filesystem_store_datadir is not used to store images but maybe something else. Danny Webb Principal OpenStack Engineer The Hut Group Tel: Email: Danny.Webb at thehutgroup.com For the purposes of this email, the "company" means The Hut Group Limited, a company registered in England and Wales (company number 6539496) whose registered office is at Fifth Floor, Voyager House, Chicago Avenue, Manchester Airport, M90 3DQ and/or any of its respective subsidiaries. Confidentiality Notice This e-mail is confidential and intended for the use of the named recipient only. If you are not the intended recipient please notify us by telephone immediately on +44(0)1606 811888 or return it to us by e-mail. Please then delete it from your system and note that any use, dissemination, forwarding, printing or copying is strictly prohibited. Any views or opinions are solely those of the author and do not necessarily represent those of the company. Encryptions and Viruses Please note that this e-mail and any attachments have not been encrypted. They may therefore be liable to be compromised. Please also note that it is your responsibility to scan this e-mail and any attachments for viruses. We do not, to the extent permitted by law, accept any liability (whether in contract, negligence or otherwise) for any virus infection and/or external compromise of security and/or confidentiality in relation to transmissions sent by e-mail. Monitoring Activity and use of the company's systems is monitored to secure its effective use and operation and for other lawful business purposes. Communications using these systems will also be monitored and may be recorded to secure effective use and operation and for other lawful business purposes. hgvyjuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Wed Jun 29 13:25:19 2022 From: pierre at stackhpc.com (Pierre Riteau) Date: Wed, 29 Jun 2022 15:25:19 +0200 Subject: Glance folder to store images In-Reply-To: References: Message-ID: You don't say which deployment tool you used to set up OpenStack, but given the content of your message, I assume you are using Kolla. In Kolla, many services have a corresponding volume used to store persistent data. Glance has a glance volume which is used by the glance_api container for the file backend. >From inside the container, the glance service would see this volume mounted at /var/lib/glance. On Wed, 29 Jun 2022 at 14:16, A Monster wrote: > I noticied in the glance configuration file after deploying openstack > that filesystem_store_datadir=/var/lib/glance/images/ however, upon > inspection, I found that images weren't stored in that folder, instead all > images where located in this folder > */var/lib/docker/volumes/glance/_data/images/.* > So I'm thinking if actually filesystem_store_datadir is not used to store > images but maybe something else. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnsomor at gmail.com Wed Jun 29 14:32:57 2022 From: johnsomor at gmail.com (Michael Johnson) Date: Wed, 29 Jun 2022 07:32:57 -0700 Subject: [tc][policy][ptl][operator] OpenSatck RBAC goal is to focus on project personas first (postpone the 'scope' implementation) In-Reply-To: References: <181abf40999.1291eff02210814.4353892754671729638@ghanshyammann.com> Message-ID: I agree. This was more aligned to what John Garbutt and I were advocating for back in 2017 and was implemented in Octavia[1] in Pike. The related proposed spec: https://review.opendev.org/c/openstack/nova-specs/+/427872 Michael [1] https://docs.openstack.org/octavia/latest/configuration/policy.html#octavia-advanced-role-based-access-control-rbac On Wed, Jun 29, 2022 at 3:26 AM Sean Mooney wrote: > > On Tue, 2022-06-28 at 15:15 -0500, Ghanshyam Mann wrote: > > Hello Everyone, > > > > We have received a good amount of feedback from the operator. In ops meetup Berlin as well > > as from KDDI (Japanese telco). I have summarized the feedback in etherpad[1]. From feedback, > > it is clear that 'scope' enable will break heat so do NFV operators and even other operators are also > > confused with the 'scope' concept. Most operators want legacy admin to work as it is (able > > to do everything in deployment). > > > > We discussed this feedback in the policy popup meeting[2] and based on feedback and our > > outstanding issue of 'scope enable break heat create_stack', we decided to postpone the > > `scope` implementation. That is the way forward to at least implement the project personas which > > is asked by many operators. Basically the below direction: > > > > * Finish delivering project personas > > This is to introduce the `member` and `reader` roles to operate things within their project. > > By default, any other project role like `foo` will not be allowed to do anything in > > the project. > > > > * Postpone the `scope` implementation for later > > Some standalone services like Ironic can still have the `scope` implementation as > > long as it does not break any cross-service communication. Other services will make sure > > they work for project scope personas even with enforce_scope enabled. > > > > We are not saying 'scope' things are not good or we will never do it but at the same time, I am not sure > > when we will do it in future. At least moving this giant goal to focus on the project personas first will > > help us to deliver one good feature (project personas) for operators otherwise we are stuck. > > > > Complete details about the reason to postpone the 'scope' implementation and projects persons > > detail are proposed in community-wide goal, please review there. > > > > - https://review.opendev.org/c/openstack/governance/+/847418 > ack ill be sure to read that. > just on the topic of scope i do think there is a better way forward then useing scope to have a similar capablity but more flexible and user friendly. > > i mentioned this on another thread but if we had the abiltiy in keystoen to scope roles to service endpoint that woudl allwo us to achive much of the > usecase scope was intended for. > > e.g. grant the neutron user the admin role scoped to just the nova project so it can call the external events api. > once we have the service role we would only grant service role scoped to nova instead of admin > > similarly we coudl do the same for nova so it has the servivce role on neutron to do port bidnign but no elevated permeission on say keystone or > horizon. > > that woudl allow operators to also subdevice there permissiosn so they could grant admin on nova to the cloud operator to define flavors but no > admin rights on say cinder which might be manged by a differnt storage admin user. > > for a netowrk monitoring systems you could for instnace grant it the reader role on a set of project but scope that role to just neutron apis. > > if we have the ablity to scope roles by keystone service cataloge entry and layer that with our ablity to assign rules ot user/groups/application > credentials on a per project basis it woudl be a very flexiable sytem to reduce the scope of acces granted to applciation. > > going one step beyond that instead of haveing system scope we can have a system role that will be delegate the ablity to do the things we planned > to make system scoped like creating flavors. > > admin would be the supper set of system, service, member and reader with extra capablity to work aroucess all tenant as we have today if desired. > > i honestly think that with a system and serivce roles and the ablity to scope roles to service endpoitn we can do everything we wanted to do with > system scope is a much simpler way that is more flexible in the long run. > > > > [1] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 > > [2] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 > > > > > > From radoslaw.piliszek at gmail.com Wed Jun 29 15:21:32 2022 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Wed, 29 Jun 2022 17:21:32 +0200 Subject: Kolla Train deliverables are going EOL Message-ID: Dear Fellow OpenStackers, Kolla Train deliverables (Kolla, Kolla-Ansible, Kayobe) are going EOL. [1] Ussuri and Victoria remain in extended maintenance. Wallaby, Xena and Yoga are our current stable releases. The primary reason for EOLing is that the branch's CI is badly broken by Python2 and CentOS7 issues and there is no interest in fixing the issues. [1] https://review.opendev.org/c/openstack/releases/+/848148 Kind regards, -yoctozepto From papathanail at uom.edu.gr Wed Jun 29 17:06:42 2022 From: papathanail at uom.edu.gr (GEORGIOS PAPATHANAIL) Date: Wed, 29 Jun 2022 20:06:42 +0300 Subject: Openstack instance is unreachable In-Reply-To: References: <20220628105723.Horde.VEeg85NrLzpcrDTFq6t70GG@webmail.nde.ag> Message-ID: I did the installation based on this https://docs.openstack.org/install-guide/openstack-services.html (queens version) In my previous installation (I use VMWare instead of XenServer) the only thing that I did is to enable promisc mode in VSphere, and the VMs were reachable. I am using ml2 plugin and linuxbridge (default installation) Does it need extra configuration? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Wed Jun 29 17:16:51 2022 From: kennelson11 at gmail.com (Kendall Nelson) Date: Wed, 29 Jun 2022 12:16:51 -0500 Subject: [PTG] [All] [Ops] [PTL] October 2022 Team Signup Message-ID: Hi Everyone, At the Berlin Summit, we announced the next PTG will be held from Monday, October 17th to Thursday, October 20th, 2022 in Columbus, OH! To signup your team, you must complete the survey[1] by August 12th at 7:00 UTC. We NEED accurate contact information for the moderator of your team?s sessions. This is because the survey information will be used to organize the schedule signups which will be done via the PTGBot. If you are not on IRC, please get setup[2] on the OFTC network and join #openinfra-events. You are also encouraged to familiarize yourself with the PTGBot documentation[3] as well. If you have any questions, please reach out! Information about signing up for timeslots will be sent to moderators mid to late August. Registration will open soon! A discounted room block at the PTG venue will be available. Continue to visit openinfra.dev/ptg for updates. -Kendall (diablo_rojo) [1] Team Survey: https://openinfrafoundation.formstack.com/forms/oct2022_ptg_team_signup [2] Setup IRC: https://docs.openstack.org/contributors/common/irc.html [3] PTGBot README: https://opendev.org/openstack/ptgbot/src/branch/master/README.rst -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Wed Jun 29 18:25:11 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Wed, 29 Jun 2022 20:25:11 +0200 Subject: [tc][policy][ptl][operator] OpenSatck RBAC goal is to focus on project personas first (postpone the 'scope' implementation) In-Reply-To: References: <181abf40999.1291eff02210814.4353892754671729638@ghanshyammann.com> Message-ID: FWIW, there was also discussion during the summit [1], where the idea was to create accounts that would be limited to pretty specific actions for services, when interacting with others. For example, nova can create port in neutron, but not manage security group rules. Which is definitely next step to what is being discussed here. Unfortunately I lost etherpad for the forum, but main suggestion there was to join biweekly meetings or just reach Ghanshyam for further discussion, as basically this topic is more suitable for PTG rather then Summit. Not sure if anyone did reach out though. [1] https://openinfra.dev/summit-schedule ??, 29 ???. 2022 ?., 16:36 Michael Johnson : > I agree. This was more aligned to what John Garbutt and I were > advocating for back in 2017 and was implemented in Octavia[1] in Pike. > > The related proposed spec: > https://review.opendev.org/c/openstack/nova-specs/+/427872 > > Michael > > [1] > https://docs.openstack.org/octavia/latest/configuration/policy.html#octavia-advanced-role-based-access-control-rbac > > On Wed, Jun 29, 2022 at 3:26 AM Sean Mooney wrote: > > > > On Tue, 2022-06-28 at 15:15 -0500, Ghanshyam Mann wrote: > > > Hello Everyone, > > > > > > We have received a good amount of feedback from the operator. In ops > meetup Berlin as well > > > as from KDDI (Japanese telco). I have summarized the feedback in > etherpad[1]. From feedback, > > > it is clear that 'scope' enable will break heat so do NFV operators > and even other operators are also > > > confused with the 'scope' concept. Most operators want legacy admin to > work as it is (able > > > to do everything in deployment). > > > > > > We discussed this feedback in the policy popup meeting[2] and based on > feedback and our > > > outstanding issue of 'scope enable break heat create_stack', we > decided to postpone the > > > `scope` implementation. That is the way forward to at least implement > the project personas which > > > is asked by many operators. Basically the below direction: > > > > > > * Finish delivering project personas > > > This is to introduce the `member` and `reader` roles to operate > things within their project. > > > By default, any other project role like `foo` will not be allowed to > do anything in > > > the project. > > > > > > * Postpone the `scope` implementation for later > > > Some standalone services like Ironic can still have the `scope` > implementation as > > > long as it does not break any cross-service communication. Other > services will make sure > > > they work for project scope personas even with enforce_scope enabled. > > > > > > We are not saying 'scope' things are not good or we will never do it > but at the same time, I am not sure > > > when we will do it in future. At least moving this giant goal to focus > on the project personas first will > > > help us to deliver one good feature (project personas) for operators > otherwise we are stuck. > > > > > > Complete details about the reason to postpone the 'scope' > implementation and projects persons > > > detail are proposed in community-wide goal, please review there. > > > > > > - https://review.opendev.org/c/openstack/governance/+/847418 > > ack ill be sure to read that. > > just on the topic of scope i do think there is a better way forward then > useing scope to have a similar capablity but more flexible and user > friendly. > > > > i mentioned this on another thread but if we had the abiltiy in keystoen > to scope roles to service endpoint that woudl allwo us to achive much of the > > usecase scope was intended for. > > > > e.g. grant the neutron user the admin role scoped to just the nova > project so it can call the external events api. > > once we have the service role we would only grant service role scoped to > nova instead of admin > > > > similarly we coudl do the same for nova so it has the servivce role on > neutron to do port bidnign but no elevated permeission on say keystone or > > horizon. > > > > that woudl allow operators to also subdevice there permissiosn so they > could grant admin on nova to the cloud operator to define flavors but no > > admin rights on say cinder which might be manged by a differnt storage > admin user. > > > > for a netowrk monitoring systems you could for instnace grant it the > reader role on a set of project but scope that role to just neutron apis. > > > > if we have the ablity to scope roles by keystone service cataloge entry > and layer that with our ablity to assign rules ot user/groups/application > > credentials on a per project basis it woudl be a very flexiable sytem to > reduce the scope of acces granted to applciation. > > > > going one step beyond that instead of haveing system scope we can have a > system role that will be delegate the ablity to do the things we planned > > to make system scoped like creating flavors. > > > > admin would be the supper set of system, service, member and reader with > extra capablity to work aroucess all tenant as we have today if desired. > > > > i honestly think that with a system and serivce roles and the ablity to > scope roles to service endpoitn we can do everything we wanted to do with > > system scope is a much simpler way that is more flexible in the long run. > > > > > > [1] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 > > > [2] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ken at jots.org Wed Jun 29 18:36:51 2022 From: ken at jots.org (Ken D'Ambrosio) Date: Wed, 29 Jun 2022 14:36:51 -0400 Subject: Rocky? Stream? Message-ID: <1c5e6be4b0d16f7c5e1397f00eed7a54@jots.org> Hey, all. My company's looking for the path forward, after the switcheroo with CentOS, and we're wondering what seems to be solidly supported now and in the future. Any ideas? Thanks! -Ken From gmann at ghanshyammann.com Wed Jun 29 20:41:47 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 29 Jun 2022 15:41:47 -0500 Subject: [tc][policy][ptl][operator] OpenSatck RBAC goal is to focus on project personas first (postpone the 'scope' implementation) In-Reply-To: References: <181abf40999.1291eff02210814.4353892754671729638@ghanshyammann.com> Message-ID: <181b132d62e.d0ff1fdc40310.1505578106462404278@ghanshyammann.com> ---- On Wed, 29 Jun 2022 05:12:29 -0500 Sean Mooney wrote --- > On Tue, 2022-06-28 at 15:15 -0500, Ghanshyam Mann wrote: > > Hello Everyone, > > > > We have received a good amount of feedback from the operator. In ops meetup Berlin as well > > as from KDDI (Japanese telco). I have summarized the feedback in etherpad[1]. From feedback, > > it is clear that 'scope' enable will break heat so do NFV operators and even other operators are also > > confused with the 'scope' concept. Most operators want legacy admin to work as it is (able > > to do everything in deployment). > > > > We discussed this feedback in the policy popup meeting[2] and based on feedback and our > > outstanding issue of 'scope enable break heat create_stack', we decided to postpone the > > `scope` implementation. That is the way forward to at least implement the project personas which > > is asked by many operators. Basically the below direction: > > > > * Finish delivering project personas > > This is to introduce the `member` and `reader` roles to operate things within their project. > > By default, any other project role like `foo` will not be allowed to do anything in > > the project. > > > > * Postpone the `scope` implementation for later > > Some standalone services like Ironic can still have the `scope` implementation as > > long as it does not break any cross-service communication. Other services will make sure > > they work for project scope personas even with enforce_scope enabled. > > > > We are not saying 'scope' things are not good or we will never do it but at the same time, I am not sure > > when we will do it in future. At least moving this giant goal to focus on the project personas first will > > help us to deliver one good feature (project personas) for operators otherwise we are stuck. > > > > Complete details about the reason to postpone the 'scope' implementation and projects persons > > detail are proposed in community-wide goal, please review there. > > > > - https://review.opendev.org/c/openstack/governance/+/847418 > ack ill be sure to read that. > just on the topic of scope i do think there is a better way forward then useing scope to have a similar capablity but more flexible and user friendly. > > i mentioned this on another thread but if we had the abiltiy in keystoen to scope roles to service endpoint that woudl allwo us to achive much of the > usecase scope was intended for. > > e.g. grant the neutron user the admin role scoped to just the nova project so it can call the external events api. > once we have the service role we would only grant service role scoped to nova instead of admin > > similarly we coudl do the same for nova so it has the servivce role on neutron to do port bidnign but no elevated permeission on say keystone or > horizon. > > that woudl allow operators to also subdevice there permissiosn so they could grant admin on nova to the cloud operator to define flavors but no > admin rights on say cinder which might be manged by a differnt storage admin user. > > for a netowrk monitoring systems you could for instnace grant it the reader role on a set of project but scope that role to just neutron apis. > > if we have the ablity to scope roles by keystone service cataloge entry and layer that with our ablity to assign rules ot user/groups/application > credentials on a per project basis it woudl be a very flexiable sytem to reduce the scope of acces granted to applciation. > > going one step beyond that instead of haveing system scope we can have a system role that will be delegate the ablity to do the things we planned > to make system scoped like creating flavors. > > admin would be the supper set of system, service, member and reader with extra capablity to work aroucess all tenant as we have today if desired. > > i honestly think that with a system and serivce roles and the ablity to scope roles to service endpoitn we can do everything we wanted to do with > system scope is a much simpler way that is more flexible in the long run. The key feedback from operators is that they only care about the admin being able to do things in complete deployment and reader roles. I am not sure splitting admin per service will be welcome compared to scope. Also, heat and NFV users will face the same issue they are currently facing with scope enable, they will require their user token to have an admin role to all the services they need which are mostly many. Anyways, let's focus on one thing at a time and as per this goal project reader is our priority otherwise this most needed role by the operator keep getting postponed because we want to solve many things together. -gmann > > > > [1] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 > > [2] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 > > > > > > > From gmann at ghanshyammann.com Wed Jun 29 20:47:16 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 29 Jun 2022 15:47:16 -0500 Subject: [tc][policy][ptl][operator] OpenSatck RBAC goal is to focus on project personas first (postpone the 'scope' implementation) In-Reply-To: References: <181abf40999.1291eff02210814.4353892754671729638@ghanshyammann.com> Message-ID: <181b137d86a.f00e1f9240416.1151705308599937073@ghanshyammann.com> ---- On Wed, 29 Jun 2022 13:25:11 -0500 Dmitriy Rabotyagov wrote --- > FWIW, there was also discussion during the summit [1], where the idea was to create accounts that would be limited to pretty specific actions for services, when interacting with others.For example, nova can create port in neutron, but not manage security group rules. Which is definitely next step to what is being discussed here. > Unfortunately I lost etherpad for the forum, but main suggestion there was to join biweekly meetings or just reach Ghanshyam for further discussion, as basically this topic is more suitable for PTG rather then Summit. Not sure if anyone did reach out though. I think you mean the 'service' role? and this etherpad https://etherpad.opendev.org/p/deprivilization-of-service-accounts ? We have that planned for phase 2 - https://review.opendev.org/c/openstack/governance/+/847418/5/goals/selected/consistent-and-secure-rbac.rst#503 And I have an action item to continue on Lance spec and update it once we finalized the goal documentation. - https://review.opendev.org/c/openstack/keystone-specs/+/818616/2 But yes, please join policy popup biweekly call and discuss it - https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popup_Team#Meeting -gmann > > [1] https://openinfra.dev/summit-schedule > > ??, 29 ???. 2022 ?., 16:36 Michael Johnson : > I agree. This was more aligned to what John Garbutt and I were > advocating for back in 2017 and was implemented in Octavia[1] in Pike. > > The related proposed spec: > https://review.opendev.org/c/openstack/nova-specs/+/427872 > > Michael > > [1] https://docs.openstack.org/octavia/latest/configuration/policy.html#octavia-advanced-role-based-access-control-rbac > > On Wed, Jun 29, 2022 at 3:26 AM Sean Mooney wrote: > > > > On Tue, 2022-06-28 at 15:15 -0500, Ghanshyam Mann wrote: > > > Hello Everyone, > > > > > > We have received a good amount of feedback from the operator. In ops meetup Berlin as well > > > as from KDDI (Japanese telco). I have summarized the feedback in etherpad[1]. From feedback, > > > it is clear that 'scope' enable will break heat so do NFV operators and even other operators are also > > > confused with the 'scope' concept. Most operators want legacy admin to work as it is (able > > > to do everything in deployment). > > > > > > We discussed this feedback in the policy popup meeting[2] and based on feedback and our > > > outstanding issue of 'scope enable break heat create_stack', we decided to postpone the > > > `scope` implementation. That is the way forward to at least implement the project personas which > > > is asked by many operators. Basically the below direction: > > > > > > * Finish delivering project personas > > > This is to introduce the `member` and `reader` roles to operate things within their project. > > > By default, any other project role like `foo` will not be allowed to do anything in > > > the project. > > > > > > * Postpone the `scope` implementation for later > > > Some standalone services like Ironic can still have the `scope` implementation as > > > long as it does not break any cross-service communication. Other services will make sure > > > they work for project scope personas even with enforce_scope enabled. > > > > > > We are not saying 'scope' things are not good or we will never do it but at the same time, I am not sure > > > when we will do it in future. At least moving this giant goal to focus on the project personas first will > > > help us to deliver one good feature (project personas) for operators otherwise we are stuck. > > > > > > Complete details about the reason to postpone the 'scope' implementation and projects persons > > > detail are proposed in community-wide goal, please review there. > > > > > > - https://review.opendev.org/c/openstack/governance/+/847418 > > ack ill be sure to read that. > > just on the topic of scope i do think there is a better way forward then useing scope to have a similar capablity but more flexible and user friendly. > > > > i mentioned this on another thread but if we had the abiltiy in keystoen to scope roles to service endpoint that woudl allwo us to achive much of the > > usecase scope was intended for. > > > > e.g. grant the neutron user the admin role scoped to just the nova project so it can call the external events api. > > once we have the service role we would only grant service role scoped to nova instead of admin > > > > similarly we coudl do the same for nova so it has the servivce role on neutron to do port bidnign but no elevated permeission on say keystone or > > horizon. > > > > that woudl allow operators to also subdevice there permissiosn so they could grant admin on nova to the cloud operator to define flavors but no > > admin rights on say cinder which might be manged by a differnt storage admin user. > > > > for a netowrk monitoring systems you could for instnace grant it the reader role on a set of project but scope that role to just neutron apis. > > > > if we have the ablity to scope roles by keystone service cataloge entry and layer that with our ablity to assign rules ot user/groups/application > > credentials on a per project basis it woudl be a very flexiable sytem to reduce the scope of acces granted to applciation. > > > > going one step beyond that instead of haveing system scope we can have a system role that will be delegate the ablity to do the things we planned > > to make system scoped like creating flavors. > > > > admin would be the supper set of system, service, member and reader with extra capablity to work aroucess all tenant as we have today if desired. > > > > i honestly think that with a system and serivce roles and the ablity to scope roles to service endpoitn we can do everything we wanted to do with > > system scope is a much simpler way that is more flexible in the long run. > > > > > > [1] https://etherpad.opendev.org/p/rbac-operator-feedback#L45 > > > [2] https://etherpad.opendev.org/p/rbac-zed-ptg#L171 > > > > > > > > > > > > From gmann at ghanshyammann.com Thu Jun 30 02:05:46 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Wed, 29 Jun 2022 21:05:46 -0500 Subject: [all][tc] Technical Committee next weekly meeting on 30 June 2022 at 1500 UTC In-Reply-To: <181a6ba323f.104aa718f133256.6630300845350952590@ghanshyammann.com> References: <181a6ba323f.104aa718f133256.6630300845350952590@ghanshyammann.com> Message-ID: <181b25b7113.10874200344021.2016151432085362344@ghanshyammann.com> Hello Everyone, Below is the agenda for Today's TC IRC meeting schedule at 1500 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting == Agenda for tomorrow's TC meeting == * Roll call * Follow up on past action items * Gate health check ** Bare 'recheck' state *** https://etherpad.opendev.org/p/recheck-weekly-summary * New ELK service dashboard: e-r service ** https://opensearch.logs.openstack.org/_dashboards/app/discover?security_tenant=global ** https://review.opendev.org/c/openstack/governance-sigs/+/835838 * RBAC feedback in ops meetup ** https://etherpad.opendev.org/p/rbac-zed-ptg#L171 ** https://review.opendev.org/c/openstack/governance/+/847418 * Create the Environmental Sustainability SIG ** https://review.opendev.org/c/openstack/governance-sigs/+/845336 * Open Reviews ** https://review.opendev.org/q/projects:openstack/governance+is:open -gmann ---- On Mon, 27 Jun 2022 14:53:49 -0500 Ghanshyam Mann wrote --- > Hello Everyone, > > The technical Committee's next weekly meeting is scheduled for 30 June 2022, at 1500 UTC. > > If you would like to add topics for discussion, please add them to the below wiki page by > Wednesday, 29 June at 2100 UTC. > > https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting > > -gmann > > > From massimo.sgaravatto at gmail.com Thu Jun 30 08:09:44 2022 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Thu, 30 Jun 2022 10:09:44 +0200 Subject: [glance][ops] [nova] Disabling an image Message-ID: Dear all What is the blessed method to avoid using an image for new virtual machines without causing problems for existing instances using that image ? If I deactivate the image, I then have problems resizing instances using that image [*]: it claims that image download is forbidden since the image was deactivated Thanks, Massimo [*] | fault | {'code': 500, 'created': '2022-06-30T07:57:30Z', 'message': 'Not authorized for image dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in download\n context, 2, \'data\', args=(image_id,))\n File "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in call\n result = getattr(controller, method)(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line 670, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, in data\n resp, body = self.http_client.get(url)\n File "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in get\n return self.request(url, \'GET\', **kwargs)\n File "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, in request\n return self._handle_response(resp)\n File "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, in _handle_response\n raise exc.from_response(resp, resp.content)\nglanceclient.exc.HTTPForbidden: HTTP 403 Forbidden: The requested image has been deactivated. Image data download is forbidden.\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 201, in decorated_function\n return function(self, context, *args, **kwargs)\n File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5950, in finish_resize\n context, instance, migration)\n File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5932, in finish_resize\n migration, request_spec)\n File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5966, in _finish_resize_helper\n request_spec)\n File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5902, in _finish_resize\n self._set_instance_info(instance, old_flavor)\n File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5890, in _finish_resize\n block_device_info, power_on)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 11343, in finish_migration\n fallback_from_host=migration.source_compute)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4703, in _create_image\n injection_info, fallback_from_host)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4831, in _create_and_inject_local_root\n instance, size, fallback_from_host)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 10625, in _try_fetch_image_cache\n trusted_certs=instance.trusted_certs)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 275, in cache\n *args, **kwargs)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 638, in create_image\n prepare_template(target=base, *args, **kwargs)\n File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 391, in inner\n return f(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 271, in fetch_func_sync\n fetch_func(target=target, *args, **kwargs)\n File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/utils.py", line 395, in fetch_image\n images.fetch_to_raw(context, image_id, target, trusted_certs)\n File "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 115, in fetch_to_raw\n fetch(context, image_href, path_tmp, trusted_certs)\n File "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 106, in fetch\n trusted_certs=trusted_certs)\n File "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1300, in download\n trusted_certs=trusted_certs)\n File "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 379, in download\n _reraise_translated_image_exception(image_id)\n File "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1031, in _reraise_translated_image_exception\n raise new_exc.with_traceback(exc_trace)\n File "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in download\n context, 2, \'data\', args=(image_id,))\n File "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in call\n result = getattr(controller, method)(*args, **kwargs)\n File "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line 670, in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, in data\n resp, body = self.http_client.get(url)\n File "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in get\n return self.request(url, \'GET\', **kwargs)\n File "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, in request\n return self._handle_response(resp)\n File "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, in _handle_response\n raise exc.from_response(resp, resp.content)\nnova.exception.ImageNotAuthorized: Not authorized for image dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.\n'} | -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Thu Jun 30 09:07:32 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 30 Jun 2022 11:07:32 +0200 Subject: [all][TC] Stats about rechecking patches without reason given Message-ID: <2224274.SIoALSJNh4@p1> Hi, During the last PTG and after it, in the TC we were discussing about CI resources usage and about "rechecks" of the CI jobs (I know, it's again the same topic). One of the things we would like to limit, or even avoid is to do "no reason rechecks" which means writing quick comment "recheck" without checking what really was wrong in the previous run. We know that putting some hard rules that only comments with "recheck" with given reason will trigger new CI jobs run will not work fine as people may simply start writing any random things there. But we want to encourage all teams to at least to investigate failures and do as many rechecks with explanation as possible. For now I prepared simple script [1] which counts how much of all rechecks are "bare rechecks". It can be checked by project (like openstack/neutron) or give summary for all projects or teams (like Quality Assurance for example). I prepared some stats for all teams listed in the https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml[1] from last 30 days: +-------------------+---------------+--------------+-------------------+ | Team | Bare rechecks | All Rechecks | Bare rechecks [%] | +-------------------+---------------+--------------+-------------------+ | skyline | 20 | 20 | 100.0 | | magnum | 2 | 2 | 100.0 | | zun | 1 | 1 | 100.0 | | mistral | 9 | 9 | 100.0 | | ec2-api | 1 | 1 | 100.0 | | barbican | 15 | 15 | 100.0 | | venus | 2 | 2 | 100.0 | | solum | 1 | 1 | 100.0 | | tacker | 30 | 30 | 100.0 | | trove | 4 | 4 | 100.0 | | rally | 2 | 2 | 100.0 | | storlets | 5 | 5 | 100.0 | | winstackers | 3 | 3 | 100.0 | | OpenStack Charms | 32 | 33 | 96.97 | | sahara | 27 | 28 | 96.43 | | keystone | 24 | 25 | 96.0 | | kuryr | 120 | 126 | 95.24 | | kolla | 134 | 142 | 94.37 | | Puppet OpenStack | 94 | 103 | 91.26 | | cloudkitty | 10 | 11 | 90.91 | | OpenStack-Helm | 29 | 32 | 90.62 | | blazar | 8 | 9 | 88.89 | | tripleo | 563 | 646 | 87.15 | | requirements | 20 | 23 | 86.96 | | Telemetry | 30 | 35 | 85.71 | | horizon | 55 | 67 | 82.09 | | ironic | 131 | 164 | 79.88 | | oslo | 11 | 14 | 78.57 | | heat | 25 | 33 | 75.76 | | cinder | 221 | 294 | 75.17 | | cyborg | 6 | 8 | 75.0 | | murano | 3 | 4 | 75.0 | | glance | 20 | 27 | 74.07 | | OpenStackSDK | 47 | 64 | 73.44 | | manila | 108 | 160 | 67.5 | | neutron | 149 | 221 | 67.42 | | senlin | 2 | 3 | 66.67 | | swift | 16 | 25 | 64.0 | | Quality Assurance | 106 | 167 | 63.47 | | nova | 41 | 71 | 57.75 | | octavia | 32 | 60 | 53.33 | | designate | 19 | 39 | 48.72 | | OpenStackAnsible | 41 | 226 | 18.14 | +-------------------+---------------+--------------+-------------------+ As You can see from that list above, there is much to improve there. I hope that if teams will be checking more reasons of the CI failures, and reporting bugs found there, we may make our CI more stable and as a result have less rechecks which will save our infra resources :) [1] https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/bare_rechecks.py[2] -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml [2] https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/bare_rechecks.py -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From smooney at redhat.com Thu Jun 30 11:42:31 2022 From: smooney at redhat.com (Sean Mooney) Date: Thu, 30 Jun 2022 12:42:31 +0100 Subject: [glance][ops] [nova] Disabling an image In-Reply-To: References: Message-ID: <259825a80b6cce7df2743acf6792ad4c598019ab.camel@redhat.com> On Thu, 2022-06-30 at 10:09 +0200, Massimo Sgaravatto wrote: > Dear all > > What is the blessed method to avoid using an image for new virtual machines > without causing problems for existing instances using that image ? > > If I deactivate the image, I then have problems resizing instances using > that image [*]: it claims that image download is forbidden since the image > was deactivated i think you mean rebuilding the instance not resizeing right? resize should not need the image since it should use the image info we embed in the nova in the instance_system_metadata table. im not sure if there is a blessed way but i proably would have changed the visablity to private. > > Thanks, Massimo > > [*] > > > | fault | {'code': 500, 'created': > '2022-06-30T07:57:30Z', 'message': 'Not authorized for image > dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.', 'details': 'Traceback (most recent > call last):\n File > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in > download\n context, 2, \'data\', args=(image_id,))\n File > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in > call\n result = getattr(controller, method)(*args, **kwargs)\n File > "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line 670, > in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File > "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, in > data\n resp, body = self.http_client.get(url)\n File > "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in > get\n return self.request(url, \'GET\', **kwargs)\n File > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, > in request\n return self._handle_response(resp)\n File > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, > in _handle_response\n raise exc.from_response(resp, > resp.content)\nglanceclient.exc.HTTPForbidden: HTTP 403 Forbidden: The > requested image has been deactivated. Image data download is > forbidden.\n\nDuring handling of the above exception, another exception > occurred:\n\nTraceback (most recent call last):\n File > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 201, in > decorated_function\n return function(self, context, *args, **kwargs)\n > File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line > 5950, in finish_resize\n context, instance, migration)\n File > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in > __exit__\n self.force_reraise()\n File > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in > force_reraise\n raise self.value\n File > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5932, in > finish_resize\n migration, request_spec)\n File > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5966, in > _finish_resize_helper\n request_spec)\n File > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5902, in > _finish_resize\n self._set_instance_info(instance, old_flavor)\n File > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in > __exit__\n self.force_reraise()\n File > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in > force_reraise\n raise self.value\n File > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5890, in > _finish_resize\n block_device_info, power_on)\n File > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 11343, > in finish_migration\n fallback_from_host=migration.source_compute)\n > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line > 4703, in _create_image\n injection_info, fallback_from_host)\n File > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4831, > in _create_and_inject_local_root\n instance, size, fallback_from_host)\n > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line > 10625, in _try_fetch_image_cache\n > trusted_certs=instance.trusted_certs)\n File > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line > 275, in cache\n *args, **kwargs)\n File > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line > 638, in create_image\n prepare_template(target=base, *args, **kwargs)\n > File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", > line 391, in inner\n return f(*args, **kwargs)\n File > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line > 271, in fetch_func_sync\n fetch_func(target=target, *args, **kwargs)\n > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/utils.py", line > 395, in fetch_image\n images.fetch_to_raw(context, image_id, target, > trusted_certs)\n File > "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 115, in > fetch_to_raw\n fetch(context, image_href, path_tmp, trusted_certs)\n > File "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 106, in > fetch\n trusted_certs=trusted_certs)\n File > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1300, in > download\n trusted_certs=trusted_certs)\n File > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 379, in > download\n _reraise_translated_image_exception(image_id)\n File > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1031, in > _reraise_translated_image_exception\n raise > new_exc.with_traceback(exc_trace)\n File > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in > download\n context, 2, \'data\', args=(image_id,))\n File > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in > call\n result = getattr(controller, method)(*args, **kwargs)\n File > "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line 670, > in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File > "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, in > data\n resp, body = self.http_client.get(url)\n File > "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in > get\n return self.request(url, \'GET\', **kwargs)\n File > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, > in request\n return self._handle_response(resp)\n File > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, > in _handle_response\n raise exc.from_response(resp, > resp.content)\nnova.exception.ImageNotAuthorized: Not authorized for image > dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.\n'} | From massimo.sgaravatto at gmail.com Thu Jun 30 12:37:53 2022 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Thu, 30 Jun 2022 14:37:53 +0200 Subject: [glance][ops] [nova] Disabling an image In-Reply-To: <259825a80b6cce7df2743acf6792ad4c598019ab.camel@redhat.com> References: <259825a80b6cce7df2743acf6792ad4c598019ab.camel@redhat.com> Message-ID: No: I really mean resize On Thu, Jun 30, 2022 at 1:42 PM Sean Mooney wrote: > On Thu, 2022-06-30 at 10:09 +0200, Massimo Sgaravatto wrote: > > Dear all > > > > What is the blessed method to avoid using an image for new virtual > machines > > without causing problems for existing instances using that image ? > > > > If I deactivate the image, I then have problems resizing instances using > > that image [*]: it claims that image download is forbidden since the > image > > was deactivated > i think you mean rebuilding the instance not resizeing right? > resize should not need the image since it should use the image info we > embed in the nova > in the instance_system_metadata table. > > im not sure if there is a blessed way but i proably would have changed the > visablity to private. > > > > > > Thanks, Massimo > > > > [*] > > > > > > | fault | {'code': 500, 'created': > > '2022-06-30T07:57:30Z', 'message': 'Not authorized for image > > dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.', 'details': 'Traceback (most > recent > > call last):\n File > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in > > download\n context, 2, \'data\', args=(image_id,))\n File > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in > > call\n result = getattr(controller, method)(*args, **kwargs)\n File > > "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line > 670, > > in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File > > "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, > in > > data\n resp, body = self.http_client.get(url)\n File > > "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in > > get\n return self.request(url, \'GET\', **kwargs)\n File > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, > > in request\n return self._handle_response(resp)\n File > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, > > in _handle_response\n raise exc.from_response(resp, > > resp.content)\nglanceclient.exc.HTTPForbidden: HTTP 403 Forbidden: The > > requested image has been deactivated. Image data download is > > forbidden.\n\nDuring handling of the above exception, another exception > > occurred:\n\nTraceback (most recent call last):\n File > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 201, in > > decorated_function\n return function(self, context, *args, **kwargs)\n > > File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line > > 5950, in finish_resize\n context, instance, migration)\n File > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in > > __exit__\n self.force_reraise()\n File > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in > > force_reraise\n raise self.value\n File > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5932, in > > finish_resize\n migration, request_spec)\n File > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5966, in > > _finish_resize_helper\n request_spec)\n File > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5902, in > > _finish_resize\n self._set_instance_info(instance, old_flavor)\n File > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in > > __exit__\n self.force_reraise()\n File > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in > > force_reraise\n raise self.value\n File > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5890, in > > _finish_resize\n block_device_info, power_on)\n File > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line > 11343, > > in finish_migration\n fallback_from_host=migration.source_compute)\n > > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", > line > > 4703, in _create_image\n injection_info, fallback_from_host)\n File > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line > 4831, > > in _create_and_inject_local_root\n instance, size, > fallback_from_host)\n > > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", > line > > 10625, in _try_fetch_image_cache\n > > trusted_certs=instance.trusted_certs)\n File > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", > line > > 275, in cache\n *args, **kwargs)\n File > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", > line > > 638, in create_image\n prepare_template(target=base, *args, > **kwargs)\n > > File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", > > line 391, in inner\n return f(*args, **kwargs)\n File > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", > line > > 271, in fetch_func_sync\n fetch_func(target=target, *args, **kwargs)\n > > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/utils.py", line > > 395, in fetch_image\n images.fetch_to_raw(context, image_id, target, > > trusted_certs)\n File > > "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 115, in > > fetch_to_raw\n fetch(context, image_href, path_tmp, trusted_certs)\n > > File "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 106, > in > > fetch\n trusted_certs=trusted_certs)\n File > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1300, in > > download\n trusted_certs=trusted_certs)\n File > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 379, in > > download\n _reraise_translated_image_exception(image_id)\n File > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1031, in > > _reraise_translated_image_exception\n raise > > new_exc.with_traceback(exc_trace)\n File > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in > > download\n context, 2, \'data\', args=(image_id,))\n File > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in > > call\n result = getattr(controller, method)(*args, **kwargs)\n File > > "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line > 670, > > in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File > > "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, > in > > data\n resp, body = self.http_client.get(url)\n File > > "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in > > get\n return self.request(url, \'GET\', **kwargs)\n File > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, > > in request\n return self._handle_response(resp)\n File > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, > > in _handle_response\n raise exc.from_response(resp, > > resp.content)\nnova.exception.ImageNotAuthorized: Not authorized for > image > > dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.\n'} | > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Jun 30 12:55:58 2022 From: smooney at redhat.com (Sean Mooney) Date: Thu, 30 Jun 2022 13:55:58 +0100 Subject: [glance][ops] [nova] Disabling an image In-Reply-To: References: <259825a80b6cce7df2743acf6792ad4c598019ab.camel@redhat.com> Message-ID: On Thu, 2022-06-30 at 14:37 +0200, Massimo Sgaravatto wrote: > No: I really mean resize i guess for resize we need to pcy the backing file which we preusmabel? are doing by redownloading the orginal image. it could technically be copied form the souce host instead but i think if you change the visiableity rahter then blocking download that would hide it form peopel lookign to create new vms with it in the image list but allow it to consiute to be used by exsiting instace for rebuild and resize. > > On Thu, Jun 30, 2022 at 1:42 PM Sean Mooney wrote: > > > On Thu, 2022-06-30 at 10:09 +0200, Massimo Sgaravatto wrote: > > > Dear all > > > > > > What is the blessed method to avoid using an image for new virtual > > machines > > > without causing problems for existing instances using that image ? > > > > > > If I deactivate the image, I then have problems resizing instances using > > > that image [*]: it claims that image download is forbidden since the > > image > > > was deactivated > > i think you mean rebuilding the instance not resizeing right? > > resize should not need the image since it should use the image info we > > embed in the nova > > in the instance_system_metadata table. > > > > im not sure if there is a blessed way but i proably would have changed the > > visablity to private. > > > > > > > > > > Thanks, Massimo > > > > > > [*] > > > > > > > > > | fault | {'code': 500, 'created': > > > '2022-06-30T07:57:30Z', 'message': 'Not authorized for image > > > dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.', 'details': 'Traceback (most > > recent > > > call last):\n File > > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in > > > download\n context, 2, \'data\', args=(image_id,))\n File > > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in > > > call\n result = getattr(controller, method)(*args, **kwargs)\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line > > 670, > > > in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, > > in > > > data\n resp, body = self.http_client.get(url)\n File > > > "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in > > > get\n return self.request(url, \'GET\', **kwargs)\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, > > > in request\n return self._handle_response(resp)\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, > > > in _handle_response\n raise exc.from_response(resp, > > > resp.content)\nglanceclient.exc.HTTPForbidden: HTTP 403 Forbidden: The > > > requested image has been deactivated. Image data download is > > > forbidden.\n\nDuring handling of the above exception, another exception > > > occurred:\n\nTraceback (most recent call last):\n File > > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 201, in > > > decorated_function\n return function(self, context, *args, **kwargs)\n > > > File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line > > > 5950, in finish_resize\n context, instance, migration)\n File > > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in > > > __exit__\n self.force_reraise()\n File > > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in > > > force_reraise\n raise self.value\n File > > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5932, in > > > finish_resize\n migration, request_spec)\n File > > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5966, in > > > _finish_resize_helper\n request_spec)\n File > > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5902, in > > > _finish_resize\n self._set_instance_info(instance, old_flavor)\n File > > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in > > > __exit__\n self.force_reraise()\n File > > > "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in > > > force_reraise\n raise self.value\n File > > > "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 5890, in > > > _finish_resize\n block_device_info, power_on)\n File > > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line > > 11343, > > > in finish_migration\n fallback_from_host=migration.source_compute)\n > > > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", > > line > > > 4703, in _create_image\n injection_info, fallback_from_host)\n File > > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line > > 4831, > > > in _create_and_inject_local_root\n instance, size, > > fallback_from_host)\n > > > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", > > line > > > 10625, in _try_fetch_image_cache\n > > > trusted_certs=instance.trusted_certs)\n File > > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", > > line > > > 275, in cache\n *args, **kwargs)\n File > > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", > > line > > > 638, in create_image\n prepare_template(target=base, *args, > > **kwargs)\n > > > File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", > > > line 391, in inner\n return f(*args, **kwargs)\n File > > > "/usr/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", > > line > > > 271, in fetch_func_sync\n fetch_func(target=target, *args, **kwargs)\n > > > File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/utils.py", line > > > 395, in fetch_image\n images.fetch_to_raw(context, image_id, target, > > > trusted_certs)\n File > > > "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 115, in > > > fetch_to_raw\n fetch(context, image_href, path_tmp, trusted_certs)\n > > > File "/usr/lib/python3.6/site-packages/nova/virt/images.py", line 106, > > in > > > fetch\n trusted_certs=trusted_certs)\n File > > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1300, in > > > download\n trusted_certs=trusted_certs)\n File > > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 379, in > > > download\n _reraise_translated_image_exception(image_id)\n File > > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 1031, in > > > _reraise_translated_image_exception\n raise > > > new_exc.with_traceback(exc_trace)\n File > > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 377, in > > > download\n context, 2, \'data\', args=(image_id,))\n File > > > "/usr/lib/python3.6/site-packages/nova/image/glance.py", line 191, in > > > call\n result = getattr(controller, method)(*args, **kwargs)\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/common/utils.py", line > > 670, > > > in inner\n return RequestIdProxy(wrapped(*args, **kwargs))\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/v2/images.py", line 255, > > in > > > data\n resp, body = self.http_client.get(url)\n File > > > "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 395, in > > > get\n return self.request(url, \'GET\', **kwargs)\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 380, > > > in request\n return self._handle_response(resp)\n File > > > "/usr/lib/python3.6/site-packages/glanceclient/common/http.py", line 120, > > > in _handle_response\n raise exc.from_response(resp, > > > resp.content)\nnova.exception.ImageNotAuthorized: Not authorized for > > image > > > dd1492d5-17a2-4dc2-a4e3-ec6c99255e4b.\n'} | > > > > From noonedeadpunk at gmail.com Thu Jun 30 12:57:44 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Thu, 30 Jun 2022 14:57:44 +0200 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: <2224274.SIoALSJNh4@p1> References: <2224274.SIoALSJNh4@p1> Message-ID: Is it possible to adjust the script a bit in the future to add the amount of changes pushed/merged or some ratio of the amount of rechecks per merged patch? I think it would also be an interesting stat to see in addition to the amount of rechecks to understand how CI is stable or not. ??, 30 ???. 2022 ?. ? 11:12, Slawek Kaplonski : > > Hi, > > > During the last PTG and after it, in the TC we were discussing about CI resources usage and about "rechecks" of the CI jobs (I know, it's again the same topic). > > One of the things we would like to limit, or even avoid is to do "no reason rechecks" which means writing quick comment "recheck" without checking what really was wrong in the previous run. > > We know that putting some hard rules that only comments with "recheck" with given reason will trigger new CI jobs run will not work fine as people may simply start writing any random things there. But we want to encourage all teams to at least to investigate failures and do as many rechecks with explanation as possible. > > > For now I prepared simple script [1] which counts how much of all rechecks are "bare rechecks". It can be checked by project (like openstack/neutron) or give summary for all projects or teams (like Quality Assurance for example). I prepared some stats for all teams listed in the https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml from last 30 days: > > > +-------------------+---------------+--------------+-------------------+ > > | Team | Bare rechecks | All Rechecks | Bare rechecks [%] | > > +-------------------+---------------+--------------+-------------------+ > > | skyline | 20 | 20 | 100.0 | > > | magnum | 2 | 2 | 100.0 | > > | zun | 1 | 1 | 100.0 | > > | mistral | 9 | 9 | 100.0 | > > | ec2-api | 1 | 1 | 100.0 | > > | barbican | 15 | 15 | 100.0 | > > | venus | 2 | 2 | 100.0 | > > | solum | 1 | 1 | 100.0 | > > | tacker | 30 | 30 | 100.0 | > > | trove | 4 | 4 | 100.0 | > > | rally | 2 | 2 | 100.0 | > > | storlets | 5 | 5 | 100.0 | > > | winstackers | 3 | 3 | 100.0 | > > | OpenStack Charms | 32 | 33 | 96.97 | > > | sahara | 27 | 28 | 96.43 | > > | keystone | 24 | 25 | 96.0 | > > | kuryr | 120 | 126 | 95.24 | > > | kolla | 134 | 142 | 94.37 | > > | Puppet OpenStack | 94 | 103 | 91.26 | > > | cloudkitty | 10 | 11 | 90.91 | > > | OpenStack-Helm | 29 | 32 | 90.62 | > > | blazar | 8 | 9 | 88.89 | > > | tripleo | 563 | 646 | 87.15 | > > | requirements | 20 | 23 | 86.96 | > > | Telemetry | 30 | 35 | 85.71 | > > | horizon | 55 | 67 | 82.09 | > > | ironic | 131 | 164 | 79.88 | > > | oslo | 11 | 14 | 78.57 | > > | heat | 25 | 33 | 75.76 | > > | cinder | 221 | 294 | 75.17 | > > | cyborg | 6 | 8 | 75.0 | > > | murano | 3 | 4 | 75.0 | > > | glance | 20 | 27 | 74.07 | > > | OpenStackSDK | 47 | 64 | 73.44 | > > | manila | 108 | 160 | 67.5 | > > | neutron | 149 | 221 | 67.42 | > > | senlin | 2 | 3 | 66.67 | > > | swift | 16 | 25 | 64.0 | > > | Quality Assurance | 106 | 167 | 63.47 | > > | nova | 41 | 71 | 57.75 | > > | octavia | 32 | 60 | 53.33 | > > | designate | 19 | 39 | 48.72 | > > | OpenStackAnsible | 41 | 226 | 18.14 | > > +-------------------+---------------+--------------+-------------------+ > > > As You can see from that list above, there is much to improve there. > > I hope that if teams will be checking more reasons of the CI failures, and reporting bugs found there, we may make our CI more stable and as a result have less rechecks which will save our infra resources :) > > > [1] https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/bare_rechecks.py > > > > -- > > Slawek Kaplonski > > Principal Software Engineer > > Red Hat From fungi at yuggoth.org Thu Jun 30 13:06:21 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 30 Jun 2022 13:06:21 +0000 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: References: <2224274.SIoALSJNh4@p1> Message-ID: <20220630130620.i5h47yddyxdypefq@yuggoth.org> On 2022-06-30 14:57:44 +0200 (+0200), Dmitriy Rabotyagov wrote: > Is it possible to adjust the script a bit in the future to add the > amount of changes pushed/merged or some ratio of the amount of > rechecks per merged patch? I think it would also be an interesting > stat to see in addition to the amount of rechecks to understand how CI > is stable or not. [...] Recheck comment volume doesn't really provide an accurate measure of CI stability, all it tells you is how often people requested rerunning tests. Their reasons for doing it can be myriad, from not believing actual failures their changes are causing, to repeatedly rechecking successful results in hopes of reproducing some rare failure condition. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From smooney at redhat.com Thu Jun 30 13:37:47 2022 From: smooney at redhat.com (Sean Mooney) Date: Thu, 30 Jun 2022 14:37:47 +0100 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: <20220630130620.i5h47yddyxdypefq@yuggoth.org> References: <2224274.SIoALSJNh4@p1> <20220630130620.i5h47yddyxdypefq@yuggoth.org> Message-ID: <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> On Thu, 2022-06-30 at 13:06 +0000, Jeremy Stanley wrote: > On 2022-06-30 14:57:44 +0200 (+0200), Dmitriy Rabotyagov wrote: > > Is it possible to adjust the script a bit in the future to add the > > amount of changes pushed/merged or some ratio of the amount of > > rechecks per merged patch? I think it would also be an interesting > > stat to see in addition to the amount of rechecks to understand how CI > > is stable or not. > [...] > > Recheck comment volume doesn't really provide an accurate measure of > CI stability, all it tells you is how often people requested > rerunning tests. Their reasons for doing it can be myriad, from not > believing actual failures their changes are causing, to repeatedly > rechecking successful results in hopes of reproducing some rare > failure condition. yep we also recheck succeful result if we think we have fixed an intermint ci failure that we could not repoduced reliably but created a patch based on code inspection. in such a case we usually recheck 3 times looking for at least 3 consecitive check +1s before we +2w rearly is also recheck if a patch is old and the logs have rotaed when im reviewing others work but genrally i just click the rebase button in that case. for example i will tend to do +2 recheck if there are already cherry picks of the patch to avoid those having to be updated. but as i said this is rare as we dont ofthen have bugfixes that sit around for 3+ months that still actully apply with out a merge confilict but it does happen. so recheck is not a a great proxy for ci stablity without knowing the reason which is why not doing bare rechecks is important. From hberaud at redhat.com Thu Jun 30 13:39:46 2022 From: hberaud at redhat.com (Herve Beraud) Date: Thu, 30 Jun 2022 15:39:46 +0200 Subject: Propose to add Takashi Kajinami as Oslo core reviewer Message-ID: Hello everybody, It is my pleasure to propose Takashi Kajinami (tkajinam) as a new member of the oslo core team. During the last months Takashi has been a significant contributor to the oslo projects. Obviously we think he'd make a good addition to the core team. If there are no objections, I'll make that happen in a week. Thanks. -- Herv? Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -------------- next part -------------- An HTML attachment was scrubbed... URL: From hberaud at redhat.com Thu Jun 30 13:43:38 2022 From: hberaud at redhat.com (Herve Beraud) Date: Thu, 30 Jun 2022 15:43:38 +0200 Subject: Propose to add Tobias Urdin as Tooz core reviewer Message-ID: Hello everybody, It is my pleasure to propose Tobias Urdin (tobias-urdin) as a new member of the Tooz project core team. During the last months Tobias has been a significant contributor to the Tooz project. Obviously we think he'd make a good addition to the core team. If there are no objections, I'll make that happen in a week. Thanks. -- Herv? Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -------------- next part -------------- An HTML attachment was scrubbed... URL: From katonalala at gmail.com Thu Jun 30 13:49:23 2022 From: katonalala at gmail.com (Lajos Katona) Date: Thu, 30 Jun 2022 15:49:23 +0200 Subject: [neutron] Drivers meeting agenda - 01.07.2022. Message-ID: Hi Neutron Drivers, The agenda for tomorrow's drivers meeting is at [1]. * [RFE] Adding the "rekey" parameter to the api for strongswan like "dpd_action" (#link https://bugs.launchpad.net/neutron/+bug/1979044) * [RFE] Firewall Group Ordering on Port Association (#link https://bugs.launchpad.net/neutron/+bug/1979816) [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda See you at the meeting tomorrow. Lajos Katona (lajoskatona) -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Thu Jun 30 14:22:08 2022 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 30 Jun 2022 14:22:08 +0000 Subject: [tc] August 2022 OpenInfra Board Sync Message-ID: <20220630142207.rwtyc3apyhd2gyjv@yuggoth.org> The Open Infrastructure Foundation Board of Directors is endeavoring to engage in regular check-ins with official OpenInfra projects. The goal is for a loosely structured discussion one-hour in length, involving members of the board and the OpenStack TC, along with other interested community members. This is not intended to be a formal presentation, and no materials need to be prepared in advance. I've started an Etherpad where participants can brainstorm potential topics of conversation, time-permitting: https://etherpad.opendev.org/p/2022-08-board-openstack-sync I've also created a Framadate poll has in order to identify a few preferred dates and times. Responses must be submitted by Friday, 2022-07-15, in order to provide the board members with time to pick from the available options. The poll can be found here: https://framadate.org/atdFRM8YeUtauSgC Feel free to reply here on the mailing list, or you can reach out to me directly by E-mail or as fungi in the #openstack-tc IRC channel, with any questions you may have. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From gmann at ghanshyammann.com Thu Jun 30 14:38:20 2022 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 30 Jun 2022 09:38:20 -0500 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> References: <2224274.SIoALSJNh4@p1> <20220630130620.i5h47yddyxdypefq@yuggoth.org> <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> Message-ID: <181b50c7269.dc6fef7798875.4040158417928524660@ghanshyammann.com> ---- On Thu, 30 Jun 2022 08:37:47 -0500 Sean Mooney wrote --- > On Thu, 2022-06-30 at 13:06 +0000, Jeremy Stanley wrote: > > On 2022-06-30 14:57:44 +0200 (+0200), Dmitriy Rabotyagov wrote: > > > Is it possible to adjust the script a bit in the future to add the > > > amount of changes pushed/merged or some ratio of the amount of > > > rechecks per merged patch? I think it would also be an interesting > > > stat to see in addition to the amount of rechecks to understand how CI > > > is stable or not. > > [...] > > > > Recheck comment volume doesn't really provide an accurate measure of > > CI stability, all it tells you is how often people requested > > rerunning tests. Their reasons for doing it can be myriad, from not > > believing actual failures their changes are causing, to repeatedly > > rechecking successful results in hopes of reproducing some rare > > failure condition. > > yep we also recheck succeful result if we think we have fixed an intermint > ci failure that we could not repoduced reliably but created a patch based on code inspection. > > in such a case we usually recheck 3 times looking for at least 3 consecitive check +1s before we +2w > > rearly is also recheck if a patch is old and the logs have rotaed when im reviewing others work > but genrally i just click the rebase button in that case. for example i will tend to do +2 recheck > if there are already cherry picks of the patch to avoid those having to be updated. but as i said this is > rare as we dont ofthen have bugfixes that sit around for 3+ months that still actully apply with out a merge confilict > but it does happen. > > so recheck is not a a great proxy for ci stablity without knowing the reason which is why not doing bare rechecks is important. I think having elastic-recheck up again will help us to check the CI stability and what bug is causing more issues. dpawlik is working on that. The TC idea about Slawek's script is to keep monitoring the bare recheck and build a culture in all OpenStack projects about not doing any bare recheck and at least finding out what bug is causing it. With that, we will be getting more attention to frequent bugs and asking the respective team to fix that. We in TC will be monitoring the same regularly and post which project is making more bare recheck. We request each project even your bare recheck is low add this monitoring in your weekly meeting also so that you keep giving the message and monitor too. I have added the same in QA meeting also. -gmann > > > From skaplons at redhat.com Thu Jun 30 14:53:19 2022 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 30 Jun 2022 16:53:19 +0200 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> References: <2224274.SIoALSJNh4@p1> <20220630130620.i5h47yddyxdypefq@yuggoth.org> <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> Message-ID: <2709079.EGI8z0umEv@p1> Hi, Dnia czwartek, 30 czerwca 2022 15:37:47 CEST Sean Mooney pisze: > On Thu, 2022-06-30 at 13:06 +0000, Jeremy Stanley wrote: > > On 2022-06-30 14:57:44 +0200 (+0200), Dmitriy Rabotyagov wrote: > > > Is it possible to adjust the script a bit in the future to add the > > > amount of changes pushed/merged or some ratio of the amount of > > > rechecks per merged patch? I think it would also be an interesting > > > stat to see in addition to the amount of rechecks to understand how CI > > > is stable or not. > > [...] > > > > Recheck comment volume doesn't really provide an accurate measure of > > CI stability, all it tells you is how often people requested > > rerunning tests. Their reasons for doing it can be myriad, from not > > believing actual failures their changes are causing, to repeatedly > > rechecking successful results in hopes of reproducing some rare > > failure condition. > > yep we also recheck succeful result if we think we have fixed an intermint > ci failure that we could not repoduced reliably but created a patch based on code inspection. > > in such a case we usually recheck 3 times looking for at least 3 consecitive check +1s before we +2w > > rearly is also recheck if a patch is old and the logs have rotaed when im reviewing others work > but genrally i just click the rebase button in that case. for example i will tend to do +2 recheck > if there are already cherry picks of the patch to avoid those having to be updated. but as i said this is > rare as we dont ofthen have bugfixes that sit around for 3+ months that still actully apply with out a merge confilict > but it does happen. > > so recheck is not a a great proxy for ci stablity without knowing the reason which is why not doing bare rechecks is important. That's true. The reason why I did script to check "bare" rechecks is to see how often people just do "recheck" without even checking reason of failures. For CI stability, some time ago I did another script https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/rechecks.py[1] which checks only merged patches and counts number of "Failed build" comments from Zuul on the last, merged patch set. That is also not perfect metric for sure but can give IMO better view of the CI stability as it will not count rechecks of the passed CI run to see intermittent failures or issues caused by the patch itself. > > > -- Slawek Kaplonski Principal Software Engineer Red Hat -------- [1] https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/rechecks.py -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: From stephenfin at redhat.com Thu Jun 30 15:51:05 2022 From: stephenfin at redhat.com (Stephen Finucane) Date: Thu, 30 Jun 2022 16:51:05 +0100 Subject: [sdk] renaming a volume with the SDK In-Reply-To: <8c8ee4d966f86466047c50cb63a2fc2be2964830.camel@redhat.com> References: <063f01d887e1$47222100$d5666300$@globalnoc.iu.edu> <080201d887f6$1c212b30$54638190$@globalnoc.iu.edu> <8c8ee4d966f86466047c50cb63a2fc2be2964830.camel@redhat.com> Message-ID: On Tue, 2022-06-28 at 21:22 +0100, Stephen Finucane wrote: > On Fri, 2022-06-24 at 14:13 -0400, jdratlif at globalnoc.iu.edu wrote: > > from keystoneauth1.adapter import Adapter > > adapter = Adapter(conn.session, service_type="block-storage") > > ? > > Adding this and passing adapter instead of conn.session to the commit call > > seems to work. > > You can also pass the proxy object itself. For example: > > cinder: BlockStorageService = conn.block_storage > volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") > volume.name = "test_rename" > volume.commit(cinder) > > Normally there would be an proxy method for this called 'update_volume' too. For > some reason this doesn't exist. If it did, you could simply do: > > cinder: BlockStorageService = conn.block_storage > volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") > cinder.update_volume(volume, name="test_rename") > > But someone needs to add that proxy method first. This is proposed at [1] and will likely form part of a future openstacksdk 1.0 release. Stephen [1] https://review.opendev.org/c/openstack/openstacksdk/+/848264/2 > > Stephen > > > ? > > --- > > John Ratliff > > Systems Automation Engineer > > GlobalNOC @ Indiana University > > ? > > > From: jdratlif at globalnoc.iu.edu > > > Sent: Friday, June 24, 2022 11:44 AM > > > To: openstack-discuss at lists.openstack.org > > > Cc: Ratliff, John > > > Subject: [sdk] renaming a volume with the SDK > > > ? > > > I?m trying to set a property on a volume, specifically the name. But when I > > > try to do this, it tells me the Session object has no default_microversion. > > > I have all my auth in environment variables. Using TOKEN authentication. > > > ? > > > What am I doing wrong here? > > > ? > > > import openstack > > > from openstack.block_storage.v3._proxy import Proxy as BlockStorageService > > > from openstack.block_storage.v3.volume import Volume > > > ? > > > # enable openstack debug logging > > > openstack.enable_logging(debug=True) > > > ? > > > conn = openstack.connect() > > > ? > > > cinder: BlockStorageService = conn.block_storage > > > volume: Volume = cinder.get_volume("b3a70014-1cf0-4fd1-bdc4-183dbfd33e79") > > > volume.name = "test_rename" > > > volume.commit(conn.session) > > > ? > > > ? > > > Thanks, > > > ? > > > --- > > > John Ratliff > > > Systems Automation Engineer > > > GlobalNOC @ Indiana University > From kdhall at binghamton.edu Thu Jun 30 16:40:11 2022 From: kdhall at binghamton.edu (Dave Hall) Date: Thu, 30 Jun 2022 12:40:11 -0400 Subject: [Openstack--Ansible] Playbook for Host Prep Message-ID: Hello, I am wondering if there is any sort of playbook or script for setting up the networking on a new physical host prior to adding it to a cluster. It is a bit cumbersome to do it by hand, especially given that bridges on one type of host need an IP while the same bridge on another type of host should not have an IP. Seeding the /etc/hosts file would be nice as well. Going a bit further, it would be awfully nice if a given host could have the same 4th octet for all host interfaces, or at least for the important ones. I believe that since the 172.29 subnets are /22 this should be possible at least for smaller clusters. Thanks. -Dave -- Dave Hall Binghamton University kdhall at binghamton.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Thu Jun 30 17:20:12 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Thu, 30 Jun 2022 19:20:12 +0200 Subject: [Openstack--Ansible] Playbook for Host Prep In-Reply-To: References: Message-ID: Hi Dave, No, unfortunately we don't have anything like that. Basically the reason is that OSA is super flexible and there's really a lot of ways on how to configure networking, depending on workload. Not saying that some ppl would leverage bare metal provisioning systems, like Maas or Ironic, which would configure networking their own way. However we have a role that is super helpful if you want to configure networking with systemd-networkd. We use that role in CI and have some example on it's usage here: https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/tasks/prepare_networking.yml#L48 So depending on your workload you just need to have a sample playbook and vars defined for hosts ??, 30 ???. 2022 ?., 18:42 Dave Hall : > Hello, > > I am wondering if there is any sort of playbook or script for setting up > the networking on a new physical host prior to adding it to a cluster. It > is a bit cumbersome to do it by hand, especially given that bridges on one > type of host need an IP while the same bridge on another type of host > should not have an IP. Seeding the /etc/hosts file would be nice as well. > > Going a bit further, it would be awfully nice if a given host could have > the same 4th octet for all host interfaces, or at least for the important > ones. I believe that since the 172.29 subnets are /22 this should be > possible at least for smaller clusters. > > Thanks. > > -Dave > > -- > Dave Hall > Binghamton University > kdhall at binghamton.edu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From noonedeadpunk at gmail.com Thu Jun 30 17:34:02 2022 From: noonedeadpunk at gmail.com (Dmitriy Rabotyagov) Date: Thu, 30 Jun 2022 19:34:02 +0200 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: <2709079.EGI8z0umEv@p1> References: <2224274.SIoALSJNh4@p1> <20220630130620.i5h47yddyxdypefq@yuggoth.org> <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> <2709079.EGI8z0umEv@p1> Message-ID: I think I need to rephrase myself a bit. Like if you have 100 patches merged and 2 rechecks, even if all of them are bare, doesn't mean that developers don't care about resources. It's more then they are so sure in their tests stability, that they absolutely sure it's infra failure. Or vice versa, if there are 20 rechecks for 2 patches, even if neither of them are bare, it's still weird and smth worth reconsidering from project perspective. I hope I explained better now what I meant. ??, 30 ???. 2022 ?., 16:56 Slawek Kaplonski : > Hi, > > Dnia czwartek, 30 czerwca 2022 15:37:47 CEST Sean Mooney pisze: > > > On Thu, 2022-06-30 at 13:06 +0000, Jeremy Stanley wrote: > > > > On 2022-06-30 14:57:44 +0200 (+0200), Dmitriy Rabotyagov wrote: > > > > > Is it possible to adjust the script a bit in the future to add the > > > > > amount of changes pushed/merged or some ratio of the amount of > > > > > rechecks per merged patch? I think it would also be an interesting > > > > > stat to see in addition to the amount of rechecks to understand how > CI > > > > > is stable or not. > > > > [...] > > > > > > > > Recheck comment volume doesn't really provide an accurate measure of > > > > CI stability, all it tells you is how often people requested > > > > rerunning tests. Their reasons for doing it can be myriad, from not > > > > believing actual failures their changes are causing, to repeatedly > > > > rechecking successful results in hopes of reproducing some rare > > > > failure condition. > > > > > > yep we also recheck succeful result if we think we have fixed an > intermint > > > ci failure that we could not repoduced reliably but created a patch > based on code inspection. > > > > > > in such a case we usually recheck 3 times looking for at least 3 > consecitive check +1s before we +2w > > > > > > rearly is also recheck if a patch is old and the logs have rotaed when > im reviewing others work > > > but genrally i just click the rebase button in that case. for example i > will tend to do +2 recheck > > > if there are already cherry picks of the patch to avoid those having to > be updated. but as i said this is > > > rare as we dont ofthen have bugfixes that sit around for 3+ months that > still actully apply with out a merge confilict > > > but it does happen. > > > > > > so recheck is not a a great proxy for ci stablity without knowing the > reason which is why not doing bare rechecks is important. > > That's true. The reason why I did script to check "bare" rechecks is to > see how often people just do "recheck" without even checking reason of > failures. > > For CI stability, some time ago I did another script > https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/rechecks.py which > checks only merged patches and counts number of "Failed build" comments > from Zuul on the last, merged patch set. That is also not perfect metric > for sure but can give IMO better view of the CI stability as it will not > count rechecks of the passed CI run to see intermittent failures or issues > caused by the patch itself. > > > > > > > > > > > > -- > > Slawek Kaplonski > > Principal Software Engineer > > Red Hat > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Thu Jun 30 17:36:53 2022 From: cboylan at sapwetik.org (Clark Boylan) Date: Thu, 30 Jun 2022 10:36:53 -0700 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: References: <2224274.SIoALSJNh4@p1> <20220630130620.i5h47yddyxdypefq@yuggoth.org> <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> <2709079.EGI8z0umEv@p1> Message-ID: On Thu, Jun 30, 2022, at 10:34 AM, Dmitriy Rabotyagov wrote: > I think I need to rephrase myself a bit. > > Like if you have 100 patches merged and 2 rechecks, even if all of them > are bare, doesn't mean that developers don't care about resources. It's > more then they are so sure in their tests stability, that they > absolutely sure it's infra failure. I'm not sure I understand why certain infra failures don't deserve a note recording why the recheck was necessary if other failures do. > Or vice versa, if there are 20 rechecks for 2 patches, even if neither > of them are bare, it's still weird and smth worth reconsidering from > project perspective. I think the idea is to create a culture of debugging and record keeping. Yes, I would expect after a few rechecks that maybe the root causes would be addressed in this case, but the first step in doing that is identifying the problem and making note of it. > > I hope I explained better now what I meant. > > ??, 30 ???. 2022 ?., 16:56 Slawek Kaplonski : >> Hi, >> >> >> Dnia czwartek, 30 czerwca 2022 15:37:47 CEST Sean Mooney pisze: >> >> > On Thu, 2022-06-30 at 13:06 +0000, Jeremy Stanley wrote: >> >> > > On 2022-06-30 14:57:44 +0200 (+0200), Dmitriy Rabotyagov wrote: >> >> > > > Is it possible to adjust the script a bit in the future to add the >> >> > > > amount of changes pushed/merged or some ratio of the amount of >> >> > > > rechecks per merged patch? I think it would also be an interesting >> >> > > > stat to see in addition to the amount of rechecks to understand how CI >> >> > > > is stable or not. >> >> > > [...] >> >> > > >> >> > > Recheck comment volume doesn't really provide an accurate measure of >> >> > > CI stability, all it tells you is how often people requested >> >> > > rerunning tests. Their reasons for doing it can be myriad, from not >> >> > > believing actual failures their changes are causing, to repeatedly >> >> > > rechecking successful results in hopes of reproducing some rare >> >> > > failure condition. >> >> > >> >> > yep we also recheck succeful result if we think we have fixed an intermint >> >> > ci failure that we could not repoduced reliably but created a patch based on code inspection. >> >> > >> >> > in such a case we usually recheck 3 times looking for at least 3 consecitive check +1s before we +2w >> >> > >> >> > rearly is also recheck if a patch is old and the logs have rotaed when im reviewing others work >> >> > but genrally i just click the rebase button in that case. for example i will tend to do +2 recheck >> >> > if there are already cherry picks of the patch to avoid those having to be updated. but as i said this is >> >> > rare as we dont ofthen have bugfixes that sit around for 3+ months that still actully apply with out a merge confilict >> >> > but it does happen. >> >> > >> >> > so recheck is not a a great proxy for ci stablity without knowing the reason which is why not doing bare rechecks is important. >> >> >> That's true. The reason why I did script to check "bare" rechecks is to see how often people just do "recheck" without even checking reason of failures. >> >> >> For CI stability, some time ago I did another script https://github.com/slawqo/rechecks-stats/blob/main/rechecks_stats/rechecks.py which checks only merged patches and counts number of "Failed build" comments from Zuul on the last, merged patch set. That is also not perfect metric for sure but can give IMO better view of the CI stability as it will not count rechecks of the passed CI run to see intermittent failures or issues caused by the patch itself. >> >> >> > >> >> > >> >> > >> >> >> >> -- >> >> Slawek Kaplonski >> >> Principal Software Engineer >> >> Red Hat >> From dms at danplanet.com Thu Jun 30 18:06:33 2022 From: dms at danplanet.com (Dan Smith) Date: Thu, 30 Jun 2022 11:06:33 -0700 Subject: [all][TC] Stats about rechecking patches without reason given In-Reply-To: (Clark Boylan's message of "Thu, 30 Jun 2022 10:36:53 -0700") References: <2224274.SIoALSJNh4@p1> <20220630130620.i5h47yddyxdypefq@yuggoth.org> <95c03dc437aef52231c2283c1d783ae8bbc99ff1.camel@redhat.com> <2709079.EGI8z0umEv@p1> Message-ID: >> Or vice versa, if there are 20 rechecks for 2 patches, even if neither >> of them are bare, it's still weird and smth worth reconsidering from >> project perspective. > > I think the idea is to create a culture of debugging and record > keeping. Yes, I would expect after a few rechecks that maybe the root > causes would be addressed in this case, but the first step in doing > that is identifying the problem and making note of it. Right, that is the goal. Asking for a message at least sets the expectation that people are looking at the reasons for the fails. Just because they don't doesn't mean they aren't, or don't care, but I think it helps reinforce the desired behavior. If nothing else, it also helps observers realize "huh, I've seen a bunch of rechecks about $reason lately, maybe we should look at that". --Dan From satish.txt at gmail.com Thu Jun 30 18:59:08 2022 From: satish.txt at gmail.com (Satish Patel) Date: Thu, 30 Jun 2022 14:59:08 -0400 Subject: [Openstack--Ansible] Playbook for Host Prep In-Reply-To: References: Message-ID: Hi Dave, In my case we have a kickstart server with some custom script handling preparing compute nodes. It kickstarted the machine and set up the interface/bridges/ips. My ip address design is based on a computer number like compute-10 (10.30.0.X). Once my compute is ready then i go ahead and run playbook to add them in openstack. We do add machines that are bulk like 100 computers in one shot and it works out pretty well. On Thu, Jun 30, 2022 at 1:22 PM Dmitriy Rabotyagov wrote: > Hi Dave, > > No, unfortunately we don't have anything like that. Basically the reason > is that OSA is super flexible and there's really a lot of ways on how to > configure networking, depending on workload. > > Not saying that some ppl would leverage bare metal provisioning systems, > like Maas or Ironic, which would configure networking their own way. > > However we have a role that is super helpful if you want to configure > networking with systemd-networkd. We use that role in CI and have some > example on it's usage here: > > https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/tasks/prepare_networking.yml#L48 > > So depending on your workload you just need to have a sample playbook and > vars defined for hosts > > ??, 30 ???. 2022 ?., 18:42 Dave Hall : > >> Hello, >> >> I am wondering if there is any sort of playbook or script for setting up >> the networking on a new physical host prior to adding it to a cluster. It >> is a bit cumbersome to do it by hand, especially given that bridges on one >> type of host need an IP while the same bridge on another type of host >> should not have an IP. Seeding the /etc/hosts file would be nice as well. >> >> Going a bit further, it would be awfully nice if a given host could have >> the same 4th octet for all host interfaces, or at least for the important >> ones. I believe that since the 172.29 subnets are /22 this should be >> possible at least for smaller clusters. >> >> Thanks. >> >> -Dave >> >> -- >> Dave Hall >> Binghamton University >> kdhall at binghamton.edu >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: