March 2022 - openstack-discuss - lists.openstack.org

Re: [TripleO] Offline Installation Support
by Anirudh Gupta 24 Mar '22

24 Mar '22

Hi Harald, Thanks for sharing the steps and apologies for the late reply. I tried setting up Local Repo Server and Registry Server on one machine. Using that I successfully deployed Openstack Ussuri Undercloud and Overcloud in an offline mode. Now, in order to test Upgrade from USSURI to VICTORIA, I updated the repo file and podman registry at my local server for Victoria. On Undercloud , repo file content is as follows: [tripleo] baseurl = http://30.30.30.8/software/repos/tripleo_victoria enabled = 1 gpgcheck = 0 module_hotfixes=1 name = tripleo where 30.30.30.8 is the IP of my reposerver. With this I have successfully updated the required packages using the command: *sudo yum update python*-tripleoclient openstack-tripleo-common openstack-tripleo-heat-templates* Also the container image prepare file, now has the following content parameter_defaults: ContainerImagePrepare: - push_destination: true set: name_prefix: openstack- name_suffix: '' namespace: myserver.com:5000/tripleovictoria neutron_driver: ovn rhel_containers: false tag: current-tripleo tag_from_label: rdo_version But once I run the command "*openstack undercloud upgrade*", I get the following error 2022-03-23 09:53:24,527 p=380392 u=root n=ansible | 2022-03-23 09:53:24.527073 | 5254007b-2040-558f-a68f-0000000000ac | FATAL | Ensure DNF modules have the right stream | undercloud | item={'module': 'container-tools', 'stream': '3.0'} | error={ "ansible_loop_var": "item", "changed": false, "invocation": { "module_args": { "allow_downgrade": false, "autoremove": false, "bugfix": false, "conf_file": null, "disable_excludes": null, "disable_gpg_check": false, "disable_plugin": [], "disablerepo": [], "download_dir": null, "download_only": false, "enable_plugin": [], "enablerepo": [], "exclude": [], "install_repoquery": true, "install_weak_deps": true, "installroot": "/", "list": null, "lock_timeout": 30, "name": [ "@container-tools:3.0/common" ], "releasever": null, "security": false, "skip_broken": false, "state": "present", "update_cache": false, "update_only": false, "validate_certs": true } }, "item": { "module": "container-tools", "stream": "3.0" }, "msg": "*No group container-tools:3.0/common available*.", "results": [] } Can you please help in resolving this issue. Regards Anirudh Gupta On Wed, Mar 23, 2022 at 2:06 PM Lokendra Rathour <lokendrarathour(a)gmail.com> wrote: > > > ---------- Forwarded message --------- > From: Harald Jensas <hjensas(a)redhat.com> > Date: Wed, Feb 23, 2022 at 9:57 PM > Subject: Re: [TripleO] Offline Installation Support > To: <openstack-discuss(a)lists.openstack.org> > > > On 2/16/22 13:58, Anirudh Gupta wrote: > > Hi Team, > > > > We have successfully done POC on deploying my TripleO Train HA Setup > > with Baremetal Provisioning on Overcloud Node as well. > > > > We would like to thank Tripleo community members for helping us and > > resolving all our concerns at all times. > > > > Moving forward, we need to deploy the setup on the Lab environment for > > which I need some support as well. > > > > We don't have internet in Lab machines, all that we have is access to a > > central server (staging) which has internet access and we can download > > all the requirements over there. > > > > Queries: > > > > * Is there any feasible method to install Undercloud and Overcloud > > Machines in complete offline mode? > > * If yes, then > > > > 1. What are the steps to download all related stuff on the staging > > server? > > I don't know the exact steps, and I am not aware of upstream > documentation page explaining this. > > You will probalby want to use the 'rsync' tool to sync RDO RPM > repositories and create a local repo on you staging machine. Sync > packages, use `createrepo` tool to generate yum/dnf repo metadata. Then > set up a webserver to make the repository available over HTTP. Search > the internets for "Create local RPM repository / mirror", there are many > generic guides that will help you. > > > For containers you need to set up a local container registry. Then I > would suggest useing the `skopeo`[1] tool to sync containers from the > public registry to your local registry. > > > 2. What modifications would be required in undercloud/overcloud and > > other configuration files to support this? > > > > You need to set up the undercloud to use the RPM repositries on the > staging server. Put config in /etc/yum.repos.d/*.repo files. Once the > overcloud is deployed you would want to add the repo config there as well. > > You need to use a custom containers-prepare file when deploying the > undercloud and overcloud, you must set the namespaces to point to your > local registry. Something like this: > > parameter_defaults: > ContainerImagePrepare: > - push_destination: false > set: > name_prefix: openstack- > name_suffix: '' > namespace: staging.example.com:5000/tripleotrain > neutron_driver: null > tag: current-tripleo > ... > > I would suggest generating the container prepare env file with the > `openstack tripleo container image prepare` command[2] and then edit the > generated file. > > > [1] https://github.com/containers/skopeo > [2] > > https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… > > > > > -- > >

3 3

[all][tc] Canceling TC next two weekly meetings
by Ghanshyam Mann 24 Mar '22

24 Mar '22

Hello Everyone, Because of OpenInfra episode of "Yoga release" on March 31st and PTG on April 7th, we agreed to cancel the TC weekly meeting for the next two weeks (31st March and 7th April). We will continue our weekly meeting from April 14th onwards. -gmann

1 0

[qa] Retire os-testr
by Stephen Finucane 24 Mar '22

24 Mar '22

Hey, I think it's time we retired os-testr. It is already effectively unmaintained and we made a concerted effort some cycles ago to migrate all projects to stestr. I think it's time we finally closed that out. For any projects still using os-testr, stestr effectively provides a drop-in replacement. I'll propose patches to do this shortly. Please let me know if anyone has any concerns about this course of action. Cheers, Stephen

5 7

[neutron] Drivers meeting agenda - 24.03.2022.
by Lajos Katona 24 Mar '22

24 Mar '22

Hi Neutron Drivers, The agenda for tomorrow's drivers meeting is at [1]. * Recheck gratuitous ARPs are not sent during master transition: ( https://bugs.launchpad.net/neutron/+bug/1952907) ** For this one there are a few wip patches: ** https://review.opendev.org/c/openstack/neutron/+/821434 ** https://review.opendev.org/c/openstack/neutron/+/821433 ** https://review.opendev.org/c/openstack/neutron/+/712474 On Demand agenda * (ralonsoh): https://bugs.launchpad.net/neutron/+bug/1964575. In particular patch https://review.opendev.org/c/openstack/neutron/+/833247. I'm raising this topic to make the community aware of this migration to SQLAlchemy 2.0 and the implications related. (if needed, I'll request 20 mins during the PTG). * (lajoskatona): PTG etherpad: https://etherpad.opendev.org/p/neutron-zed-ptg [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda See you at the meeting tomorrow. Lajos Katona (lajoskatona)

1 0

[docs] toctree directive + hidden
by Peter Matulis 24 Mar '22

24 Mar '22

Greetings to any doc peeps, I've been struggling with my project's documentation table of contents for a while. What I want to do is NOT publish a TOC on the page where it is defined (via the 'toctree' directive) but I DO want the sidebar/navigation (left menu) to be populaed. According to the upstream documentation [0] the way forward is via the toctree 'hidden' option. However, by doing so I also lose the sidebar aspect - I filed a bug against the openstackdocstheme theme [2]. Help? p.s. I found an old workaround on Stack Overflow [1]. Thanks, Peter [0]: https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html [1]: https://stackoverflow.com/a/20841272/12801299 [2]: https://bugs.launchpad.net/openstack-doc-tools/+bug/1965063

1 0

[kolla] Moving weekly meeting time
by Michał Nasiadka 24 Mar '22

24 Mar '22

Hello Koalas! As agreed on the weekly meeting yesterday - we’re moving the weekly Kolla Wednesday meeting one hour earlier starting next week (March 30th). The reason for this change is daylight savings time change (Sunday March 13th for US, and Sunday March 27th for EU). https://review.opendev.org/c/opendev/irc-meetings/+/835020 Michal

1 0

[all][tc] Technical Committee next weekly meeting on Mar 24th at 1500 UTC
by Ghanshyam Mann 23 Mar '22

23 Mar '22

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Mar 24th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Mar 23rd, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[cinder] Bug deputy report for week of 03-23-2022
by Sofia Enriquez 23 Mar '22

23 Mar '22

This is a bug report from 03-09-2022 to 03-23-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium - https://bugs.launchpad.net/cinder/+bug/1964396 "Volume left in detaching status." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1965847 "Attempt to import duplicate backup record deletes existing record." Confirmed. - NVMe-oF: - https://bugs.launchpad.net/os-brick/+bug/1965954 "NVMe-oF: Extend replicated for in-use doesn't grow the raid." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964590 "NVMe-oF: Extend in-use volume not working." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964395 "NVMe-oF: Does not properly wait for a block device." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964389 "NVMe-oF: Shows misleading traceback." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964388 "NVMe-oF: Connects wrong volume to instance." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964385 "NVMe-oF: Devices disappearing from the host." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964388 "NVMe-oF: Connects wrong volume to instance." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964383 "NVMe-oF: Only connects to the first volume." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964380 "NVMe-oF: Does not work if driver shares the subsystem." Unassigned. - https://bugs.launchpad.net/os-brick/+bug/1964379 "Encryption broken for the nvmeof connector." Unassigned. Low - https://bugs.launchpad.net/cinder/+bug/1965799 "Cinder gets out of memory at json.dumps when doing a backup of a large volume." Fix proposed to master. - https://bugs.launchpad.net/cinder/+bug/1964394 "LVM nvmet: Exporting volume creates kernel messages." Unassigned. - https://bugs.launchpad.net/cinder/+bug/1964391 "LVM nvmet: Disconnects all volumes on map/unmap." Unassigned. Incomplete - https://bugs.launchpad.net/cinder/+bug/1965952 "Netapp copyoffload provider location None." Unassigned. Cheers -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

2 3

[kolla] [rabbitmq] RMQ trouble after config update
by Albert Braden 23 Mar '22

23 Mar '22

We’re running kolla-ansible Train. I followed the recommendations in [1] and ended up with the following config: definitions.json (in the rabbitmq container): { "vhosts": [ {"name": "/"} ], "users": [ {"name": "openstack", "password": "<password>", "tags": "administrator"} ], "permissions": [ {"user": "openstack", "vhost": "/", "configure": ".*", "write": ".*", "read": ".*"} ], "policies":[ {"vhost": "/", "name": "notifications-expire", "pattern": "^(notifications_designate|versioned_notifications).*", "apply-to": "queues", "definition": {"message-ttl":600000,"expires":1200000}, "priority":0}, {"vhost": "/", "name": "ha-all", "pattern": "^(?!(amq\.)|(.*_fanout_)|(reply_)).*", "apply-to": "all", "definition": {"ha-mode":"all","ha-promote-on-shutdown": "always", "ha-sync-mode":"automatic"}, "priority":0} ] } etc/kolla/config/global.conf: [oslo_messaging_rabbit] amqp_durable_queues = True This fixed some issues, but we seem to have a new issue so I must be missing a setting. When we stop the RMQ container on ctrl1, designate stops working (DNS records are not created nor deleted) and I see this in designate-sink.log: 2022-03-17 19:21:29.261 28 ERROR oslo.messaging._drivers.impl_rabbit [req-2c0cd9f4-5331-4697-9c3e-eece475a52af - - - - -] Failed to consume message from queue: Queue.declare: (404) NOT_FOUND - home node 'rabbit@de6-ctrl1' of durable queue 'notifications_designate.info' in vhost '/' is down or inaccessible: amqp.exceptions.NotFound: Queue.declare: (404) NOT_FOUND - home node 'rabbit@de6-ctrl1' of durable queue 'notifications_designate.info' in vhost '/' is down or inaccessible When I look at the notifications_designate.info queue in the web interface, it appears to have moved to ctrl2: Features durable: true Policy notifications-expire Operator policy Effective policy definition expires: 1200000 message-ttl: 600000 Node rabbit@qde3-ctrl2 When I look at designate.conf in the designate_sink containers I don’t see anything pointing only to ctrl1: transport_url = rabbit://openstack:<pwd>@<ctrl1>:5672,openstack:<pwd>@<ctrl2>:5672,openstack:<pwd>@<ctrl3>:5672// But it appears that Designate still tries to use the queue on ctrl1. After I bring up ctrl1, the notifications_designate.info queue remains on ctrl2, but Designate starts working. What am I missing? [1] https://wiki.openstack.org/wiki/Large_Scale_Configuration_Rabbit

3 9

[devstack][ubuntu20.04 focal] Failed to fetch after apt-get install
by 韩光宇 23 Mar '22

23 Mar '22

Hi all, I used devstack to deploy OpenStack with ironic in Ubuntu20.4. But I always got ``` Err:1 http://nova.clouds.archive.ubuntu.com/ubuntu focal/universe amd64 exfat-fuse amd64 1.3.0-1 Connection failed [IP: 91.189.88.245 80] Err:2 http://nova.clouds.archive.ubuntu.com/ubuntu focal/universe amd64 libf2fs5 amd64 1.11.0-1.1ubuntu1 Connection failed [IP: 91.189.91.121 80] Err:3 http://nova.clouds.archive.ubuntu.com/ubuntu focal/universe amd64 libf2fs-format4 amd64 1.11.0-1.1ubuntu1 Connection failed [IP: 91.189.88.245 80] Err:4 http://nova.clouds.archive.ubuntu.com/ubuntu focal/universe amd64 f2fs-tools amd64 1.11.0-1.1ubuntu1 Connection failed [IP: 91.189.91.121 80] Err:5 http://nova.clouds.archive.ubuntu.com/ubuntu focal-updates/main amd64 fonts-noto-mono all 20200323-1build1~ubuntu20.04.1 Connection failed [IP: 91.189.88.245 80] Err:6 http://nova.clouds.archive.ubuntu.com/ubuntu focal/main amd64 fonts-urw-base35 all 20170801.1-3 Connection failed [IP: 91.189.91.121 80] Err:7 http://nova.clouds.archive.ubuntu.com/ubuntu focal/main amd64 libxi6 amd64 2:1.7.10-0ubuntu1 Connection failed [IP: 91.189.88.245 80] Err:8 http://nova.clouds.archive.ubuntu.com/ubuntu focal/main amd64 libxinerama1 amd64 2:1.1.4-2 Connection failed [IP: 91.189.91.121 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/universe/f/fuse-exfat/exf… Connection failed [IP: 91.189.88.245 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/universe/f/f2fs-tools/lib… Connection failed [IP: 91.189.91.121 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/universe/f/f2fs-tools/lib… Connection failed [IP: 91.189.88.245 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/universe/f/f2fs-tools/f2f… Connection failed [IP: 91.189.91.121 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/main/f/fonts-noto/fonts-n… Connection failed [IP: 91.189.88.245 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/main/f/fonts-urw-base35/f… Connection failed [IP: 91.189.91.121 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/main/libx/libxi/libxi6_1.… Connection failed [IP: 91.189.88.245 80] E: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/main/libx/libxinerama/lib… Connection failed [IP: 91.189.91.121 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ``` This is my local.conf: https://paste.opendev.org/show/bsJYZ7fcLSlAxZkr5fsV/ This is my /etc/apt/sources.list: https://paste.opendev.org/show/bRAlj2UoeqNv6U7dVdwG/ I have added /etc/resolv.conf: ``` nameserver 127.0.0.53 options edns0 trust-ad search openstacklocal nameserver 114.114.114.114 nameserver 8.8.8.8 nameserver 223.5.5.5 nameserver 223.6.6.6 ``` $ sudo apt-get update Hit:1 http://nova.clouds.archive.ubuntu.com/ubuntu focal InRelease Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease Hit:3 http://nova.clouds.archive.ubuntu.com/ubuntu focal-updates InRelease Hit:4 http://nova.clouds.archive.ubuntu.com/ubuntu focal-backports InRelease Reading package lists... Done $ sudo apt-get update --fix-missing Hit:1 http://nova.clouds.archive.ubuntu.com/ubuntu focal InRelease Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease Hit:3 http://nova.clouds.archive.ubuntu.com/ubuntu focal-updates InRelease Hit:4 http://nova.clouds.archive.ubuntu.com/ubuntu focal-backports InRelease Reading package lists... Done But still got error. I know this is a apt install error, But I always can't resolve it. Do you konw how can I resolve this? I will very appreciate any help and advice. HanGuangyu

2 1