July 2023 - openstack-discuss - lists.openstack.org

[puppet] Proposing core team clean up
by Takashi Kajinami 23 Aug '23

23 Aug '23

Hello, I went through the current core reviewers and found out that the current list includes some people no longer working in the OpenStack community. I'd like to propose removing the following persons from the core reviewer team as we haven't seen these people making any activity in the community recently. * Pradeep Kilambi - puppet-aodh, puppet-ceilometer, puppet-gnocchi, puppet-panko * Richard Megginson - puppet-keystone * Denis Egorenko - puppet-murano * Serg Melikyan - puppet-murano * Sergey Kolekonov - puppet-neutron * David Moreau Simard - puppet-openstack-integration * Yu xingchao - all modules Also, I confirmed that Wes Hayutin is no longer working for OpenStack and he agreed to retire from the core reviewer position. Thus I'll remove him as well. I'll be open for any feedback for one week to hear any thoughts/feedback about this removal. I'd like to take this moment to thank all people for their past work. We always welcome them back if they resume their work in this project. Thank you, Takashi KAJINAMI

3 4

[charms] Nominate Martin Kalcok for charms-core
by Frode Nordahl 21 Aug '23

21 Aug '23

Hello all, I would like to nominate Martin Kalcok for charms-core. He has been part of the extended charms teams for some time now, and has provided many contributions to OpenStack, Ceph and OVN charms [0]. He is diligent and thorough in his contributions, and has recently become a full time member of the focused OVN team. 0: https://review.opendev.org/q/owner:martin.kalcok%2540canonical.com -- Frode Nordahl

5 4

[olso.messaging] RabbitMQ and streams
by Arnaud Morin 16 Aug '23

16 Aug '23

Hey all, Is there any chance that someone already worked on integrating RabbitMQ streams (see [1]) to replace fanouts in oslo rabbitmq driver? We are wondering if this would help lowering the rabbit usage, and maybe increase the rabbit stability? Our deployment is relying a lot on fanouts to delivers messages to computes (mostly neutron, e.g. secgroup update / remote cache population), this is leading to a massive number of messages beeing delivered by seconds. Cheers, Arnaud [1] https://www.rabbitmq.com/streams.html

2 2

LDAP failover fails
by Albert Braden 14 Aug '23

14 Aug '23

We are experiencing the LDAP failover issue described in [1]. Redhat’s solution is to not bother fixing the bug, and to tell customers to put the LDAP server behind a load-balancer. According to Redhat, that is not a good solution for FreeIPA, as explained in [2] and further elucidated in the blog post [3] that it references. I see that the community has a bug open for this [4] and the bug is being worked on here [5] but there has been no activity since 10/22. What is the status of this bugfix? Does it just need someone to review and merge it, or is there more work to be done? How are other FreeIPA users working around this problem? [1] https://bugzilla.redhat.com/show_bug.cgi?id=2024602#c3 [2] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/ht… [3] http://ssimo.org/blog/id_019.html [4] https://bugs.launchpad.net/keystone/+bug/1953622 [5] https://review.opendev.org/c/openstack/keystone/+/821086

2 4

[openstack][neutron[nova][kolla-ansible]instance cannot ping after live migrate
by Nguyễn Hữu Khôi 02 Aug '23

02 Aug '23

Hello guys. I am using openstack yoga with kolla ansible. When I migrate: instance1 from host A to host B after that I cannot ping this instance(telnet also). I must restart neutron_openvswitch_agent or move this instance back to host B then this problem has gone. this is my settings: ----------------- neutron.conf ----------------- [nova] live_migration_events = True ------------------------------------------------ ----------------- nova.conf ----------------- [DEFAULT] vif_plugging_timeout = 600 vif_plugging_is_fatal = False debug = True [compute] live_migration_wait_for_vif_plug = True [workarounds] enable_qemu_monitor_announce_self = True ----------------- openvswitch_agent.ini----------------- [securitygroup] firewall_driver = openvswitch [ovs] openflow_processed_per_port = true I check nova, neutron, ops logs but they are ok. Thank you. Nguyen Huu Khoi

4 12

Tap as a service on openstack with Juju and MaaS
by Jean-Bernard Altidor 31 Jul '23

31 Jul '23

Hello, I've been struggling to configure tap as a service on openstack for a few months now. I can't find any guide or anything about it. Can you suggest a method of port mirroring? I've even via the ovs bridge but I really need a way to mirror data and send it to an instance for analysis as I'm working on an IDS. Thanks in advance for any ideas ! JB

2 2

[nova] Scheduler Optimiser
by Dominik Danelski 31 Jul '23

31 Jul '23

Hello, I would like to introduce you to the tool developed under the working title "Scheduler Optimiser". It is meant to test the effectiveness of different Scheduler configurations, both weights and filters on a given list of VM orders and in a semi-realistic infrastructure. My company - CloudFerro - has been preparing in-house for the last few months and foresees publishing the project as FOSS once it reaches the MVP stage. To make the final result more useful to the community and speed up the development (and release), I humbly ask for your expertise: Are you aware of previous similar efforts? Do you notice some flaws in the current approach? What, in your opinion, are more important aspects of the infrastructure behaviour, and what can be relatively safely ignored in terms of the effect on Scheduler results/allocation? Project objectives: * Use Devstack (or another OpenStack deployer) with a real Scheduler to replay a list of compute VM orders, either real from one's infrastructure or artificially created. * Assess the effectiveness of the scheduling in various terms like: "How many machines of a given type can still be allocated at the moment?" using plug-in "success meters". In a strict sense, the project does not simulate THE Scheduler but interacts with it. * Use fake-virt to emulate huge architectures on a relatively tiny test bench. * Have as little as possible, and ideally no changes to the Devstack's code that could not be included in the upstream repository. The usage should be as simple as: 1. Install Devstack. 2. Configure Devstack's cluster with its infrastructure information like flavours and hosts. 3. Configure Scheduler for a new test case. 4. Replay VM orders. 5. Repeat steps 3 and 4 to find better Scheduler settings. * Facilitate creating a minimal required setup of the test bench. Not by replacing standard Devstack scripts, but mainly through tooling for quick rebuilding data like flavours, infrastructure state, and other factors relevant to the simulation. Outside of the scope: * Running continuous analysis on the production environment, even if some plug-ins could be extracted for this purpose. * Retaining information about users and projects when replaying orders. * (Probably / low priority) replaying actions other than VM creation/deletion as they form a minority of operations and ignoring them should not have a distinct effect on the comparison experiments. Current state: Implemented: * Recreating flavours from JSON file exported via OpenStack CLI. * Replaying a list of orders in the form of (creation_date, termination_date, resource_id (optional), flavor_id) with basic flavour properties like VCPU, RAM, and DISK GB. The orders are replayed consecutively. * Plug-in success-rater mechanism which runs rater classes (returning quantified success measure) after each VM add/delete action, retains their intermediate history and "total success" - how it is defined is implementation dependent. First classes interacting with Placement like: "How many VMs of flavours x (with basic parameters for now) can fit in the cluster?" or "How many hosts are empty?". Missing: * Recreating hosts, note the fake-virt remark from "Risks and Challenges". * Tools facilitating Scheduler configuration. * Creating VMs with more parameters like VGPU, traits, and aggregates. * (Lower priority) saving the intermediate state of the cluster during simulation i.e. allocations to analyse it without rerunning the experiment. Currently, only the quantified meters are saved. * Gently failing and saving all information in case of resource depletion: close to completion, handling one exception type in upper layers is needed. * More success meters. Risks and Challenges: * Currently, the tool replays actions one by one, it waits for each creation and deletion to be complete before running success raters and taking another order. Thus, the order of actions is important, but not their absolute time and temporal density. This might skip some side-effects of a realistic execution. * Similarly, to the above, fake-virt provides simple classes that will not reproduce some behaviours of real-world hypervisors. An explicit Scheduler avoids hosts that had recently failed to allocate a VM, but most likely fake-virt will not mock such behaviour. * Fake-virt should reproduce a real diverse infrastructure instead of x copies of the same flavour. This might be the only, but very important change to the OpenStack codebase. If successful, it could benefit other projects and tests as well. Even though the list of missing features is seemingly larger, the most important parts of the program are already there, so we hope to finish the MVP development in a relatively short amount of time. We are going to publish it as FOSS in either case, but as mentioned your observations would be very much welcome at this stage. I am also open to answering more questions about the project. Kind regards Dominik Danelski

3 3

[neutron] bug deputy report for week of 2023-07-24
by Bence Romsics 31 Jul '23

31 Jul '23

Hi All, We had the following new bugs in the last week or so: Unassigned: * https://bugs.launchpad.net/neutron/+bug/2028651 [ovn] IPv6 VIPs broken with ML2/OVN * https://bugs.launchpad.net/neutron/+bug/2028442 Support DNS for ovn_{nb,sb}_connection * https://bugs.launchpad.net/neutron/+bug/2028285 [unit test][xena+] test_port_deletion_prevention fails when runs in isolation Likely valid and high, but my dvr-fu is weak. dvr folks, please have a look: * https://bugs.launchpad.net/neutron/+bug/2028795 Restarting OVS with DVR creates a network loop fix proposed: https://review.opendev.org/c/openstack/neutron/+/889752 High: * https://bugs.launchpad.net/neutron/+bug/2028185 Large number of FIPs causes slow sync_routers response fix proposed: https://review.opendev.org/c/openstack/neutron/+/888915 * https://bugs.launchpad.net/neutron/+bug/2028846 FIP PF don't works with vlan tenant network and ovn backend fix proposed: https://review.opendev.org/c/openstack/neutron/+/889871 Medium: * https://bugs.launchpad.net/neutron/+bug/2028161 [ovn-octavia-provider] FIP not included into LogicalSwitchPortUpdate event handler method fix merged on master, backports proposed: https://review.opendev.org/q/Ibee3906e8e9575fba7811e989e3e111a026ce45b Low: * https://bugs.launchpad.net/neutron/+bug/2028724 neutron should forbid configuring agent_down_time that is known to crash due to CPython epoll limitation fix proposed: https://review.opendev.org/c/openstack/neutron/+/889373 RFE: * https://bugs.launchpad.net/neutron/+bug/2028660 [fwaas][rfe] support list type of port_range for firewall rule Incomplete: * https://bugs.launchpad.net/neutron/+bug/2028338 Router with l3_agent state unknown Invalid: * https://bugs.launchpad.net/neutron/+bug/2028544 dhcp agent binding count greather than dhcp_agents_per_network Cheers, Bence (rubasov)

1 0

[OSA] Failover at the storage level
by Murilo Morais 29 Jul '23

29 Jul '23

Good morning everybody! I'm trying to raise a scenario with redundancy at the storage level. I'm using CEPH with RBD-mirror. Cinder has a pretty clear documentation about it, but I didn't find anything about Glance and the "nova_libvirt_images_rbd_pool" and the "cinder_service_backup_driver". What would be the best approach? Only use Cinder with Mirror? If by chance the "nova_libvirt_images_rbd_pool" dies, if you are using config-drive the disk will die, correct? In this case, would it be better to leave local? I can't see what would be the best scenario for Failover cases with OSA. Thanks in advance!

2 1

[openstack-helm] [ceph deployment failed] error:- MountVolume.SetUp failed for volume "ceph-client-admin-keyring" secret "ceph-client-admin-keyring" not found
by Viswanath Ediga 29 Jul '23

29 Jul '23

Team, I am trying to deploy openstack using OSH. I am following documentation https://docs.openstack.org/openstack-helm/latest/install/multinode.html I am getting error while "Deploying ceph" phase. container creation is failing. Below is the error that I see in POD description Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedMount 24m (x73 over 3h7m) kubelet Unable to attach or mount volumes: unmounted volumes=[ceph-client-admin-keyring], unattached volumes=[], failed to process volumes=[]: timed out waiting for the condition Warning FailedMount 4m18s (x99 over 3h9m) kubelet MountVolume.SetUp failed for volume "ceph-client-admin-keyring" : secret "ceph-client-admin-keyring" not found Regards, Any help is appreciated. Kasi Viswanath

1 0