May 2022 - openstack-discuss - lists.openstack.org

[ops][octavia][neutron] Distributed Virtual Routing: Floating IPs attached to virtual IP addresses are assigned on network nodes
by Jan Horstmann 09 May '22

09 May '22

Hello! When we initially deployed openstack we thought that using distributed virtual routing with ml2/ovs-dvr would give us the ability to automatically scale our network capacity with the number of hypervisors we use. Our main workload are kubernetes clusters which receive ingress traffic via octavia loadbancers (configured to use the amphora driver). So the idea was that we could increase the number of loadbalancers to spread the traffic over more and more compute nodes. This would imply that any volume based (distributed) denial of service attack on a single loadbalancer would just saturate a single compute node and leave the rest of the system functional. We have recently learned that, no matter the loadbalancer topology, a virtual IP is created for it by octavia. This, and probably all virtual IPs in openstack, are reserved by an unbound and disabled port and then set as an allowed address pair on any server's port which might hold it. Up to this point our initial assumption should be true, as the server actually holding the virtual IP would reply to any ARP requests and thus any traffic should be routed to the node with the virtual machine of the octavia amphora. However, we are using our main provider network as a floating IP pool and do not allow direct port creation. When a floating IP is attached to the virtual IP it is assigned to the SNAT router namespace on a network node. Naturally in high traffic or even (distributed) denial of service situations the network node might become a bottleneck. A situation we thought we could avoid by using distributed virtual routing in the first place. This leads me to a rabbit hole of questions I hope someone might be able to help with: Is the assessment above correct or am I missing something? If it is correct, do we have any other options than vertically scaling our network nodes to handle traffic? Do other ml2 drivers (e.g. OVN) handle this scenario differently? If our network nodes need to handle most of the traffic anyway, do we still have any advantage using distributed virtual routing? Especially when considering the increased complexity compared to a non- distributed setup? Has anyone ever explored non-virtual IP based high availability options like e.g. BGP multipathing in a distributed virtual routing scenario? Any input is highly appreciated. Regards, Jan

3 3

Glance browser error
by Derek O keeffe 09 May '22

09 May '22

Hi all, Second mail about this issue. I'm getting the following error in the chrome console: Refused to connect to 'https://10.xx.xx.xxx:9292/v2/images/f7df36f3-33d8-49b6-95b6-b16e95bc3225/fi…' because it violates the following Content Security Policy directive: "default-src 'self'". Note that 'connect-src' was not explicitly set, so 'default-src' is used as a fallback I can't upload an image through horizon but I can do it from the cli with no issues. I can't find anything online to help with the issue so any input would be greatly appreciated. Regards,Derek

3 2

[magnum] Proposing Mohammed Naser for core-reviewer
by Spyros Trigazis 09 May '22

09 May '22

Dear all, I would like to nominate Mohammed Naser for core reviewer in the magnum project. Mohammed has a proven knowledge of the codebase with frequent contributions and reviews. Mohammed will be of great help in fast and reliable reviews. Cheers, Spyros

1 0

Data Center Survival in case of Disaster / HW Failure in DC
by KK CHN 09 May '22

09 May '22

List, We are having an old cloud setup with OpenStack Ussuri usng Debian OS, (Qemu KVM ). I know its very old and we can't upgrade to to new versions right now. The Deployment is as follows. A. 3 Controller in (cum compute nodes . VMs are running on controllers too..) in HA mode. B. 6 separate Compute nodes C. 3 separate Storage node with Ceph RBD Question is 1. In case of any Sudden Hardware failure of one or more controller node OR Compute node OR Storage Node what will be the immediate redundant recovery setup need to be employed ? 2. In case H/W failure our recovery need to as soon as possible. For example less than30 Minutes after the first failure occurs. 3. Is there setup options like a hot standby or similar setups or what we need to employ ? 4. To meet all RTO (< 30 Minutes down time ) and RPO(from the exact point of crash all applications and data must be consistent) . 5. Please share your thoughts for reliable crash/fault resistance configuration options in DC. We have a remote DR setup right now in a remote location. Also I would like to know if there is a recommended way to make the remote DR site Automatically up and run ? OR How to automate the service from DR site to meet exact RTO and RPO Any thoughts most welcom. Regards, Krish

4 5

[baremetal-sig][ironic] Tue May 10, 2022, 2pm UTC: "Looking back and forward: The Xena and Zed Cycles for Ironic"
by Arne Wiebalck 09 May '22

09 May '22

Dear all, The Bare Metal SIG will meet tomorrow Tue May 10, 2022, 2pm UTC featuring a topic of the day presentation by Ironic PTL Iury Gregory: "Looking back and forward: The Xena and Zed Cycles for Ironic" Everyone is welcome, all details on how to join can be found on the SIG's etherpad: https://etherpad.opendev.org/p/bare-metal-sig Hope to see you there! Arne -- Arne Wiebalck CERN IT

1 0

[all][tc] What's happening in Technical Committee: summary May 6th, 22: Reading: 5 min
by Ghanshyam Mann 07 May '22

07 May '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on May 5. Most of the meeting discussions are summarized in this email. Meeting summary logs are available @https://meetings.opendev.org/meetings/tc/2022/tc.2022-05-05-15.00.log.html As it was a video call, the full recording is here @https://www.youtube.com/watch?v=kJdoWcBiMuY&t=205s * Next TC weekly meeting will be on May 12 Thursday at 15:00 UTC, feel free to add the topic on the agenda[1] by May 11. 2. What we completed this week: ========================= * Add the cinder-three-par charm to Openstack charms[2] * Update milestones for FIPS goal[3] * Add gmann to oslo tact-sig liaison[4] 3. Activities In progress: ================== TC Tracker for Zed cycle ------------------------------ * Zed tracker etherpad includes the TC working items[5], we have started the many items. Open Reviews ----------------- * Nine open reviews for ongoing activities[6]. CFN overview from Jie Niu, China Mobile Research Institute ----------------------------------------------------------------------- As you might have seen CFN proposal on ML[7], China Mobile team(Yu and Jie) joined the TC meeting and presented the CFN idea, architecture, and roadmap in detail. We had a few question-answer at the end. As the discussion is going on between China Mobile team and the Foundation, TC did not take any decision on being CFN as SIG in OpenStack or not. We are open to ideas coming from the discussion with Foundation. 'tick' 'tock' release cadence open items ---------------------------------------------- We are discussing two things here: 1. release notes strategy: This is not concluded yet but the discussion point is how we should make tock release notes noticeable in the next tick release for operators upgrading to tick-tick release? Whether we should copy the key deprecated/upgrade release notes or use like or mention about reading the previous tock release notes if you upgrading from tick->tick? We will continue the discussion in next meeting and conclude. 2. tick tock name using as per legal check >From foundation legal checks, it seems it is not so clear how we should use tick or tock or better we should not use these names, discussion is on gerrit[8]. We discussed a few ideas about using SLU|SLURP (Skip level upgrade release process) but it is not agreed upon by all the TC members. We are stuck with the same "horror naming" show in this too as we are in release naming. We are open to ideas and if you have any better ideas, feel free to add them in Gerrit review[8] Improve project governance --------------------------------- As discussed in PTG (yoga and zed) we are going to try a new framework "Technical-preview" to catch the inactive project or new project moving towards the OpenStack way. Slawek has the proposal up and ready for review[9]. Change OpenStack release naming policy proposal ----------------------------------------------------------- No concrete update on this than what we are discussing on ML[10] and Gerrit[11]. Migration from old ELK service to new Dashboard ----------------------------------------------------------- The new dashboard[12] is ready to use, we encourage community members to use it and provide feedback if any. We discussed the e-r service and repo maintenance. Ananya from tripleO team joined the meeting and they are ok to merge the rdo branch with master. Once that is done we will figure out where we can host this service, Dainel has some ideas about it and will continue the discussion in the TC channel or meeting. Consistent and Secure Default RBAC ------------------------------------------- We had our weekly call on Tuesday 3rd May and discussed the need for 'service' role. There are points that we should not make 'service' role as a light admin but at the same time, it is needed for the inter-service communication. Discussion still going on and we will continue it next Tuesday's call along with heat discussion. Notes are in etherpad[13] and the next meeting details are in wiki page [14], FIPs community-wide goal ------------------------------- Ade proposal of the updated milestone has been merged and goal selection proposal is up[15]. 2021 User Survey TC Question Analysis ----------------------------------------------- No update on this. The survey summary is up for review[16]. Feel free to check and provide feedback. Zed cycle Leaderless projects ---------------------------------- No updates on this. Only Adjutant project is leaderless/maintainer-less. We will check Adjutant's the situation again on ML and hope Braden will be ready with their company side permission[17]. Fixing Zuul config error ---------------------------- Requesting projects with zuul config error to look into those and fix them which should not take much time[18]. Project updates ------------------- * None 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[19]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [20] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/837781 [3] https://review.opendev.org/c/openstack/governance/+/838601 [4] https://review.opendev.org/c/openstack/governance/+/840351 [5] https://etherpad.opendev.org/p/tc-zed-tracker [6] https://review.opendev.org/q/projects:openstack/governance+status:open [7] http://lists.openstack.org/pipermail/openstack-discuss/2022-April/028345.ht… [8] https://review.opendev.org/c/openstack/governance/+/840354 [9] https://review.opendev.org/c/openstack/governance/+/839880 [10] http://lists.openstack.org/pipermail/openstack-discuss/2022-April/028354.ht… [11] https://review.opendev.org/c/openstack/governance/+/839897 [12] http://lists.openstack.org/pipermail/openstack-discuss/2022-April/028346.ht… [13] https://etherpad.opendev.org/p/rbac-zed-ptg#L103 [14] https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… [15] https://review.opendev.org/c/openstack/governance/+/840920 [16] https://review.opendev.org/c/openstack/governance/+/836888 [17] http://lists.openstack.org/pipermail/openstack-discuss/2022-March/027626.ht… [18] https://etherpad.opendev.org/p/zuul-config-error-openstack [19] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [20] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

Openstack OSA Latest - Glance
by Derek O keeffe 06 May '22

06 May '22

Hi all, Could any one tell me if they have come across this error (in the chrome console) while trying to create a new image in horizon: Refused to connect to 'https://10.37.110.104:9292/v2/images/3e553b60-0a6c-48a0-b855-60088839201e/f…' because it violates the following Content Security Policy directive: "default-src 'self'". Note that 'connect-src' was not explicitly set, so 'default-src' is used as a fallback If so, any advice? I can scp the image file to the utility container and upload it from there no problem. Also, journalctl -f on the glance container shows nothing when the process fails. We have set the internal and external lb vip addresses differently but both are on the same subnet (internal) as it's an in house deploy. So two addresses on the mgmt interface to be precise. Maybe that's causing an issue. Regards,Derek

2 1

[manila] Zed cycle bug squash
by Carlos Silva 06 May '22

06 May '22

Greetings Zorillas and interested stackers! As mentioned in the previous weekly meetings, we will soon be meeting for the first bugsquash of the Zed release! The event will be held from May 2nd to May 6th, providing an extended contribution window. May 2nd 15:00 - 16:00 UTC - Kick off May 5th 15:00 - 16:00 UTC - Mid term checkpoint (we won't have our regular Manila meeting on this day) May 6th 15:00 - 15:30 UTC - Wrap Up We will use a meetpad for these meetings [1]. The main idea of this event is to go over the list of stale bugs and act on them, either seeing if they are incomplete or invalid at this point or working on them. The stale bugs list will be available on [2]. [1] https://meetpad.opendev.org/ManilaZed1Bugsquash [2] https://ethercalc.openstack.org/1nesczgjufb9 See you next week! Thank you, carloss

1 1

[nova][placement] Incomplete Consumers return negative value after upgrade
by Jan Wasilewski 06 May '22

06 May '22

Hi, after an upgrade from Stein to Train, I hit an issue with negative value during upgrade check for placement: # placement-status upgrade check+------------------------------------------------------------------+| Upgrade Check Results |+------------------------------------------------------------------+| Check: Missing Root Provider IDs || Result: Success || Details: None |+------------------------------------------------------------------+| Check: Incomplete Consumers || Result: Warning || Details: There are -20136 incomplete consumers table records for || existing allocations. Run the "placement-manage db || online_data_migrations" command. |+------------------------------------------------------------------+ Seems that negative value is a result that I get such values from tables consumer and allocations: mysql> select count(id), consumer_id from allocations group by consumer_id;...1035 rows in set (0.00 sec) mysql> select count(*) from consumers;+----------+| count(*) |+----------+| 21171 |+----------+1 row in set (0.04 sec) Unfortunately such warning cannot be solved by execution of suggested command( placement-manage db online_data_migrations) as it seems it adds records to consumers table - not to allocations, which looks like to be a problem here. I was following recommendations from this discussion: http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018536… but unfortunately it doesn't solve the issue(even not changing a negative value). I'm just wondering if I skipped something important and you can suggest some (obvious?) solution. Thank you in advance for your time and help. Best regards, Jan

2 3

[Kolla][CloudKitty] Wrong version in ubuntu-binary-cloudkitty-processor:yoga
by Rafael Weingärtner 06 May '22

06 May '22

Hello guys, I was checking the upstream containers for CloudKittty for Yoga release; and, I noticed that the CloudKitty containers are using version 11.0.1, which is not the Yoga version of CloudKitty. I also checked Xena, and the same thing happens there. Do we need to define these component versions somewhere else for the Kolla containers that are built upstream? -- Rafael Weingärtner

3 4