April 2023 - openstack-discuss - lists.openstack.org

Re: multi-region deployment within datacenter scenario
by Satish Patel 06 Apr '23

06 Apr '23

Hi Piotr, Basically we use a vlan base tenant so the network node has very less usage. All my vms talk directly to the physical network and go out via the edge router. Our VM churn rate is also low because it's a private cloud. I am thinking of creating 3x API nodes and 3x RabbitMQ/DB nodes (total 6 nodes control plane and hope it works fine). One big advantage of having multiple pods (deployments) so during upgrade or any issue we have backup to work. On Thu, Apr 6, 2023 at 3:42 PM Piotr Misiak <piotrmisiak1984(a)gmail.com> wrote: > Hi Satish, > > Nova is well designed to handle hundreds of nodes. I would take a look at > Neutron instead, because it doesn’t scale very well. It depends on how your > networking configuration and use cases look like. > > BR, > Piotr > > On Wed, 5 Apr 2023 at 17:46 Satish Patel <satish.txt(a)gmail.com> wrote: > >> Folks, >> >> This is a very broad question and could have many answers but still >> wanted to ask to see what people are thinking and doing. >> >> We have DC with multiple racks and around 600 nodes so we are planning to >> create 3 private openstack and each with 200 nodes. In other data centers >> we have multiple 200 nodes openstack clusters and life is good. In the new >> datacenter I am thinking of deploying the same way openstack but shared >> keystone between multiple openstack environments to have a single place for >> identification and I am going to call it 3 regions even it's in a single >> DC. Hope shared keystone is helpful/easier for doing terrafrom work and >> selecting region easily. >> >> Other option I was thinking about to deploy cellv2 technology and put all >> 600 nodes in single cloud but always worried for operation point of view. >> Now sure how many people running cellv2 successfully without pain. (Except >> CERN) >> >> If people running multi-region openstack cloud then please share some >> thought and idea to make life easier. Not looking for best solution but >> just clue to get idea to deploy and manage multiple openstack. >> >> Thanks in advance. >> S >> >> >>

1 0

multi-region deployment within datacenter scenario
by Satish Patel 06 Apr '23

06 Apr '23

Folks, This is a very broad question and could have many answers but still wanted to ask to see what people are thinking and doing. We have DC with multiple racks and around 600 nodes so we are planning to create 3 private openstack and each with 200 nodes. In other data centers we have multiple 200 nodes openstack clusters and life is good. In the new datacenter I am thinking of deploying the same way openstack but shared keystone between multiple openstack environments to have a single place for identification and I am going to call it 3 regions even it's in a single DC. Hope shared keystone is helpful/easier for doing terrafrom work and selecting region easily. Other option I was thinking about to deploy cellv2 technology and put all 600 nodes in single cloud but always worried for operation point of view. Now sure how many people running cellv2 successfully without pain. (Except CERN) If people running multi-region openstack cloud then please share some thought and idea to make life easier. Not looking for best solution but just clue to get idea to deploy and manage multiple openstack. Thanks in advance. S

3 7

[interop][ptg] Interop Bobcat vPTG summary
by Martin Kopec 06 Apr '23

06 Apr '23

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Current pain points and challenges of interop testing == The current guidelines don't contain complex enough tests - apart from api tests we should also have more functional tests which mock users' interactions with the cloud - we should test as a real user would use the cloud. People could fake the refstack results - after a discussion we came to an agreement that those people would always find a way, there is no need for coming up with sophisticated ways to prevent it. The point of Interop is to provide a check for vendors/users to see where they stand with their product based on the agreed standards. Use of Interop's tooling is complicated - there have been reports that users struggle with refstack-client and tempest. There is an ongoing effort to create a refstack container which would contain all necessary tools and could be run with just one command. Passing the Interop's guidelines means interoperability on which level? Passing the guidelines should mean being in compliance with the beforehand community agreed upon standards. Getting vendors running the interop tests more often - we only came up with the strategies which are already being used, such as expirations and guideline versioning. == Future strategy == The future strategy could be summarized in 3 points: * discuss the possibility for using sdk-based and tobiko testing * better describe to users the expectations they should have for clouds which assert interoperability * look into expiration of older standards and require that compliant providers indicate to users which version of the standard they last tested/passed [1] https://etherpad.opendev.org/p/bobcat-ptg-interop Regards, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

2 1

[tacker][ptg] Tacker PTG Summary
by Yasufumi Ogawa 06 Apr '23

06 Apr '23

Hi team, Thank you for joining our three days vPTG sessions for Tacker bobcat. It was three days and we had about 20 attendees each day. You can find all the result of discussions in [1] and cross project session with Keystone, Ironic and Barbican for supporting external authz for OAuth2.0 [2]. Here is a summary of our vPTG. - We shared several issues of functional tests have been suffered us for unexpected failures through the latest cycles and agree to revise the tests. Current target include refactoring messed directory structure and useless logs, reducing too much timeouts or redefining test scenarios. - For docs, there is no instruction for supported versions of k8s, helm or prometheus introduced in Tacker in the latest releases. There is also something missing instructions or too old contents. We've agreed to make a plan to update the docs and complete TODOs in bobcat. - Enable tacker to get error messages of management driver as a requirement from operators. - Share the current status of secure RBAC support in Tacker and remained tasks mainly for implementing unit and functional tests and documentation. - We agreed to introduct a new feature for supporting external key manager for authn credentials with encryption for security and custom resources for the latest k8s cluster APIs. - Discussed a requirement for supporting terraform infra-driver for some usecases deloying telco systems on cloud-based infra, and roughly agreed to include the feature into Tacker although we need to have some more surveys for implementation details. - We've had several proposals for improving our dev scheme such as introducing type annotation, plantUML for docs or so. - Shared the current status of Pypi maintainer cleanup. [1] https://etherpad.opendev.org/p/tacker-bobcat-ptg [2] https://etherpad.opendev.org/p/bobcat-ptg-ext_oauth2_server

1 0

[tc] May 2023 OpenInfra Board Sync
by Jeremy Stanley 06 Apr '23

06 Apr '23

The Open Infrastructure Foundation Board of Directors is endeavoring to engage in regular check-ins with official OpenInfra projects. The goal is for a loosely structured discussion one-hour in length, involving members of the board and the OpenStack TC, along with other interested community members. This is not intended to be a formal presentation, and no materials need to be prepared in advance. I've started an Etherpad where participants can brainstorm potential topics of conversation, time-permitting: https://etherpad.opendev.org/p/2023-05-board-openstack-sync Immediately following the February 8 discussion, I proposed to schedule the next call for 20:00 UTC on Wednesday, May 10 and heard no objections, so I've attached a calendar file which can serve as a convenient schedule hold for this, in case anyone needs it. -- Jeremy Stanley

1 0

[nova][cinder] why volumes can only be attached and dettached for running instances
by kaqiu pi 06 Apr '23

06 Apr '23

hello, I can only attached and dettached volumes for running instances in openstack. I would like to ask the reason for this design. And is there any danger if this restriction is turned on through custom modification in a private scene? Thank you so much. Yours

3 2

(no subject)
by Richard Buregeya 06 Apr '23

06 Apr '23

Hello Team, Hope you are doing well. Any one with an idea if there is a modification in COA exam, I heard it is now a multiple question exams instead of practical exam, Anyone with current information, kindly update me. Thanks a lot! Regards

3 6

[nova][cinder] future of rebuild without reimaging
by Bence Romsics 06 Apr '23

06 Apr '23

Hi All! We have users who use 'rebuild' on volume booted servers before nova microversion 2.93, relying on the behavior that it keeps the volume as is. And they would like to keep doing this even after the openstack distro moves to a(n at least) zed base (sometime in the future). As a naive user, it seems to me both behaviors make sense. I can easily imagine use cases for rebuild with and without reimaging. However since the implementation of https://specs.openstack.org/openstack/nova-specs/specs/zed/implemented/volu… rebuild without reimaging is only possible using an old microversion (<2.93). With that change merged, rebuild without reimaging seems to be a somewhat less than fully supported feature. A few examples of what I mean by that: First, there's this warning: https://opendev.org/openstack/python-openstackclient/src/commit/5eb89e4ca1c… In which it is unclear to me what exactly will become an error in a future release. Rebuild with a different image? Or any rebuild with microversion <2.93? Then old nova microversions may get dropped. Though what I heard from nova folks, this is unlikely to happen. Then there are a few hypothetical situations like: a) Rebuild gets a new api feature (in a new microversion) which can never be combined with the do-not-reimage behavior. b) Rebuild may have a bug, whose fix requires a microversion bump. This again can never be combined with the old behavior. What do you think, are these concerns purely theoretical or real? If we would like to keep having rebuild without reimaging, can we rely on the old microversion indefinitely? Alternatively shall we propose and implement a nova spec to explicitly expose the choice in the rebuild api (just to express the idea: osc server rebuild --reimage|--no-reimage)? If the topic is worth further discussion beyond the ML, I can also bring it to the nova ptg. Thanks in advance, Bence Romsics (rubasov) ps: I'll be afk for a few days, but I'll follow up next Tuesday.

7 16

[qa][ptg] QA Bobcat vPTG summary
by Martin Kopec 06 Apr '23

06 Apr '23

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Antelope Retrospective == Good things: kudos to all who are still around and help with reviews, patches, bug fixes Bad things: unstable gate due to tox update, timeouts, job flakiness (mainly multinode job) == uWSGI is in the maintenance mode == This doesn't require an immediate action as it's still in the maintenance mode = any bugs should be fixed and until we need new features we don't have to rush into replacing it. We should have it on the watchlist though and potentially test some alternatives - I've put this as one of the priority items for this cycle [2]. == FIPS interop testing == We had a FIPS goal since the last PTG [3]. The patches [4] and [5] were to have FIPS on ubuntu and these [6] and [7] are to add FIPS on a centos job. The next goal is to have a FIPS job on Rocky linux as it's more stable than centos. == Retiring Patrole == We are going to retire Patrole project. 2 main facts that speak in favor of retiring: Fact 1: plugins have enough RBAC tests (and are writing more to test SRBAC) - therefore patrole is not needed, the testing coverage got by the plugins is enough. Fact 2: patrole gates have been broken for a long time and no one was affected by that. == Additional PyPi maintainer cleanup == Only 2 projects from QA left with additional maintainers: * openstack/os-performance-tools - additional maintainer SpamapS (Clint Byrum) * openstack/patrole - additional maintainer DavidPurcell If you know any of them, please, tell them to reach out to me. I've sent emails to them, but maybe the email addresses I found aren't checked regularly anymore. == Tempest/Devstack gate stability == Several precautions have been done such as reduce mysql memory [8] or reduce the number of tests executed per job, increase of a timeout etc [9]. However, those precautions are mainly workarounds. The important note is that the instability was only highlighted by our testing, not caused by it. Many problems are inefficiencies in openstack software, e.g.: * openstack-client issues a new token for every operation * python startup time - entrypoint lookup * privesp processes in aggregate use more memory than nova (every service runs their own copy) * inefficient ansible code, etc Any volunteers who would like to first analyse the situation, find which of those inefficiencies are the biggest culprits and how big effect they have on the system? [1] https://etherpad.opendev.org/p/qa-bobcat-ptg [2] https://etherpad.opendev.org/p/qa-bobcat-priority [3] https://etherpad.opendev.org/p/antelope-ptg-interop [4] https://review.opendev.org/c/zuul/zuul-jobs/+/866881 [5] https://review.opendev.org/c/zuul/zuul-jobs/+/873893 [6] https://review.opendev.org/c/openstack/devstack/+/871606 [7] https://review.opendev.org/c/openstack/tempest/+/873697 [8] https://review.opendev.org/c/openstack/devstack/+/873646 [9] https://review.opendev.org/q/topic:bug%252F2004780 Regards, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

[cinder] 2023.2 Bobcat PTG Summary
by Rajat Dhasmana 06 Apr '23

06 Apr '23

Hello, Please find the summary of 2023.2 Bobcat PTG in the wiki page[1]. It includes recordings, summary and action items for each topic. More details can be found in the PTG etherpad[2]. I would recommend everyone to please look for the action items assigned to you to plan it according to the cycle deadlines. [1] https://wiki.openstack.org/wiki/CinderBobcatPTGSummary [2] https://etherpad.opendev.org/p/bobcat-ptg-cinder Thanks Rajat Dhasmana

1 0