September 2023 - openstack-discuss

RE: ***UNCHECKED*** openstack-discuss Digest, Vol 59, Issue 10 | Openstack Web Interface Issue
by Asma Naz Shariq 04 Sep '23

04 Sep '23

Hi Team, I have deployed Openstack through manual installation-Yoga release. I encountered the attached error and can't have access to Openstack dashboard. When I entered the login credentials it displays something went wrong and you don't have permission to use /horizon/. Can anyone have a solution for this error. -----Original Message----- From: openstack-discuss-request(a)lists.openstack.org <openstack-discuss-request(a)lists.openstack.org> Sent: Monday, September 4, 2023 8:27 PM To: openstack-discuss(a)lists.openstack.org Subject: ***UNCHECKED*** openstack-discuss Digest, Vol 59, Issue 10 Send openstack-discuss mailing list submissions to openstack-discuss(a)lists.openstack.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss or, via email, send a message with subject or body 'help' to openstack-discuss-request(a)lists.openstack.org You can reach the person managing the list at openstack-discuss-owner(a)lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: 1. Re: Live migration error (Karera Tony) 2. What is the difference between 4 and 8 virtual sockets to physical sockets? (Jorge Visentini) 3. [all][stable][ptl] Propose to EOL Stein series (El?d Ill?s) ---------------------------------------------------------------------- Message: 1 Date: Mon, 4 Sep 2023 14:11:37 +0200 From: Karera Tony <tonykarera(a)gmail.com> To: smooney(a)redhat.com Cc: Danny Webb <Danny.Webb(a)thehutgroup.com>, openstack-discuss <openstack-discuss(a)lists.openstack.org> Subject: Re: Live migration error Message-ID: <CA+69TL3=7GwfgRgAjfxvx5q4aZCstXxtBeB2mGb9ofa0JCZmbA(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hello Smooney, I can confirm that it is disabled on all the servers. Regards Tony Karera On Mon, Sep 4, 2023 at 1:25?PM <smooney(a)redhat.com> wrote: > On Mon, 2023-09-04 at 12:59 +0200, Karera Tony wrote: > > Hello Danny, > > > > I am actually using Broadwell-noTSX-IBRS on all the servers but I > > have > seen > > that some flags are not there in the older servers. > > > > The flags below to be specific. > > > > hwp hwp_act_window hwp_pkg_req > so hwp is apprentlly intels pstate contol based on > https://unix.stackexchange.com/a/43540 > > i dont belive this is normally exposed to a guest but if you are > seeign differnce between hosts then that likely means you have > disabled pstates also know as "intel speedstep" on some of the host > and not others. > > > > > Regards > > > > Tony Karera > > > > > > > > > > On Fri, Sep 1, 2023 at 4:05?PM <smooney(a)redhat.com> wrote: > > > > > On Fri, 2023-09-01 at 14:56 +0200, Karera Tony wrote: > > > > Hello Danny, > > > > > > > > Thanks for the feedback. > > > > > > > > use a lowest common denominator cpu_model : Does this mean > > > > changing > the > > > > servers ? > > > > > > it means tha tif you have a mix of cpus in the deployment you > > > shoudl hardcod to the older of the diffent cpu models i.e skylake > > > or whatever it may > be > > > in your case. > > > > > > > > > > https://docs.openstack.org/nova/latest/configuration/config.html#libvi > rt.cpu_models > > > > > > [libvirt] > > > cpu_mode=custom > > > cpu_models=skylake > > > > > > in more recent release we replaced the older cpu_model with > > > cpu_models with is a comma sperated list of models in prefernce > > > order. i.e. nova will use the first > cpu_modle > > > in the list that work > > > on the current host. not that while this give extra flexiblity it > > > vs > just > > > using the oldest cpu model > > > it does limit the set of hosts you can migrate too but means you > > > can > have > > > better perfromance so its > > > a tradeoff. performance vs portablity. > > > > > > the error you mentioned can also be caused by micorocode updates. > > > intel remove TSX and MPX i beilve in the last 2 years form come of > there > > > cpus > > > that breaks live migration if a vm was create with access to that > > > cpu instruction > > > > > > the only way to resolve that is to cold migrate the guest. > > > i.e. if a vm is currently booted with cpu_model X it cannot be > > > modifed while the guest is running so you either need to update > > > the config option and reboot all the vms > or > > > more particlaly update the config > > > and then cold migrate them which will allow you to move the > instnace(your > > > orginal goal) while allowing > > > the new cpu model to take effect. > > > > > > novas default for the cpu model when not set is with our default > cpu_mode > > > is host_model > > > meaning whatever model best matches the current hosts cpu. > > > > > > that is the correct default in general but if you have mixed cpu > > > generatiosn in your cloud its not ideal. > > > > > > hopefuly that helps > > > > > > > > > > > Regards > > > > > > > > Tony Karera > > > > > > > > > > > > > > > > > > > > On Fri, Sep 1, 2023 at 12:37?PM Danny Webb < > Danny.Webb(a)thehutgroup.com> > > > > wrote: > > > > > > > > > It means that you have CPUs with incompatible flags or you've > > > > > got differing kernel versions that expose different flags or > > > > > you've got differing libvirt versions that expose different > > > > > flags. You either > > > need to > > > > > use a lowest common denominator cpu_model or do a cold migration. > > > > > ------------------------------ > > > > > *From:* Karera Tony <tonykarera(a)gmail.com> > > > > > *Sent:* 01 September 2023 10:13 > > > > > *To:* openstack-discuss > > > > > <openstack-discuss(a)lists.openstack.org> > > > > > *Subject:* Live migration error > > > > > > > > > > > > > > > * CAUTION: This email originates from outside THG * > > > > > ------------------------------ Hello Team, > > > > > > > > > > I am trying to migrate instances from one host to another but > > > > > I am > > > getting > > > > > this error. > > > > > > > > > > *Error: *Failed to live migrate instance to host "compute1". > Details > > > > > <http://10.10.13.31/admin/instances/#message_details> > > > > > Migration pre-check error: Unacceptable CPU info: CPU doesn't > > > > > have compatibility. 0 Refer to > > > > > > http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult > > > > > (HTTP 400) (Request-ID: > > > > > req-9dfdfe2e-70fa-481a-bd3d-c2c76a6e93da) > > > > > > > > > > Any idea on how to fix this? > > > > > > > > > > Regards > > > > > > > > > > Tony Karera > > > > > > > > > > > > > > > > > > > > > > >

1 0

Need assistance
by Gk Gk 04 Sep '23

04 Sep '23

Hi, We have Yoga version setup of OSA. When a user is trying to use a certain flavor which has some extra specs, same as that which is mentioned for an aggregate, the vm creation is failing for the user. But I created a vm using that same flavor and it succeeded for a test user account. When I analyzed the logs, the nova scheduler is not filtering or checking the individual hosts from that target aggregate in the case of that user. But it is validating the target aggregate hosts in the case of a test user and able to select a host for vm creation. So what could be the reason that the scheduler is unable to find the hosts from that aggregate in the case of that user as opposed to that of the test user ? Is there any connection between a project and an aggregate ? Thanks Kumar

2 1

[all][qa] Devstack dropping support for Ubuntu Focal
by Ghanshyam Mann 04 Sep '23

04 Sep '23

Hello Everyone, As you know, this release 2023.2 (bobcat) does not mandate testing the ubuntu Focal[1] and with Nova bumped the libvirt version, focal jobs are failing. Devstack was planning to drop the Focal support in the next cycle but we have to do it now only[2]. This email is a heads-up if you are still wondering about the Focal job failure. If your project still running any of the jobs on focal, please plan to move to Jammy. [1] https://governance.openstack.org/tc/reference/runtimes/2023.2.html [2] https://review.opendev.org/c/openstack/devstack/+/885468 -gmann

1 0

[all][stable][ptl] Propose to EOL Stein series
by Elõd Illés 04 Sep '23

04 Sep '23

Hi, Core projects announced and deleted their stable/stein branches for some time already. Furthermore, most of the projects, where stable/stein is still open, have broken gates. So, the same way as we transitioned Rocky series to End of Life [1][2], now is the time to do the same with Stein. I'll propose the stein-eol transition patches for the rest of the open projects. I ask again the teams to signal their decision (with a +1 if the team is ready for the transition). Thanks in advance! (Please note, that the Extended Maintenance process is phasing out / transforming soon, see details in Technical Committee's resolution [3], which was accepted and merged in August.) Thanks, Előd Illés irc: elodilles @ #openstack-stable / #openstack-release [1] https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031922… [2] https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033386.h… [3] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc…

1 0

Live migration error
by Karera Tony 04 Sep '23

04 Sep '23

Hello Team, I am trying to migrate instances from one host to another but I am getting this error. *Error: *Failed to live migrate instance to host "compute1". Details <http://10.10.13.31/admin/instances/#message_details> Migration pre-check error: Unacceptable CPU info: CPU doesn't have compatibility. 0 Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult (HTTP 400) (Request-ID: req-9dfdfe2e-70fa-481a-bd3d-c2c76a6e93da) Any idea on how to fix this? Regards Tony Karera

3 6

[PTL][release] 2023.2 Bobcat Cycle Highlights
by Herve Beraud 04 Sep '23

04 Sep '23

Hi, This is a reminder that Cycle highlights need to be added to deliverable yamls so that they can be included in release marketing preparations. (See the details about how to add them at the project team guide [1]) Note that the deadline was the Feature Freeze date in previous cycles, but due to requests of PTLs it was decided to postpone one week in the future (to R-4 week). This is better for PTLs as they are usually very busy around Feature Freeze, on the other hand gives less time to marketing team to process the highlights. [1] https://docs.openstack.org/project-team-guide/release-management.html# cycle-highlights Thanks, -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[neutron] Bug Deputy Report August 28 - September 3
by Elvira Garcia Ruiz 03 Sep '23

03 Sep '23

Hi! I was the bug deputy last week. Find the summary below: High ------ - https://bugs.launchpad.net/neutron/+bug/2033508 - [HA] "neutron-dynamic-routing" HA unittest failing Unassigned - https://bugs.launchpad.net/neutron/+bug/2033493 - [ndr] Unit tests failing due to a missing patch, still unreleased, in Neutron Fix proposed: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/893189 Assigned to Rodolfo Alonso Hernandez - https://bugs.launchpad.net/neutron/+bug/2033305 - OVN metadata agent keeps restarting Assigned to Miro Tomaska Fix proposed: https://review.opendev.org/c/openstack/neutron/+/890986 - https://bugs.launchpad.net/neutron/+bug/2033556 - Documentation for DNS integration is incomplete Assigned to Dr. Jens Harbott - https://bugs.launchpad.net/neutron/+bug/2033887 - [OVN][Trunk] The cold migration process is broken since patch 882581 Assigned to Rodolfo Fix proposed: https://review.opendev.org/c/openstack/neutron/+/893447 + backports - https://bugs.launchpad.net/neutron/+bug/2033752 - test_reboot_server_hard fails with AssertionError: time.struct_time() not greater than time.struct_time() This is affecting us but the failure is on Nova. Yatin will monitor this. - https://bugs.launchpad.net/neutron/+bug/2033932 - Add support for OVN MAC_Binding aging Fix proposed: https://review.opendev.org/c/openstack/neutron/+/893575 Assigned to Terry Medium ---------- - https://bugs.launchpad.net/neutron/+bug/2033281 - [OVN] ovn_hash_ring table cleanup In progress. Fix merged on master: https://review.opendev.org/c/openstack/neutron/+/893030 Assigned to Lucas Low ------ - https://bugs.launchpad.net/neutron/+bug/2033651 - [fullstack] Reduce the CI job time Unassigned Incomplete --------------- - https://bugs.launchpad.net/neutron/+bug/2033293 - dns integration saying plugin does not match requirements Unassigned Left a comment since I think this might not be a bug. Also contacted mlavalle to get more information. Undecided --------------- https://bugs.launchpad.net/neutron/+bug/2033683 - openvswitch.agent.ovs_neutron_agent fails to Cmd: ['iptables-restore', '-n'] I’m not very sure about how to confirm this one. Unassigned Kind regards, Elvira

1 0

[SDK][CLI][API][OpenAPI] Machine readable OpenStack API spec - time to do a next step?
by Artem Goncharov 01 Sep '23

01 Sep '23

Hey, Some of you may know that for certain time I was doing experiments on the sdk/cli/ansible_mod side with regards to the code generation. Reason for that is simple: huge codebases and too few resources. While not being the primary goal of this initiative, but still a pointer into the same direction is that most of the tools we need to maintain do the same stuff with some adaptations necessary for the framework in question. I did a lot of diverse attempts trying to take current OpenStackSDK codebase as a “source of truth” regarding what exists in which service and use it to generate code (OSC, AnsibleModule, etc). Pretty quickly it became clear that we do not have enough information to cover all the different needs: - when query parameter has been added into or dropped from the API - what is the type of the query parameter and whether in the case it is list it should be rendered as "?tags=a,b" or "?tags=a&tags=b” - which fields are necessary to be send in the request, which are optional and how to deal when you send name but get back UUID in the same field - what are the parameter/resource fields types in detail with respect to strongly typed programming languages (“int” is not good to differentiate between u8 and f64) - how to determine list of supported “actions” on the resource which are above the plain CRUD Firstly I tried to extend our current model of things in SDK, but even that was not so helpful. And then I decided to give another look at OpenAPI and our desires since multiple years to move into that direction. First of all it provides all the required information to do what is necessary. And so I tried to gather all the things we said about OpenAPI at OpenStack and which challenges we have and compare it against the OpenAPI 3.1. Problems: 1) OpenAPI does not support model of microversions. To be more precise on that you can’t describe different request/response body based on the header differentiator 2) URL + Method are representing unique combination making it impossible to represent different actions based on the content 3) All relevant Headers are supposed to be described in the spec. For example Swift is using X-Object-Meta-* prefix to allow user customisations what is not currently supported by the spec Actually that’s it, everything else was doable. Have I missed something? So what can we do about it: 1) OpenAPI 3.1 is now based upon jsonschema, what allows huge flexibility in describing object schema including inheritance and polymorphism (with composition, discriminator, etc). So we can describe the necessary Microversion Header and schema using “oneOf” whenever necessary. Moreover it is possible to extend type schema with “x-“ prefixed custom properties. So I went and started adding “x-openstack-“ prefixed properties where necessary 2) I personally agree with OpenAPI that URL+method should be enough to uniquely identify what is expected (while not a big deal for the API consumer, it would have been easier on the server side to have URL uniquely tied with the data and operation instead of dynamically looking at the request to figure out what to do. This is actually a most complex thing to deal with and the only solution that comes to my mind would be not to try to put everything into single spec, but rather have a dedicated specs for different operations. It makes it anyway easier to maintain such spec. Maybe at some next OpenAPI iteration this gets resolved and we can adopt changes. 3) A tricky thing, but there is nothing preventing us specifying headers as “x-object-meta-*” and defining custom processors for things like that to for example compose/decompose this into dictionary on the client side OpenAPI now even allows defining spec extensions if necessary, but I think at the moment we can cover all issues without that. Demo time: - server live migration (in my eyes one of the trickiest things I have seen so far from the OpenStack APIs). Here you have an action requiring a dedicated spec, multiple revisions with micro versions and even parameter type bool which can be a string as well: [2] https://paste.opendev.org/show/bcxFi2CrNX1YNEuTzoEh/ More real-life examples in [4] - image create and upload: [3] https://paste.opendev.org/show/bNN1t0qVmHEdVHu0Mtrn/ Here it is possible to put few methods into the single spec. You can also see here samples of different media-types, capture headers, readOnly/writeOnly properties (You can actually take those specs and paste them into https://editor-next.swagger.io/ and see how it looks like) Closing notes: I have implemented a WIP change [1] in SDK that consumes list of specs and performs requested operation. Series of SDK commands are converted to the new way dealing with APIs. This is backwards compatible, but opens a road to heavily reduce amount of code in SDK (precisely by dropping all those customisations that are necessary now). Honestly speaking my drafts even show performance improvement on SDK side in the area of 100-300ms for diverse API operations, but I am still not really able to explain why. Maybe because of avoiding most black magic of SDK. I assume after completion we could get another performance boost by dropping other unnecessary logic and abstractions. The work is not completed as the change title states. Once we have specs for the commands code generators can be updated to consume this data and produce much better code. BTW, I have now also a prototype of the new CLI written with Rust and this is a hypersonic bullet compared to the current OSC. But please do not push me more on that, it is still in a very early stages and depends heavily on the available specs. I can only say that it is now designed in the way that every API call can have a thin CLI coverage just by providing a spec, when additional logic is desired - surely will require human implementation. Code generators in the pipe: OSC, AnsibleModules, RustSDK (sync/async), RustCLI. Next thing that are on the radar: gopher cloud, terraform modules, async python sdk, JS SDK(?) If all of that gets executed properly and with some community traction we can all have following things covered: - improve standardisation of OpenStack internals and externals: glance and nova (at least those 2) are already using jsonschema internally in different areas to describe requests/responses. Why not to make this standard reaching the service consumers? - getting rid of api-ref work by updating our sphinx machinery to consume our customised specs and produce nice docs matching the reality - sharing specs between teams to improve interface (not like currently we need to read the api-ref with tons of bugs plus source code to understand how to cover new feature in service X). Maybe even a central repo with the specs per release. - there are plenty of code generators and server bindings for OpenAPI specs so that we can potentially align frameworks used by different teams to maintain less - less work for all of us who needs services talking to each other (not immediately right now, but once the code is switched on consuming specs) - request verification already on the client side not waiting for the response - finally show something to customers often annoying asking “where are your openapi specs” (no offence here ;-))? I know it is a long message. But I am pretty excited with the progress and would like to hear community opinions. For the more detailed discussion consider this as a pre-announcement of the topic for PTG in sdk/cli slots. Huge invest but huge outcome P.S. it can result in a good chunk of relatively easy work for students Regards, Artem [1] https://review.opendev.org/c/openstack/openstacksdk/+/892161 [2] https://paste.opendev.org/show/bcxFi2CrNX1YNEuTzoEh/ [3] https://paste.opendev.org/show/bNN1t0qVmHEdVHu0Mtrn/ [4] https://review.opendev.org/c/openstack/openstacksdk/+/893365

4 5

[release] Release countdown for week R-4, Sep 04-08
by Elõd Illés 01 Sep '23

01 Sep '23

Development Focus ----------------- We just passed feature freeze! Until release branches are cut, you should stop accepting featureful changes to deliverables following the cycle-with-rc release model, or to libraries. Exceptions should be discussed on separate threads on the mailing-list, and feature freeze exceptions approved by the team's PTL. Focus should be on finding and fixing release-critical bugs, so that release candidates and final versions of the 2023.2 Bobcat deliverables can be proposed, well ahead of the final 2023.2 Bobcat release date. General Information ------------------- We are still finishing up processing a few release requests, but the 2023.2 Bobcat release requirements are now frozen. If new library releases are needed to fix release-critical bugs in 2023.2 Bobcat, you must request a Requirements Freeze Exception (RFE) from the requirements team before we can do a new release to avoid having something released in 2023.2 Bobcat that is not actually usable. This is done by posting to the openstack-discuss mailing list with a subject line similar to: [$PROJECT][requirements] RFE requested for $PROJECT_LIB Include justification/reasoning for why a RFE is needed for this lib. If/when the requirements team OKs the post-freeze update, we can then process a new release. A soft String freeze is now in effect, in order to let the I18N team do the translation work in good conditions. In Horizon and the various dashboard plugins, you should stop accepting changes that modify user-visible strings. Exceptions should be discussed on the mailing-list. By September 28th, 2023 this will become a hard string freeze, with no changes in user-visible strings allowed. Actions ------- stable/2023.2 branches should be created soon for all not-already-branched libraries. You should expect 2-3 changes to be proposed for each: a .gitreview update, a reno update (skipped for projects not using reno), and a tox.ini constraints URL update. Please review those in priority so that the branch can be functional ASAP. The Prelude section of reno release notes is rendered as the top level overview for the release. Any important overall messaging for 2023.2 Bobcat changes should be added there to make sure the consumers of your release notes see them. Upcoming Deadlines & Dates -------------------------- RC1 deadline: September 14th, 2023 (R-3 week) Final RC deadline: September 28th, 2023 (R-1 week) Final 2023.2 Bobcat release: October 4th, 2023 2024.1 Caracal Virtual PTG - October 23-27, 2023 Előd Illés irc: elodilles @ #openstack-release

1 0

[cinder][FFE] Granting FFE to features (8th September)
by Rajat Dhasmana 01 Sep '23

01 Sep '23

Hi All, We had 6 feature requests during the 2023.2 (Bobcat) development cycle[1]. Few of the features had other patch dependencies amounting to significantly more patches. With limited review bandwidth, we were not able to review all the features on time, hence I would like to grant FFE to the following listed features. The reason so many changes are granted FFE are due to the following reasons: * It has gone through iterations of updates and most of them has +2s so it should just require another core to merge * The response time for addressing review comments is fast which is a good thing * The changes are driver specific and isolated to a particular driver that have been tested by the vendor team and third party CI 1. Fujitsu Driver: Add QoS support Patch: https://review.opendev.org/c/openstack/cinder/+/847730 2. NetApp ONTAP: Added support to Active/Active mode in NFS driver Patch: https://review.opendev.org/c/openstack/cinder/+/886869 Dependent patches: https://review.opendev.org/c/openstack/cinder/+/798384 https://review.opendev.org/c/openstack/cinder/+/889826 3. [NetApp] LUN space-allocation support for iSCSI Patch: https://review.opendev.org/c/openstack/cinder/+/893106 4. [Pure Storage] Replication-Enabled and Snapshot Consistency Groups Patch: https://review.opendev.org/c/openstack/cinder/+/891234 5. [HPE XP] Support HA and data deduplication Patch: https://review.opendev.org/c/openstack/cinder/+/892608 Dependent patches: https://review.opendev.org/c/openstack/cinder/+/885986 https://review.opendev.org/c/openstack/cinder/+/879830 https://review.opendev.org/c/openstack/cinder/+/877672 https://review.opendev.org/c/openstack/cinder/+/891937 [1] https://etherpad.opendev.org/p/cinder-2023.2-bobcat-features The deadline after FFE is going to be 8th September, 2023 but we will try to get the changes reviewed and merged at the earliest. Let me know if there are any queries or concerns. Thanks Rajat Dhasmana

1 0