November 2023 - openstack-discuss

[tripleo] stepping down from TripleO core
by Takashi Kajinami 27 Nov '23

27 Nov '23

Hello, As you might know or notice, I recently joined a different company, and my focus has been changed. I'm still involved in the OpenStack community but I may no longer be involved in TripleO. To clarify my status I've removed myself from TripleO core. # Please be aware that I'm removing myself from release liason, # someone from the core team might want to replace me, though # I don't think there may be multiple releases created in the future. I'd like to thank all members in the team (current members and ex-menmbers) for your great collaboration and work in the project. Thank you, Takashi Kajinami

3 2

[puppet] Proposing EOL of stable/wallaby and xena
by Takashi Kajinami 24 Nov '23

24 Nov '23

Hello, Some of you may know this but currently CentOS 8 Stream integration jobs in stable branches are all broken. I've looked into some of these failures but tasks randomly get stuck which eventually caused timeout and I've not yet found any potential cause. While we may be able to spend some time to dig into the problem, I'm wondering if anyone still requires wallaby and xena which support CentOS 8 Stream. Most of the backports submitted to these old branches were for TripleO,but I've seen some patches indicating TripleO wallaby is being retired so I doubt anyone still need keeping these branches open for backports. Please let me know if you have any concern about the proposed EOL and are interested in fixing these jobs. The stable/yoga branch has already passed the EM phase, so is another candidate for EOL but my intention is to keep it for now and re-consider its EOL early next year. We may have to make all CentOS 8 jobs non-voting, though. (We have CentOS 9 jobs in this branch). Thank you, Takashi Kajinami

2 2

[kolla] Upgrading from Train to Wallaby
by Albert Braden 24 Nov '23

24 Nov '23

Is it possible to upgrade kolla-ansible from Train to Wallaby in one step, or do we need to step through Ussuri and Victoria?

4 3

[keystone] temporary workaround for deprecated _member_ role?
by Christian Stelter 23 Nov '23

23 Nov '23

Hi! As someone who hasn't yet delved very deeply into keystone and the policies of the individual OpenStack services, I wondered whether setting member as the implied role for _member_ could pick up all those users who didn't manage to rotate their application credentials before switching to Zed Antelope. Is this a valid approach or could it cause problems? Kind regards, Christian

4 8

[ironic] Multiple vendor drivers being deprecated
by Julia Kreger 23 Nov '23

23 Nov '23

Greetings everyone, During Ironic's last PTG[0], we discussed the future path for drivers and the overall direction in the vendor ecosystem. The overall consensus was that DMTF Redfish is largely achieving its overall goal to standardize interfaces for interactions with Baseboard Management Controllers. Most of the vendors we collaborate with are in the process of determining their support and migration paths to Redfish. But as with any Open Source project with drivers, we also haven't heard from some vendors recently for a variety of reasons. It is with that in mind, we've made the decision to deprecate and eventually remove[1] some of the vendor drivers in cases where we've not heard from the maintainers and there is a forward path. * ibmc - We've not heard from the maintainer in some time due to geopolitical circumstances, but when we last heard from the developer/maintainer, they indicated work was actively underway to achieve redfish compliance in their employer's hardware. * xclarity - This driver was a driver which was designed to talk to the Lenovo xClarity Controller. This driver largely had no forward path, as we believe the underlying API support it used was also deprecated by Lenovo with the release of xClarity Controller v3. We have received numerous reports that the upstream redfish hardware type works as expected against Lenovo hardware, and at this point it just seems logical to move beyond this early driver. * iDRAC WSMAN driver flavors - The Dell team at a point long in the past decided Redfish was the future direction, and they created a set of "Redfish" interfaces in their driver, which replaced "WebServices for Management", better known as WSMAN interfaces which hail back to the days of iDRAC 6 (2008). At this point, given iDRAC7 (2012) firmware updates are available with Redfish support, we believe Redfish is generally suitable moving forward. The specific modules going away are the "-wsman" variant interfaces under the idrac hardware type. In case you are now wondering about the Fujitsu iRMC and HPE iLO drivers, we've heard from those teams recently and they too are focusing on Redfish as their forward path. The exact shape and form remains to be determined as each vendor has unique constraints and requirements to work with. As a result of this, their driver interfaces are not impacted by this decision by the community. If you have any questions or concerns, please feel free to reach out or join the Ironic team during our weekly meeting[2]. -Julia [0]: https://etherpad.opendev.org/p/ironic-ptg-october-2023 [1]: https://review.opendev.org/c/openstack/ironic/+/901501 [2]: https://wiki.openstack.org/wiki/Meetings/Ironic

1 0

Re: [SDK][CLI][API][OpenAPI] Status Update
by Artem Goncharov 22 Nov '23

22 Nov '23

Hi all, as promised here is status update of the OpenAPI work, and sorry in advance for another long message Short version ========== I have extended codegerator [1] to build OpenAPI specs from the sources of the services. All the exposed URLs with wherever possible query parameters/request/body are placed into the OpenAPI spec and whenever available generated information is extended with descriptions from rendered API-Ref html to keep life easier. This helped me to spot a variety of issues: unknown/undesired routes exposed, undocumented routes, documentation for unexposed routes, etc Sadly not all services are having all the required information to build the spec so the generator is created in the way that supports extension and overrides for every service. - Nova has route versions, request_body and response codes, but no information about response json - Glance is not bad, but not every request/response info is available directly on the controller. - Cinder is not having query parameters information, no response body schema - Keystone is not having much of useful information directly attached to the routes, but is having useful part schemas in the code - Neutron is very special - Octavia is having request and response info available (as wsme, but I have written wsme=>jsonschema convertor) Current state of schemas can be found under [2]. Schemas are still with gaps, but general structure can be seen [1] https://review.opendev.org/c/openstack/openstacksdk/+/882326 [2] https://github.com/gtema/openstack-openapi/tree/main/specs Long version ========= codegenerator is able to inspect source code of services in diverse ways (requires service to be "pip installed" into the env of codegenerator). Generally I am looking at the api router/mapper of the service and start traversing from there. This ensures we document all the routes and not what we think about. It helped me to figure out i.e. Cinder is exposing all routes with and without project_id (this was honestly new to me and would have helped to reduce insanity in project_id handling in the SDK code), Octavia exposes all routes with /v2, /v2/lbaas, /v2.0 and /v2.0/lbaas prefixes (altogether 4 times same operations). It is also possible to provide rendered html of the API-Ref and codegenerator will try to extract all methods descriptions and even try to get parameters description as long as it is possible (no magic, best effort approach). Nova: uses typical wsgi + routes. All exposed routes in all versions can be seen. There are validations attached to all routes what provides jsonschemas about query parameters and request body. There is some non standard mess with flavors actions (flavor delete/create/update are exposed as wsgi actions what under the hood exposes /flavors/{flavor_id}/action with corresponding json bodies. Validation for those fails, but that results in a wrong errors being returned to the user. I assume this is rather a bug and not a feature. Sadly there are no schemas describing api responses at all, so this needs to be provided separately/manually. Cinder: similar to Nova, except that also no query parameters information is available. Glance: Glance is sometimes having useful information attached to the routes by having serializer/deserializer classes available. This helped getting nearly all metadefs documented at no effort. Sadly there are also negative exceptions: image data operations are not having content-type information attached directly, so this requires a hack in the generator to build a correct spec for those operations. Still I was positively surprised with the amount of useful info I was able to get out of the source code without "hacks". Keystone: Flask is a tricky thing. Looking at all the routes exposed I got specs for all the resources in HEAD mode. Current API Ref mentions that, but they are not described at all. But that is not a big deal. A more severe problem is that the same controller is exposed for /<RESOURCE> and /<RESOURCE>/{resource_id} routes. This leads to the fact that user can send i.e. DELETE/PUT to /projects which will fail with 500 because project_id is a positional argument in the controller and not having default. But that heavily depends on the function signature and in my eyes this is a huge potential vulnerability and a quality gap (what would happen when somebody accidentally adds default value to that parameter?). This is the same for all routes exposed and not only relevant to projects. Sadly no extractable information about query parameters is available, but jsonschemas of resources are there (even for the /auth/tokens ). This requires manual mapping in the generator to know where schemas can be obtained from. Neutron: This one caused me a 4 day long headache. I have solved most of the issues, but there is still work to be done - inspecting the app requires neutron to be running (with a db created, all the schemas populated, config with all the plugins/extensions enabled, ....) Here the problem is to have actually all the plugins configured (some of them appears just don't like to work together with others) - some routes are exposed as wsgi app (the extensions app), but basic resources (networks, subnets, ports) are exposed as not inspectable pecan stuff (using dynamic _lookup to find route controller is just too complex for inspection) - neutron is having api definition which can be treated as a schema, but requires sophisticated logic (describing bulk requests is actually a no-brainer this way either) - still current biggest item is to have a max configuration so that codegenerator can look at every possible plugin Octavia: It is also using the Pecan framework with dynamic lookups which is a huge problem for the inspection, but the amount of those is rather an exception that allows me to traverse through "static" paths and supplement them with "exceptions". In contrast to all the other services Octavia is attaching full request and response definitions to the routes. Those are not jsonschema (wsme types), but jsonschema can be generated out of that with ~100 magic lines Heat: have not started, but the first look tells it is also using routes lib so it should be easy to get exposed routes. However it doesn't seem to have any usable api schemas in the code. Swift: I created the spec manually before I started working on generating specs from source. Luckily Swift is not having any/major api changes so it should not be a problem. With API-Ref info flowing into the spec we can get very usable specs automatically. Ideally all this information should be in docstrings (this is where initial descriptions are coming from and stay if api-ref match is not found). In the meanwhile I got requests about the topic from some other contributors which is why I feel myself confirmed I am not working on something nobody needs. As such we might need to restructure how and where the work is done and eventually get a few new projects to host codegenerator and openapi specs. Ideally I would like openapi specs to be belonging to services themselves, but I think getting them together separately is a better choice for the current phase until generation is finalized. Anyway, having a generator allows us to create jobs generating new specs for the services. With the generated specs I am already able to generate cli code (and sdk and ansible modules and etc) Next steps ======== - decide on where to host the generator (currently under the openstacksdk project) and where to store the specs - continue adding missing information to the already generated specs - decide on how to deal with sphinx-openapi. It is present and Stephen seems to be a maintainer of it, but even ignoring OpenStack API specialities it just produces not the output we used to have with current os-api-ref. So eventually a new project for rendering specs in the current style is necessary. Doesn't look like a complex task though - continue looking at other services (heat, magnum, etc) Feel free to ping me directly for collaboration and/or questions. Regards, Artem

2 1

[manila] Cancelling Nov 23rd weekly meeting
by Carlos Silva 22 Nov '23

22 Nov '23

Hello Zorillas! As a lot of people from our regular audience have holidays during this week, we are cancelling the Nov 23rd Manila weekly meeting. In the Nov 30th meeting, we would like to use most of the time to review backports, so please join us to help getting some fixes landing in the stable branches. Thanks and regards, carloss

1 0

[publiccloud-sig] Summary meeting CW 47
by Felix Kronlage-Dammers 22 Nov '23

22 Nov '23

Hi, in today’s publiccloud-sig meeting we’ve continued the topic of “flavor discoverability”, a topic that has been part of our discussion for a while. Before I dive into that, a quick heads-up: So far we’ve been meeting in the odd weeks. We’ll be switching that to the even weeks. This means, we’ll be meeting again next week (Nov. 29th). For today’s meeting: The minutes for the meeting are here: https://etherpad.opendev.org/p/publiccloud-sig-meeting Since this was a videocall (and as such no meetingbot logs), here a quick summary (thanks @joek-office). ======= 0. Discussion on flavor/image/volume "search engine" aka flavor-discovery - (gtema) The most informations that can be interessting for clients to search resources are at the respective services (nova/glance/cinder) in place - nova has traits but this is not searchable (possible extension - nova mentioned this as a tricky development), especially for clients - the traits has the ability to give out internal informations about the infrastructure, so cloud provider's possibly not happy with this solution - (gtema) problems with finding the right flavor/image in a openstack cloud - The flavor definitions are more or less a range in that nova scheduler will do its best to meet the criteria, no guarantee to get what you want @gtema pls update/delete this - The architecture of the instance is more defined by the image than from the flavor - thats because in the image are the appropriate sources for a cpu architecture - at the moment two possible ideas idea 1: extend the services with the appropriate features to meet the requirements cons: - long development pros: - no moderation/weighting of results possible idea 2: build up a new service with required metadata based on the .wellknown concept supported by sdk right now (or discoverable via service catalogue - that's not in scope of the service catalog, but the new "discovery" service may be added into the catalog) cons: pros: - the service can moderate/weight results to meet criterias from cloud provider AND client side - available for older OpenStack versions ========= For people interested in this subject, we (at least gtema and me) are going to continue this discussion tomorrow at 1400 CET here: https://conf.scs.koeln:8443/OIF-public-cloud-sig felix -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[kolla] Weekly meeting cancelled today (22nd Nov 2023)
by Michał Nasiadka 22 Nov '23

22 Nov '23

Hello, Due to key participants absence - the weekly Kolla meeting is cancelled today. Let’s meet next week Thanks, Michal

1 0

[cinder][backup] backup_max_operations configuration is not working
by Pureewat Kaewpoi 22 Nov '23

22 Nov '23

Hi communities I have set cinder backup with these configurations backup_max_operations = 4 backup_workers = 4 When I create 20 backups simultaneously According to logs It still creating backups at the same time. 2023-11-07 06:17:42.029 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:42.038 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:44.328 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:44.334 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:46.665 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:46.672 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:49.005 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:49.018 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:51.378 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:51.389 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:54.004 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:54.017 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:56.094 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:56.113 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:17:58.523 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:17:58.536 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:01.017 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:01.027 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:03.146 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:03.162 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:05.622 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:05.649 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:07.800 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:07.818 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:10.129 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:10.148 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:12.684 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:12.775 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:16.356 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:16.400 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:17.480 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:17.575 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:20.227 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:20.249 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:22.115 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:22.165 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:24.379 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:24.435 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:27.681 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:28.074 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:29.440 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:30.324 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) 2023-11-07 06:18:31.846 555360 INFO cinder.backup.manager [] Create backup started, backup: <BACKUP-ID> volume: <VOLUME-ID> 2023-11-07 06:18:31.874 555360 INFO cinder.backup.manager [] Call Volume Manager to get_backup_device for Backup(*TRUNCRATED*) The expected result is creating 4 backups simultaneously Have anyone face this issue ? or Do I config a wrong configuration ? Thanks Pureewat Kaewpoi

3 3