November 2019 - openstack-discuss

[ptg] Auto-generated etherpad links !
by Thierry Carrez 13 Nov '19

13 Nov '19

Hi everyone, The PTGbot grew a new feature over the summer. It now dynamically generates the list of PTG track etherpads. You can find that list at: http://ptg.openstack.org/etherpads.html If you haven't created your etherpad already, just follow the link there to create your etherpad. If you have created your track etherpad already under a different name, you can overload the automatically-generated name using the PTGbot. Just join the #openstack-ptg channel and (as a Freenode authenticated user) send the following command: #TRACKNAME etherpad <URL for your etherpad> Example: #keystone etherpad https://etherpad.openstack.org/p/awesome-keystone-pad That will update the link on that page automatically. Hoping to see you in Shanghai! -- Thierry Carrez (ttx)

2 1

[kuryr] [tc] kuryr project mission
by Michał Dulko 13 Nov '19

13 Nov '19

Hi, It's been more than a year after fuxi [1] and fuxi-kubernetes [2] were declared as retired. First one was supposed to integrate Docker with Cinder and Manila. The second was aiming to do the same with Kubernetes. I'm not really sure how to serve the first use case, but it seems like there are some alternatives [3], [4]. fuxi-kubernetes use case is now served by Cloud Provider OpenStack in Cinder [5] and Manila [6] CSI plugins, which seem like a much better fit. Currently the only maintained deliverables in kuryr project are kuryr, kuryr-libnetwork and kuryr-kubernetes. All of them are related to networking. Given that I'd like to propose rephrasing Kuryr mission statement from: > Bridge between container framework networking and storage models > to OpenStack networking and storage abstractions. to > Bridge between container framework networking models > to OpenStack networking abstractions. effectively getting storage out of project scope. Are there any thoughts or objections? Maybe someone sees a better phrasing? Thanks, Michał [1] https://opendev.org/openstack/fuxi [2] https://opendev.org/openstack/fuxi-kubernetes [3] https://rexray.readthedocs.io/en/stable/user-guide/schedulers/docker/plug-i… [4] https://github.com/j-griffith/cinder-docker-driver [5] https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usi… [6] https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usi…

5 4

[tc][horizon][all] Horizon plugins maintenance
by Ivan Kolodyazhny 13 Nov '19

13 Nov '19

Hi team, As you may know, we've got a pretty big list of Horizon Plugins [1]. Unfortunately, not all of them are in active development due to the lack of resources in projects teams. As a Horizon team, we understand all the reasons, and we're doing our best to help other teams to maintain plugins. That's why we're proposing our help to maintain horizon plugins. We raised this topic during the last Horizon weekly meeting [2] and we'll have some discussion during the PTG [3] too. There are a lot of Horizon changes which affect plugins and horizon team is ready to help: - new Django versions - dependencies updates - Horizon API changes - etc. To get faster fixes in, it would be good to have +2 permissions for the horizon-core team for each plugin. We helped Heat team during the last cycle adding horizon-core to the heat-dashboard-core team. Also, we've got +2 on other plugins via global project config [4] and via Gerrit configuration for (neutron-*aas-dashboard, tuskar-ui). Vitrage PTL agreed to do the same for vitrage-dashboard during the last meeting [5]. Of course, it's up to each project to maintain horizon plugins and it's responsibilities but I would like to raise this topic to the TC too. I really sure, that it will speed up some critical fixes for Horizon plugins and makes users and operators experience better. [1] https://docs.openstack.org/horizon/latest/install/plugin-registry.html [2] http://eavesdrop.openstack.org/meetings/horizon/2019/horizon.2019-10-16-15.… [3] https://etherpad.openstack.org/p/horizon-u-ptg [4] http://codesearch.openstack.org/?q=horizon-core&i=nope&files=&repos=opensta… [5] http://eavesdrop.openstack.org/meetings/vitrage/2019/vitrage.2019-10-23-08.… Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

9 9

[ptg][glance] PTG summary
by Abhishek Kekane 13 Nov '19

13 Nov '19

Hi All, I attended OpenInfra summit and PTG at Shanghai last week. It was an interesting event with lots of discussion happening around different OpenStack projects. I was mostly associated with Glance and cross-projects work related to Glance. There were other topics around Edge, UI and QA. During summit Me and Erno gave a Glance project update where we discussed what we achieved in Train cycle and what we are going to do in Ussuri cycle. As multiple stores feature is stabilized during Train, In Ussuri main focus of glance is on enhancing /v2/import API to import single image into multiple stores and copying existing images to multiple stores to avoid the manual efforts required by operator to copy the image across the stores. New delete API will also be added to delete the image from single store, also cinder driver of glance_store needs refactoring so that it can use multiple backends configured by cinder. Efforts will be continued for cluster awareness of glance API during this cycle as well. Apart from these edge related work, Glance team will also work on removing deprecated registry and related functional tests, removing of sheepdog driver from glance_store, adding s3 driver with multiple stores support in glance_store and some urgent bug fixes. Cross-Project work: In this PTG we had discussion with Nova and Cinder regarding the adoption of multiple store feature of Glance. As per discussion we have finalized the design and Glance team will work together with Nova and Cinder towards adding multiple store support feature in Train cycle. Support for Glance multiple stores in Cinder: As per discussion, volume-type will be used to add which store the image will be uploaded on upload-to-image operation, also cinder will send base image id to glance as a header using which glance will upload the image created from volume to all those stores in which base image is present. Nova snapshots to dedicated store: Agreement is, Nova will send us a base image id to glance as a header using which glance will upload the instance snapshot to all those stores in which base image is present. Talk with QA team: Glance has also talked with QA team for adding new tempest coverage for newly added features in the last couple of cycles, Glance team will work with tempest to add below new tempest tests. 1. New import workflow (image conversion, inject metadata etc.) - Depends on https://review.opendev.org/#/c/545483/ devstack patch 2. Hide old images 3. Multiple stores: https://review.opendev.org/#/c/689104/ in devstack 3.1 Devstack patch + zuul job to setup multiple stores and the job will run on glance and run glance api and scenario tests 4. Delete barbican secrets from glance images 4.1 add the tests the in barbican-tempest-plugin 4.2 run as part of barbican gate using their job 4.3 run that tests with new job (multi stores) on glance gate. do not run barbican job on glance. Below is the Ussuri cycle planning and deadline for Glance. Ussuri milestone planning: Ussuri U1 - December 09-13: 1. Import image in multiple stores (Specs + Implementation) 2. Copy existing image in multiple stores (Specs + Implementation) 3. S3 driver for glance 4. remove sheepdog driver from glance_store 5. Fix subunit parser error 6. Modify existing nova and cinder specs Ussuri U2 - February 10-14 1. Cluster awareness of glance API nodes 2. remove registry code 3. Delete image from single store 4. Nova and Cinder upload snapshot/volume to glance 5. image-import.conf parsing issue with uwsgi Ussuri U3 - April 06-10 1. Multiple cinder store support in glance_store (specs + implementation) 2. Creating image from volume using ceph (slow uploading issue) 3. Image encryption 4. Tempest work Glance PTG planning etherpad: https://etherpad.openstack.org/p/Glance-Ussuri-PTG-planning Let me know if you guys need more details on this. Thanks & Best Regards, Abhishek Kekane

1 0

[nova][ops][stable] Any interest in backporting --dry-run and/or --instance options for heal_allocations?
by Matt Riedemann 13 Nov '19

13 Nov '19

I was helping someone recover from a stuck live migration today where the migration record was stuck in pre-migrating status and somehow the request never hit the compute or was lost. The guest was stopped on the guest and basically the live migration either never started or never completed properly (maybe rabbit dropped the request or the compute service was restarted, I don't know). I instructed them to update the database to set the migration record status to 'error' and hard reboot the instance to get it running again. Then they pointed out they were seeing this in the compute logs: "There are allocations remaining against the source host that might need to be removed" That's because the source node allocations are still tracked in placement by the migration record and the dest node allocations are tracked by the instance. Cleaning that up is non-trivial. I have a troubleshooting doc started for manually cleaning up that kind of stuff here [1] but ultimately just told them to delete the allocations in placement for both the migration and the instance and then run the heal_allocations command to recreate the allocations for the instance. Since this person's nova deployment was running Stein, they don't have the --dry-run [2] or --instance [3] options for the heal_allocations command. This isn't a huge problem but it does mean they could be healing allocations for instances they didn't expect. They could work around this by installing nova from train or master in a VM/container/virtual environment and running it against the stein setup, but that's maybe more work than they want to do. The question I'm posing is if people would like to see those options backported to stein and if so, would the stable team be OK with it? I'd say this falls into a gray area where these are things that are optional, not used by default, and are operational tooling so less risk to backport, but it's not zero risk. It's also worth noting that when I wrote those patches I did so with the intent that people could backport them at least internally. [1] https://review.opendev.org/#/c/691427/ [2] https://review.opendev.org/#/c/651932/ [3] https://review.opendev.org/#/c/651945/ -- Thanks, Matt

5 5

[nova][ptg] PCI refactoring needs and a strawman proposal inside
by Sylvain Bauza 13 Nov '19

13 Nov '19

Based on some Forum discussions, we had a de facto conversation at the Nova PTG about any potential things we could do for helping our operators, but also potentially Cyborg since they use the PCI passthrough capabilities. The feedback from ops (thanks mnaser) was that PCI passthrough works pretty smoothly but there are some ugly issues where we could improve the UX. The agreement in the room was, at least for Ussuri, to start collecting some ideas on how we could model Placement usage for PCI devices and also write motivations for such things in a spec. We also agreed on a smooth upgrade plan where PCITracker would still be present for a couple of releases until we're able to close the feature gap. Bauzas volunteered for drafting the spec and gibi, stephenfin and bauzas started to sketch up the Placement modeling on a whiteboard, picture to be shared in a follow-up.

3 2

[nova] spec is ready to review and for feature liasion needed
by Brin Zhang(张百林) 13 Nov '19

13 Nov '19

Hi everyone: This spec comment by melwitt, and already updated, wait for review and need a feature liaison. Proposal for a safer noVNC console with password authentication https://review.opendev.org/#/c/623120 Brin Zhang

2 2

[stable][cinder] Proposal to remove John Griffith from cinder-stable-maint
by Brian Rosmaita 12 Nov '19

12 Nov '19

John Griffith has taken on other commitments and stepped down as a cinder-core recently, so it doesn't make sense for him to continue on the cinder-stable-maint list. I'd like to acknowledge his role as "The Father of Cinder", though, and express my thanks on behalf of the Cinder team for all his past work on the project. cheers, brian

2 1

[ptg][nova][cinder] x-p meeting minutes
by Sylvain Bauza 12 Nov '19

12 Nov '19

Excerpts taken from https://etherpad.openstack.org/p/shanghai-ptg-cinder We discussed two items : 1/ Improving replication - When a failover happens volumes are no longer usable in Nova. - Nova should start re-attaching volumes after a failover? - Why can't we detach and attach the volume? Data that is in flight would be lost. - Question about boot from volume. In that case the instance is dead anyway because access to the volume has been lost. - Could go through the shutdown, detach, attach, reboot path. - Problem is that detach is going to fail. Need to force it or handle the failure. - We aren't sure that Nova will allow a detach of a boot volume. - Also don't currently have a force detach API. - AGREE: Need to figure out how to pass the force to os-brick to detach volume and when rebooting a volume. 2/ Nova bug for images created from encrypted volumes - Nova is not creating a new key for encrypted images but the deletion policy metadata can allow a key to be deleted wehn it is still in use by other images or even volumes - Nova needs to clone the keys when doing create image. - The nova team thinks that we have found a bug that needs to be fixed. We just need to open a bug. - ACTION Cinder to open a bug against Nova. (can you also ping lyarwood when it's open?) -S

3 2

[keystone] Shanghai forum/PTG recap
by Colleen Murphy 12 Nov '19

12 Nov '19

While the keystone team didn't itself meet at last week's PTG, I posted a recap of the event from a keystone perspective here: http://www.gazlene.net/shanghai-forum-ptg.html Hope it's a useful summary for those who couldn't attend in-person. Colleen

1 0