May 2019 - openstack-discuss - lists.openstack.org

[nova][CI] GPUs in the gate
by Artom Lifshitz 13 May '19

13 May '19

Hey all, Following up on the CI session during the PTG [1], I wanted to get the ball rolling on getting GPU hardware into the gate somehow. Initially the plan was to do it through OpenLab and by convincing NVIDIA to donate the cards, but after a conversation with Sean McGinnis it appears Infra have access to machines with GPUs. >From Nova's POV, the requirements are: * The machines with GPUs should probably be Ironic baremetal nodes and not VMs [*]. * The GPUs need to support virtualization. It's hard to get a comprehensive list of GPUs that do, but Nova's own docs [2] mention two: Intel cards with GVT [3] and NVIDIA GRID [4]. So I think at this point the question is whether Infra can support those reqs. If yes, we can start concrete steps towards getting those machines used by a CI job. If not, we'll fall back to OpenLab and try to get them hardware. [*] Could we do double-passthrough? Could the card be passed through to the L1 guest via the PCI passthrough mechanism, and then into the L2 guest via the mdev mechanism? [1] https://etherpad.openstack.org/p/nova-ptg-train-ci [2] https://docs.openstack.org/nova/rocky/admin/virtual-gpu.html [3] https://01.org/igvt-g [4] https://docs.nvidia.com/grid/5.0/pdf/grid-vgpu-user-guide.pdf

5 8

[requirements] bandit bump to 1.6.0
by Herve Beraud 13 May '19

13 May '19

Hello, FYI bandit 1.6.0 was released and changes the behavior of the '-x' option so that it now supports glob patterns. Many openstack projects will facing bandit issues due to these changes. Two possibilities exists: - pin your bandit version to < 1.6.0 - accept 1.6.0 and modify your bandit call by passing a patterns like this https://review.opendev.org/#/c/658319/1 We also need to update openstack/requirements ( https://review.opendev.org/#/c/658767/) I think the better approach is to use 1.6.0 now and to fix the bandit command to avoid issues in the future, and avoid undesired reviews on this topic. Regards -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

2 2

[nova][ptg] Summary: Implicit trait-based filters
by Eric Fried 13 May '19

13 May '19

Summary: In keeping with the first proposed cycle theme [1] (though we didn't land on that until later in the PTG), we would like to be able to add required traits to the GET /allocation_candidates query to reduce the number of results returned - i.e. do more filtering in placement rather than in the scheduler (or worse, the compute). You can already do this by explicitly adding required traits to flavor/image; we want to be able to do it implicitly based on things like: - If the instance requires multiattach, make sure it lands on a compute that supports multiattach [2]. - If the image is in X format, make sure it lands on a compute that can read X format [3]. Currently the proposals in [2],[3] work by modifying the RequestSpec.flavor right before select_destinations calls GET /allocation_candidates. This just happens to be okay because we don't persist that copy of the flavor back to the instance (which we wouldn't want to do, since we don't want these implicit additions to e.g. show up when we GET server details, or to affect other lifecycle operations). But this isn't a robust design. What we would like to do instead is exploit the RequestSpec.requested_resources field [4] as it was originally intended, accumulating all the resource/trait/aggregate/etc. criteria from the flavor, image, *and* request_filter-y things like the above. However, gibi started on this [5] and it turns out to be difficult to express the unnumbered request group in that field for... reasons. Action: Since gibi is going to be pretty occupied and unlikely to have time to resolve [5], aspiers has graciously (been) volunteered to take it over; and then follow [2] and [3] to use that mechanism once it's available. efried [1] https://review.opendev.org/#/c/657171/1/priorities/train-priorities.rst@13 [2] https://review.opendev.org/#/c/645316/ [3] https://review.opendev.org/#/q/topic:bp/request-filter-image-types+(status:… [4] https://opendev.org/openstack/nova/src/commit/5934c5dc6932fbf19ca7f3011c4cc… [5] https://review.opendev.org/#/c/647396/

3 5

[neutron] Bug deputy report 2019-05-06 - 2019-05-13
by Lajos Katona 13 May '19

13 May '19

Hi Neutrinos, This week's bug deputy report for networking projects: Highlights: * There are 2 ryu related bugs with Rocky. * Unassigned bugs: 1828205, 1828547, 1828605, 1828375, 1828406 * 3 RFEs HIGH MEDIUM #1828053 GRE tunnels between VMs don't work when openvswitch firewall is used https://bugs.launchpad.net/neutron/+bug/1828053 Fix Released #1828205 "network-segment-ranges" doesn't return the project_id https://bugs.launchpad.net/neutron/+bug/1828205 Unassigned #1828543 Routed provider networks: placement API handling errors https://bugs.launchpad.net/neutron/+bug/1828543 Assigned #1828605 [l3][scale issue] unrestricted hosting routers in network node increase service operating pressure https://bugs.launchpad.net/neutron/+bug/1828605 Unassigned, related to #1828494 #1828721 [VPNaaS]: Check restart_check_config enabled https://bugs.launchpad.net/neutron/+bug/1828721 In Progress LOW #1828363 FdbInterfaceTestCase interface names should be randomly generated https://bugs.launchpad.net/neutron/+bug/1828363 In Progress #1828375 Bulk creation of subports fails with StaleDataError https://bugs.launchpad.net/neutron/+bug/1828375 Reported from Ocata, need more effort to reproduce on master/other branch #1828406 neutron-dynamic-routing bgp ryu hold timer expired but never tried to recover https://bugs.launchpad.net/neutron/+bug/1828406 Rocky with ryu, Unassigned #1828423 Very often in DHCP agent we have ""Duplicate IP addresses found, DHCP cache is out of sync" https://bugs.launchpad.net/neutron/+bug/1828423 Assigned #1828437 Remove unneeded compatibility conversions https://bugs.launchpad.net/neutron/+bug/1828437 In Progress #1828473 Dnsmasq spawned by neutron-dhcp-agent should use bind-dynamic option instead of bind-interfaces https://bugs.launchpad.net/neutron/+bug/1828473 In Progress #1828547 neutron-dynamic-routing TypeError: argument of type 'NoneType' is not iterable https://bugs.launchpad.net/neutron/+bug/1828547 Rocky with ryu, Unassigned RFE #1828367 [RFE] Ironic notifier - Notify Ironic on port status changes https://bugs.launchpad.net/neutron/+bug/1828367 In Progress #1828494 [RFE][L3] l3-agent should have its capacity https://bugs.launchpad.net/neutron/+bug/1828494 In Progress #1828607 [RFE] DVR Enhancements https://bugs.launchpad.net/neutron/+bug/1828607 In Progress Regards Lajos

1 0

Baremetal attach volume in Multi-tenancy
by zack chen 13 May '19

13 May '19

Hi, I am looking for a mechanism that can be used for baremetal attach volume in a multi-tenant scenario. In addition we use ceph as the backend storage for cinder. Can anybody give me some advice?

4 10

[devstack][stein][masakari]Using Masakari in multihost devstack stein setup
by Sanjay K 13 May '19

13 May '19

Hi all, I have been trying to setup Masakari in a 3 node setup - 1 controller + 2 computes with minimal openstack services installed using devstack stable/stein version. All these 3 nodes are Ubuntu 16.04 VMs. I have included Masakari plugin in controller's local.conf file and included masakari-monitor in both compute node's local.conf file. I want to test out VM/process failure in my test environment. To do so, when I kill one of the qemu process created for one instance (cirros 256 flavor VMs) on one of the compute with root user login, I did not see any notification under my horizon/instance-ha section for this (I have already created Segments, Hosts under instance-ha in horizon). Also the killed process is not restarted by Masakari. I have asked the same question in openstack forum, but not got a single response. masakari-notification-on-process-failure <https://ask.openstack.org/en/question/121490/masakari-notification-on-proce…> Whether any additional services related to Masakari need to be configured on compute hosts to detect failures since I did not find exactly the documentation related inside devstack for this. Please let me know if I am missing anything my set up. Any helps and pointers are most appreciated. thank you for your reply. best regards

2 4

[oslo] PTG Summary
by Ben Nemec 13 May '19

13 May '19

Hi, You can find the raw notes on the etherpad (https://etherpad.openstack.org/p/oslo-train-topics), but hopefully this will be an easier to read/understand summary. Pluggable Policy ---------------- Spec: https://review.opendev.org/#/c/578719/ Since this sort of ran out of steam last cycle, we discussed the option of not actually making it pluggable and just explicitly adding support for other policy backends. The specific one that seems to be of interest is Open Policy Agent. To do this we would add an option to enable OPA mode, where all policy checks would be passed through to OPA by default. An OPACheck class would also be added to facilitate migration (as a rule is added to OPA, switch the policy to OPACheck. Once all rules are present, remove the policy file and just turn on the OPA mode). However, after some further investigation by Patrick East, it was not clear if users were asking for this or if the original spec was more of a "this might be useful" thing. He's following up with some OPA users to see if they would use such a feature, but at this point it's not clear whether there is enough demand to justify spending time on it. Image Encryption/Decryption Library ----------------------------------- I mention this mostly because the current plan is _not_ to create a new Oslo library to enable the feature. The common code between services is expected to live in os-brick, and there does not appear to be a need to create a new encryption library to support this (yay!). oslo.service SIGHUP bug ----------------------- This is a problem a number of people have run into recently and there's been some ongoing, but spotty, discussion of how to deal with it. In Denver we were able to have some face-to-face discussions and hammer out a plan to get this fixed. I think we have a fix identified, and now we just need to get it proposed and tested so we don't regress this in the future. Most of the prior discussion and a previously proposed fix are at https://review.opendev.org/#/c/641907/ so if you want to follow this that's the place to do it. In case anyone is interested, it looks like this is a bug that was introduced with mutable config. Mutable config requires a different type of service restart, and that was never implemented. Now that most services are using mutable config, this is much bigger problem. Unified Limits and Policy ------------------------- I won't try to cover everything in detail here, but good progress was made on both of these topics. There isn't much to do from the Oslo side for the policy changes, but we identified a plan for an initial implementation of oslo.limit. There was general agreement that we don't necessarily have to get it 100% right on the first attempt, we just need to get something in the repo that people can start prototyping with. Until we release a 1.0 we aren't committed to any API, so we have flexibility to iterate. For more details, see: https://etherpad.openstack.org/p/ptg-train-xproj-nova-keystone oslo.service profiling and pypy ------------------------------- Oslo has dropped support for pypy in general due to lack of maintainers, so although the profiling work has apparently broken oslo.service under pypy this isn't something we're likely to address. Based on our conversation at the PTG game night, it sounds like this isn't a priority anymore anyway because pypy didn't have the desired performance improvement. oslo.privsep eventlet timeout ----------------------------- AFAICT, oslo.privsep only uses eventlet at all if monkey-patching is enabled (and then only to make sure it returns the right type of pipe for the environment). It's doubtful any eventlet exceptions are being raised from the privsep code, and even if they are they would go away once monkey-patching in the calling service is disabled. Privsep is explicitly not depending on eventlet for any of its functionality so services should be able to freely move away from eventlet if they wish. Retrospective ------------- In general, we got some major features implemented that unblocked things either users or services were asking for. We did add two cores during the cycle, but we also lost a long-time Oslo core and some of the other cores are being pulled away on other projects. So far this has probably resulted in a net loss in review capacity. As a result, our primary actions out of this were to continue watching for new candidates to join the Oslo team. We have at least one person we are working closely with and a number of other people approached me at the event with interest in contributing to one or more Oslo projects. So while this cycle was a bit of a mixed bag, I have a cautiously optimistic view of the future. Service Healthchecks and Metrics -------------------------------- Had some initial hallway track discussions about this. The self-healing SIG is looking into ways to improve the healthcheck and metric situation in OpenStack, and some of them may require additions or changes in Oslo. There is quite a bit of discussion (not all of which I have read yet) related to this on https://review.opendev.org/#/c/653707/ On the metrics side, there are some notes on the SIG etherpad (currently around line 209): https://etherpad.openstack.org/p/DEN-self-healing-SIG It's still a bit early days for both of these things so plans may change, but it seems likely that Oslo will be involved to some extent. Stay tuned. Endgame ------- No spoilers, I promise. If you made it all the way here then thanks and congrats. :-) I hope this was helpful, and if you have any thoughts about anything above please let me know. Thanks. -Ben

2 1

[octavia][lbaas][neutron-lbaas] Octavia v1 API and neutron-lbaas retirement
by Adam Harwell 12 May '19

12 May '19

Hi Octavia folks, This email is to announce that we are currently in the process of retiring both neutron-lbaas and the Octavia v1 API. We said this would happen in September 2019 or the OpenStack "U" release, whichever came first. Because September of 2019 occurs during the Train release cycle, there will be no Train release of neutron-lbaas. We will therefore be retiring the neutron-lbaas repository and the Octavia v1 API effective immediately, as there is no reason to continue to maintain them if no release is forthcoming. Reminder: The Octavia v1 API was used by the neutron-lbaas Octavia provider driver and was not an end-user API. The Octavia v1 API is a proprietary API and is not compatible with the LBaaS v2 API specification or any other LBaaS API specification. If you have questions about this deprecation cycle please see the FAQ wiki: https://wiki.openstack.org/wiki/Neutron/LBaaS/Deprecation If you have questions or concerns not covered by the FAQ, please reach out to the Octavia team either by using the "[octavia]" subject prefix on this mailing list or in our IRC channel #openstack-lbaas. We look forward to working with you moving forward with Octavia LBaaS! --Adam

2 3

[devstack-plugin-container][zun][kuryr] Extend core team for devstack-plugin-container
by Hongbin Lu 11 May '19

11 May '19

Hi all, I propose to add Zun and Kuryr core team into devstack-plugin-container. Right now, both Zun and Kuryr are using that plugin and extending the core team would help accelerating the code review process. Please let me know if there is any concern of the proposal. Best regards, Hongbin

1 1

[tc][searchlight] What does Maintenance Mode mean for a project?
by Trinh Nguyen 11 May '19

11 May '19

Hi, Currently, in the project details section of Searchlight page [1], it says we're in the Maintenance Mode. What does that mean? and how we can update it? Thanks, [1] https://www.openstack.org/software/releases/rocky/components/searchlight -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

2 2