May 2019 - openstack-discuss - lists.openstack.org

[ironic][neutron][ops] Ironic multi-tenant networking, VMs
by Jeremy Freudberg 05 May '19

05 May '19

Hi all, I'm wondering if anyone has any best practices for Ironic bare metal nodes and regular VMs living on the same network. I'm sure if involves Ironic's `neutron` multi-tenant network driver, but I'm a bit hazy on the rest of the details (still very much in the early stages of exploring Ironic). Surely it's possible, but I haven't seen mention of this anywhere (except the very old spec from 2015 about introducing ML2 support into Ironic) nor is there a gate job resembling this specific use. Ideas? Thanks, Jeremy

3 4

[qa][form][ptg] QA Summary for Forum & PTG
by Ghanshyam Mann 05 May '19

05 May '19

Hello Everyone, We had a good discussion at QA forum and PTG. I am summarizing those and will start the separate thread for the few topics which needs more feedback. Summit: QA Forum sessions: 1. OpenStack QA - Project Update: Tuesday, April 30, 2:35pm-2:55pm We gave the updates on what we finished on Stein and draft plan for Train cycle. The good thing to note is we still have a lot of activity going on in QA. As overall QA projects, we did >3000 reviews and 750 commits. Video is not up still so I am copying the slide link below. Slides: https://docs.google.com/presentation/d/10zupeFZuOlxroAMl29qVJl78nD4_YWHkQxA… 2. OpenStack QA - Project Onboarding : Wednesday, May 1, 9:00am-9:40am We did host the QA onboarding sessions but there were only 3 attendees and no new contributor. I think it is hard to see any new contributor in summits now so I am thinking whether we should host the onboarding sessions from next time. Etherpad: https://etherpad.openstack.org/p/DEN-qa-onboarding 3. Users / Operators adoption of QA tools / plugins : Wednesday, May 1, 10:50am-11:30am3. As usual, we had more attendees in this session and useful feedback. There are few tooling is being shared by attendees: 1. Python hardware module for bare metal detailed hardware inspection & anomaly detection https://github.com/redhat-cip/hardware 2. Workload testing: https://opendev.org/x/tobiko/ Another good idea from Doug was plugin feature in openstack-health dashboard. That is something we discussed in PTG. For more details on this, refer the PTG " OpenStack-health improvement" section. Etherpad: https://etherpad.openstack.org/p/Den-forum-qa-ops-user-feedback QA PTG: 2nd - 3rd May: We were 3-4 attendee in the room always and others attended per topics. Good discussions and few good improvement ideas about gate stability and dashboard etc. 1. Topic: Stein Retrospective We collect good and need improvement things in this session. In term of good things, we completed the OpenStack gate migration fro Xenial to Bionic, lot of reviews and code. Doug from AT&T mentioned about to add tempest and patrole to gates and check in their production deployment process, "Thank you for all of the hard work from the QA team!!!" Slow reviews are a concern as we have a good number of the incoming request. This is something we should improve in Train. Action items: gmann: start the plan for backlogs especially for review and doc cleanup. masayukig: plan to have resource leakage check in gate. ds6901:will work with his team to clean up leaks and submit bugs 2. Topic: Keystone system-scope testing QA and Keystone team gathered together in this cross-project session about next steps on system scope testing. We talked on multiple points about how to cover all new roles for system scope and how to keep the backward compatibility testing for stable branches still testing the without system scope. We decided to move forward for system_admin as of now and fall back the system_admin to project scope if there is no system_scope testing flag is true on Tempest side (this will cover the stable branch testing unaffected). We agreed : - To move forward with system admin - https://review.opendev.org/#/c/604909/ - Add tempest job to test system scope - https://review.opendev.org/#/c/614484/ - Then add to tempest full - gmann - Then add testing for system reader - Investigate more advanced RBAC testing with Patrole - gmann Etherpad: https://etherpad.openstack.org/p/keystone-train-ptg-testing-system-scope-in… 3. Topic: new whitebox plugin for tempest: This is a new idea from artom about testing things outside of Tempest's scope (currently mostly used to check instance XML for NFV use case tests). Currently, this tool does ssh into VM and fetch the xml for further verification etc. We agreed on point to avoid any duplicate test verification from the Tempest or nova functional tests This is good to tests from more extra verification by going inside VM like after migration data, CPU pinning etc. As next step artom to propose the QA spec with details and proposal of this plugin under QA program. 4. Topic: Document the QA process or TODO things for releases, stable branch cut: Idea is to start a centralized doc page for QA activities and process etc. we want to use the qa-specs repo to publish the content to doc.openstack.org/qa/. This can be not so easy and need few tweaks on doc jobs. I will get into the details and then discuss with infra team. This is a low priority for now. 5. Topic: Plugin sanity check Current tempest-plugins-sanity job is not stable and so it is n-v. We want to make it voting by only installing the active plugins. many plugins are failing either they are dead or not so active. We agreed on: - make faulty plugins as blacklist with bug/patch link and notify the same on ML every time we detect any failure - Publish the blakclist on plugins-registry doc. - After that make this job voting, make the process of fixing and removing the faulty plugin which unblocks the tempest gate with author self-approve. - Make sanity job running on plugins which are dependent on each other. For example, congress-tempest-plugin use neutron-tempest-plugin, mistral-tempest-plugin etc so all these plugins should have a sanity job which can install and list these plugins tests only not all the plugins. 6. Topic: Planning for Patrole Stable release: We had a good amount of discussions for Patrole improvements area to release it stable. Refer the below ML thread for details and further discussions on this topic: - http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005870.html 7. Topic: How to make tempest-full stable ( Don't fail integrated job when not related test will fail ) Current integrated-gate jobs (tempest-full) is not so stable for various bugs specially timeout. We discussed the few ideas to improve it. Refer the below ML thread for details and further discussions on this topic : http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html 8. Topic: OpenStack-Health Improvement Doug from AT&T has few improvement ideas for health dashboard which has been discussed in PTG: - Test Grouping - Define groups - Assigned test to groups - filter by groups - Compare 2 runs = Look into push AQuA report to subunit2SQL as a tool Action Items: - Doug is going to write the spec for plugin approach. All other ideas can be done after we have the plugin approach ready. - filter - presentation 9. Topic: Stein Backlogs & Train priorities & Planning We collected the Train items in below mentioned etherpad with the assignee. Anyone would like to help on any of the item, ping me on IRC or reply here. Etherpad: https://etherpad.openstack.org/p/qa-train-priority 10. Topic: grenade zuulv3 jobs review/discussions We did not get the chance to review these. Let's continue it after PTG. Full Detail discussion: https://etherpad.openstack.org/p/qa-train-ptg -gmann

1 0

[fenix][ptg] summary
by Juvonen, Tomi (Nokia - FI/Espoo) 04 May '19

04 May '19

Fenix Train PTG, What to do next, prioritizing https://etherpad.openstack.org/p/DEN2019-fenix-PTG Two non-Telco users would like to use Fenix to maintain their cloud. For this, Fenix need to prioritize work so we can provide production ready framework without Telco features first. Work is now prioritized in the Etherpad and missing things should also be added to storyboard next week. Fenix and ETSI NFV synch https://etherpad.openstack.org/p/DEN2019-fenix-ETSI-NFV-PTG There was also a discussion about supporting ETSI NFV defined constraints. Some instance and anti-affinity group constraints could be in Nova. Anyhow, for Fenix to be generic for any cloud, it would make sense to have more information kept within Fenix. This needs further investigation. Then there was a proposal of having direct subscription to Fenix from VNFM side instead of using subscription to AODH to have event alarm from Fenix notification to VNFM. One bad thing here was that VIM shouldn't have direct API call to external system. The current notification / AODH was also nice for any user to build simple manager just to receive the hint what new capability is coming to application (non-Telco) or to still have some simple solution to interact at the time of maintenance. Thanks, Tomi (tojuvone)

1 0

[nova][ptg] Summary/Outcome: Train Cycle Themes
by Eric Fried 04 May '19

04 May '19

Etherpad: https://etherpad.openstack.org/p/nova-train-themes Summary: In Stein, we started doing cycle themes instead of priorities. The distinction being that themes should represent tangible user (as in OpenStack consumer) facing value, whereas priorities represent what work items we want to do. Outcome: We decided on themes around: (1) The use of placement (2) Cyborg integration (3) Docs I have curated the etherpad and discussions and proposed these themes to the nova-specs repository at https://review.opendev.org/657171 efried .

1 0

[nova][ptg] Summary: Secure Boot support for QEMU- and KVM-based Nova instances
by Kashyap Chamarthy 04 May '19

04 May '19

Spec: https://review.opendev.org/#/c/506720/ -- Add "Secure Boot support for KVM & QEMU guests" spec Summary: - Major work in all the lower-level dependencies: OVMF, QEMU and libvirt is ready. Nova can now start integrating this feature. (Refer to the spec for the details.) - [IN-PROGRESS] Ensure that the Linux distributions Nova cares about ship the OVMF firmware descriptor files. (Requires QEMU 4.1, coming out in August. Refer this QEMU patch series; merged in Git master: https://lists.nongnu.org/archive/html/qemu-devel/2019-04/msg03799.html bundle edk2 platform firmware with QEMU.) - NOTE: This is not a blocker for Nova. We can parallely hammer away at the work items outlined in the spec. - [IN-PROGRESS] Kashyap is working with Debian folks to ship a tool ('ovmf-vars-generator') to enroll default UEFI keys for Secure Boot. - Filed a Debian "RFP" for it https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=927414 - Fedora already ships it; Ubuntu is working on it (https://launchpad.net/ubuntu/+source/edk2/0~20190309.89910a39-1ubuntu1) - NOTE: This is not a blocker, but a nice-to-have, because distributions already ship an OVMF "VARS" (variable store file) with default UEFI keys enrolled. - ACTION: John Garbutt and Chris Friesen to review the Nova spec. (Thanks!) -- /kashyap

1 0

[placement][nova][ptg] Summary: Testing PlacementFixture effectively
by Chris Dent 04 May '19

04 May '19

See also: https://etherpad.openstack.org/p/ptg-train-xproj-nova-placement Nova uses the PlacementFixture (provided by placement) to be able to do functional tests with a real placement API and database. This works pretty well but we discovered during the run to the end of Stein that seemingly unrelated changes in placement could break the fixture and bring nova's gate to a halt. Bad. Decision: Placement will run nova's functional tests in its own gate on each change. If it proves to save some time the api_sample tests will be blacklisted. We do not want to whitelist as that will lead to trouble in the future. There was discussion of doing this for osc-placement as well, but since we just saved a bunch of elapsed time with functional tests in osc-placement with https://review.opendev.org/#/c/651939/ and there's no integrated gate criticality with osc-placement, we decided not to. Action: cdent will make a story and do this -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

1 1

[tripleo] Nominate Cédric Jeanneret (Tengu) for tripleo-validations core
by Gaël Chamoulaud 04 May '19

04 May '19

Hi TripleO devs, The new Validation Framework is a big step further for the tripleo-validations project. In many ways, it improves the way of detecting & reporting potential issues during a TripleO deployment. As the mastermind of this new framework, Cédric brought a new lease of life to the tripleo-validations project. That's why we would highly benefit from his addition to the core reviewer team. Assuming that there are no objections, we will add Cédric to the core team next week. Thanks, Cédric, for your excellent work! =Gaël

10 10

[Vitrage] add datasource kapacitor for vitrage
by hoanq13＠viettel.com.vn 03 May '19

03 May '19

Hi, In our system, we use monitor by TICK stack (include: Telegraf for collect metric, InfluxDB for storage metric, Chronograf for visualize and Kapacitor alarming), which is popular monitor solution. We hope can integrate vitrage in, so we decide to write kapacitor datasource contribute for vitrage. The work is almost done , you can review in: https://review.opendev.org/653416 So i send this mail hope for more review, ideal,... Appreciate it. also ask: have any step i miss in pipeline of contribute datasource vitrage? like create blueprints, vitrage-spec,vv.. Should i do it?

2 2

[ZUN] Proxy on Docker + Zun
by Alejandro Ruiz Bermejo 03 May '19

03 May '19

I'm still working on my previous error of the openstack appcontainer run error state: I have Docker working behind a Proxy. As you can see in the Docker info i attach to this mail. I tried to do the curl http://10.8.9.54:2379/health with the proxy environment variable and i got timeout error (without it the curl return the normal healthy state for the etcd cluster). So my question is if i'm having a problem with the proxy configuration and docker commands when i'm executing the openstack appcontainer run. And if you know any use case of someone working with Docker behind a proxy and Zun in the Openstack environment. This is the outputh of # systemctl show --property Environment docker Environment=HTTP_PROXY=http://10.8.7.60:3128/ NO_PROXY=localhost, 127.0.0.0/8,10.8.0.0/16 HTTPS_PROXY=http://10.8.7.60:3128/ And this is the one of root@compute /h/team# docker info Containers: 9 Running: 0 Paused: 0 Stopped: 9 Images: 7 Server Version: 18.09.5 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84 runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30 init version: fec3683 Security Options: apparmor seccomp Profile: default Kernel Version: 4.15.0-48-generic Operating System: Ubuntu 18.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 8 Total Memory: 15.66GiB Name: compute ID: W35H:WCPP:AM3K:NENH:FEOR:S23C:N3FZ:QELB:LLUR:USMJ:IM7W:YMFX Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false HTTP Proxy: http://10.8.7.60:3128/ HTTPS Proxy: http://10.8.7.60:3128/ No Proxy: localhost,127.0.0.0/8,10.8.0.0/16 Registry: https://index.docker.io/v1/ Labels: Experimental: false Cluster Store: etcd://10.8.9.54:2379 Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine WARNING: API is accessible on http://compute:2375 without encryption. Access to the remote API is equivalent to root access on the host. Refer to the 'Docker daemon attack surface' section in the documentation for more information: https://docs.docker.com/engine/security/security/#docker-daemon-attack-surf… WARNING: No swap limit support

2 1

[nova][ptg] Summary: docs
by Eric Fried 03 May '19

03 May '19

Summary: Nova docs could use some love. Agreement: Consider doc scrub as a mini-theme (cycle themes to be discussed Saturday) to encourage folks to dedicate some amount of time to reading & validating docs, and opening and/or fixing bugs for discovered issues. efried .

2 1