October 2019 - openstack-discuss - lists.openstack.org

[keystone] Keystone Team Update - Week of 7 October 2019
by Colleen Murphy 12 Oct '19

12 Oct '19

# Keystone Team Update - Week of 7 October 2019 ## News ### RC2 We ended up cutting a second RC in order to remove the policy.v3cloudsample.json file[1] and to include placeholder schema migrations[2]. That RC should become the Train release next week[3]. [1] https://review.opendev.org/687639 [2] https://review.opendev.org/687775 [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010074.… ## Office Hours When there are topics to cover, the keystone team holds office hours on Tuesdays at 17:00 UTC. There won't be office hours next week. Add topics you would like to see covered during office hours to the etherpad: https://etherpad.openstack.org/p/keystone-office-hours-topics ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 17 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 32 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Milestone Outlook https://releases.openstack.org/train/schedule.html Next week is the Train release! ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[neutron][security group][IPv6] IPv6 ICMPv6 port security in security group
by Xing Zhang 11 Oct '19

11 Oct '19

Hi all, When using neutron on CentOS 7 with OVSHybridIptablesFirewallDriver, create a vm with IPv4/IPv6 dual stack port, then remove all security group, we can get response with ping dhcp or router using IPv6 address in vm, while IPv4 can't. IPv6 works different with IPv4 in some cases and some useful function must work with ICMPv6 like NDP, NS, NA. Checking these two links below, neutron only drop IPv6 RA from vm, and allow all ICMPv6 ICMPv6 Type 128 Echo Request and Type 129 Echo Reply are allowed by default. Should we try to restrict ICMPv6 some types or there are some considerations and just follow ITEF 4890? IETF 4890 [section 4.3.2. Traffic That Normally Should Not Be Dropped] mentioned that: As discussed in Section 3.2 <https://tools.ietf.org/html/rfc4890#section-3.2>, the risks from port scanning in an IPv6 network are much less severe, and it is not necessary to filter IPv6 Echo Request messages. [section 3.2. Probing] However, the very large address space of IPv6 makes probing a less effective weapon as compared with IPv4 provided that addresses are not allocated in an easily guessable fashion. https://github.com/openstack/neutron/commit/a8a9d225d8496c044db7057552394af… https://www.ietf.org/rfc/rfc4890.txt Commands are: neutron port-update --no-security-groups 0307f016-0cc8-468b-bf3e-36ebe50e13ac ping6 from vm to dhcp ip6tables rules in compute node: PS: seems rules for type 131/135/143 are included in the rule # ip6tables-save | grep 08a0812a -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 143 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 134 -m comment --comment "Drop IPv6 Router Advts from VM Instance." -j DROP -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback full rules are at Ref #3 REF #1 ml2_config.ini [securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver Ref #2 Chain neutron-openvswi-o08a0812a-9 (2 references) pkts bytes target prot opt in out source destination 0 0 RETURN icmpv6 * * :: ff02::/16 ipv6-icmptype 131 /* Allow IPv6 ICMP traffic. */ 1 72 RETURN icmpv6 * * :: ff02::/16 ipv6-icmptype 135 /* Allow IPv6 ICMP traffic. */ 2 152 RETURN icmpv6 * * :: ff02::/16 ipv6-icmptype 143 /* Allow IPv6 ICMP traffic. */ 5 344 neutron-openvswi-s08a0812a-9 all * * ::/0 ::/0 0 0 DROP icmpv6 * * ::/0 ::/0 ipv6-icmptype 134 /* Drop IPv6 Router Advts from VM Instance. */ 5 344 RETURN icmpv6 * * ::/0 ::/0 /* Allow IPv6 ICMP traffic. */ 0 0 RETURN udp * * ::/0 ::/0 udp spt:546 dpt:547 /* Allow DHCP client traffic. */ 0 0 DROP udp * * ::/0 ::/0 udp spt:547 dpt:546 /* Prevent DHCP Spoofing by VM. */ 0 0 RETURN all * * ::/0 ::/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */ 0 0 DROP all * * ::/0 ::/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */ 0 0 neutron-openvswi-sg-fallback all * * ::/0 ::/0 /* Send unmatched traffic to the fallback chain. */ Ref #3 # ip6tables-save | grep 08a0812a -A neutron-openvswi-PREROUTING -m physdev --physdev-in qvb08a0812a-9e -m comment --comment "Set zone for 812a-9ef7-45e3-9d81-9463dd80e63e" -j CT --zone 4104 -A neutron-openvswi-PREROUTING -i qvb08a0812a-9e -m comment --comment "Set zone for 812a-9ef7-45e3-9d81-9463dd80e63e" -j CT --zone 4104 -A neutron-openvswi-PREROUTING -m physdev --physdev-in tap08a0812a-9e -m comment --comment "Set zone for 812a-9ef7-45e3-9d81-9463dd80e63e" -j CT --zone 4104 :neutron-openvswi-i08a0812a-9 - [0:0] :neutron-openvswi-o08a0812a-9 - [0:0] :neutron-openvswi-s08a0812a-9 - [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-out tap08a0812a-9e --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain -A neutron-openvswi-FORWARD -m physdev --physdev-in tap08a0812a-9e --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain -A neutron-openvswi-INPUT -m physdev --physdev-in tap08a0812a-9e --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-openvswi-o08a0812a-9 -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 130 -j RETURN -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j RETURN -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j RETURN -A neutron-openvswi-i08a0812a-9 -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 134 -j RETURN -A neutron-openvswi-i08a0812a-9 -d 20ff::c/128 -p udp -m udp --sport 547 --dport 546 -j RETURN -A neutron-openvswi-i08a0812a-9 -d fe80::/64 -p udp -m udp --sport 547 --dport 546 -j RETURN -A neutron-openvswi-i08a0812a-9 -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP -A neutron-openvswi-i08a0812a-9 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 143 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -j neutron-openvswi-s08a0812a-9 -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 134 -m comment --comment "Drop IPv6 Router Advts from VM Instance." -j DROP -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -p udp -m udp --sport 546 --dport 547 -m comment --comment "Allow DHCP client traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -p udp -m udp --sport 547 --dport 546 -m comment --comment "Prevent DHCP Spoofing by VM." -j DROP -A neutron-openvswi-o08a0812a-9 -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN -A neutron-openvswi-o08a0812a-9 -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP -A neutron-openvswi-o08a0812a-9 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback -A neutron-openvswi-s08a0812a-9 -s 20ff::c/128 -m mac --mac-source FA:16:3E:7C:D8:C0 -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN -A neutron-openvswi-s08a0812a-9 -s fe80::f816:3eff:fe7c:d8c0/128 -m mac --mac-source FA:16:3E:7C:D8:C0 -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN -A neutron-openvswi-s08a0812a-9 -m comment --comment "Drop traffic without an IP/MAC allow rule." -j DROP -A neutron-openvswi-sg-chain -m physdev --physdev-out tap08a0812a-9e --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-i08a0812a-9 -A neutron-openvswi-sg-chain -m physdev --physdev-in tap08a0812a-9e --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-o08a0812a-9

2 1

Port creation times out for some VMs in large group
by Albert Braden 11 Oct '19

11 Oct '19

When I create 100 VMs in our prod cluster: openstack server create --flavor s1.tiny --network it-network --image cirros-0.4.0-x86_64 --min 100 --max 100 alberttest Most of them build successfully in about a minute. 5 or 10 will stay in BUILD status for 5 minutes and then fail with " BuildAbortException: Build of instance <UUID> aborted: Failed to allocate the network(s), not rescheduling." If I build smaller numbers, I see less failures, and no failures if I build one at a time. This does not happen in dev or QA; it appears that we are exhausting a resource in prod. I tried reducing various config values in dev but am not able to duplicate the issue. The neutron servers don't appear to be overloaded during the failure. What config variables should I be looking at? Here are the relevant log entries from the HV: 2019-09-26 10:10:43.001 57008 INFO os_vif [req-dea54d9a-3f3e-4d47-b901-a4f41b1947a8 d28c3871f61e4c8c8f8c7600417f7b14 e9621e3b105245ba8660f434ab21016c - default 4fb72165eee4468e8033cdc7d506ddf0] Successfully plugged vif VIFBridge(active=False,address=fa:16:3e:8b:45:07,bridge_name='brq49cbe55d-51',has_traffic_filtering=True,id=18f4e419-b19c-4b62-b6e4-152ec78e72bc,network=Network(49cbe55d-5188-4183-b5ad-e65f9b46f8f2),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap18f4e419-b1') 2019-09-26 10:15:44.029 57008 WARNING nova.virt.libvirt.driver [req-dea54d9a-3f3e-4d47-b901-a4f41b1947a8 d28c3871f61e4c8c8f8c7600417f7b14 e9621e3b105245ba8660f434ab21016c - default 4fb72165eee4468e8033cdc7d506ddf0] [instance: dc58f154-00f9-4c45-8986-94b10821cbc9] Timeout waiting for [('network-vif-plugged', u'18f4e419-b19c-4b62-b6e4-152ec78e72bc')] for instance with vm_state building and task_state spawning.: Timeout: 300 seconds More logs and data: http://paste.openstack.org/show/779524/

4 17

[keystone][edge][k8s] Keystone - StarlingX integration feedback
by Ildiko Vancsa 11 Oct '19

11 Oct '19

Hi, I wanted to point you to a thread that’s just started on the edge-computing mailing list: http://lists.openstack.org/pipermail/edge-computing/2019-October/000642.html The mail contains information about a use case that StarlingX has to use Keystone integrated with Kubernetes which I believe is valuable information to the Keystone team to see if there are any items to discuss further/fix/implement. Thanks, Ildikó

2 1

monitoring openstack
by Ali Ebrahimpour 11 Oct '19

11 Oct '19

hi guys i want to install monitoring in my horizon Ui and i'm confused in setting up ceilometer or gnocchi or aodh or monasca in my project because all of them where deprecated. i setup openstack with ansible <https://docs.openstack.org/project-deploy-guide/openstack-ansible/stein/>and i want to monitor the usage of cpu and ram and etc in my dashboard and i also want to know how much resources each customer used for one hour and day. Thanks in advance for your precise guidance.

2 1

[nova] rescue instances with volumes
by Ionut Biru 11 Oct '19

11 Oct '19

Hello guys, How do you guys rescue instances that are booted from volumes or have volumes attached to them? If i use nova rescue on instances booted from volume, api returns that it's not compatible and if i rescue an instance that has a volume attached, the drive is not available in the OS. -- Ionut Biru - https://fleio.com

2 1

[Openstack] FOSDEM 2020 Virtualization & IaaS Devroom CfP
by Piotr Kliczewski 11 Oct '19

11 Oct '19

We are excited to announce that the call for proposals is now open for the Virtualization & IaaS devroom at the upcoming FOSDEM 2020, to be hosted on February 1st 2020. This year will mark FOSDEM’s 20th anniversary as one of the longest-running free and open source software developer events, attracting thousands of developers and users from all over the world. FOSDEM will be held once again in Brussels, Belgium, on February 1st & 2nd, 2020. This devroom is a collaborative effort, and is organized by dedicated folks from projects such as OpenStack, Xen Project, oVirt, QEMU, KVM, and Foreman. We would like to invite all those who are involved in these fields to submit your proposals by December 1st, 2019. About the Devroom The Virtualization & IaaS devroom will feature session topics such as open source hypervisors and virtual machine managers such as Xen Project, KVM, bhyve, and VirtualBox, and Infrastructure-as-a-Service projects such as KubeVirt, Apache CloudStack, OpenStack, oVirt, QEMU and OpenNebula. This devroom will host presentations that focus on topics of shared interest, such as KVM; libvirt; shared storage; virtualized networking; cloud security; clustering and high availability; interfacing with multiple hypervisors; hyperconverged deployments; and scaling across hundreds or thousands of servers. Presentations in this devroom will be aimed at developers working on these platforms who are looking to collaborate and improve shared infrastructure or solve common problems. We seek topics that encourage dialog between projects and continued work post-FOSDEM. Important Dates Submission deadline: 1 December 2019 Acceptance notifications: 10 December 2019 Final schedule announcement: 15th December 2019 Devroom: 1st February 2020 Submit Your Proposal All submissions must be made via the Pentabarf event planning site[1]. If you have not used Pentabarf before, you will need to create an account. If you submitted proposals for FOSDEM in previous years, you can use your existing account. After creating the account, select Create Event to start the submission process. Make sure to select Virtualization and IaaS devroom from the Track list. Please fill out all the required fields, and provide a meaningful abstract and description of your proposed session. Submission Guidelines We expect more proposals than we can possibly accept, so it is vitally important that you submit your proposal on or before the deadline. Late submissions are unlikely to be considered. All presentation slots are 30 minutes, with 20 minutes planned for presentations, and 10 minutes for Q&A. All presentations will be recorded and made available under Creative Commons licenses. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example: "If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>." In the Submission notes field, please also confirm that if your talk is accepted, you will be able to attend FOSDEM and deliver your presentation. We will not consider proposals from prospective speakers who are unsure whether they will be able to secure funds for travel and lodging to attend FOSDEM. (Sadly, we are not able to offer travel funding for prospective speakers.) Submission Guidelines Mentored presentations will have 25-minute slots, where 20 minutes will include the presentation and 5 minutes will be reserved for questions. The number of newcomer session slots is limited, so we will probably not be able to accept all applications. You must submit your talk and abstract to apply for the mentoring program, our mentors are volunteering their time and will happily provide feedback but won't write your presentation for you! If you are experiencing problems with Pentabarf, the proposal submission interface, or have other questions, you can email our devroom mailing list[2] and we will try to help you. How to Apply In addition to agreeing to video recording and confirming that you can attend FOSDEM in case your session is accepted, please write "speaker mentoring program application" in the "Submission notes" field, and list any prior speaking experience or other relevant information for your application. Code of Conduct Following the release of the updated code of conduct for FOSDEM, we'd like to remind all speakers and attendees that all of the presentations and discussions in our devroom are held under the guidelines set in the CoC and we expect attendees, speakers, and volunteers to follow the CoC at all times. If you submit a proposal and it is accepted, you will be required to confirm that you accept the FOSDEM CoC. If you have any questions about the CoC or wish to have one of the devroom organizers review your presentation slides or any other content for CoC compliance, please email us and we will do our best to assist you. Call for Volunteers We are also looking for volunteers to help run the devroom. We need assistance watching time for the speakers, and helping with video for the devroom. Please contact devroom mailing list [2] for more information. Questions? If you have any questions about this devroom, please send your questions to our devroom mailing list. You can also subscribe to the list to receive updates about important dates, session announcements, and to connect with other attendees. See you all at FOSDEM! [1] <https://penta.fosdem.org/submission/FOSDEM17> https://penta.fosdem.org/submission/FOSDEM20 [2] iaas-virt-devroom at lists.fosdem.org

1 0

[release] Release countdown for week R-0, October 14-18
by Sean McGinnis 11 Oct '19

11 Oct '19

Development Focus ----------------- We will be releasing the coordinated OpenStack Train release next week, on Wednesday October 16th. Thanks to everyone involved in the Train cycle! We are now in pre-release freeze, so no new deliverable will be created until final release, unless a release-critical regression is spotted. Otherwise, teams attending the PTG in Shanghai should start to plan what they will be discussing there, by creating and filling team etherpads. You can find the list of etherpads at: http://ptg.openstack.org/etherpads.html General Information ------------------- On release day, the release team will produce final versions of deliverables following the cycle-with-rc release model, by re-tagging the commit used for the last RC. A patch doing just that will be proposed. PTLs and release liaisons should watch for that final release patch from the release team. While not required, we would appreciate having an ack from each team before we approve it on the 16th. Upcoming Deadlines & Dates -------------------------- Final Train release: October 16 Forum+PTG at Shanghai summit: November 4

1 1

Networking-vpp 19.08.1 for VPP 19.08.1 is now available
by Naveen Joy (najoy) 11 Oct '19

11 Oct '19

Hello All, We'd like to invite you to try out Networking-vpp 19.08.1. As many of you may already know, VPP is a fast user space forwarder based on the DPDK toolkit. VPP uses vector packet processing algorithms to minimize the CPU time spent on each packet to maximize throughput. Networking-vpp is a ML2 mechanism driver that controls VPP on your control and compute hosts to provide fast L2 forwarding under Neutron. This latest version of Networking-vpp is updated to work with VPP 19.08.1 In this release, we've worked on the below updates: - We've added ERSPAN support for Tap-as-a-Service (TaaS). Since ERSPAN provides remote port mirroring, you can now mirror your OpenStack traffic to a destination outside of OpenStack or to a remote OpenStack VM. This is a customized version of OpenStack TaaS. We will be working with the community to push our custom TaaS extensions upstream. In the meantime, you can access our modified TaaS code here[2]. For further info on installation and usage, you can read the TaaS-README[3]. - We've updated the code to be compatible with VPP 19.08 & 19.08.1 API changes. - We've updated the unit test framework to support python3.5 onwards. - We've added security-group support for Trunk subports. We've added support for neutron trunk_details extension. - We've fixed some bugs in our Trunk and L3 plugins that caused a race condition during port binding. - We've migrated our repo from OpenStack to OpenDev. - A recent change in nova caused live migration to fail for instances with NUMA characteristics. We've found that this is a limitation in nova and not VPP/Networking-vpp. It is still possible to use live migration with VPP/Networking-vpp. Please refer to the README[1] for further details. - We've been doing the usual round of bug fixes and updates. The code will work with VPP 19.08.1 and has been updated to keep up with Neutron Rocky and Stein. The README [1] explains how you can try out VPP with Networking-vpp using Devstack: the Devstack plugin will deploy the mechanism driver and VPP and should give you a working system with a minimum of hassle. We will be continuing our development for VPP's 20.01 release. We welcome anyone who would like to come help us. -- Naveen, Ian & Jerome [1] https://opendev.org/x/networking-vpp/src/branch/master/README.rst [2] https://github.com/jbeuque/tap-as-a-service [3] https://opendev.org/x/networking-vpp/src/branch/master/README_taas.txt

1 0

[charms] placement charm
by Corey Bryant 10 Oct '19

10 Oct '19

Hi All, I'd like to see if I can get some input on the current state of the Placement API split. For some background, the nova placement API was removed from nova in train, and it's been split into its own project. It's mostly just a basic API charm. The tricky part is the migration of tables from the nova_api database to the placement database. Code is located at: https://github.com/coreycb/charm-placement https://github.com/coreycb/charm-interface-placement https://review.opendev.org/#/q/topic:charms-train-placement+(status:open+OR… Test scenarios I've been testing with: 1) deploy nova-cc et al train, configure keystonev3, deploy instance 2) deploy nova-cc et al stein, configure keystonev3, deploy instance 1, deploy placement train, deploy instance 2, upgrade nova-cc to train, deploy instance 3 There is currently an issue with the second test scenario where instance 2 creation errors because nova-scheduler can't find a valid placement candidate (not sure of the exact error atm). However if I delete instance 1 before creating instance 2 it is created successfully. It feels like a DB related issue but I'm really not sure so I'll keep digging. Thanks! Corey

2 6