February 2020 - openstack-discuss

[neutron] Team meeting 25.02.2020 cancellee
by Slawek Kaplonski 24 Feb '20

24 Feb '20

Hi, I’m on PTO tomorrow and I will not be able to chair our team meeting. So lets cancel it. See You all on the meeting next week. One reminder, if You are planning to attend Vancouver PTG in June, please add Your name to the list in [1]. I need to give number of rough number of attendees by Sunday, March 2nd. Please add Your name there even if it’s not yet confirmed. [1] https://etherpad.openstack.org/p/neutron-victoria-ptg — Slawek Kaplonski Senior software engineer Red Hat

1 0

Neutron metadata service not responding due to rabbitmq dropping messages
by Grant Morley 24 Feb '20

24 Feb '20

Hi all, We have recently come across an issue where our metadata service stops responding. If you try to curl the service from within an instance you get: % curl http://169.254.169.254 <html><body><h1>504 Gateway Time-out</h1> The server didn't respond in time. </body></html> After doing some digging around on our neutron nodes I noticed we were getting loads of RabbitMQ timeout errors whilst trying to process message requests: 2020-02-24 07:28:09.747 26378 ERROR neutron.common.rpc [-] Timeout in RPC method get_ports. Waiting for 26 seconds before next attempt. If the server is not down, consider increasing the rpc_response_timeout option as Neutron server(s) may be overloaded and unable to respond quickly enough.: MessagingTimeout: Timed out waiting for a reply to message ID a14c4a1395864cd980c1ec563a5c48aa The servers are fairly busy, however we do not have a massive installation >1500 instances and roughly 850 routers. However if I restart the "neutron-metadata-agent" service and the "neutron-server" service it seems to fix the issue for a while but ultimately it comes back. I did increase the "rpc_timeout" on the netutron nodes to 120 seconds but that seems quite long to me. Likewise the RabbitMQ servers are not overly busy, we seem to get a constant stream of only 40+ messages in the queue at one time and that can spike depending on workload. Does anyone know of any tuning or tweaking we can do to the metadata service in either Neutron or Nova that might help? We are running OpenStack Queens if that helps. Many thanks, Grant

1 0

RHOSP-like installation
by Volodymyr Litovka 24 Feb '20

24 Feb '20

Dear colleagues, while having a good experience with Openstack on Ubuntu, we're facing a plenty of questions re RHOSP installation. The primary requirement for our team re RHOSP is to get a knowledge on RHOSP - how to install it and maintain. As far as I understand, RDO is the closest way to reach this target but which kind of installation it's better to use? - * plain RDO installation as described in generic Openstack guide at https://docs.openstack.org/install-guide/index.html (specifics in RHEL/CentOS sections) * or TripleO installation as described in http://tripleo.org/install/ * or, may be, it is possible to use RHOSP in kind of trial mode to get enough knowledge on this platform? Our lab consists of four servers (64G RAM, 16 cores@2GHz) which we're going to use in "ultraconverged" mode - as both controller and agent (compute/network/storage) nodes (controllers, though, can be in virsh-controlled VMs). In case of TripleO scenario, 4th server can be used for undercloud role. This installation is intended not for production use, but rather for learning purposes, so no special requirements for productivity. The only special requirement - to be functionally as much as close to canonical RHOSP platform. I will highly appreciate your suggestions on this issue. Thank you. -- Volodymyr Litovka "Vision without Execution is Hallucination." -- Thomas Edison

2 1

[oslo] Meeting Time Poll
by Ben Nemec 24 Feb '20

24 Feb '20

Hello again, We have a few regular attendees of the Oslo meeting who have conflicts with the current meeting time. As a result, we would like to find a new time to hold the meeting. I've created a Doodle poll[0] for everyone to give their input on times. It's mostly limited to times that reasonably overlap the working day in the US and Europe since that's where most of our attendees are located (yes, I know, that's a self-fulfilling prophecy). If you attend the Oslo meeting, please fill out the poll so we can hopefully find a time that works better for everyone. Thanks! -Ben /me finally checks this one off the action items for next week :-) 0: https://doodle.com/poll/zmyhrhewtes6x9ty

1 1

[ironic][neutron][ops] Ironic multi-tenant networking with VXLAN Overlay
by Austin Cormier 24 Feb '20

24 Feb '20

We are using Openstack + Ironic to test routing software which requires the use of VLANs. Using VXLan overlays allows the VM's to have nested VLANs without issue as they are encapsulated into a VXLAN and sent across. However, we are struggling to find a path forward in extending the VXLAN overlay out to the hardware device. Unfortunately, the networking-generic-switch driver does not support VXLAN but this seems to fairly easy to extend (given the right switch). The challenge I'm facing is that with Neutron OVS + L2Pop, there seems to be no easy way to extend the L2 population to include an external device. The only path forward that I can see is to use the Linux Bridge driver + VXLAN multicast and find a switch that also supports VXLAN multicast so it can participate in the overlay dynamically. The HPE FlexFabric seems to do this from the documentation: https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-c04567545 Can anyone help confirm that the switch above will work and or give alternative suggestions on approaching this problem? -Austin

1 0

[openstack-helm][stein]Cinder pods stuck in init state after deployed in k8s cluster
by Sanjay K 24 Feb '20

24 Feb '20

Hello openstack-helm team, I am trying to deploy stein cinder service in my k8s cluster using persistent volume and persistent volume claim for local storage or NFS storage. However after deploying cinder in my cluster, the cinder pods remains in Init state even though the PV and PVC are created. Please look at my below post on openstack forum and guide me how to resolve this issue. https://ask.openstack.org/en/question/126191/cinder-pods-in-init-state-when… thank you for your help and support on this. best regards, Sanjay

2 3

[largescale-sig] Next meeting: Feb 26, 9utc
by Thierry Carrez 24 Feb '20

24 Feb '20

Hi everyone, The Large Scale SIG will have a meeting this week on Wednesday, Feb 26 at 9 UTC[1] in #openstack-meeting on IRC: [1] https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200226T09 As always, the agenda for our meeting is available at: https://etherpad.openstack.org/p/large-scale-sig-meeting Feel free to add topics to it! A quick reminder of the TODOs from our last meeting: - masahito to update spec based on initial feedback - everyone to review and comment on https://review.opendev.org/#/c/704733/ Talk to you all on Wednesday, -- Thierry Carrez

1 0

[neutron] Bug deputy report (week starting on 2020-02-17)
by Bernard Cafarelli 24 Feb '20

24 Feb '20

Picking up after amotoki's report [0], here is last week's report. This includes all bugs reported up to 1864374 (included) Some gate issues (python2 is still fighting us back, etc), new "interesting" bugs on different topics. Bugs marked with "***" do not have current assignee Critical * socket.timeout error in dvr CI jobs cause SSH issues - https://bugs.launchpad.net/neutron/+bug/1863858 Under investigation by slaweq - neutron-tempest-plugin-designate-scenario fails on stable branches with "SyntaxError: invalid syntax" installing dnspython - https://bugs.launchpad.net/neutron/+bug/1864015 Patch merged to pin designate plugin - https://review.opendev.org/708825 - "ping" prepended to ip netns commands - https://bugs.launchpad.net/neutron/+bug/1864186 Patch merged: https://review.opendev.org/709100 High * When updating the network segment ID, the project_id should be provided - https://bugs.launchpad.net/neutron/+bug/1863619 Needed parameter to get private ranges belonging to the project, fix in progress: https://review.opendev.org/708196/ * [OVN] Provider driver IPv6 traffic doesn't work - https://bugs.launchpad.net/neutron/+bug/1863892 IPv6 load balancers with OVN provider do not work, in progress by maciejjozefczyk * [OVN] DHCP doesn't work while instance has disabled port security - https://bugs.launchpad.net/neutron/+bug/1864027 Behaviour introduced by a recent fix, root cause may be in core OVN *** IP allocation for stateless IPv6 does not filter on segment when fixed-ips contain a subnet_id - https://bugs.launchpad.net/neutron/+bug/1864225 Title sums it up nicely, stateless IPv6 + segments issue *** When adding another stateless subnet, implicit address allocation happen for port with deferred ip allocation - https://bugs.launchpad.net/neutron/+bug/1864333 IPv6 allocation happens for deffered allocation port Medium *** Unhandled error - WSREP has not yet prepared node for application use - https://bugs.launchpad.net/neutron/+bug/1863579 Unassigned. Transient startup error seen in kolla-ansible upgrade CI when database is not ready, we can probably add a retry here * [neutron-tempest-plugin] test_trunk.TrunkTestInheritJSONBase.test_add_subport fails if unordered - https://bugs.launchpad.net/neutron/+bug/1863707 Results order failure in test, fix in progress: https://review.opendev.org/708305/ * [OVN]Could not support more than one qos rule in one policy - https://bugs.launchpad.net/neutron/+bug/1863852 Fix in progress: https://review.opendev.org/708586 * Missing information in log message for Invalid tunnel type - https://bugs.launchpad.net/neutron/+bug/1863888 Add some log extra information, fix in progress: https://review.opendev.org/708634 * [OVN] OVN LoadBalancer VIP shouldn't have addresses set - https://bugs.launchpad.net/neutron/+bug/1863893 Regression fix, in progress by maciejjozefczyk * [OVN] Remove dependency on port_object - https://bugs.launchpad.net/neutron/+bug/1863987 Suggested fix: https://review.opendev.org/708806/ Low * Default security group rules not created during list of SG rules - https://bugs.launchpad.net/neutron/+bug/1864171 Fix in progress: https://review.opendev.org/709070 *** BGP dynamic routing in neutron - No Info about capability to receiv/learn dynamic routes - https://bugs.launchpad.net/neutron/+bug/1864219 Doc enhancement request on BGP limitations *** ml2 ovs does not flush iptables switching to FW ovs - https://bugs.launchpad.net/neutron/+bug/1864374 Switching from iptables to ovs firewall is not a trivial task, this may end up being documentation bug too Related to neutron * Upgrade from Rocky to Stein, router namespace disappear - https://bugs.launchpad.net/neutron/+bug/1863982 Issue in kolla-ansible upgrade Rocky->Stein, failing to restore iptables rules. It does not seem to be neutron-specific, but opinions welcome [0] http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012610… -- Bernard Cafarelli

1 0

[neutron] Network segment ranges feature
by Rodolfo Alonso 24 Feb '20

24 Feb '20

Hello Neutrinos: First of all, a reference: https://bugs.launchpad.net/neutron/+bug/1863423 I detected some problems with this feature. The service plugin, when enabled and according to the spec [1], should segregate the segmentation range in several ranges. Each one (or many) of those ranges could be assigned to a project. When a network is created in a specific project, should receive a segmentation ID from the project network segment ranges. In case of not having any range assigned, a segmentation ID from a default range will be provided. How the current implementation actually works: When a driver is loaded (VLAN, VXLAN, GRE or Geneve), it creates a default network segment range, based on the static configuration provided in the plugin ("network_vlan_ranges", "vni_ranges", etc). Then the admin can create project specific network segment ranges (remember: always per driver type). Those project specific ranges cannot overlap but the project specific ranges CAN overlap the default one. A valid state could be: VLAN: - Default: 100:1000 - Project1: 100:200 - Project2: 200:300 When assigning segmentation IDs to new networks, the driver will query [2]: - The existing ranges per project. - The default range (always present). The result is that, if the project network segment range runs out of segment IDs, it will retrieve those ones from the default range. That will lead, eventually, to a clash between ranges. Project 1 will be able to allocate a network with a segmentation ID from Project 2 range. E.g.: [3] The problem is how the queries are done: [4] --> select one of each commented line to reproduce "query_project_id" and "query_shared" in [2]. Now we know the feature is not properly working, my question is what the proper behaviour should be. Some alternatives: 1) If a project has one or more network segment ranges, the segmentation ID should be retrieved ONLY from those ranges. If a project does not have any range, then it will retrieve from the shared pool BUT NEVER selecting a segment ID belonging to other project ID (ufff this query is going to be complex). --> IMO, the best solution. 2) If a project has one or more network segment ranges, the segmentation ID should be retrieved first from those ranges and then from the shared one, BUT NEVER selecting a segment ID belonging to other project ID. Same for range-less projects. --> IMO, if the administrator has assigned a range for a project, the project should ONLY use this pool. This is not a good solution. Can I have your feedback? Regards. [1] https://specs.openstack.org/openstack/neutron-specs/specs/stein/network-seg… [2]https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/driv… [3]http://paste.openstack.org/show/789872/ [4]http://paste.openstack.org/show/789873/

3 3

[openstack-discuss][neutron] is neutron-openvswitch-agent needed for network connection on provider network?
by Chen CH Ji 24 Feb '20

24 Feb '20

2 2