May 2019 - openstack-discuss - lists.openstack.org

[openstack-dev] [Neutron] Train PTG Summary
by Miguel Lavalle 22 May '19

22 May '19

Dear Neutron team, Thank you very much for your hard work during the PTG in Denver. Even though it took place at the end of a very long week, we had a very productive meeting and we planned and prioritized a lot of work to be done during the cycle. Following below is a high level summary of the discussions we had. If there is something I left out, please reply to this email thread to add it. However, if you want to continue the discussion on any of the individual points summarized below, please start a new thread, so we don't have a lot of conversations going on attached to this update. You can find the etherpad we used during the PTG meetings here: https://etherpad.openstack.org/p/openstack-networking-train-ptg Retrospective ========== * The team considered positive points during the Stein cycle the following: - Implemented and merged all the targeted blueprints. - Minted several new core team members through a mentoring program. The new core reviewers are Nate Johnston, Hongbin Lu, Liu Yulong, Bernard Caffarelli (stable branches) and Ryan Tidwell (neutron-dynamic-routing) - Very good cross project cooperation with Nova ( https://blueprints.launchpad.net/neutron/+spec/strict-minimum-bandwidth-sup…) and StarlingX ( https://blueprints.launchpad.net/neutron/+spec/network-segment-range-manage… ) - The team got caught up with all the community goals - Added non-voting jobs from the Stadium projects, enabling the team to catch potential breakage due to changes in Neutron - Successfully forked the Ryu SDN framework, which is used by Neutron for Openflow programming. The original developer is not supporting the framework anymore, so the Neutron team forked it as os-ken ( https://opendev.org/openstack/os-ken) and adopted it seamlessly in the code * The team considered the following as improvement areas: - At the end of the cycle, we experienced failures in the gate that impacted the speed at which code was merged. Measures to solve this problem were later discussed in the "CI stability" section below - The team didn't make much progress adopting Storyboard. Given comments of lack of functionality from other projects, a decision was made to evaluate progress by other teams before moving ahead with Storyboard - Lost almost all the key contributors in the following Stadium projects: https://opendev.org/openstack/neutron-fwaas and https://opendev.org/openstack/neutron-vpnaas. Miguel Lavalle will talk to the remaining contributors to asses how to move forward - Not too much concrete progress was achieved by the performance and scalability sub-team. Please see the "Neutron performance and scaling up" section below for next steps - Engine facade adoption didn't make much progress due to the loss of all the members of the sub-team working on it. Miguel Lavalle will lead this effort during Train. Nate Johnston and Rodolfo Alonso volunteered to help. The approach will be to break up this patch into smaller, more easily implementable and reviewable chunks: https://review.opendev.org/#/c/545501/ Support more than one segment per network per host ======================================== The basic value proposition of routed networks is to allow deployers to offer their users a single "big virtual network" without the performance limitations of large L2 broadcast domains. This value proposition is currently limited by the fact that Neutron allows only one segment per network per host: https://github.com/openstack/neutron/blob/77fa7114f9ff67d43a1150b52001883fa…. As a consequence, as demand of IP addresses exceeds the limits of a reasonably sized subnets (/22 subnets is a consensus on the upper limit), it becomes necessary to allow hosts to be connected to more than one segment in a routed network. David Bingham and Kris Lindgren (GoDaddy) have been working on PoC code to implement this (https://review.opendev.org/#/c/623115). This code has helped to uncover some key challenges: * Change all code that assumes a 1-1 relationship between network and segment per host into a 1-many relationship. * Generate IDs based on segment_id rather than network_id to be used in naming software bridges associated with the network segments. * Ensure new 1-many relationship (network -> segment) can be supported by ML2 drivers implementors. * Provide migration paths for current deployments of routed networks. The agreements made were the following: * We will write a spec reflecting the learnings of the PoC * The spec will target all the supported ML2 backends, not only some of them * We will modify and update ML2 interfaces to support the association of software bridges with segments, striving to provide backwards compatibility * We will try to provide an automatic migration option that only requires re-starting the agents. If that proves not to be possible, a set of migration scripts and detailed instructions will be created The first draft of the spec is already up for review: https://review.opendev.org/#/c/657170/ Neutron CI stability ============== At the end of the Stein cycle the project experienced a significant impact due to CI instability. This situation has improved recently but there is still gains to be achieved. The team discussed to major areas of improvement: make sure we don't have more tests that are necessary (simplification of jobs) and fix recurring problems. - To help the conversation on simplification of jobs, Slawek Kaplonski shared this matrix showing what currently is being tested: https://ethercalc.openstack.org/neutron-ci-jobs * One approach is reducing the services Neutron is tested with in integrated-gate jobs (tempest-full), which will reduce the number of failures not related to Neutron. Slawek Kaplonski represented Neutron in the QA PTG session where this approach was discussed. The proposal being socialized in the mailing list ( http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005871.html ) involves: # Run only dependent service tests on project gate # Tempest gate will keep running all the services tests as the integrated gate at a centeralized place without any change in the current job # Each project can run a simplified integrated gate job template tailored to its needs # All the simplified integrated gate job templates will be defined and maintained by QA team # For Neutron there will be an "Integrated-gate-networking". Tests to run in this template: Neutron APIs , Nova APIs, Keystone APIs. All scenario currently running in tempest-full in the same way (means non-slow and in serial). The improvement will be to exclude the Cinder API tests, Glance API tests and Swift API tests * Another idea discussed was removing single node jobs that are very similar to multinode jobs # One possibility is consolidating grenade jobs. There is a proposal being socialized in the mailing list: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006146.html # Other consolidation of single node - multinode jobs will require stabilizing the corresponding multinode job - One common source of problems is ssh failures in various scenario tests * Several team members are working on different aspects of this issue * Slawek Kaplonski is investigating authentication failures. As of the writing of this summary, it has been determined that there is a slowdown in the metadata service, either on the Neutron or the Nova side. Further investigation is going on * Miguel Lavalle is adding tcpdump commands to router namespaces to investigate data path disruptions netwroking-omnipath ================ networking-omnipath (https://opendev.org/x/networking-omnipath) is a ML2 mechanism driver that integrates OpenStack Neutron API with an Omnipath backend. It enables Omnipath switching fabric in OpenStack cloud and each network in the Openstack networking realm corresponds to a virtual fabric on the omnipath side. - Manjeet Singh Bhatia proposed to make networking-omnipath a Neutron Stadium project - The agreement was that Miguel Lavalle and Manjeet will work together in determining whether networking-omnipath meet the Stadium project criteria, as outlined here: https://docs.openstack.org/neutron/latest/contributor/stadium/governance.ht… - In case the criteria is not met, a remediation plan will be defined Cross networking project topics ======================= - Cross networking project topics * Neutron the only projects not using WSGI * We have to make it the default option in DevStack, although this will require some investigation * We already have a check queue non-voting job for WSGI. It is failing constantly, although the failures are all due to a singe test case (neutron_add_remove_fixed_ip). Miguel Lavalle will investigate and fix it * Target is to adopt WSGI as the default by Train-2 - Adoption of neutron-classifier (https://opendev.org/x/neutron-classifier) * David Shaughnessy has two PoC patches that demonstrate the adoption of neutron-classifier into Neutron's QoS. David will continue refining these patches and will bring them up for discussion in the QoS sub-team meeting on May 14th * It was also agreed to start the process of adding neutron-classifier the the Neutron Stadium. David Shaughnessy and Miguel Lavalle will work on this per the criteria defined in https://docs.openstack.org/neutron/latest/contributor/stadium/governance.ht… - DHCP agent configured with mismatching domain and host entries * Since the merge of https://review.opendev.org/#/c/571546/, we have a confusion about what exactly the dns_domain field of a network is for. Historically, dns_domain for use with external DNS integration in the form of designate, but that delineation has become muddied with the previously mentioned change. * Miguel Lavalle will go back to the original spec of DNS integration and make a decision as to how to move forward - Neutron Events for smartNIC-baremetal use-case * In smartNIC baremetal usecase, Ironic need to know when agent is/is-not alive (since the neutron agent is running on the smartNIC) and when a port becomes up/down * General agreement was to leverage the existing notifier mechanism to emit this information for Ironic to consume (requires implementation of an API endpoint in Ironic). It was also agreed that a spec will be created * The notifications emitted can be leveraged by Ironic for other use-cases. In fact, in a lunch with Ironic team members (Julia Kreger, Devananda van der Veen and Harald Jensås), it was agreed to use use it also for the port bind/unbind completed notification. Neutron performance and scaling up =========================== - Recently, a performance and scalability sub-team ( http://eavesdrop.openstack.org/#Neutron_Performance_sub-team_Meeting) has been formed to explore ways to improve performance overall - One of the activities of this sub-team has been adding osprofiler to the Neutron Rally job (https://review.opendev.org/#/c/615350). Sample result reports can be seen here: http://logs.openstack.org/50/615350/38/check/neutron-rally-task/0a4b791/res… and http://logs.openstack.org/50/615350/38/check/neutron-rally-task/0a4b791/res… - Reports analysis indicate that port creation takes on average in the order of 9 seconds, even without assigning IP addresses to it and without binding it. The team decided to concentrate its efforts in improving the entire port creation - binding - wiring cycle. One step necessary for this is the addition of a Rally scenario, which Bence Romsics volunteered to develop. - Another area of activity has been EnOS ( https://github.com/BeyondTheClouds/enos ), which is a framework that deploys OpenStack (using Kolla Ansible) and then runs Rally based performance experiments on that deployment ( https://enos.readthedocs.io/en/stable/benchmarks/index.html) * The deployment can take place on VMs (Vagrant provider) or in large clouds such as Grid5000 testbed: https://www.grid5000.fr/w/Grid5000:Home * The Neutron performance sub-team and the EnOS developers are cooperating to define a performance experiment at scale * To that end, Miguel Lavalle has built a "big PC" with an AMD Threadripper 2950x processor (16 cores, 32 threads) and 64 GB of memory. This machine will be used to experiment with deployments in VMs to refine the scenarios to be tested, with the additional benefit that the Rally results will not be affected by variability in the OpenStack CI infrastructure. * Once the Neutron and EnOS team reach an agreement on the scenarios to be tested, an experiment will be run Grid5000 * The EnOS team delivered on May 6th the version that supports the Stein release - Miguel Lavalle will create a wiki page to record a performance baseline and track subsequent progress DVR Enhancements =============== - Supporting allowed_address_pairs for DVR is a longstanding issue for DVR: https://bugs.launchpad.net/neutron/+bug/1774459. There are to patches up for review to address this issue: * https://review.opendev.org/#/c/616272/ * https://review.opendev.org/#/c/651905/ - The team discussed the current state of DVR functionality and identified the following missing features that would be beneficial for operators: * Distributed ingress/egress for IPv6. Distributed ingress/egress (AKA "fast-exit") would be implemented for IPv6. This would bypass the centralized router in a network node * Support for smart-NIC offloads. This involves pushing all DVR forwarding policy into OVS and implementing it via OpenFlow * Distributed DHCP. Rather than having DHCP for a given network be answered centrally, OpenFlow rules will be programmed into OVS to provide static, locally generated responses to DHCP requests received on br-int * Distributed SNAT. This involves allowing SNAT to happen directly on the compute node instead of centralizing it on a network node. * There was agreement that these features are needed and Ryan Tidwell agreed to develop a spec as the next step. The spec is already up for review: https://review.opendev.org/#/c/658414 - networking-ovn team members pointed out that some of the above features are already implemented in their Stadium project. This led to the discussion of why duplicate efforts implementing the same features and instead explore the possibility of a convergence between the ML2 / agents based reference implementation and the ML2 / OVN implementation. * This discussion is particularly relevant in the context where the OpenStack community is rationalizing its size and contributors are scarcer * Such a convergence would most likely play out over several development cycles * The team agreed to explore how to achieve this convergence. To move forward, we will need visibility and certainty that the following is feasible: # Feature parity with what the reference implementation offers today # Ability to support all the backends in the current reference implementation # Offer verifiable substantial gains in performance and scalability compared to the current reference implementation # Broaden the community of developers contributing to the ML2 / OVN implementation * To move ahead in the exploration of this convergence, three actions were agreed: # Benchmarking of the two implementations will be carried out with EnOS, as part of the performance and scaling up activities described above # Write the necessary specs to address feature parity, support all the backends in the current reference implementation and provide migration paths # An item will be added to the weekly Neutron meeting to track progress # Make every decision along this exploration process with approval of the broader community Policy topics / API ============== - Keystone has a now a system scope. A system-scoped token implies the user has authorization to act on the deployment system. These tokens are useful for interacting with resources that affect the deployment as a whole, or exposes resources that may otherwise violate project or domain isolation * Currently in Neutron, if users have an admin role they, can access all the resources * In order to maintain alignment with the community. Akihiro Motoki will review the Neutron code and determine how the admin role is used to interact with deployment resources. He will also monitor how Nova's adoption of the system scope progresses - During the policy-in-code work, some improvements and clean ups were left pending, which are Items 2.3, 2.4 and 4 in https://etherpad.openstack.org/p/neutron-policy-in-code - The Neutron approach to use new extensions to make any change to the ReST API discoverable, has resulted in the proliferation of "shim extensions" to introduce small changes such as the addition of an attribute * To eliminate this issue, Akihiro Motoki proposed to maintain the overall extensions approach but micro version the extensions so that every feature added does not result in another extension * The counter argument from some in the room was: "extensions are messy, but it's a static mess. Putting in Micro versions creates a mess in the code with lots of conditionals on micro version enablement" * It was decided to explore simple versioning of extensions. The details will be fleshed out in the following spec: https://review.opendev.org/#/c/656955 Neutron - Nova cross project planning ============================= This session was summarized in the following messages to the mailing list: - http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005844.html summarizes the following topics * Optional NUMA affinity for neutron ports * Track neutron ports in placement * os-vif to be extended to contain new fields to record the connectiviy type and ml2 driver that bound the vif * Boot vms with unaddressed port - Leaking resources when ports are deleted out-of-band is summarized in this thread: http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005837.html - Melanie Witt asked if Neutron would support implementing transferring ownership of its resources. The answer was yes and as next step, she is going to send a message to the mailing list to define the next steps Code streamlining proposals ====================== - Streamlining IPAM flow. As a result of bulk port creation work done in Stein by Nate Johnston, it is clear that there are opportunities to improve the IPAM code. The team brainstormed several possible approaches and the following proposals were put forward: * When allocating blocks of IP addresses where strategy is 'next ip' then ensure it happens as a single SQL insert * Create bulk versions of allocate_ip_from_port_and_store etc. so that bulk can be preserved when pushed down to the IPAM driver to take advantage of the previous item * Add profiling code to the IPAM call so that we can log the time duration for IPAM execution, as a PoC - Streamlining use of remote groups in security groups. Nate Johnston pointed out that there is a performance hit when using security groups that are keyed to a remote_group_id, because when a port is added to a remote group it triggers security group rule updates for all of the members of the security group. On deployments with 150+ ports, this can take up to 5 mins to bring up the port * After discussion, the proposed next step is for Nate Johnston to create a PoC for a new approach where a nested security group creates a new iptables table/ovs flow table (let's call it a subtable) that can be used as an abstraction for the nested group relationship. Then the IP addresses of the primary security group will jump to the new table, and the new table can represent the contents of the remote security group # In a case where a primary security group has 170 members and lists itself as a remote security group (indicating members can all talk amongst themselves) when adding an instance to the security group that causes 171 updates, since each member needs the address of the new instance and a record needs to be created for the new one # With the proposed approach there would only be 2 updates: creating an entry for the new instance to jump to the subtable representing the remote security group, and adding an entry to the subtable Train community goals ================= The two community goals accepted or Train are: - PDF doc generation for project docs: https://review.opendev.org/#/c/647712/ * Akihiro Motoki will track this goal - IPv6 support and testing goal: https://review.opendev.org/#/c/653545/ * Good blog entry on overcoming metadata service shortcomings in this scenario: https://superuser.openstack.org/articles/deploying-ipv6-only-tenants-with-o… neutron-lib topics ============= - To help expedite the merging the of neutron-lib consumption patches it was proposed to the team that neutron-lib-current projects must get their dependencies for devstack based testing jobs from source, instead of PyPI. * For an example of an incident motivating this proposal, please see: https://bugs.launchpad.net/tricircle/+bug/1816644 * This refers to inter-project dependencies, for example networking-l2gw depending on networking-sfc. It does not apply to *-lib projects, those will still be based on PyPI release * The team agreed to this proposal * When creating a new stable branch the Zuul config would need to be updated to point to the stable releases of the other projects it depends on. May include a periodic job that involves testing master and stable branches against PyPI packages * Boden Russel will make a list of what jobs need to be updated in projects that consume neutron-lib (superset of stadium) - Boden reminded the team we have a work items list for neutron-lib https://etherpad.openstack.org/p/neutron-lib-volunteers-and-punch-list Requests for enhancement ===================== - Improve extraroute API * Current extraroute API does not allow atomic additions/deletions of particular routing table entries. In the current API the routes attribute of a router (containing all routing table entries) must be updated at once, leading to race conditions on the client side * The team debated several alternatives: an API extension that makes routers extra routes first level resources, solve the concurrency issue though "compare and swap" approach, seek input from API work group or provide better guidelines for the use of the current API * The decision was made to move ahead with a spec proposing extra routes as first level API resources. That spec can be found here: https://review.opendev.org/#/c/655680 - Decouple placement reporting service plugin from ML2 * The placement reporter service plugin as merged in Stein depends on ML2. The improvement idea is to decouple it, by a driver pattern as in the QoS service plugin * We currently don't have a use case for this decoupling. As a consequence, it was decided to postpone it Various topics ========== - Migration of stadium projects CI jobs to python 3 * We have an etherpad recording the work items: https://etherpad.openstack.org/p/neutron_stadium_python3_status * Lajos Katona will take care of networking-odl * Miguel Lavalle will talk to Takashi Yamamoto about networking-midonet * Nate Johnston will continue working on networking-bagpipe and neutron-fwaas patches * A list of projects beyond the Stadium will be collected as part of the effort for neutron-lib to start pulling requirements from source - Removal of deprecated "of_interface" option * The option was deprecated in Pike * In some cases, deployers might experience a few seconds of data plane down time when the OVS agent is restarted without the option * A message was sent to the ML warning of this possible effect: http://lists.openstack.org/pipermail/openstack-dev/2018-September/134938.ht…. There has been no reaction from the community * We will move ahead with the removal of the option. Patch is here: https://review.opendev.org/#/c/599496 Status and health of some Stadium and non-Stadium projects ============================================== - Some projects have experienced loss of development team: * networking-old. In this case, Ericsson is interested in continuing maintaining the project. The key contact is Lajos Katona * networking-l2gw is also interesting for Ericsson (Lajos Katona). Over the pas few cycles the project has been maintained by Ricardo Noriega of Red Hat. Miguel Lavalle will organize a meeting with Lajos and Ricardo to decide how to move ahead with this project * neutron-fwaas. In this case, Miguel Lavalle will send a message to the mailing list describing the status of the project and requesting parties interested in continuing maintaining the project

2 1

[Watcher] team meeting and agenda
by li.canwei2＠zte.com.cn 22 May '19

22 May '19

Hi, Watcher will have a meeting at 08:00 UTC today in the #openstack-meeting-alt channel. The agenda is available on https://wiki.openstack.org/wiki/Watcher_Meeting_Agenda#05.2F22.2F2019 feel free to add any additional items. Thanks! Canwei Li

1 0

[auto-scaling] PTG summary
by Rico Lin 21 May '19

21 May '19

Sorry for the late summary here. As the following action should get started. Let me try to point out actions and discussions we have from Summit + PTG. If you care about auto-scaling with/around OpenStack please join us All our discussion should be able to found in Etherpad https://etherpad.openstack.org/p/DEN-auto-scaling-SIG For more information about auto-scaling SIG: - repo: https://opendev.org/openstack/auto-scaling-sig - storyboard: https://storyboard.openstack.org/#!/project/openstack/auto-scaling-sig - irc: #openstack-auto-scaling - meetings: http://eavesdrop.openstack.org/#Auto-scaling_SIG_Meeting - use the ics file so you don't miss the schedule Call for help: we just formed so we need reviewers and helpers. Let us know if you would like to be core reviewer for SIG repo, would like to help on any tasks, or whould like th share any idea with us. As we try to find out what's the most important action we should take to initial this SIG, here are some init scope we will try to improve on: - Documentation: As we all agreed, document to record the current auto-scaling situation and collect use cases or relative resources, so users can have a better landscape. We will have to ask other teams to help to provide related use cases as one of our first steps. Here are some identified tasks: https://storyboard.openstack.org/#!/story/2005751 - Integration tests: Once we set up a basic test environment (which will happen soon), we will also try to talk with projects on having a cross-project gating CI job. At PTG, we already consult some projects and generally got positive feedback. It will be harder to create a CI job for autoscaling OpenStack services like nova-compute, so that part is not included in the initial plan for now. Here are tasks for now, and please suggest more if you see available: https://storyboard.openstack.org/#!/story/2005752 - Initial repo setup: Thanks for Adam, we now have our repository and irc bot ready, if you check out https://docs.openstack.org/auto-scaling-sig/latest/ and https://opendev.org/openstack/auto-scaling-sig you will find the doc and repo entry is ready. - Make a list to identify missing features: Which we should add into the docs to really identify missing features that might be some potential targets we can ask if project teams would like to have. Or help users to avoid mistaken how OpenStack autoscaling might works. - Consistent our discussion and task tracking: We not set up missions and looks like we have a lot of things need to be done in this cycle, so we should keep tracking and make sure we not missing anything - Collect Use Cases and user feedbacks: We need to identify use cases and any feedback, so if you like to add any use case or scenario to our document, please help with added it in https://opendev.org/openstack/auto-scaling-sig/src/branch/master/use-cases -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

1 0

[tripleo] retiring ansible-role-k8s-* repos
by Alex Schultz 21 May '19

21 May '19

FYI, We're retiring the following repos that were used during the initial investigation of k8s + tripleo. openstack/ansible-role-k8s-cinder openstack/ansible-role-k8s-cookiecutter openstack/ansible-role-k8s-glance openstack/ansible-role-k8s-keystone openstack/ansible-role-k8s-mariadb openstack/ansible-role-k8s-rabbitmq openstack/ansible-role-k8s-tripleo Please let me know if there are any objections. We're shooting for the end of this week (by May 24th) depending on patch merging. Thanks, -Alex

2 1

[tripleo] TripleO CI Summary: Sprint 30
by Rafael Folco 21 May '19

21 May '19

Greetings, The TripleO CI team has completed Sprint 30 / Unified Sprint 9 (Apr 25 thru May 15). The following is a summary of completed work during this sprint cycle: - Tested and created container build jobs for RDO on RHEL 7 in the internal instance of Software Factory. - Moved collect-logs into a separate repository to be consumed by other projects. - Ran a test day for zuul reproducer, filing and fixing bugs. - Continued on-boarding baremetal hardware to the internal Software Factory for running promotion jobs. - Promotion status: green on all branches at most of the sprint - PTG was attended by weshay, marios, sagi, quique The planned work for the next sprint [1] are: - Complete RDO on RHEL7 work by having an independent pipeline running container and image build, standalone and ovb featureset001 jobs. - Bootstrap OSP 15 standalone job on RHEL8 running in the internal Software Factory. The Ruck and Rover for this sprint are Marios Andreou (marios) and Sorin Gabriele Cerami (panda). Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes are being tracked in etherpad [2]. Thanks, rfolco [1] https://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-10 [2] https://etherpad.openstack.org/p/ruckroversprint10

1 0

Re: [Release-job-failures] Release of openstack/puppet-vswitch failed
by Tony Breeds 21 May '19

21 May '19

On Thu, May 16, 2019 at 11:11:20PM +0000, zuul(a)openstack.org wrote: > Build failed. > > - release-openstack-puppet http://logs.openstack.org/56/56ba10b449c6bed9d468d90b12ee8046c77cbb29/relea… : POST_FAILURE in 2m 46s I didn't see this one discussed. From[1]: Forge API failed to upload tarball with code: 400 errors: The dependency 'camptocamp/kmod' in metadata.json is empty. All dependencies must have a 'version_requirement'. Yours Tony. [1] http://logs.openstack.org/56/56ba10b449c6bed9d468d90b12ee8046c77cbb29/relea…

2 2

Devstack cannot deploy an openstack environment in master branch
by Fang, Liang A 21 May '19

21 May '19

Hi Devstack cannot deploy an openstack environment in master branch. It always stick in starlingx horizon. Here are some logs: Every 10.0s: cat stack.sh.log.summary Mon May 20 19:29:52 2019 2019-05-21 02:14:20.727 | stack.sh log /opt/stack/logs/stack.sh.log.2019-05-20-191420 2019-05-21 02:14:21.969 | Installing package prerequisites 2019-05-21 02:15:31.441 | Starting RabbitMQ 2019-05-21 02:15:36.870 | Installing OpenStack project source 2019-05-21 02:17:07.343 | Installing Tempest 2019-05-21 02:17:45.010 | Configuring and starting MySQL 2019-05-21 02:17:50.678 | Starting Keystone 2019-05-21 02:19:42.599 | Configuring Horizon 2019-05-21 02:19:51.117 | Configuring Glance 2019-05-21 02:19:57.830 | Configuring Neutron 2019-05-21 02:20:31.993 | Configuring Cinder 2019-05-21 02:20:38.434 | Configuring placement 2019-05-21 02:20:51.430 | Configuring Nova 2019-05-21 02:21:49.456 | Starting Glance 2019-05-21 02:21:51.681 | Uploading images 2019-05-21 02:21:54.614 | Starting Nova API 2019-05-21 02:21:57.146 | Starting Neutron 2019-05-21 02:21:59.473 | Starting Placement 2019-05-21 02:22:03.514 | Creating initial neutron network elements 2019-05-21 02:22:28.396 | Starting Nova 2019-05-21 02:22:45.696 | Starting Cinder 2019-05-21 02:22:50.307 | Starting Horizon ++:: openstack --os-cloud devstack-admin --os-region RegionOne compute service list --host ubuntu16vmliang --service nova-compute -c ID -f value +:: ID= +:: [[ '' == '' ]] +:: sleep 1 +functions:wait_for_compute:449 rval=124 +functions:wait_for_compute:461 time_stop wait_for_service +functions-common:time_stop:2317 local name +functions-common:time_stop:2318 local end_time +functions-common:time_stop:2319 local elapsed_time +functions-common:time_stop:2320 local total +functions-common:time_stop:2321 local start_time +functions-common:time_stop:2323 name=wait_for_service +functions-common:time_stop:2324 start_time=1558405396860 +functions-common:time_stop:2326 [[ -z 1558405396860 ]] ++functions-common:time_stop:2329 date +%s%3N +functions-common:time_stop:2329 end_time=1558405457017 +functions-common:time_stop:2330 elapsed_time=60157 +functions-common:time_stop:2331 total=5598 +functions-common:time_stop:2333 _TIME_START[$name]= +functions-common:time_stop:2334 _TIME_TOTAL[$name]=65755 +functions:wait_for_compute:463 [[ 124 != 0 ]] +functions:wait_for_compute:464 echo 'Didn'\''t find service registered by hostname after 60 seconds' Didn't find service registered by hostname after 60 seconds +functions:wait_for_compute:465 openstack --os-cloud devstack-admin --os-region RegionOne compute service list +----+------------------+-----------------+----------+---------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+-----------------+----------+---------+-------+----------------------------+ | 2 | nova-scheduler | ubuntu16vmliang | internal | enabled | up | 2019-05-21T02:24:16.000000 | | 5 | nova-consoleauth | ubuntu16vmliang | internal | enabled | up | 2019-05-21T02:24:17.000000 | | 6 | nova-conductor | ubuntu16vmliang | internal | enabled | up | 2019-05-21T02:24:09.000000 | | 1 | nova-conductor | ubuntu16vmliang | internal | enabled | up | 2019-05-21T02:24:09.000000 | +----+------------------+-----------------+----------+---------+-------+----------------------------+ +functions:wait_for_compute:467 return 124 +lib/nova:is_nova_ready:1 exit_trap +./stack.sh:exit_trap:525 local r=124 ++./stack.sh:exit_trap:526 jobs -p +./stack.sh:exit_trap:526 jobs= +./stack.sh:exit_trap:529 [[ -n '' ]] +./stack.sh:exit_trap:535 '[' -f /tmp/tmp.IbKRmNIAwU ']' +./stack.sh:exit_trap:536 rm /tmp/tmp.IbKRmNIAwU +./stack.sh:exit_trap:540 kill_spinner +./stack.sh:kill_spinner:435 '[' '!' -z '' ']' +./stack.sh:exit_trap:542 [[ 124 -ne 0 ]] +./stack.sh:exit_trap:543 echo 'Error on exit' Error on exit +./stack.sh:exit_trap:545 type -p generate-subunit +./stack.sh:exit_trap:546 generate-subunit 1558404858 599 fail +./stack.sh:exit_trap:548 [[ -z /opt/stack/logs ]] +./stack.sh:exit_trap:551 /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs neutron-dhcp-agent: no process found neutron-l3-agent: no process found neutron-metadata-agent: no process found neutron-openvswitch-agent: no process found stack@ubuntu16vmliang:~/devstack$ +./stack.sh:exit_trap:560 exit 124 Regards Liang

1 0

[cinder] Help with a review please
by Sam Morrison 21 May '19

21 May '19

Hi, I’ve had a review going on for over 8 months now [1] and would love to get this in, it’s had +2s over the period and keeps getting nit picked, finally being knocked back due to no spec which there now is [2] This is now stalled itself after having a +2 and it is very depressing. I have had generally positive experiences contributing to openstack but this has been a real pain, is there something I can do to make this go smoother? Thanks, Sam [1] https://review.opendev.org/#/c/599866/ <https://review.opendev.org/#/c/599866/> [2] https://review.opendev.org/#/c/645056/ <https://review.opendev.org/#/c/645056/>

3 3

[masakari] propose to skip 5/21 IRC meeting
by Sam P 20 May '19

20 May '19

Hi all, Since most of the members are absent in this week, I would like to skip 5/21 irc meeting. Please use this mailing for share any issues or discussions for masakari. Best regards, Sampath

1 0

[neutron] Bug deputy report 2019-05-14 - 2019-05-20
by Ryan Tidwell 20 May '19

20 May '19

Hello Neutrinos, Here is this week's bug deputy report for neutron-related issues. *High* N/A* * *Medium* * https://bugs.launchpad.net/neutron/+bug/1829332 - neutron-server report DhcpPortInUse ERROR log o https://review.opendev.org/659523 <https://review.opendev.org/659523> and https://review.opendev.org/659524 <https://review.opendev.org/659524> have been proposed against neutron and neutron-lib respectively. * https://bugs.launchpad.net/neutron/+bug/1829261 - Duplicate quota entry for project_id/resource causes inconsistent behaviour * https://bugs.launchpad.net/neutron/+bug/1829357 - Firewall-as-a-Service (FWaaS) v2 scenario in neutron o This is a documentation issue, the docs suggest commands be invoked via neutron CLI when this is not supported. https://review.opendev.org/659721 <https://review.opendev.org/659721> has been proposed to fix this. * https://bugs.launchpad.net/neutron/+bug/1829387 - no way for non admin users to get networks * https://bugs.launchpad.net/neutron/+bug/1829414 - Attribute filtering should be based on all objects instead of only first o https://review.opendev.org/659617 <https://review.opendev.org/659617> has been proposed to address this. *Low* N/A* * *RFE* * https://bugs.launchpad.net/neutron/+bug/1829449*- *Implement consistency check and self-healing for SDN-managed fabrics o A potentially interesting RFE around enabling feedback mechanisms for ML2 mech drivers. I'm not sure what to make of this yet, but I think it warrants some further discussion. ** *Filed and Fix Released: * * https://bugs.launchpad.net/neutron/+bug/1829304 - Neutron returns HttpException: 500 on certain operations with modified list of policies for non-admin user o https://review.opendev.org/#/c/659397/ was merged on master, with backports proposed to stable/stein and stable/rocky *Unassigned:* https://bugs.launchpad.net/neutron/+bug/1829261 and https://bugs.launchpad.net/neutron/+bug/1829387 -Ryan Tidwell

2 2