October 2019 - openstack-discuss - lists.openstack.org

octavia 5.0.0.0rc2 (train)
by no-reply＠openstack.org 07 Oct '19

07 Oct '19

Hello everyone, A new release candidate for octavia for the end of the Train cycle is available! You can find the source code tarball at: https://tarballs.openstack.org/octavia/ Unless release-critical issues are found that warrant a release candidate respin, this candidate will be formally released as the final Train release. You are therefore strongly encouraged to test and validate this tarball! Alternatively, you can directly test the stable/train release branch at: https://opendev.org/openstack/octavia/src/branch/stable/train Release notes for octavia can be found at: https://docs.openstack.org/releasenotes/octavia/ If you find an issue that could be considered release-critical, please file it at: https://storyboard.openstack.org/#!/project/908 and tag it *train-rc-potential* to bring it to the octavia release crew's attention.

1 0

Re: [Airship-discuss] Fwd: OOK,Airship
by Roman Gorshunov 07 Oct '19

07 Oct '19

Thanks Ashlee! Charles, A few companies who work on development of Airship do use it, including production uses: AT&T, SUSE, Mirantis, Ericsson, SK Telekom and others. Many of those companies (if not all) use Airship + OpenStack Helm as well. Airship, as you have mentioned, is a collection of components for undercloud control plane, which helps to deploy nodes with OS+Docker+Kubernetes on it, configure/manage it all in GitOps way, and then help to maintain the configuration. It also allows to manage deploys and maintenance of whatever runs on top of Kubernetes cluster, would that be OpenStack Helm or other software packaged in Helm format. OpenStack Helm does not really require to be running on Airship-managed cluster. It could run standalone. Yes, you can roll out an open source production grade Airship/Openstack Helm deployment today. Good example of production grade configuration could be found in airship/treasuremap repository [0] as 'seaworthy' site definition. You are welcome to try, of course. For the questions - reach out to us on IRC #airshipit@Freenode of via Airship-discuss mailing list. [0] https://opendev.org/airship/treasuremap Best regards, -- Roman Gorshunov On Wed, Oct 2, 2019 at 9:27 PM Ashlee Ferguson <ashlee(a)openstack.org> wrote: > > Hi Charles, > > Glad to hear you’re interested! Forwarding this to the Airship ML since there may be folks on this mailing list that will have pointers who didn't see the openstack-discuss post. > > Ashlee > > > > Begin forwarded message: > > From: Charles <cems(a)ebi.ac.uk> > Subject: OOK,Airship > Date: October 2, 2019 at 5:39:16 PM GMT+2 > To: openstack-discuss(a)lists.openstack.org > > Hi, > > > We are interested in OOK and Openstack Helm. > > Has anyone any experience with Airship (now that 1.0 is out)? > > Noticed that a few Enterprise distributions are looking at managing the Openstack control plane with Kubernetes and have been testing Airship with a view to rolling it out (Mirantis,SUSE) > > Is this a signal that there is momentum around Openstack Helm? > > Is it possible to roll out an open source production grade Airship/Openstack Helm deployment today, or is it too early? > > > Thoughts? > > > Charles > > > > > _______________________________________________ > Airship-discuss mailing list > Airship-discuss(a)lists.airshipit.org > http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss

3 2

[kolla] Feature freeze
by Mark Goddard 07 Oct '19

07 Oct '19

Hello Koalas, We are now in feature freeze for the Train release. Cores, please do not approve feature patches on the master branch until we have created the stable/train branch. We will allow some exceptions which must be approved by the core team. Currently, we have nova cells support and IPv6-only mode. Please apply for feature freeze exceptions either on openstack-discuss or during the weekly IRC meeting. The deadline for merging features with exceptions is Friday 18th October. Please now focus on bug fixing and testing. Thanks, Mark

1 0

[neutron] Bug deputy report (week starting on 2019-09-30)
by Bernard Cafarelli 07 Oct '19

07 Oct '19

Hello Neutrinos, train is almost ready to leave the station, and it is time for a new bug deputy rotation cycle! I was on duty last week, triaging bugs up to 1846703 included A quiet week, with most bugs having potential fixes or good discussions. First one listed could benefit from another pair of eyes Undecided: * neutron-openvswitch-agent and IPv6 - https://bugs.launchpad.net/neutron/+bug/1846494 Can not use an IPv6 address for OpenFlow connections listening address (of_listen_address) High: * Pyroute2 can return dictionary keys in bytes instead of strings - https://bugs.launchpad.net/neutron/+bug/1846360 Fix in progress: https://review.opendev.org/686206 * [mysql8] Unknown column 'public' in 'firewall_rules_v2' - https://bugs.launchpad.net/neutron/+bug/1846606 neutron-fwaas db creation failing with mysql 8 Fix in progress: https://review.opendev.org/686753 <https://review.opendev.org/#/c/686753/> Medium: * Designate integration not fully multi region safe - https://bugs.launchpad.net/neutron/+bug/1845891 Fix released: https://review.opendev.org/684854 RFE: * routed network for hypervisor - https://bugs.launchpad.net/neutron/+bug/1846285 Proposition to have routed networks separation at hypervisor level directly, apparently already running in-house at bug reporter's Wishlist: * Avoid neutron to return error 500 when deleting port if designate is down - https://bugs.launchpad.net/neutron/+bug/1846703 Another bug for Designate support, port create and delete operations do not react the same when designate is down Some discussions also in https://review.opendev.org/685644 Opinion: * ovs VXLAN over IPv6 conflicts with linux native VXLAN over IPv4 using standard port - https://bugs.launchpad.net/neutron/+bug/1846507 Configuration issue in kolla-ansible CI, ovs-agent and CI configuration competing for IPv6 binding address. Neutron listed for possible insights on the issue Invalid: * ha router appear double vip - https://bugs.launchpad.net/neutron/+bug/1845900 Kolla issue with HA controllers when stopping L3 agent - keepalived processes are in same container and are killed at same time as agent, added Kolla to affected projects * packet loss during active L3 HA agent restart - https://bugs.launchpad.net/neutron/+bug/1846198 Similar issue for openstack-ansible, it kills all processes in control group (including keepalived processes) when restarting the systemd unit - added OSA as affected project Thanks! Passing the deputy role to slaweq -- Bernard Cafarelli

1 0

ANNOUNCE: Train packages repository for Debian Buster is now available and tested
by Thomas Goirand 06 Oct '19

06 Oct '19

Hi, It's been a few days already, there's some fully working (and tested) Debian repositories backported to Buster for train. The URLs are using the usual scheme: deb http://buster-train.debian.net/debian/ buster-train-backports main deb-src http://buster-train.debian.net/debian/ buster-train-backports main deb http://buster-train.debian.net/debian/ buster-train-backports-nochange main deb-src http://buster-train.debian.net/debian/ buster-train-backports-nochange main Early last week, I was able to test this, doing my first deployment, and starting my first VM on it. I haven't run tempest on this yet, though my manual tests went well (ie: floating IP, ssh to instance, mounting a cinder volume over Ceph and LVM, etc.). Please do test this, and report any eventual issue. If everything goes as planned, I'll be at the Debian cloud sprint in Boston the week of the release, discussing the Debian official images for the cloud. So I will only be able to upload the final versions of projects for Train only then after (probably, during the week-end, so it's available on Monday). Cheers, Thomas Goirand (zigo)

2 1

Neutron Dhcp-agent
by Jyoti Dahiwele 06 Oct '19

06 Oct '19

Dear Team, Please clarify on how can I use dhcp-agent of neutron as a relay and to use existing dhcp for allocation of IPs to instances?

3 2

[keystone] Keystone Team Update - Week of 30 September 2019
by Colleen Murphy 04 Oct '19

04 Oct '19

# Keystone Team Update - Week of 30 September 2019 ## News Quiet week as we wait for the final release and start preparing for Forum and next cycle. ## Action Items Team members: see action required regarding the new roadmap tracker[1]. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/009942.… ## Office Hours When there are topics to cover, the keystone team holds office hours on Tuesdays at 17:00 UTC. We won't plan to hold office hours next week. Add topics you would like to see covered during office hours to the etherpad: https://etherpad.openstack.org/p/keystone-office-hours-topics ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 7 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 33 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Bugs This week we opened 1 new bugs and closed 3. Bugs opened (1) Bug #1846817 (keystone:Medium) opened by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1846817 Bugs fixed (3) Bug #968696 (keystone:High) fixed by Colleen Murphy https://bugs.launchpad.net/keystone/+bug/968696 Bug #1630434 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1630434 Bug #1806762 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1806762 Notably, we closed #968696 *for keystone*, as we have completed the migration of our policies to understand system scope and, when [oslo_policy]/enforce_scope is set to true and deprecated policies are overridden, system-wide requests won't respond to project-scoped tokens. This does not mean the "admin"-ness problem is solved across OpenStack, as it will have to be addressed on a service-by-service basis. ## Milestone Outlook https://releases.openstack.org/train/schedule.html Next week will be the last chance to release another RC if we need one. Please help triage and address any RC-critical bugs should they come up. Also, the release schedule for Ussuri has been published: https://releases.openstack.org/ussuri/schedule.html ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[oslo][nova] Revert of oslo.messaging JSON serialization change
by Ben Nemec 04 Oct '19

04 Oct '19

Hi, I've just proposed https://review.opendev.org/#/c/685724/ which reverts a change that recently went in to make the fake driver in oslo.messaging use jsonutils for message serialization instead of json.dumps. As explained in the commit message on the revert, this is problematic because the rabbit driver uses kombu's default serialization method, which is json.dumps. By changing the fake driver to use jsonutils we've made it more lenient than the most used real driver which opens us up to merging broken changes in consumers of oslo.messaging. We did have some discussion of whether we should try to override the kombu default and tell it to use jsonutils too, as a number of other drivers do. The concern with this was that the jsonutils handler for things like datetime objects is not tz-aware, which means if you send a datetime object over RPC and don't explicitly handle it you could lose important information. I'm open to being persuaded otherwise, but at the moment I'm leaning toward less magic happening at the RPC layer and requiring projects to explicitly handle types that aren't serializable by the standard library json module. If you have a different preference, please share it here. Thanks. -Ben

4 5

Proposed Forum Schedule
by Jimmy Mcarthur 04 Oct '19

04 Oct '19

The forum schedule is now live: https://www.openstack.org/summit/shanghai-2019/summit-schedule/global-searc… If you'd prefer to use the spreadsheet view: https://drive.google.com/file/d/1qp0I9xnyOK3mhBitQnk2a7VuS9XClvyF Please let Kendall Nelson or myself know as soon as possible if you see any conflicts. Cheers, Jimmy

1 0

[Glance][PTG]Shanghai PTG planning
by Abhishek Kekane 04 Oct '19

04 Oct '19

Hello Everyone, I have prepared an etherpad [1] to plan the Shanghai PTG discussion topics for glance. The etherpad contains template to add the topic for discussion. It has also references of previous PTG planning etherpads. Even if anyone is not going to attend the PTG but wants there topic needs to be discussed can add as well. Kindly add your topics. [1] https://etherpad.openstack.org/p/Glance-Ussuri-PTG-planning Thanks, Abhishek Kekane

1 0

[release] Release countdown for week R-1, October 7-11
by Sean McGinnis 03 Oct '19

03 Oct '19

Development Focus ----------------- We are on the final mile of this Train ride! (You can thank Thierry for that one ^) Remember that the Train final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. Thursday, October 10th is the deadline for final Train release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on October 16th. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the Ussuri development cycle, including discussing Forum and PTG sessions content, in preparation of the Summit in Shanghai next month. Actions --------- Watch for any translation patches coming through on the stable/train branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/train branch before triggering a new release. Please drop by #openstack-release with any questions or concerns about the upcoming release! Upcoming Deadlines & Dates -------------------------- Final Train release: October 16 Forum+PTG at Shanghai summit: November 4

1 0

[keystone] Ussuri roadmap
by Colleen Murphy 03 Oct '19

03 Oct '19

Hi team, In past cycles we used Trello for tracking our goals throughout a cycle, which gave us flexibility and visibility over the cycle plans as a whole in conjunction with specs and launchpad bugs. Trello is a proprietary platform and in the last few months changed its ToS to limit the number of public boards an organization can have, and the keystone team has reached that limit. Rather than try to backup and archive our old boards or create another team or a non-team board, for Ussuri I would like to try using a board on a different platform, Taiga: https://tree.taiga.io/project/keystone-ussuri-roadmap/kanban Taiga is AGPLv3 and has no restrictions on its hosted version that I've discovered yet. Many thanks to Morgan for discovering and researching it. I've copied over our incomplete stories from the Train roadmap[1] and arranged the kanban board more or less the same way as the old Trello board, but the platform seems to be very flexible and we could change the layout and workflows in any way that makes sense. For instance, while I only enabled the kanban feature, there is also a sprints/backlog mode if we wanted to take advantage of that. I can grant administrator privileges to anyone who is interested in investigating all the configuration options (or you can create your own sandbox projects to play with). The main deficiency seems to be the lack of support for "teams" or "organizations"[2], but users can be added to the board individually. Action required: * If you were a member of the old Trello keystone team and would to be a member of this board, send me an email address that I can send an invite to * Once you have an account and are added to the board, please have a look at the stories that are already there an assign yourself to the ones you are working on or plan to work on, and update their status or add relevant reviews as comments. Feel free to play with the platform's features and provide feedback in this thread or at next week's team meeting. Please also let me know if you have concerns about using this platform. Colleen [1] https://trello.com/b/ClKW9C8x/keystone-train-roadmap [2] https://tree.taiga.io/project/taiga/us/2129

1 0

[requirements][mistral][amqp] Failing “docs” job due to the upper constraint conflict for amqp
by Renat Akhmerov 03 Oct '19

03 Oct '19

Hi, We have a failing “docs” ([1]) CI job that fails because it implicitly brings amqp 2.5.2 but this lib is not allowed to be higher than 2.5.1 in the upper-constraings.txt in the requirements project ([2]). We see that there’s the patch [3] generated by the proposal bot that bumps the constraint to 2.5.2 for amqp (among others) but it was given -2. Please assist on how to address in the best way. Should we bump only amqp version in upper constraints for now? [1] https://zuul.opendev.org/t/openstack/build/6fe7c7d3e60b40458d2a98f3a293f412… [2] https://github.com/openstack/requirements/blob/master/upper-constraints.txt… [3] https://review.opendev.org/#/c/681382 Thanks Renat Akhmerov @Nokia

2 3

Proposed Forum Schedule
by Jimmy McArthur 03 Oct '19

03 Oct '19

Hello! I'm attaching a PDF of the proposed Shanghai Forum Schedule. I'll publish the same on the actual website later this afternoon. However, there is still time for feedback/time changes, assuming there aren't conflicts for speakers/moderators. This is also available for download here: https://drive.google.com/file/d/1qp0I9xnyOK3mhBitQnk2a7VuS9XClvyF/view?usp=… Please respond to this thread with any concerns. Cheers, Jimmy

1 0

[nova][kolla] questions on cells
by Mark Goddard 03 Oct '19

03 Oct '19

Hi, I'm currently working (with dougsz) on adding support for nova cells v2 in Kolla Ansible. I have a few questions, and things that I'd like confirming. 1. Is there any benefit to not having a superconductor? Presumably it's a little more efficient in the single cell case? Also IIUC it only requires a single message queue so is a little simpler? 2. Do console proxies need to live in the cells? This is what devstack does in superconductor mode. I did some digging through nova code, and it looks that way. Testing with novncproxy agrees. This suggests we need to expose a unique proxy endpoint for each cell, and configure all computes to use the right one via e.g. novncproxy_base_url, correct? 3. Should I upgrade the superconductor or conductor service first? 4. Does the cell conductor need access to the API DB? 5. What DB configuration should be used in nova.conf when running online data migrations? I can see some migrations that seem to need the API DB, and others that need a cell DB. If I just give it the API DB, will it use the cell mappings to get to each cell DB, or do I need to run it once for each cell? 6. After an upgrade, when can we restart services to unpin the compute RPC version? Looking at the compute RPC API, it looks like the super conductor will remain pinned until all computes have been upgraded. For a cell conductor, it looks like I could restart it to unpin after upgrading all computes in that cell, correct? 7. Which services require policy.{yml,json}? I can see policy referenced in API, conductor and compute. I'm trying to design it such that it's possible to deploy a new cell in an existing environment, and upgrade one cell at a time. Getting the ordering right for this is fiddly, and I expect I'll be back to get some eyes on my plan. Cheers, Mark

6 16

[all] Planned Ussuri release schedule published
by Sean McGinnis 03 Oct '19

03 Oct '19

Hey everyone, The proposed release schedule for Ussuri was up for a few weeks with only cosmetic issues to address. The proposed schedule has now been merged and published to: https://releases.openstack.org/ussuri/schedule.html Barring any new issues with the schedule being raised, this should be our schedule for the Ussuri development cycle. The planned Ussuri release date is May 13, 2020. Thanks! Sean

6 11

[i18n] Request to be added as Vietnamese translation group coordinators
by Trinh Nguyen 03 Oct '19

03 Oct '19

Hi i18n team, Dai and I would like to volunteer as the coordinators of the Vietnamese translation group. If you find us qualified, please let us know. Bests, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

5 7

[all] [tc] [docs] [release] [ptls] Docs as SIG: Ownership of docs.openstack.org
by Alexandra Settle 03 Oct '19

03 Oct '19

Hi all, Quick recap: The documentation team is set to disband as an official project, and make the leap to becoming a SIG (Special Interest Group). This decision is not one we have made lightly, but with the changes of direction since Pike (project documentation belonging with the projects, etc) the need for a centralised documentation team as an official project is no longer integral to producing the software. The transition of the docs team to a SIG has already begun [1] and [2]. The openstackdocstheme, openstack-doc-tools, os-api-ref and whereto have been placed within the remit of the Oslo team and approved. The remaining individuals working on the docs team are sorting through what's left and where it is best placed. In [1], Doug has rightfully pointed out that whilst the documentation team as it stands today is no longer "integral", the docs.openstack.org web site is. An owner is required. The suggestion is for the Release Management team is the "least worst" (thanks, tonyb) place for the website manaagement to land. As Tony points out, this requires learning new tools and processes but the individuals working on the docs team currently have no intention to leave, and are around to help manage this from the SIG. Open to discussion and suggestions, but to summarise the proposal here: docs.openstack.org ownership is to transition to be the responsibility of the Release Management team officially provided there are no strong objections. Thanks, Alex IRC: asettle [1] https://review.opendev.org/#/c/657142/ [2] https://review.opendev.org/#/c/657141/

5 11

[kolla-ansible] migration
by Ignazio Cassano 02 Oct '19

02 Oct '19

Hello, Anyone have tried to migrate an existing openstack installation to kolla containers? Thanks Ignazio

5 16

[openstack-ansible] Stepping down as core reviewer
by Jesse Pretorius 02 Oct '19

02 Oct '19

Hi everyone, While I had hoped to manage keeping up with OSA reviews and some contributions, unfortunately there is too much on my plate in my new role to allow me to give OSA sufficient time and I feel that it's important to not give any false promises. I am therefore stepping down as a core reviewer for OSA. My journey with OpenStack-Ansible started with initial contributions before it was an official OpenStack project, went on to helping lead the project to becoming an official project in the big tent, then on to becoming a successful project with diverse contributors of which I was proud to be a part. Over time I learned a heck of a lot about building and leading an Open Source community, about developing Ansible playbooks and roles at significant scale, and about building, packaging and deploying python software. It has been a very valuable experience through which I have grown personally and professionally. This community's strengths are in its leadership by operators, its readiness to assist newcomers and in striving to maintain a deployment system which is easy to understand and use (while somehow also being ridiculously flexible). As Jean-Philippe Evrard has recently expressed, this is the DNA which makes the community special. As you should all be aware, I am always ready to help when asked and I can also share historical context if there is a need for that so please feel free to ping me on IRC or add me to a review and I'll do my best. My journey onward is working with TripleO in the upgrades team, so you'll still find me contributing to OpenStack as a whole. I'll be hanging out in #tripleo and #openstack-dev on IRC if you're looking for me. All the best, Jesse (odyssey4me)

2 1

OOK,Airship
by Charles 02 Oct '19

02 Oct '19

Hi, We are interested in OOK and Openstack Helm. Has anyone any experience with Airship (now that 1.0 is out)? Noticed that a few Enterprise distributions are looking at managing the Openstack control plane with Kubernetes and have been testing Airship with a view to rolling it out (Mirantis,SUSE) Is this a signal that there is momentum around Openstack Helm? Is it possible to roll out an open source production grade Airship/Openstack Helm deployment today, or is it too early? Thoughts? Charles

1 0

[kuryr][kuryr-libnetwork] Nominating Hongbin Lu to kuryr-libnetwork and kuryr core
by Michał Dulko 02 Oct '19

02 Oct '19

Hi, I'd like to nominate Hongbin Lu to be core reviewer in both kuryr- libnetwork and kuryr projects. Besides saying that he's doing a great job maintaining kuryr-libnetwork I'm simply surprised he don't have +2/-2 rights there and that should definitely get fixed. As there isn't a lot of people maintaining those projects anymore, I'll just skip the voting part and add Hongbin to core teams immediately. Thanks, Michał

1 0

glance 19.0.0.0rc1 (train)
by no-reply＠openstack.org 02 Oct '19

02 Oct '19

Hello everyone, A new release candidate for glance for the end of the Train cycle is available! You can find the source code tarball at: https://tarballs.openstack.org/glance/ Unless release-critical issues are found that warrant a release candidate respin, this candidate will be formally released as the final Train release. You are therefore strongly encouraged to test and validate this tarball! Alternatively, you can directly test the stable/train release branch at: https://opendev.org/openstack/glance/src/branch/stable/train Release notes for glance can be found at: https://docs.openstack.org/releasenotes/glance/ If you find an issue that could be considered release-critical, please file it at: https://bugs.launchpad.net/glance/+bugs and tag it *train-rc-potential* to bring it to the glance release crew's attention.

1 0

issues creating a second vm with numa affinity
by Manuel Sopena Ballesteros 01 Oct '19

01 Oct '19

Dear Openstack user community, I have a compute node with 2 numa nodes and I would like to create 2 vms, each one using a different numa node through numa affinity with cpu, memory and nvme pci devices. pci passthrough whitelist [root@zeus-53 ~]# tail /etc/kolla/nova-compute/nova.conf [notifications] [filter_scheduler] enabled_filters = enabled_filters = RetryFilter, AvailabilityZoneFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, PciPassthroughFilter available_filters = nova.scheduler.filters.all_filters [pci] passthrough_whitelist = [ {"address":"0000:06:00.0"}, {"address":"0000:07:00.0"}, {"address":"0000:08:00.0"}, {"address":"0000:09:00.0"}, {"address":"0000:84:00.0"}, {"address":"0000:85:00.0"}, {"address":"0000:86:00.0"}, {"address":"0000:87:00.0"} ] alias = { "vendor_id":"8086", "product_id":"0953", "device_type":"type-PCI", "name":"nvme"} Openstack flavor openstack flavor create --public xlarge.numa.perf.test --ram 200000 --disk 700 --vcpus 20 --property hw:cpu_policy=dedicated --property hw:emulator_threads_policy=isolate --property hw:numa_nodes='1' --property pci_passthrough:alias='nvme:4' The first vm is successfully created openstack server create --network hpc --flavor xlarge.numa.perf.test --image centos7.6-image --availability-zone nova:zeus-53.localdomain --key-name mykey kudu-1 However the second vm fails openstack server create --network hpc --flavor xlarge.numa.perf --image centos7.6-kudu-image --availability-zone nova:zeus-53.localdomain --key-name mykey kudu-4 Errors in nova compute node 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [req-b5a25c73-8c7d-466c-8128-71f29e7ae8aa 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] Instance failed to spawn: libvirtError: internal error: qemu unexpectedly closed the monitor: 2019-09-27T06:45:19.118089Z qemu-kvm: kvm_init_vcpu failed: Cannot allocate memory 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] Traceback (most recent call last): 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2369, in _build_resources 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] yield resources 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2133, in _build_and_run_instance 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] block_device_info=block_device_info) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3142, in spawn 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] destroy_disks_on_failure=True) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5705, in _create_domain_and_network 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] destroy_disks_on_failure) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] self.force_reraise() 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] six.reraise(self.type_, self.value, self.tb) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5674, in _create_domain_and_network 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] post_xml_callback=post_xml_callback) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5608, in _create_domain 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] guest.launch(pause=pause) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] self._encoded_xml, errors='ignore') 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] self.force_reraise() 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] six.reraise(self.type_, self.value, self.tb) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] return self._domain.createWithFlags(flags) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] result = proxy_call(self._autowrap, f, *args, **kwargs) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] rv = execute(f, *args, **kwargs) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] six.reraise(c, e, tb) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] rv = meth(*args, **kwargs) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1110, in createWithFlags 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] libvirtError: internal error: qemu unexpectedly closed the monitor: 2019-09-27T06:45:19.118089Z qemu-kvm: kvm_init_vcpu failed: Cannot allocate memory 2019-09-27 16:45:19.785 7 ERROR nova.compute.manager [instance: ebe4e78c-501e-4535-ae15-948301cbf1ae] Numa cell/node 1 (the one assigned on kudu-4) has enough cpu, memory, pci devices and disk capacity to fit this vm. NOTE: below is the information relevant I could think of that shows resources available after creating the second vm. [root@zeus-53 ~]# numactl -H available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 28 29 30 31 32 33 34 35 36 37 38 39 40 41 node 0 size: 262029 MB node 0 free: 52787 MB node 1 cpus: 14 15 16 17 18 19 20 21 22 23 24 25 26 27 42 43 44 45 46 47 48 49 50 51 52 53 54 55 node 1 size: 262144 MB node 1 free: 250624 MB node distances: node 0 1 0: 10 21 1: 21 10 NOTE: this is to show that numa node/cell 1 has enough resources available (also nova-compute logs shows that kudu-4 is assigned to cell 1) [root@zeus-53 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/md127 3.7T 9.1G 3.7T 1% / ... NOTE: vm disk files goes to root (/) partition [root@zeus-53 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 59.6G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 16G 0 part [SWAP] loop0 7:0 0 100G 0 loop └─docker-9:127-6979358884-pool 253:0 0 100G 0 dm ├─docker-9:127-6979358884-4301cee8d0433729cd6332ca2b6111afc85f14c48d4ce2d888a1da0ef9b5ca01 253:1 0 10G 0 dm ├─docker-9:127-6979358884-d59208adcb7cee3418f810f24e6c3a55d39281f713c8e76141fc61a8deba8a2b 253:2 0 10G 0 dm ├─docker-9:127-6979358884-106bc0838e37442eca84eb9ab17aa7a45308b7e3a38be3fb21a4fa00366fe306 253:3 0 10G 0 dm ├─docker-9:127-6979358884-7e16b5d012ab8744739b671fcdc8e47db5cc64e6c3d5a5fe423bfd68cfb07b20 253:4 0 10G 0 dm ├─docker-9:127-6979358884-f1c2545b4edbfd7b42d2a492eda8224fcf7cefc3e3a41e65d307c585acffe6a8 253:5 0 10G 0 dm ├─docker-9:127-6979358884-e7fd6c7b3f624f387bdb3746a7944a30c92d8ee5395e75c76288b281bd009d90 253:6 0 10G 0 dm ├─docker-9:127-6979358884-95a818cc7afd9867385bb9a9ea750d4cc6e162916c6ae3a157097af74578e1e4 253:7 0 10G 0 dm ├─docker-9:127-6979358884-9a7f28d396c149119556f382bf5c19f5925eed5d18b94407649244c7adabb4b3 253:8 0 10G 0 dm ├─docker-9:127-6979358884-b25941b6f115300caea977911e2d7fd3541ef187c9aa5736fe10fad638ecd0d1 253:9 0 10G 0 dm ├─docker-9:127-6979358884-122b201c6ad24896a205f8db4a64759ba8fbd5bbe245d0f98984268a01e6a0c4 253:10 0 10G 0 dm └─docker-9:127-6979358884-bc04120ba59a1b393f338a1cef64b16d920cf4e73400198e4b999bb72a42ff90 253:11 0 10G 0 dm loop1 7:1 0 2G 0 loop └─docker-9:127-6979358884-pool 253:0 0 100G 0 dm ├─docker-9:127-6979358884-4301cee8d0433729cd6332ca2b6111afc85f14c48d4ce2d888a1da0ef9b5ca01 253:1 0 10G 0 dm ├─docker-9:127-6979358884-d59208adcb7cee3418f810f24e6c3a55d39281f713c8e76141fc61a8deba8a2b 253:2 0 10G 0 dm ├─docker-9:127-6979358884-106bc0838e37442eca84eb9ab17aa7a45308b7e3a38be3fb21a4fa00366fe306 253:3 0 10G 0 dm ├─docker-9:127-6979358884-7e16b5d012ab8744739b671fcdc8e47db5cc64e6c3d5a5fe423bfd68cfb07b20 253:4 0 10G 0 dm ├─docker-9:127-6979358884-f1c2545b4edbfd7b42d2a492eda8224fcf7cefc3e3a41e65d307c585acffe6a8 253:5 0 10G 0 dm ├─docker-9:127-6979358884-e7fd6c7b3f624f387bdb3746a7944a30c92d8ee5395e75c76288b281bd009d90 253:6 0 10G 0 dm ├─docker-9:127-6979358884-95a818cc7afd9867385bb9a9ea750d4cc6e162916c6ae3a157097af74578e1e4 253:7 0 10G 0 dm ├─docker-9:127-6979358884-9a7f28d396c149119556f382bf5c19f5925eed5d18b94407649244c7adabb4b3 253:8 0 10G 0 dm ├─docker-9:127-6979358884-b25941b6f115300caea977911e2d7fd3541ef187c9aa5736fe10fad638ecd0d1 253:9 0 10G 0 dm ├─docker-9:127-6979358884-122b201c6ad24896a205f8db4a64759ba8fbd5bbe245d0f98984268a01e6a0c4 253:10 0 10G 0 dm └─docker-9:127-6979358884-bc04120ba59a1b393f338a1cef64b16d920cf4e73400198e4b999bb72a42ff90 253:11 0 10G 0 dm nvme0n1 259:8 0 1.8T 0 disk └─nvme0n1p1 259:9 0 1.8T 0 part └─md127 9:127 0 3.7T 0 raid0 / nvme1n1 259:6 0 1.8T 0 disk └─nvme1n1p1 259:7 0 1.8T 0 part └─md127 9:127 0 3.7T 0 raid0 / nvme2n1 259:2 0 1.8T 0 disk nvme3n1 259:1 0 1.8T 0 disk nvme4n1 259:0 0 1.8T 0 disk nvme5n1 259:3 0 1.8T 0 disk NOTE: this is to show that there are 4 nvme disks (nvme2n1, nvme3n1, nvme4n1, nvme5n1) available for the second vm What "emu-kvm: kvm_init_vcpu failed: Cannot allocate memory" means in this context? Thank you very much NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

3 3

[qa][stable] tempest.api.volume.test_versions.VersionsTest.test_show_version fails on stable/pike
by Matt Riedemann 01 Oct '19

01 Oct '19

A new test was recently added to tempest [1] which uses the volume API v3 client to list and show versions and compare those against the response schema. It's failing against stable/pike changes [2] presumably because the volume v3 API does not show the (deprecated) v1 API, though I'm a bit confused because the test is doing a GET / and then for each version in the response it's doing a GET /{version}/ so I'm not sure what's up with cinder's pike API version interface where it would list a version but not be able to show it. Anyway, it sounds like this is another case where we're going to have to pin tempest to a tag in devstack on stable/pike to continue running tempest jobs against stable/pike changes, similar to what recently happened with stable/ocata [3]. [1] https://review.opendev.org/#/c/680790/ [2] https://145b0d5be77f0cbe05ff-d67a6226f6eb6d34ab4efd951744a16f.ssl.cf5.rackc… [3] https://review.opendev.org/#/c/681950/ -- Thanks, Matt

3 5

[OSA][openstack-ansible] Stepping down from core reviewer
by Jean-Philippe Evrard 01 Oct '19

01 Oct '19

Hello OSA friends, It's with great sadness that announcing I will be stepping down from OpenStack-Ansible's core role. OSA has been the place where I grew from contributor to OpenStack core for the first time, so it will always keep a special place in my mind :) It's also where I met contributors I can now consider personal friends. It's a project I've helped grow and prosper the last 4 years. I am very happy of what we have all achieved. My last goals in OSA were to simplify it further and a focus on bare metal. Those efforts are either merged or advanced enough nowadays. I consider I have achieved what I wanted to: OSA is now easier than ever to manage, contribute, and deliver. With more experienced people leaving and new people joining, I sure hope the DNA of the project will stay the same: An always welcoming and friendly community, with a no-bullshit and not-too-serious attitude. A project focusing on operator issues and use cases, mentoring contributors to be great members of the OpenStack community. Again, I want to thank you for being an amazing community, and it's been great working with all of you. I think there are still plenty of things OSA can achieve. If you want my opinion, you shouldn't hesitate to contact me. I just don't have enough time to keep up with the reviews, nor am I actively contributing enough to stay at core. All the best, Jean-Philippe Evrard (evrardjp)

5 4

[tc] Weekly update
by Jean-Philippe Evrard 01 Oct '19

01 Oct '19

Hello friends, Here's what need attention for the OpenStack TC this week: 1. We should ensure we have two TC members focusing on next cycle goal selection process. Only Ghanshyam is dealing with this, and we must help him on the way! Any volunteers? Thanks again gmann for working on that. 2. Jimmy McArthur sent us the results of the OpenStack User survey on the ML [1]. We currently haven't analyzed the information yet. Any volunteer to analyse the information (in order to extract action items) is welcomed. It would be great if we could discuss this at our next official meeting, or at least discuss the next steps. 3. Our next meeting date will be the Thursday 10 October. I will be travelling that day, so it would be nice to have a volunteer to host the meeting. For that, our next meeting agenda needs clarifications. It would be great if you could update the agenda (please also write if your absent) on the wiki [2], so that I can send the invite to the ML. I will send the invite on Thursday. 4. We still haven't finished the conversationg about naming releases. There are a few new ideas floated around, so we should maybe drop the current process to take count of the newly proposed ideas (The large cities lists proposed by Nate, the movie quotes proposed by Thierry [9])? Alternatively, if we can't find consensus, should we just entrust the release naming process to the release team? 5. We should decide to deprecate or not the PowerVMStackers team [3] and move it as a SIG. The votes don't reflect this. Thank you everyone! [1]: http://lists.openstack.org/pipermail/openstack-discuss/2019-September/00950… [2]: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [3]: https://review.opendev.org/680438 [4]: https://review.opendev.org/680985 [5]: https://review.opendev.org/681260 [6]: https://review.opendev.org/681480 [7]: https://review.opendev.org/681924 [8]: https://review.opendev.org/682380 [9]: https://review.opendev.org/684688

3 3

networking-midonet 9.0.0.0rc2 (train)
by no-reply＠openstack.org 01 Oct '19

01 Oct '19

Hello everyone, A new release candidate for networking-midonet for the end of the Train cycle is available! You can find the source code tarball at: https://tarballs.openstack.org/networking-midonet/ Unless release-critical issues are found that warrant a release candidate respin, this candidate will be formally released as the final Train release. You are therefore strongly encouraged to test and validate this tarball! Alternatively, you can directly test the stable/train release branch at: https://opendev.org/openstack/networking-midonet/src/branch/stable/train Release notes for networking-midonet can be found at: https://docs.openstack.org/releasenotes/networking-midonet/ If you find an issue that could be considered release-critical, please file it at: https://bugs.launchpad.net/networking-midonet/+bugs and tag it *train-rc-potential* to bring it to the networking-midonet release crew's attention.

1 0

[nova][ops] Removing Debug middleware
by Eric Fried 01 Oct '19

01 Oct '19

Deployers- There's a Debug middleware in nova that's not used in the codebase since 2010, so we're removing it [1]. BUT Theoretically, deployments could be making use of it by injecting it into the paste pipeline (a ~3LOC edit to your local api-paste.ini). If this is you, and you're really relying on this behavior, let us know. We can either revert the change or show you how to carry it locally (which would be really easy to do). Thanks, efried [1] https://review.opendev.org/#/c/662506/

1 0

[all] [tc] [ptls] PDF Goal Aftermath
by Alexandra Settle 01 Oct '19

01 Oct '19

Hi all, Thanks to all those who worked super hard to achieve the PDF Enablement goal for Train. Things are looking great, and I've already received feedback from downstream about how useful the PDFs are! So that's fantastic. For the sake of the goal, the intention was to have potentially imperfect PDFs and to ensure they're building. In some cases, this does leave room for improvement. In openstack-doc, we've been approached a few times regarding moving forward when there are issues with no known workarounds. So, asking all those that have been working on/building their PDFs and are experiencing issues that can be fixed post-mortem to ensure everything is documented in our Common Problems etherpad [1]. Ensuring everything is documented, means we can work together to identify appropriate workarounds or fixes in the future. We will review all items without a workaround in the short term. Cheers, Alex [1] https://etherpad.openstack.org/p/pdf-goal-train-common-problems -- Alexandra Settle <a.settle(a)outlook.com> IRC: asettle

1 0