June 2019 - openstack-discuss - lists.openstack.org

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[openstack-ansible] dropping suse support
by Mohammed Naser 05 May '20

05 May '20

Hi everyone, I've been trying to avoid writing this email for the longest time ever. It's really painful to have to reach this state, but I think we've hit a point where we can no longer maintain support for SUSE within OpenStack Ansible. Unfortunately, with the recent layoffs, the OSA team has taken a huge hit in terms of contributors which means that there is far less contributors within the project. This means that we have less resources to go around and it forces us to focus on a more functional, reliable and well tested set of scenarios. Over the past few cycles, there has been effort to add SUSE support to OpenStack Ansible, during the time that we had maintainers, it was great and SUSE issues were being fixed promptly. In addition, due to the larger team at the time, we found ourselves having some extra time where we can help unbreak other gates. Jesse used to call this a "labour of love", which I admired at the time and hoped we continue to do as much as we can of. However, the lack of a committed maintainer for OpenSUSE has resulted in constantly failing jobs[1][2] (which were moved to non-voting, wasting CI resources as no one fixed them). In addition, it's causing several gate blocks for a few times with no one really finding the time to clean them up. We are resource constrained at this point and we need the resource to go towards making the small subset of supported features functional (i.e. CentOS/Ubuntu). We struggle with that enough, and there seems to be no deployers that are running SUSE in real life at the moment based on bugs submitted. With that, I propose that we drop SUSE support this cycle. If anyone would like to volunteer to maintain it, we can review that option, but that would require a serious commitment as we've had maintainers step off and it hurts the velocity of the project as no one can merge code anymore. ..Really wish I didn't have to write this email Mohammed [1]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct… [2]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct…

6 9

[kolla-ansible] migration
by Ignazio Cassano 02 Oct '19

02 Oct '19

Hello, Anyone have tried to migrate an existing openstack installation to kolla containers? Thanks Ignazio

5 16

[nova] critical bug around reload/upgrades
by Mohammed Naser 23 Aug '19

23 Aug '19

Hello: I've discussed this for quite sometime with Dan over IRC and a bit with Zane as well, but basically, Nova thinks that when it gets a reload (aka SIGHUP), nothing else has occurred. However, oslo.service actually calls stop(), reload() then start() again, which potentially kills all RPC. This has caused a pretty big issue in our gates and it also means that the whole idea behind 'reload nova-compute while upgrading to refresh info' concept is fundamentally broken. I tried to do some work on this here, however, I wasn't really able to get to the bottom of it. There seems to be a decision that needs to be taken in terms of .. do we change what reload() actually means in oslo_service (it actually is more like a restart, not a reload) or does Nova (and other projects) change their implementation in assuming what reload() does? https://review.openstack.org/#/c/641907/ This seems to have been floating around for a really long time, so I'd be happy to work with someone to find the fix (and we can totally test it inside OpenStack Ansible by reloading instead of restarting). Thanks! Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

4 4

[nova][ops] What should the compute service delete behavior be wrt resource providers with allocations?
by Matt Riedemann 22 Aug '19

22 Aug '19

Before [1] when deleting a compute service in the API we did not check to see if the compute service was hosting any instances and just blindly deleted the service and related compute_node(s) records which orphaned the resource provider(s) for those nodes. With [2] we built on that and would cleanup the (first [3]) compute node resource provider by first deleting any allocations for instances still on that host - which because of the check in [1] should be none - and then deleted the resource provider itself. [2] forgot about ironic where a single compute service can be managing multiple (hundreds or even thousands) of baremetal compute nodes so I wrote [3] to delete *all* resource providers for compute nodes tied to the service - again barring there being any instances running on the service because of the check added in [1]. What we've failed to realize until recently is that there are cases where deleting the resource provider can still fail because there are allocations we haven't cleaned up, namely: 1. Residual allocations for evacuated instances from a source host. 2. Allocations held by a migration record for an unconfirmed (or not yet complete) migration. Because the delete_resource_provider method isn't checking for those, we can get ResourceProviderInUse errors which are then ignored [4]. Since that error is ignored, we continue on to delete the compute service record [5], effectively orphaning the providers (which is what [2] was meant to fix). I have recreated the evacuate scenario in a functional test here [6]. The question is what should we do about the fix? I'm getting lost thinking about this in a vacuum so trying to get some others to help think about it. Clearly with [1] we said you shouldn't be able to delete a compute service that has instances on it because that corrupts our resource tracking system. If we extend that to any allocations held against providers for that compute service, then the fix might be as simple as not ignoring the ResourceProviderInUse error and fail if we can't delete the provider(s). The question I'm struggling with is what does an operator do for the two cases mentioned above, not-yet-complete migrations and evacuated instances? For migrations, that seems pretty simple - wait for the migration to complete and confirm it (reverting a cold migration or resize would put the instance back on the compute service host you're trying to delete). The nastier thing is the allocations tied to an evacuated instance since those don't get cleaned up until the compute service is restarted [7]. If the operator never intends on restarting that compute service and just wants to clear the data, then they have to manually delete the allocations for the resource providers associated with that host before they can delete the compute service, which kind of sucks. What are our options? 1. Don't delete the compute service if we can't cleanup all resource providers - make sure to not orphan any providers. Manual cleanup may be necessary by the operator. 2. Change delete_resource_provider cascade=True logic to remove all allocations for the provider before deleting it, i.e. for not-yet-complete migrations and evacuated instances. For the evacuated instance allocations this is likely OK since restarting the source compute service is going to do that cleanup anyway. Also, if you delete the source compute service during a migration, confirming or reverting the resize later will likely fail since we'd be casting to something that is gone (and we'd orphan those allocations). Maybe we need a functional recreate test for the unconfirmed migration scenario before deciding on this? 3. Other things I'm not thinking of? Should we add a force parameter to the API to allow the operator to forcefully delete (#2 above) if #1 fails? Force parameters are hacky and usually seem to cause more problems than they solve, but it does put the control in the operators hands. If we did remove allocations for an instance when deleting it's compute service host, the operator should be able to get them back by running the "nova-manage placement heal_allocations" CLI - assuming they restart the compute service on that host. This would have to be tested of course. Help me Obi-Wan Kenobi. You're my only hope. [1] https://review.opendev.org/#/q/I0bd63b655ad3d3d39af8d15c781ce0a45efc8e3a [2] https://review.opendev.org/#/q/I7b8622b178d5043ed1556d7bdceaf60f47e5ac80 [3] https://review.opendev.org/#/c/657016/ [4] https://github.com/openstack/nova/blob/cb0cfc90e1e03e82c42187ec60f46fb8fd59… [5] https://github.com/openstack/nova/blob/cb0cfc90e1e03e82c42187ec60f46fb8fd59… [6] https://review.opendev.org/#/c/663737/ [7] https://github.com/openstack/nova/blob/cb0cfc90e1e03e82c42187ec60f46fb8fd59… -- Thanks, Matt

6 16

[storyboard] email notification on stories/tasks of subscribed projects
by Akihiro Motoki 22 Aug '19

22 Aug '19

Hi, Is there a way to get email notification on stories/tasks of subscribed projects in storyboard? In launchpad, I configure launchpad bug notifications of all bug changes in interested projects. After migration to storyboard (for example, openstackclient), I failed to find a similar way. Thanks, Akihiro Motoki (irc: amotoki)

5 8

[tripleo][openstack-ansible] Integrating ansible-role-collect-logs in OSA
by Chandan kumar 12 Aug '19

12 Aug '19

Hello, With os_tempest project, TripleO and Openstack Ansible projects started collaborating together to reuse the tools developed by each other to avoid duplicates and enable more collaboration. During Denver Train 2019 PTG, we decided to unifying the CI logs for both the projects by providing a unified experience to the developers while browsing the CI logs making sure we have a similar logs structure for both the projects, so that one can easily navigate through the logs without scratching the heads and also the logs tree structure should say what logs are present where. In TripleO, we have ansible-role-collect-logs[1.] role for the same and in OSA we have logs_collect.sh[2.] script for the same. But once the logs gets collected at each other projects, It is very hard to navigate and find out where is the respective files. A little about ansible-role-collect-logs roles: * A role for aggregating logs from different nodes * Provide a list for files to collect at one places in playbook [3] * Hyperlink the logs with description about log files * Once logs collection is done, it pushes to a particular log server For example, tempest.html or stestr_results.html is the common file for viewing tempest results. In TripleO, we keep it under logs folder but in OSA logs/openstack/aio1-utility/stestr_results.html. If a new user contributes to another project, he tries to follow the same pattern for find logs as he seen in current project. By Keeping the same structure at all places It would be easier. So moving ahead what we are going to do: * Refactor collect-logs role to pass defaults list of files at one place * Pass the list of different logs files based on deployment tools * Put system/containers related commands at one place * Replace the collect_logs.sh script with playbook in OSA and replace it. Thanks for reading, We are looking for the comments on the above suggestion. Links: [1.] https://opendev.org/openstack/ansible-role-collect-logs/ [2.] https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/l… [3.] https://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/… Thanks, Chandan Kumar

4 4

[ironic][ops] Taking ironic nodes out of production
by Arne Wiebalck 09 Aug '19

09 Aug '19

Dear all, One of the discussions at the PTG in Denver raised the need for a mechanism to take ironic nodes out of production (a task for which the currently available 'maintenance' flag does not seem appropriate [1]). The use case there is an unhealthy physical node in state 'active', i.e. associated with an instance. The request is then to enable an admin to mark such a node as 'faulty' or 'in quarantine' with the aim of not returning the node to the pool of available nodes once the hosted instance is deleted. A very similar use case which came up independently is node retirement: it should be possible to mark nodes ('active' or not) as being 'up for retirement' to prepare the eventual removal from ironic. As in the example above, ('active') nodes marked this way should not become eligible for instance scheduling again, but automatic cleaning, for instance, should still be possible. In an effort to cover these use cases by a more general "quarantine/retirement" feature: - are there additional use cases which could profit from such a "take a node out of service" mechanism? - would these use cases put additional constraints on how the feature should look like (e.g.: "should not prevent cleaning") - are there other characteristics such a feature should have (e.g.: "finding these nodes should be supported by the cli") Let me know if you have any thoughts on this. Cheers, Arne [1] https://etherpad.openstack.org/p/DEN-train-ironic-ptg, l. 360

5 7

Re: [telemetry] Team meeting agenda for tomorrow
by Joseph Davis 05 Aug '19

05 Aug '19

On 5/8/19 7:12 AM, openstack-discuss-request(a)lists.openstack.org wrote: > Hello Trinh, > Where does the meeting happen? Will it be via IRC Telemetry channel? Or, in > the Etherpad (https://etherpad.openstack.org/p/telemetry-meeting-agenda)? I > would like to discuss and understand a bit better the context behind > the Telemetry > events deprecation. Unfortunately, I have a conflict at that time and will not be able to attend. I do have a little bit of context on the Events deprecation to share. First, you will note the commit message from the commit [0] when Events were deprecated: " Deprecate event subsystem This subsystem has never been finished and is not maintained. Deprecate it for future removal. " I got the impression from jd at the time that there were a number of features in Telemetry, including Panko, that were not really "finished" and that the engineers who had worked on them had moved on to other things, so the features had become unsupported. In late 2018 there was an effort to clean up things that were not well maintained or didn't fit the direction of Telemetry. See also: https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/ Events is one feature that often gets requested, but the use cases and demand for it are not expressed strongly or well understood by most people. If the Telemetry project has demand to de-deprecate Event handling (including Panko), I'd suggest a review of the requirements for event handling and possibly choosing a champion for maintaining the Panko service. Also note: over in Monasca we have a spec [1] for handling Events ingestion which I hope we will be completing in Train. Contributions and comments welcome. :) joseph [0] https://github.com/openstack/ceilometer/commit/8a0245a5b3e1357d35ad6653be37… [1] https://github.com/openstack/monasca-specs/blob/master/specs/stein/approved… > > On Wed, May 8, 2019 at 12:19 AM Trinh Nguyen<dangtrinhnt(a)gmail.com> wrote: > >> Hi team, >> >> As planned, we will have a team meeting at 02:00 UTC, May 9th on >> #openstack-telemetry to discuss what we gonna do for the next milestone >> (Train-1) and continue what we left off from the last meeting. >> >> I put here [1] the agenda thinking that it should be fine for an hour >> meeting. If you have anything to talk about, please put it there too. >> >> [1]https://etherpad.openstack.org/p/telemetry-meeting-agenda >> >> >> Bests, >> >> -- >> *Trinh Nguyen* >> *www.edlab.xyz<https://www.edlab.xyz>* >> >>

14 25

[Airship-Seaworthy] Deployment of Airship-Seaworthy on Virtual Environment
by Anirudh Gupta 05 Aug '19

05 Aug '19

Hi Team, We want to test Production Ready Airship-Seaworthy in our virtual environment The link followed is https://airship-treasuremap.readthedocs.io/en/latest/seaworthy.html<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairship-t…> As per the document we need 6 DELL R720xd bare-metal servers: 3 control, and 3 compute nodes. But we need to deploy our setup on Virtual Environment. Does Airship-Seaworthy support Installation on Virtual Environment? We have 2 Rack Servers with Dual-CPU Intel® Xeon® E5 26xx with 16 cores each and 128 GB RAM. Is it possible that we can create Virtual Machines on them and set up the complete environment. In that case, what possible infrastructure do we require for setting up the complete setup. Looking forward for your response. Regards अनिरुद्ध गुप्ता (वरिष्ठ अभियंता) Hughes Systique Corporation D-23,24 Infocity II, Sector 33, Gurugram, Haryana 122001 DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus.

3 5

Magnum Kubernetes openstack-cloud-controller-manager unable not resolve master node by DNS
by Pawel Konczalski 31 Jul '19

31 Jul '19

Hi, i try to deploy a Kubernetes cluster with OpenStack Magnum but the openstack-cloud-controller-manager pod fails to resolve the master node hostname. Does magnum require further parameter to configure the DNS names of the master and minions? DNS resolution in the VMs works fine. Currently there is no Designate installed in the OpenStack setup. openstack coe cluster template create kubernetes-cluster-template1 \ --image Fedora-AtomicHost-29-20190429.0.x86_64 \ --external-network public \ --dns-nameserver 8.8.8.8 \ --master-flavor m1.kubernetes \ --flavor m1.kubernetes \ --coe kubernetes \ --volume-driver cinder \ --network-driver flannel \ --docker-volume-size 25 openstack coe cluster create kubernetes-cluster1 \ --cluster-template kubernetes-cluster-template1 \ --master-count 1 \ --node-count 2 \ --keypair mykey # kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE kube-system coredns-78df4bf8ff-mjp2c 0/1 Pending 0 36m <none> <none> <none> kube-system heapster-74f98f6489-tgtzl 0/1 Pending 0 36m <none> <none> <none> kube-system kube-dns-autoscaler-986c49747-wrvz4 0/1 Pending 0 36m <none> <none> <none> kube-system kubernetes-dashboard-54cb7b5997-sk5pj 0/1 Pending 0 36m <none> <none> <none> kube-system openstack-cloud-controller-manager-dgk64 0/1 CrashLoopBackOff 11 36m <none> kubernetes-cluster1-vulg5fz6hg2n-master-0 <none> # kubectl -n kube-system logs openstack-cloud-controller-manager-dgk64 Error from server: Get https://kubernetes-cluster1-vulg5fz6hg2n-master-0:10250/containerLogs/kube-…: dial tcp: lookup kubernetes-cluster1-vulg5fz6hg2n-master-0 on 8.8.8.8:53: no such host BR Pawel

2 3

[forum][sdk][nova] Closing compute API feature gaps in the openstack CLI - recap
by Matt Riedemann 29 Jul '19

29 Jul '19

I wanted to give a quick recap of this Forum session for those that couldn't attend and also find owners. Please reply to this thread if you'd like to sign up for any specific item. The etherpad [1] has the details. To restate the goal: "Identify the major issues and functional gaps (up through Mitaka 2.25) and prioritize which to work on and what the commands should do." We spent the majority of the time talking about existing issues with compute API functionality in openstack CLI, primarily boot-from-volume, live migration and lack of evacuate support (evacuate as in rebuild on a new target host because the source host is dead, not drain a host with live migrations [2]). We then talked through some of the microversion gaps and picked a few to focus on. Based on that, the agreements and priorities are: **High Priority** 1. Make the boot-from-volume experience better by: a) Support type=image for the --block-device-mapping option. b) Add a --boot-from-volume option which will translate to a root --block-device-mapping using the provided --image value (nova will create the root volume under the covers). Owner: TBD (on either) 2. Fix the "openstack server migrate" command We're going to deprecate the --live option and add a new --live-migration option and a --host option. The --host option can be used for requesting a target host for cold migration (omit the --live/--live-migration option for that). Then in a major release we'll drop the --live option and intentionally not add a --force option (since we don't want to support forcing a target host and bypassing the scheduler). Owner: TBD (I would split the 2.56 cold migration --host support from the new --live-migration option review-wise) **Medium Priority** Start modeling migration resources in the openstack CLI, specifically for microversions 2.22-2.24, but note that the GET /os-migrations API is available since 2.1 (so that's probably easiest to add first). The idea is to have a new command resource like: openstack compute migration (list|delete|set) [--server <server_id>] Owner: TBD (again this is a series of changes) **Low Priority** Closing other feature gaps can be done on an as-needed basis as we've been doing today. Sean Mooney is working on adding evacuate support, and there are patches in flight (see [3]) for other microversion-specific features. I would like to figure out how to highlight these to the OSC core team on a more regular basis, but we didn't really talk about that. I've been trying to be a type of liaison for these patches and go over them before the core team tries to review them to make sure they match the API properly and are well documented. Does the OSC core team have any suggestions on how I can better socialize what I think is ready for core team review? [1] https://etherpad.openstack.org/p/DEN-osc-compute-api-gaps [2] http://www.danplanet.com/blog/2016/03/03/evacuate-in-nova-one-command-to-co… [3] https://etherpad.openstack.org/p/compute-api-microversion-gap-in-osc -- Thanks, Matt

2 2

[dev][magnum] Magnum event notifications not having relevant content related to clusters
by Emnauel Andrecut 23 Jul '19

23 Jul '19

I'm writing about Magnum event notifications that do not contain any relevant information about the related Cluster (missing cluster id, other attributes..) As I've seen the PyCADF library receives no targetId and so the id (and mostly any other id) is randomly generated. Do you plan to add more information about the cluster on create success/pending events. Or would you accept a patch that does that?

3 2

[all] [ptls] [tc] [nova] [neutron] Volunteers that know TeX for PDF community goal
by Alexandra Settle 19 Jul '19

19 Jul '19

Hi all, The work for the Train community goal - PDF support for project docs - is well underway. [1] Now, we're looking for volunteers to help test the implementation. We'll need someone to help build the docs into PDFs and determine things we can fix through tweaks to our docs, or if they're bugs in Sphinx. AKA: We need a troubleshoot artist. If you can volunteer, please add yourself to the wiki table here [2]. I've added neutron and nova specifically here as we need someone who is familiar with the project and it's dependencies to help us get that setup. Any questions? Reach out. Cheers, Alex [1] https://review.opendev.org/#/q/topic:build-pdf-docs+(status:open+OR+status:… [2] https://wiki.openstack.org/wiki/Documentation#PDF_for_Project_Docs_-_Commun…

8 12

Barbican support for Window
by Garaga Rayudu 18 Jul '19

18 Jul '19

Hi Team, Is it supported for window OS. If Yes, please let me know more details about installation. Also let me know should I integrate with our product freely to support key management. Since it look like open source product. Thanks, Rayudu

2 1

OpenInfra Days Vietnam 2019 (Hanoi) - Call For Presentations
by Trinh Nguyen 12 Jul '19

12 Jul '19

Hello, Hope you're doing well :) The OpenInfra Days Vietnam 2019 [1] is looking for speakers in many different topics (e.g., container, CI, deployment, edge computing, etc.). If you would love to have a taste of Hanoi, the capital of Vietnam, please join us this one-day event and submit your presentation [2]. *- Date:* 24 AUGUST 2019 *- Location:* INTERCONTINENTAL HANOI LANDMARK72, HANOI, VIETNAM Especially this time, we're honored to have the Upstream Institute Training [3] hosted by the OpenStack Foundation on the next day (25 August 2019). [1] http://day.vietopeninfra.org/ [2] https://forms.gle/iiRBxxyRv1mGFbgi7 [3] https://docs.openstack.org/upstream-training/upstream-training-content.html See you in Hanoi! Bests, On behalf of the VietOpenInfra Group. -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

2 2

[tc] Assuming control of GitHub organizations
by Jim Rollenhagen 10 Jul '19

10 Jul '19

Hi, The opendev team reached out to me about handing off administrative access of the "openstack" and related organizations on GitHub. They think it would be best if the TC took control of that, or at least took control of delegating that access. In general, the goal here is to support OpenStack's presence and visibility on GitHub. Per Jim Blair: > In the long run, this shouldn't entail a lot of work, generally creating new > repos on GitHub to accept mirroring from opendev systems, performing renames, > handling transfer requests when repos move out of the openstack namespace, > setting and updating descriptions, and curating the list of pinned > repositories. In the short term, we have some archiving and moving to do.[0] Do TC members want to manage this, or should we delegate? One thing to figure out is how to grant that access. The opendev team uses a shared account with two-factor authentication provided by a shared shell account. This mitigates accidental pushes or settings changes when an admin is using their usual GitHub account. The TC (or its delegates) probably doesn't have a shared shell account to do this with. Some options: * each admin creates a second GitHub account for this purpose use a shared * account without 2FA use a shared account with 2FA, share the one time secret * with everyone to configure their own token generator use personal accounts * but be very careful Thoughts on these options? [0] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006829.html // jim

8 15

[nova][ironic] Lock-related performance issue with update_resources periodic job
by Jason Anderson 09 Jul '19

09 Jul '19

Hey OpenStackers, I work on a cloud that allows users to reserve and provision bare metal instances with Ironic. We recently performed a long-overdue upgrade of our core components, all the way from Ocata up through Rocky. During this, we noticed that instance build requests were taking 4-5x (!!) as long as before. We have two deployments, one with ~150 bare metal nodes, and another with ~300. These are each managed by one nova-compute process running the Ironic driver. After investigation, the root cause appeared to be contention between the update_resources periodic task and the instance claim step. There is one semaphore "compute_resources" that is used to control every access within the resource_tracker. In our case, what was happening was the update_resources job, which runs every minute by default, was constantly queuing up accesses to this semaphore, because each hypervisor is updated independently, in series. This meant that, for us, each Ironic node was being processed and was holding the semaphore during its update (which took about 2-5 seconds in practice.) Multiply this by 150 and our update task was running constantly. Because an instance claim also needs to access this semaphore, this led to instances getting stuck in the "Build" state, after scheduling, for tens of minutes on average. There seemed to be some probabilistic effect here, which I hypothesize is related to the locking mechanism not using a "fair" lock (first-come, first-served) by default. Our fix was to drastically increase the interval this task runs at--from every 1 minute to every 12 hours. We only provision bare metal, so my rationale was that the periodic full resource sync was less important and mostly helpful for fixing weird things where somehow Placement's state got out of sync with Nova's somehow. I'm wondering, after all this, if it makes sense to rethink this one-semaphore thing, and instead create a per-hypervisor semaphore when doing the resource syncing. I can't think of a reason why the entire set of hypervisors needs to be considered as a whole when doing this task, but I could very well be missing something. TL;DR: if you have one nova-compute process managing lots of Ironic hypervisors, consider tweaking the update_resources_interval to a higher value, especially if you're seeing instances stuck in the Build state for a while. Cheers, Jason Anderson Cloud Computing Software Developer Consortium for Advanced Science and Engineering, The University of Chicago Mathematics & Computer Science Division, Argonne National Laboratory

5 11

[keystone] Virtual Midcycle Planning
by Colleen Murphy 08 Jul '19

08 Jul '19

Hi team, As discussed in today's meeting, we will be having a virtual midcycle some time around milestone 2. We'll do two days with one three-hour session (with breaks) each day. We will do this over a video conference session, details of how to join will follow closer to the event. I've started a brainstorming etherpad: https://etherpad.openstack.org/p/keystone-train-midcycle-topics Please add discussion topics or hacking ideas to the etherpad and I will try to sort them. We need to decide on when exactly to hold the midcycle. I've created a doodle poll: https://doodle.com/poll/wr7ct4uhpw82sysg Please select times and days that you're available and then we'll try to schedule two back-to-back days (or at least two days in the same week) for the midcycle. Let me know if you have any questions or concerns. Colleen

1 1

[qa][ptg][nova][cinder][keystone][neutron][glance][swift][placement] How to make integrated-gate testing (tempest-full) more stable and fast
by Ghanshyam Mann 07 Jul '19

07 Jul '19

Current integrated-gate jobs (tempest-full) is not so stable for various bugs specially timeout. We tried to improve it via filtering the slow tests in the separate tempest-slow job but the situation has not been improved much. We talked about the Ideas to make it more stable and fast for projects especially when failure is not related to each project. We are planning to split the integrated-gate template (only tempest-full job as first step) per related services. Idea: - Run only dependent service tests on project gate. - Tempest gate will keep running all the services tests as the integrated gate at a centeralized place without any change in the current job. - Each project can run the below mentioned template. - All below template will be defined and maintained by QA team. I would like to know each 6 services which run integrated-gate jobs 1."Integrated-gate-networking" (job to run on neutron gate) Tests to run in this template: neutron APIs , nova APIs, keystone APIs ? All scenario currently running in tempest-full in the same way ( means non-slow and in serial) Improvement for neutron gate: exlcude the cinder API tests, glance API tests, swift API tests, 2."Integrated-gate-storage" (job to run on cinder gate, glance gate) Tests to run in this template: Cinder APIs , Glance APIs, Swift APIs, Nova APIs and All scenario currently running in tempest-full in the same way ( means non-slow and in serial) Improvement for cinder, glance gate: excluded the neutron APIs tests, Keystone APIs tests 3. "Integrated-gate-object-storage" (job to run on swift gate) Tests to run in this template: Cinder APIs , Glance APIs, Swift APIs and All scenario currently running in tempest-full in the same way ( means non-slow and in serial) Improvement for swift gate: excluded the neutron APIs tests, - Keystone APIs tests, - Nova APIs tests. Note: swift does not run integrated-gate as of now. 4. "Integrated-gate-compute" (job to run on Nova gate) tests to run is : Nova APIs, Cinder APIs , Glance APIs ?, neutron APIs and All scenario currently running in tempest-full in same way ( means non-slow and in serial) Improvement for Nova gate: excluded the swift APIs tests(not running in current job but in future, it might), Keystone API tests. 5. "Integrated-gate-identity" (job to run on keystone gate) Tests to run is : all as all project use keystone, we might need to run all tests as it is running in integrated-gate. But does keystone is being unsed differently by all services? if no then, is it enough to run only single service tests say Nova or neutron ? 6. "Integrated-gate-placement" (job to run on placement gate) Tests to run in this template: Nova APIs tests, Neutron APIs tests + scenario tests + any new service depends on placement APIs Improvement for placement gate: excluded the glance APIs tests, cinder APIs tests, swift APIs tests, keystone APIs tests Thoughts on this approach? The important point is we must not lose the coverage of integrated testing per project. So I would like to get each project view if we are missing any dependency (proposed tests removal) in above proposed templates. - https://etherpad.openstack.org/p/qa-train-ptg -gmann

7 11

[masakari] Run masakari-hostmonitor into Docker container
by gaetan.trellu＠incloudus.com 06 Jul '19

06 Jul '19

Hi, I'm integrating Masakari into Kolla and Kolla Ansible projects but I'm facing an issue related to masakari-hostmonitor. Based on masakari-monitors code[1], "systemctl status" command is used to check if pacemaker, pacemaker-remote and corosync are running. Having systemd running into Docker container is not the best solution. Does any of you has been able to run masakari-monitor into Docker container ? Thanks for your help. Gaëtan - [1] https://github.com/openstack/masakari-monitors/blob/26d558333d9731ca06da09b…

3 2

[placement] db query analysis
by Chris Dent 05 Jul '19

05 Jul '19

One of the queries that has come up recently with placement performance is whether there may be opportunities to gain some improvement by making fewer queries to the db. That is: * are there redundant queries * are there places where data is gathered in multiple queries that could be in one (or at least fewer) * are there queries that are not doing what we think to that end I've done some analysis of logs produced when [placement_database]/connection_debug is set to 50 (which dumps SQL queries to the INFO log). The collection of queries made during a single request to GET /allocation_candidates is at http://paste.openstack.org/show/753183/ The data set is a single resource resource provider of the same form as used in the placement-perfload job (where 1000 providers are used). Only 1 is used in this case as several queries use 'IN' statements that list all the resource provider ids currently in play, and that gets dumped to the log making it inscrutable. I've noted in the paste where this happens. Each block of SQL is associated with the method that calls it. The queries are in the order they happen. One query that happens three times (once for each resource class requested) is listed once. Observations: * The way we use IN could be improved using a bindparam: https://docs.sqlalchemy.org/en/13/core/sqlelement.html?highlight=expanding%… * That we use IN in that fashion at all, where we are carrying lists of rp ids around and making multiple queries, instead of one giant one, might be an area worth exploring. * There are a couple of places where get get a trait id (via name) in a separate query from using the trait id. * What can you see? Please have a look to see if anything looks odd, wrong, etc. Basically what we're after is trying to find things that violate our expectations. Note that this is just one of several paths through the database. When there are sharing or nested providers things change. I didn't bother to do a more complex set of queries at this time as it seemed starting simple would help us tease out how best to communicate these sorts of things. Related to that, I've started working on a nested-perfload at https://review.opendev.org/665695 -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

2 2

[kolla] Proposing yoctozepto as core
by Mark Goddard 05 Jul '19

05 Jul '19

Hi, I would like to propose adding Radosław Piliszek (yoctozepto) to kolla-core and kolla-ansible-core. While he has only recently started working upstream in the project, I feel that he has made some valuable contributions already, particularly around improving and maintaining CI. His reviews generally provide useful feedback, sometimes in advance of Zuul! Core team - please vote, and consider this my +1. I will keep this vote open for a week or until all cores have responded. Cheers, Mark

5 5

[kolla][kayobe] vote: kayobe as a kolla deliverable
by Mark Goddard 05 Jul '19

05 Jul '19

Hi, In the most recent kolla meeting [1] we discussed the possibility of kayobe becoming a deliverable of the kolla project. This follows on from discussion at the PTG and then on here [3]. The two options discussed are: 1. become a deliverable of the Kolla project 2. become an official top level OpenStack project There has been some positive feedback about option 1 and no negative feedback that I am aware of. I would therefore like to ask the kolla community to vote on whether to include kayobe as a deliverable of the kolla project. The electorate is the kolla-core and kolla-ansible core teams, excluding me. The opinion of others in the community is also welcome. If you have questions or feedback, please respond to this email. Once you have made a decision, please respond with your answer to the following question: "Should kayobe become a deliverable of the kolla project?" (yes/no) Thanks, Mark [1] http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-06-19-15.00.l… [2] https://etherpad.openstack.org/p/kolla-train-ptg [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006901.html

4 6

[tc][ops] reviving osops- repos
by Mohammed Naser 02 Jul '19

02 Jul '19

Hi everyone, So we've realized that we have a few operational tools that we keep around that live all over the place and it would probably be the best for the community to have access to them. I did a bit of research and it looks like the openstack/osops-* repos used to exist between operators to maintain things like this, however, they've gone pretty abandoned over time. I'd like to pick up this effort and as part of doing that, I'd like to move them into a GitHub organization and rename them out of x/ to make them not seem so .. 'dead'. I've requested to move x/osops-* to openstack-operators/* in GitHub so that I can setup the appropriate mirroring in post pipeline (and then propose a patch to rename things inside Gerrit as well). There was concern around the idea of using 'openstack' as it's a copyrighted term and a TC discussion was brought up as an idea which makes sense. I'd like to ask if everyone else feels like it's okay so move things into that org and host them there. Thanks, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

9 15

[cinder] Deprecating driver versions
by Erlon Cruz 01 Jul '19

01 Jul '19

Hey folks, Driver versions has being a source of a lot of confusions with costumers. Most of our drivers have a version number and history that are updated as the developers adds new fixes and features. Drivers also have a VERSION variable in the version class that should be bumped by developers. The problem with that is: - sometimes folks from the community just push patches on drivers, and its hard to bump every vendor version correctly; - that relies in the human factor to remember adding it, and usually that fails; - if we create a bugfix and bump the version, the backport to older branches will carry the version, which will not reflect the correct driver code; So, the solution I'm proposing for this is that we use the Cinder versions[1] and remove all version strings for drivers. Every new release we get a version. For stable versions, from time to time the PTL bumps the stable version and we have an accurate ways to describe the code. If we need to backport and send something to the costumer, we can do the backport, poke the PTL, and he will generate another version which can be downloaded on github or via PIP, and present the version to our costumers. So, what are your thought around this? Anyone else has had problems with that? What would be the implications of removing the driver version strings? Erlon [1] https://releases.openstack.org/teams/cinder.html [2] https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/solid…

4 5

glance image upload error [Errno 32] Broken pipe
by Satish Patel 01 Jul '19

01 Jul '19

I have installed opnestack-ansible and integrate glance with ceph storage, first day when i upload image it works but today when i am trying to upload image i am getting this error [root@ostack-infra-2-1-utility-container-c166f549 ~]# openstack image create --file cirros-0.3.4-x86_64-disk.raw --container-format bare --disk-format raw --public cirros-0.3.4-tmp Error finding address for http://172.28.8.9:9292/v2/images/8f3456aa-52fc-4b4a-8b11-dfbadb8e88ca/file: [Errno 32] Broken pipe I do have enough space on ceph storage, i am not seeing any error on glance logs also which help me. [root@ostack-infra-01-ceph-mon-container-692bea95 root]# ceph df detail GLOBAL: SIZE AVAIL RAW USED %RAW USED OBJECTS 9314G 3526G 5787G 62.14 245k POOLS: NAME ID QUOTA OBJECTS QUOTA BYTES USED %USED MAX AVAIL OBJECTS DIRTY READ WRITE RAW USED images 6 95367M N/A 22522M 3.62 584G 2839 2839 2594k 36245 67568M vms 7 N/A N/A 1912G 76.58 584G 248494 242k 6567M 3363M 5738G volumes 8 N/A N/A 0 0 584G 0 0 0 0 0 backups 9 N/A N/A 0 0 584G 0 0 0 0 0 metrics 10 N/A N/A 0 0 584G 0 0 0 0 0

2 1

[kolla] [ceph] Ceph using cluster network for replication
by vladimir franciz blando 30 Jun '19

30 Jun '19

Hi, In my globals.yml file --- cluster_interface: "bond1" --- but after deployment, there's nothing on the ceph.conf file that indicates that it's using the cluster interface --- # docker exec ceph_mon cat /etc/ceph/ceph.conf [global] log file = /var/log/kolla/ceph/$cluster-$name.log log to syslog = false err to syslog = false log to stderr = false err to stderr = false fsid = f8ff1404-bead-422b-9076-21583012ad30 mon initial members = 172.16.43.22, 172.16.43.11, 172.16.43.12 mon host = 172.16.43.22, 172.16.43.11, 172.16.43.12 mon addr = 172.16.43.22:6789, 172.16.43.11:6789, 172.16.43.12:6789 auth cluster required = cephx auth service required = cephx auth client required = cephx setuser match path = /var/lib/ceph/$type/$cluster-$id [mon] mon compact on start = true mon cluster log file = /var/log/kolla/ceph/$cluster.log --- I was expecting cluster network = <cluster_network_cidr>. I even tried making a custom config (/etc/kolla/config/ceph/ceph.conf) which includes the cluster network = <cluster_network_cidr> public network = <public_network_cidr> and run "kolla-ansible -i multinode reconfigure" but it did nothing, was there something missing on what I did? - Vlad ᐧ

2 1

[keystone] Keystone Team Update - Week of 24 June 2019
by Colleen Murphy 28 Jun '19

28 Jun '19

# Keystone Team Update - Week of 24 June 2019 ## News ### Virtual Midcycle Planning We've begun planning our midcycle[1], which we will hold virtually sometime around or after milestone 2. Please participate in suggesting topics and the scheduling poll. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007344.html ## Open Specs Train specs: https://bit.ly/2uZ2tRl Ongoing specs: https://bit.ly/2OyDLTh ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 13 changes this week, including some advances in the oslo.limit and application credential access rules implementations. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 45 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Bugs This week we opened 3 new bugs and closed none. Bugs opened (3) Bug #1834294 (keystone:Undecided) opened by Eivinas https://bugs.launchpad.net/keystone/+bug/1834294 Bug #1834304 (keystone:Undecided) opened by Rohit Londhe https://bugs.launchpad.net/keystone/+bug/1834304 Bug #1834342 (keystone:Undecided) opened by Benoît Knecht https://bugs.launchpad.net/keystone/+bug/1834342 ## Milestone Outlook https://releases.openstack.org/train/schedule.html Spec freeze is in 4 weeks. Our midcycle will be held some time around or just after that, and feature proposal freeze closely follows. ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[nova] API updates week 19-26
by Ghanshyam Mann 28 Jun '19

28 Jun '19

Hello Everyone, Please find the Nova API updates of this week. API Related BP : ============ COMPLETED: 1. Support adding description while locking an instance: - https://blueprints.launchpad.net/nova/+spec/add-locked-reason Code Ready for Review: ------------------------------ 1. Add host and hypervisor_hostname flag to create server - Topic: https://review.opendev.org/#/q/topic:bp/add-host-and-hypervisor-hostname-fl… - Weekly Progress: patch is updated with review comment. ready for re-review. I will re-review it tomorrow. 2. Specifying az when restore shelved server - Topic: https://review.opendev.org/#/q/topic:bp/support-specifying-az-when-restore-… - Weekly Progress: Review comments is fixed and ready to re-review. 3. Nova API cleanup - Topic: https://review.opendev.org/#/c/666889/ - Weekly Progress: Code is up for review. A lot of files changed but should be ok to review. I have pushed a couple of patches for missing tests of previous microversions. 4. Detach and attach boot volumes: - Topic: https://review.openstack.org/#/q/topic:bp/detach-boot-volume+(status:open+O… - Weekly Progress: No Progress Spec Ready for Review: ----------------------------- 1. Nova API policy improvement - Spec: https://review.openstack.org/#/c/547850/ - PoC: https://review.openstack.org/#/q/topic:bp/policy-default-refresh+(status:op… - Weekly Progress: Under review and updates. 2. Support for changing deleted_on_termination after boot -Spec: https://review.openstack.org/#/c/580336/ - Weekly Progress: No update this week. 3. Support delete_on_termination in volume attach api -Spec: https://review.openstack.org/#/c/612949/ - Weekly Progress: No updates this week. 4. Add API ref guideline for body text - ~8 api-ref are left to fix. Previously approved Spec needs to be re-proposed for Train: --------------------------------------------------------------------------- 1. Servers Ips non-unique network names : - https://blueprints.launchpad.net/nova/+spec/servers-ips-non-unique-network-… - https://review.openstack.org/#/q/topic:bp/servers-ips-non-unique-network-na… 2. Volume multiattach enhancements: - https://blueprints.launchpad.net/nova/+spec/volume-multiattach-enhancements - https://review.openstack.org/#/q/topic:bp/volume-multiattach-enhancements+(… Bugs: ==== No progress report in this week too. I will start the bug triage next week. NOTE- There might be some bug which is not tagged as 'api' or 'api-ref', those are not in the above list. Tag such bugs so that we can keep our eyes. -gmann

3 3

[openstack-dev] [neutron] Cancelling Neutron Drivers meeting
by Miguel Lavalle 28 Jun '19

28 Jun '19

Dear Neutrinos, Next week, the 4th of July Holiday (Independence Day) will be observed in the USA. Since several members of the drivers team live in that country and will enjoy a long weekend, we are cancelling the meeting on the 5th. We will resume our meeting normally on the 12th. Best regards Miguel

1 0

[nova] Last chance to weigh in on nova adopting the SDK for inter-service comms
by Matt Riedemann 28 Jun '19

28 Jun '19

John and I are +2 and Eric is +1 (would be +2 if not involved in the effort) on the nova spec [1] to adopt the openstacksdk for nova communicating with other services rather than the python-*clients. There are several motivations within the spec itself and in review comments for doing this, but since it's a major change to nova across the board I want to make sure other cores and contributors are aware and at least not against this before it's approved. I think a reasonable timeline for waiting to approve is until EOD July 2 (next week) since that's our third and final spec review sprint. [1] https://review.opendev.org/#/c/662881/ -- Thanks, Matt

1 0

[infra] Stepping down as infra-root
by Paul Belanger 28 Jun '19

28 Jun '19

Greetings! 10 months ago my role at Red Hat changed, which saw my day to day job move away from OpenStack, specifically the Infrastructure team. Since then, I've be contributing to OpenStack in my spare time however it isn't going as well I as would like. Unfortunately, there is never enough time in the day to do everything and as a result I believe it is best for me to step day, and remove myself as infra-root: https://review.opendev.org/668192 In the 7 years I've been contributing to the infra team, I've learned a great many things (and still learning). Thank you to all the awesome humans, past and present, for making my day to day much better. To the community, I do ask if you are at all interested in how the infrastructure works / operates, reach out to the #openstack-infra channel and get involved. I refer you to the help wanted list: https://governance.openstack.org/tc/reference/help-most-needed.html#communi… Paul

2 1

https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/ds_util.py
by Keith Berger 28 Jun '19

28 Jun '19

Gary, I hope you are doing well. I was trying to understand the logic in this code when using datastore_regex. It seems even if you set that parameter and it matches some N datastores, the "free capacity " being returned by nova-compute is only from a single/best datastore. def get_available_datastores(session, cluster=None, datastore_regex=None): """Get the datastore list and choose the first local storage.""" ds = session._call_method(vutil, "get_object_property", cluster, "datastore") ... ... if _is_datastore_valid(propdict, datastore_regex, allowed_ds_types): new_ds = ds_obj.Datastore( ref=obj_content.obj, name=propdict['summary.name'], capacity=propdict['summary.capacity'], freespace=propdict['summary.freeSpace']) # favor datastores with more free space if (best_match is None or new_ds.freespace > best_match.freespace): best_match = new_ds return best_match The issue we are seeing is a customer has for example 5 x 10 GB data stores (DS1,..DS5.) For whatever reason , DS1 was filled more fully and once it hit close to 100% no new instances could launch as the were blocked by the Nova DiskFilter. I know we can use the disk_allocation to "fool" this but why is not using to total cumulative storage as opposed to the one with most free or first one. I am wondering if there is some configuration issue. If it is supposed to use best_match, should that dynamically change among the matched regex Datastore to the one with most free space? Could it be it is somehow skipping the "best_match" and always using " first local storage" I appreciate any suggestions or comments you have. Keith -- Keith Berger Master Software Engineer SUSE (P)+1 470.237.2012 (M)+1 404.664.9610 keith.berger(a)suse.com

1 0

[kolla] How can I change package version when building the image?
by Eddie Yen 28 Jun '19

28 Jun '19

Hi, I'm using stable/rocky from git now, but I want to build the prometheus images with latest version since there're few exporters that too previous that can't use most Grafana dashboards in default. I already know I have to edit templates in kolla-ansible since the newer exporters has using the different launch commands. But Idk how to set the version because the document didn't say how to override the Dockerfile. Is that OK to change package version inside Dockerfile? Many thanks, Eddie.

3 4

performance issue with opendev gitea interface.
by Sean Mooney 27 Jun '19

27 Jun '19

i have started this as a separate thread form the Github organization management but it has been in the back of my mind as we are considering not syncing to gitub going forward. for larger project like nova https://opendev.org/openstack/nova preforms quite poorly in both firefox and chome i am seeing ~ 15second before the first response form opendev.org for nova. os-vif or other smaller projects seem to respond ok but for nova it makes navigating the code or liking code to other via opendev quite hard. i brought this up on the infra irc a few weeks ago and asked if gitea had any kind of caching and while the initial response was "we do not believe so". before archiving or stopping syncing to github i was wondering if we could explore options to improve the performace. if opendev.org is not currently fronted by a cdn perhaps that would help. Similarly looking at https://docs.gitea.io/en-us/config-cheat-sheet/ it may be possible to either change the cache or database parameters to improve performance. I really don't know how gitea has been deployed but at present the web interface is not usable in nova in a responsive manner so i have continued to use github when linking code to others but it would be nice to be able to use opendev instead. regards sean.

4 3

[sig][api] API-SIG office hours cancelled for 4 July
by Michael McCune 27 Jun '19

27 Jun '19

due the holiday in the united states and travel by our other members, the api-sig will not hold office hours next thursday, 4 july. thank you =) peace o/

1 0

[security sig] Weekly Newsletter June 27th & July 4th Meeting Canceled
by Gage Hugo 27 Jun '19

27 Jun '19

Just a heads up, but since next Thursday is the 4th of July, we will be canceling next week's Security SIG meeting. Have a safe holiday everyone! #Week of: 27 June 2019 - Security SIG Meeting Info: http://eavesdrop.openstack.org/#Security_SIG_meeting - Weekly on Thursday at 1500 UTC in #openstack-meeting - Agenda: https://etherpad.openstack.org/p/security-agenda - https://security.openstack.org/ - https://wiki.openstack.org/wiki/Security-SIG #Meeting Notes - Summary: http://eavesdrop.openstack.org/meetings/security/2019/security.2019-06-27-1… - Slow week, discussed the TC approving the Image Encryption pop-up team & TPM passthrough - Next week's meeting will be canceled due to July 4th 🎆 ## News - The TC approved adding "Image Encryption" as a pop-up team - https://review.opendev.org/#/c/661983/ - https://governance.openstack.org/tc/reference/popup-teams.html#image-encryp… - Physical TPM passthrough PoC - https://review.opendev.org/#/c/667926/ - spec - https://review.opendev.org/#/c/667928/ # VMT Reports - A full list of publicly marked security issues can be found here: https://bugs.launchpad.net/ossa/ - No new public security bugs this week

1 0

[sahara] Cancelling Sahara meeting July 4
by Jeremy Freudberg 27 Jun '19

27 Jun '19

Hi all, There will be no Sahara meeting this upcoming Thursday, July 4. Holler if you need anything. Thanks, Jeremy

1 0

Re: [nova] Questions on Force refresh instance info_cache during heal
by Matt Riedemann 27 Jun '19

27 Jun '19

On 6/26/2019 4:39 AM, 胡丽娜 wrote: > Dear Matt Riedemann : > I am quiet interested in your recently published > patch https://review.opendev.org/#/c/591607/ , there is a question i > met ，If we spawn VM with port id and then update db , set > network_info='[]' , Afert periodic task_heal_instance_info_cache , > Preserve On Delete turn into False , in face ,it's corret value is True . > Steps to reproduce: > 1.Spawn vm with port id : nova boot --flavor xx --nic port-id --image xx > vm_name > 2.View the value of Preserve On Delete , it'value is True > 3.Update the DB row ,drop interface_list > 4.View the value of Preserve On Delete , it turns into False > Have you ever had this question ? Look forward to your reply ! > Best wishes ! > > 胡丽娜 BC-EC > 中国移动苏州研发中心云计算产品部 > 中移（苏州）软件技术有限公司 > 苏州高新区科技城昆仑山路58号中移软件园 +openstack-discuss mailing list The problem in this case is that nova relies on the network info cache to determine which ports were created by nova and which were pre-existing and provided by the user, either during server create or attached to the server after it was created. When building the network info model we determine the pre-existing port IDs here [1] and then pass that to _build_vif_model here [2]. We then determine the preserve_on_delete value if the port ID is in that pre-existing ports set [3]. What we should probably be doing is if we have lost the cache is always set preserve_on_delete=True since we don't have the cache information to know if we should delete the port when the server is deleted, and it could be an SR-IOV port or something that the user does not want deleted. The worst case in this scenario is the server is deleted and some ports are unbound and left hanging around that the user has to manually cleanup but that's arguably better than deleting something they didn't want to be deleted. I used some similar logic in this related patch [4]. If people agree with this (being conservative when we've lost the cache), then please report a bug and we can make the change (it should be pretty simple). [1] https://review.opendev.org/#/c/591607/25/nova/network/neutronv2/api.py@2826 [2] https://review.opendev.org/#/c/591607/25/nova/network/neutronv2/api.py@2906 [3] https://review.opendev.org/#/c/591607/25/nova/network/neutronv2/api.py@2764 [4] https://review.opendev.org/#/c/640516/ -- Thanks, Matt

1 1

[nova] TPM passthrough
by Jim Rollenhagen 27 Jun '19

27 Jun '19

Hey y'all, We have an internal use case which requires a VM with a TPM, to be used to store a private key. Libvirt has two ways to present a TPM to a VM: passthrough or emulated. Per kashyap and the #qemu IRC channel, libvirt stores the TPM's state on disk, unencrypted. Our risk profile includes "someone walks away with a disk", so this won't work for our use case. The QEMU devs have asked for RFEs to implement vTPMs where the state never touches the disk, so I have hopes that this will be done eventually. However, I suspect that this will still take some time, especially as nobody has volunteered to actually do the work yet. So, I'd like to propose we implement TPM passthrough in Nova. My team is happy to do the work, but I'd love some guidance as to the best way to implement this so we can get a spec done (I assume it's "just another resource class"?). If Nova doesn't want this feature in, and would rather just wait for the features in QEMU, we'll carry it downstream, I guess. :) Thoughts? // jim

5 9

[dev][requirements] Upcoming changes to constraints handling in tox.ini
by Tony Breeds 27 Jun '19

27 Jun '19

Hi folks, This is a heads-up to describe 3 sets of changes you'll start seeing starting next week. 1) lower-constraints.txt handling TL;DR: Make sure projects do not specify a constraint file in install_command 2) Switch to the new canonical constraints URL on master TR;DR: Make sure you use https://releases.openstack.org/constraints/upper/master 3) Switch to the new canonical constraints URL on stable branches TR;DR: Make sure you use https://releases.openstack.org/constraints/upper/$series These will be generated from a member of the requirements team[1], and will be on the gerrit topic constraints-updates. We'll start next week to give y'all a few days to digest this email Now for slightly more details: 1) lower-constraints.txt handling If a projects has an install_command that includes a constraint file *and* the also have a lower-constraints test env what may happen is something like: given a tox.ini that contains: --- [testenv] install_command = pip install -U -c upper-constraints.txt {opts} {deps} deps = -r requirements.txt -r test-requirements.txt [testenv:lower-constraints] deps = -c lower-constrtaints.txt -r requirements.txt -r test-requirements.txt --- pyXX: virtualenv -p $python_version .tox/$testenv_name .tox/$testenv_name/bin/pip install -U -c upper-constratints.txt -r requirements.txt -rtest-requirements.txt .tox/$testenv_name/bin/pip install . lower-constraints: virtualenv -p $python_version .tox/$testenv_name .tox/$testenv_name/bin/pip install -U -c upper-constratints.txt -c lower-constrtaints.txt -r requirements.txt -rtest-requirements.txt .tox/$testenv_name/bin/pip install . pip will constrain a library to the first item it encounters, which pretty much means that the lower-constraints testenv is still testing against upper-constraints.txt[3] The fix is to move all constraints into deps rather than install_command. This impacts ~40 projects and will be fixed by hand, rather than a tool so we can preserve each projects style for tox.ini. There's a pretty good chance that this change will break the lower-constraints testenv, project teams are encouraged to help identify the incorrect values in lower-constraints, the requirements team will pick the current value in upper-constraints which almost certainly raises a value well past the minimum. https://review.opendev.org/#/c/601188/ is the change we did for keystone way back when Some projects have upper-constraints.txt in install_command but explicitly override it in lower-constratints.txt this works and wont be changed at this point but ideally all projects will use the same format to reduce the impact in switching between projects. 2) Switch to the new canonical constraints URL on master Way back at the first Denver PTG thr requirements, release and infrastructure teams discussed avoiding using gitweb urls for constraints urls. This was discussed[4] in more detail after the liberty EOL broke consumers still wanting to use tox as: https://git.openstack.org/cgit/openstack/requirements/plain/upper-constrain… 'vanished' The idea was re-raised[5] in Feb. this year where we devised a plan to create URLs that abstract the git hosting and allow us to switch between branches and tags at the right times in the OpenStack life-cycle. Other, more dynamic, ideas were discussed but over complicate things and make OpenStack slightly harder to understand or create more work. for example we could: a. Add an install_script that looks up $metadata and grab the right constraints We're trying avoid scripts like this as it very rapidly gets us to the point where we're doing "managed copy-and-paste" between projects *or* projects install scripts diverge b. Add to tox itself the ability to derive the URL from $metadata This is doable but frankly requires more time than we have right now. So what we have is: redirect 301 /constraints/upper/master http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/train http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/stein http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/rocky http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/queens http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/pike http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/ocata http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/newton http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/mitaka http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/liberty http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/kilo http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/juno http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… [Yes I know we need to use opendev that's tomorrows problem] so that means that: http{s,}://releases.openstack.org/constraints/upper/master http{s,}://releases.openstack.org/constraints/upper/$series will always do the right thing based on the state of each OpenStack release. So we're going to generate changes that point to the right place. For master we have 2 options: a. Use .../train If we use train in the constraints URLs now things will work and when we branch for 'U' we'd need to update master to point to the right series name. This is very doable. The release team already generates a change on master at this point in time to update reno. We also need to update the new stable branch so that .gitreview points to the new branch. b. Use .../master If we use 'master' in the constratints URLs master will always work and when we branch for 'U' we'd need to update the stable/train branch to point to train. The release team already does this today along with the .gitreview change. So the bottom line is that using master (over train) is *slighty* less work for the release team and *slightly* less work for each project team. as such that's what we're going to do. There are several changes in the works that set the constraints url to either https://opendev.org/openstack/requirements/raw/branch/master/upper-constrai… or https://releases.openstack.org/constraints/upper/train These will work and will not break anything when the project branches, however for the sake of OpenStack wide consistency, and avoiding merge conflicts project teams are discouraged from merging them, or if the change also fixes other references to git.openstack.org just have the tox.ini updates removed. Once tox.ini is done the requirements team will look for additional places where the 'old style' urls are used and follow-up but right now there's a lot of noise to sift through 3) Switch to the new canonical constraints URL on stable branches Hopefully with all the background from 2 above this should be pretty self-explanatory. This is however much lower priority and probably wont start until the bulk of 1 and 2 is complete Yours Tony. [1] https://review.opendev.org/#/admin/groups/131,members [2] [2] Probably me actually ;P [3] There is some nuance here but this is the general rule [4] In the thread that starts with: http://lists.openstack.org/pipermail/openstack-dev/2017-September/122333.ht… [5] In the thread that starts with: http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002682…

6 7

Backup solutions
by Alfredo De Luca 27 Jun '19

27 Jun '19

Hi all. Just a quick one. Other than freezer openstack project as backup solution, are there any other opensource/commercial project/software/solutions for that? Cheers -- *Alfredo*

3 3

[glance] Openstack Glance Image custom properties issue
by Hemant Sonawane 27 Jun '19

27 Jun '19

Hello, My self Hemant Sonawane. Recently I am working on openstack and I am getting some strange error when I installed openstack dashboard and tried to edit the visibility of images and update their metadata it dosent work I get *Forbidden : Redirecting to login *and *Unable to edit the image custom properties *such errors. I also tried to see the glance-api logs and found this *"2019-06-26 13:11:51,705.705 1 WARNING glance.api.v2.images [-] Could not find schema properties file schema-image.json. Continuing without custom properties"* Is there any glance policy in horizon? that might cause this error I think. The detail log file attached for your ready reference. So please look into my concern and let me know if there is any solution to resolve my issue. Thanks and Regards, Hemant Sonawane

1 0

Reply： [nova]Bug #1829696: qemu-kvm process takes 100% CPU usage whenrunning redhat/centos 7.6 guest OS
by 钱熙 26 Jun '19

26 Jun '19

Thanks very much. I will upgrade the virt tools software on the host and check if it still happens. Later, I hope I could update you guys. Thanks again. Best regards QianXi------------------ 原始邮件 ------------------ 发件人: "Kashyap Chamarthy"<kchamart(a)redhat.com> 发送时间: 2019年6月21日(星期五) 凌晨0:32 收件人: "钱熙"<qianxi416(a)foxmail.com>; 抄送: "openstack-discuss"<openstack-discuss(a)lists.openstack.org>; 主题: Re: [nova]Bug #1829696: qemu-kvm process takes 100% CPU usage whenrunning redhat/centos 7.6 guest OS On Thu, Jun 20, 2019 at 10:06:11PM +0800, 钱熙 wrote: > Hi there, Hi, > > I am struggling with qemu-kvm 100% CPU usage problem. > When running redhat or centos 7.6 guest os on vm, > the cpu usage is very low on vm(almost 100% idle when no tasks run), but on the host, > qemu-kvm reports 100% cpu busy usage. > > > https://bugs.launchpad.net/nova/+bug/1829696 > I opened the bug above, however it did not find some interest. > > > After searching some related bugs report, > I suspect that it is due to the clock settings in vm's domain xml. > My settings are as follows: > <clock offset='utc'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='no'/> > </clock> > And details about the bug, please see https://bugs.launchpad.net/nova/+bug/1829696 I don't think it is related to clock settings at all. > It shows that only the version 7.6 of redhat or centos affected by this bug behavior. > In my cluster, it is OK for versions from redhat or centos 6.8 to 7.5. > > > Any clue or suggestion? Please see DanPB's response in comment#2; I agree with it. (Note that I've changed the bug component from 'openstack-nova' --> 'qemu') https://bugs.launchpad.net/qemu/+bug/1829696/comments/2 -- /kashyap

2 1

[goal][python3] Train unit tests weekly update (goal-11)
by Corey Bryant 26 Jun '19

26 Jun '19

This is the goal-11 weekly update for the "Update Python 3 test runtimes for Train" goal [1]. There are 11 weeks remaining for completion of Train community goals [2]. == What's the Goal? == To ensure (in the Train cycle) that all official OpenStack repositories with Python 3 unit tests are exclusively using the 'openstack-python3-train-jobs' Zuul template or one of its variants (e.g. 'openstack-python3-train-jobs-neutron') to run unit tests, and that tests are passing. This will ensure that all official projects are running py36 and py37 unit tests in Train. For complete details please see [1]. == Ongoing Work == Open patches needing reviews: https://review.openstack.org/#/q/topic:python3-train+is:open Failing patches: https://review.openstack.org/#/q/topic:python3-train+status:open+(+label:Ve… Patch automation scripts needing review: https://review.opendev.org/#/c/666934 Some notes on 2 issues I came across this week: 1) I've updated the patch automation scripts to drop py34,py35,py36 and add py37 to default tox targets as discussed with smcginnis, fungi, and gmann in #openstack-tc this week. 2) While trying to proposed and urge the move to py37-only default tox targets, some projects have decided to keep both py36 and py37 targets and some projects have decided to run with the generic py3-only target. == Completed Work == Merged patches: https://review.openstack.org/#/q/topic:python3-train+is:merged == How can you help? == Please take a look at the failing patches and help fix any failing unit tests for your project(s). Python 3.7 unit tests will be self-testing in Zuul. If you're interested in helping submit patches, please let me know. == Reference Material == [1] Goal description: https://governance.openstack.org/tc/goals /train/python3-updates.html [2] Train release schedule: https://releases.openstack.org/train/schedule.html (see R-5 for "Train Community Goals Completed") Storyboard: https://storyboard.openstack.org/#!/story/2005924 Porting to Python 3.7: https://docs.python.org/3/whatsnew/3.7.html#porting-to-python-3-7 Python Update Process: https://opendev.org/openstack/governance/src/branch/master/resolutions/2018… Train runtimes: https://opendev.org/openstack/governance/src/branch/master/reference/runtim… Thanks, Corey

1 0

[stable][sahara] Changes to the stable team
by Tony Breeds 26 Jun '19

26 Jun '19

Hello all, In the middle of last week Telles asked[1] for a couple of the current Sahara core team (Jeremy and Luigi) to be added to the stable group. Upon review there isn't enough data (either reviews or backports) to verify that Jeremy and Luigi understand the stable policy. So I'd like to try an experiment and I've added both to the stable core team but I ask neither +W a backport and ping me (or another stable team member) for the final approval. I'd like to suggest dropping: * Elise Gafford egafford(a)redhat.com * Sergey Lukjanov slukjanov(a)mirantis.com * Sergey Reshetnyak sreshetniak(a)mirantis.com * Vitaly Gridnev gridnevvvit(a)gmail.com from the sahara-stable-maint group given[2] is empty. Yours Tony. [1] http://eavesdrop.openstack.org/irclogs/%23openstack-stable/%23openstack-sta… [2] https://review.opendev.org/#/q/(project:openstack/python-saharaclient+OR+pr…

2 2

[nova] Spec Review Day: Tuesday July 2nd
by Eric Fried 26 Jun '19

26 Jun '19

Esteemed Nova contributors and maintainers- With about a month until spec freeze [1], we need to fish or cut bait [2] on open specs. To that end, I have recycled the Spec Review Day etherpad [3] and populated it with all open Train specs (those with a file under specs/train/approved) [4]. I seeded their initial status in the etherpad, including noting from whom the next action is expected. Authors: Look for your specs, especially if they say "Action: author", and update them ASAP. (Having done so, you can toggle to "Action: reviewer".) Reviewers and Cores: Scan the list for Action: "reviewer" and/or "approver" and dive in. (You may wish to start by intersecting with [5], which narrows [4] down to specs you've already reviewed at some point.) You do not need to wait until, or limit your activity to, Tuesday. Especially if you're going to be unavailable then. Thanks, efried [1] https://wiki.openstack.org/wiki/Nova/Train_Release_Schedule [2] https://en.wiktionary.org/wiki/fish_or_cut_bait [3] https://etherpad.openstack.org/p/nova-spec-review-day [4] https://review.opendev.org/#/q/project:openstack/nova-specs+status:open+pat… [5] https://review.opendev.org/#/q/project:openstack/nova-specs+status:open+pat…

1 1

[stackalytics] Reported numbers seem inaccurate
by Mark Goddard 26 Jun '19

26 Jun '19

Hi, I've heard a couple of people say they don't feel the numbers reported by Stackalytics accurately reflect reality. I've been trying to gather a few stats for my Kolla update session in Denver, and am finding the same. I'll try to give some concrete examples. Reviews for all kolla deliverables in Stein [1]. Here the company stats don't reflect the individual stats. Also, the total reviews in the 'Kolla Official' module does not equal the sum of the reviews of its submodules (kolla, kolla-ansible, kolla-cli). If I look at the contribution summary for Kolla Official in the last 90 days [2], they are actually greater than those for the last 180 days [3]! There are also similar issues with commit metrics, and none seem to match what I see in git. Thanks, Mark [1] https://www.stackalytics.com/?metric=marks&module=kolla-group [2] https://www.stackalytics.com/report/contribution/kolla-group/90 [3] https://www.stackalytics.com/report/contribution/kolla-group/180

10 11

[nova] FYI for out-of-tree virt drivers: deprecating non-update-provider-tree compat
by Matt Riedemann 26 Jun '19

26 Jun '19

I have a change up [1] to deprecate the backward compatibility code in the ResourceTracker for virt drivers that do not implement the update_provider_tree interface. Starting in Train you'll get a warning in the nova-compute logs every time the update_available_resource periodic runs (we could only log on startup, that's up for debate in the change) until the driver is updated and in the U release we'll drop that compatibility code. The only out-of-tree virt driver I checked was nova-lxd and it does *not* implement update_provider_tree so for those that care about that driver you should implement the interface. The patch has a link to the reference docs on implementing that interface for your driver (it's pretty simple). [1] https://review.opendev.org/#/c/667442/ -- Thanks, Matt

1 0

Shanghai CFP CLOSING Soon!
by Jimmy McArthur 26 Jun '19

26 Jun '19

Sorry everyone. I completely messed up the subject. So for real... it's already here, but it won't be for long! Details below. The July 2 deadline to submit a presentation [1] for the Open Infrastructure Summit [2] in Shanghai is in less than one week! Submit your session today and join the global community in Shanghai, November 4-6, 2019. Sessions will be presented in both Mandarin and English, so you may submit your presentation in either language. Submit your presentations, panels, and hands-on workshops [3] before July 2 at 11:59 pm PT (July 3, 2019 at 15:00 China Standard Time). Tracks [4]: Container Infrastructure Hands-on Workshops AI, Machine Learning & HPC Private & Hybrid Cloud Public Cloud 5G, NFV & Edge Open Development Getting Started CI/CD Security Upcoming Shanghai Summit Deadlines * Register now [5] before the early bird registration deadline in early August (USD or RMB options available) * Apply for Travel Support [6] before August 8. For more information on the Travel Support Program, go here [7]. * Interested in sponsoring the Summit? [8]. * The content submission process for the Forum and Project Teams Gathering will be managed separately in the upcoming months. We look forward to your submissions! Cheers, Jimmy [1] https://cfp.openstack.org/ [2] https://www.openstack.org/summit/shanghai-2019/ [3] https://cfp.openstack.org/ [4] https://www.openstack.org/summit/shanghai-2019/summit-categories/ [5] https://www.openstack.org/summit/shanghai-2019/ [6] https://openstackfoundation.formstack.com/forms/travelsupportshanghai [7] https://www.openstack.org/summit/shanghai-2019/travel/ [8] https://www.openstack.org/summit/shanghai-2019/sponsors/ > >

1 0

Shanghai CFP Coming Soon!
by Jimmy McArthur 26 Jun '19

26 Jun '19

Hi Everyone! The July 2 deadline to submit a presentation [1] for the Open Infrastructure Summit [2] in Shanghai is in less than one week! Submit your session today and join the global community in Shanghai, November 4-6, 2019. Sessions will be presented in both Mandarin and English, so you may submit your presentation in either language. Submit your presentations, panels, and hands-on workshops [3] before July 2 at 11:59 pm PT (July 3, 2019 at 15:00 China Standard Time). Tracks [4]: Container Infrastructure Hands-on Workshops AI, Machine Learning & HPC Private & Hybrid Cloud Public Cloud 5G, NFV & Edge Open Development Getting Started CI/CD Security Upcoming Shanghai Summit Deadlines * Register now [5] before the early bird registration deadline in early August (USD or RMB options available) * Apply for Travel Support [6] before August 8. For more information on the Travel Support Program, go here [7]. * Interested in sponsoring the Summit? [8]. * The content submission process for the Forum and Project Teams Gathering will be managed separately in the upcoming months. We look forward to your submissions! Cheers, Jimmy [1] https://cfp.openstack.org/ [2] https://www.openstack.org/summit/shanghai-2019/ [3] https://cfp.openstack.org/ [4] https://www.openstack.org/summit/shanghai-2019/summit-categories/ [5] https://www.openstack.org/summit/shanghai-2019/ [6] https://openstackfoundation.formstack.com/forms/travelsupportshanghai [7] https://www.openstack.org/summit/shanghai-2019/travel/ [8] https://www.openstack.org/summit/shanghai-2019/sponsors/

3 2

[magnum] weekly meetings
by Bharat Kunwar 26 Jun '19

26 Jun '19

Hello all I was wondering if there was a reason there hasn’t been a regular weekly meeting recently. The last one I traced back in the IRC log was back in 4 June. It would be good if there was an up to date schedule somewhere with definite dates where there is a chair available. Best Bharat

2 2

[nova] Strict isolation of group of hosts for image
by Kamde, Vrushali 26 Jun '19

26 Jun '19

Hi, Working on implementation of 'Support filtering of allocation_candidates by forbidden aggregates' spec. Here we are trying to Test granular resource request. Facing issue in configuring inventories for 'CUSTOM_RESOURCE_CLASS' and resulting into 'No Valid host found' whereas expecting host from placement. So kindly help me here how one can set inventory for 'CUSTOM_RESOURCE_CLASS' because when the inventory is added for the custom resource class 'CUSTOM_RESOURCE_CLASS', it's getting removed when the compute service periodic task updates the inventory. FYI, here are the steps followed: 1. Data configured at nova: * Create three aggregates 'agg1', 'agg2', 'agg3' by using: 'POST'-- /os-aggregates(Create aggregate) * Setting metadata on aggregates: i. Setting metadata (trait:HW_CPU_X86_SGX) on the aggregate agg1 by using: 'POST'-- /os-aggregates/{aggregate_id}/action(set metadata) ii. Setting metadata (trait:STORAGE_DISK_SSD) on the aggregate agg2 by using: 'POST'-- /os-aggregates/{aggregate_id}/action(set metadata) iii. Setting metadata (trait:CUSTOM_MAGIC, trait:HW_CPU_X86_MMX) on the aggregate agg3 by using: 'POST'-- /os-aggregates/{aggregate_id}/action(set metadata) * Associate aggregates 'agg3' to host say 'RP1' by using: 'POST'-- /os-aggregates/{aggregate_id}/action(Add host) * Setting Extra-specs on Flavor(trait:HW_CPU_X86_MMX, trait1:CUSTOM_MAGIC, resources1:CUSTOM_RESOURCE_CLASS): 'POST'-- /flavors/{flavor_id}/os-extra_specs 1. Data configured at placement: * Setting traits (CUSTOM_MAGIC, HW_CPU_X86_MMX) on host 'RP1'. 'PUT'-- /resource_providers/{uuid}/traits * Create a new Resource class 'CUSTOM_RESOURCE_CLASS': 'POST'-- /resource_classes 1. Assigned inventories to the new resource class: openstack resource provider inventory set --resource CUSTOM_RESOURCE_CLASS:total=78 <RP1_uuid> 'PUT'-- /resource_providers/{uuid}/inventories/{resource_class} 1. When we boot the instance : * The final url of allocation_candidates which we get is as follows: '/allocation_candidates?limit=1000&member_of=%21in%3A442ca580-fd07-433c-9b67-c18fdeccbca1%2C61b8bb02-01e0-4e28-af4c-0ea4c828539a &required=HW_CPU_X86_MMX&required1=CUSTOM_MAGIC&resources=DISK_GB%3A1%2CMEMORY_MB%3A512%2CVCPU%3A1&resources1=CUSTOM_RESOURCE_CLASS%3A1' * Observed after the inventory is added for the custom resource class 'CUSTOM_RESOURCE_CLASS', it's getting removed when the compute service periodic task updates the inventory. Therefore the instance is not getting booted on 'RP1'. The error faced which we get is 'No Valid host found'. Regards, Vrushali Kamde. Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.

2 1

Best practices to restart / repair broken Octavia LoadBalancer
by Pawel Konczalski 26 Jun '19

26 Jun '19

2 1

Re: [Nova][Ironic] Reset Configurations in Baremetals Post Provisioning
by Alex Xu 26 Jun '19

26 Jun '19

Mark Goddard <mark(a)stackhpc.com> 于2019年6月12日周三下午2:45写道： > > > On Wed, 12 Jun 2019, 06:23 Alex Xu, <soulxu(a)gmail.com> wrote: > >> >> >> Mark Goddard <mark(a)stackhpc.com> 于2019年6月12日周三上午1:39写道： >> >>> On Mon, 10 Jun 2019 at 06:18, Alex Xu <soulxu(a)gmail.com> wrote: >>> > >>> > >>> > >>> > Eric Fried <openstack(a)fried.cc> 于2019年6月7日周五上午1:59写道： >>> >> >>> >> > Looking at the specs, it seems it's mostly talking about changing >>> VMs resources without rebooting. However that's not the actual intent of >>> the Ironic use case I explained in the email. >>> >> > Yes, it requires a reboot to reflect the BIOS changes. This reboot >>> can be either be done by Nova IronicDriver or Ironic deploy step can also >>> do it. >>> >> > So I am not sure if the spec actually satisfies the use case. >>> >> > I hope to get more response from the team to get more clarity. >>> >> >>> >> Waitwait. The VM needs to be rebooted for the BIOS change to take >>> >> effect? So (non-live) resize would actually satisfy your use case just >>> >> fine. But the problem is that the ironic driver doesn't support resize >>> >> at all? >>> >> >>> >> Without digging too hard, that seems like it would be a fairly >>> >> straightforward thing to add. It would be limited to only "same host" >>> >> and initially you could only change this one attribute (anything else >>> >> would have to fail). >>> >> >>> >> Nova people, thoughts? >>> >> >>> > >>> > Contribute another idea. >>> > >>> > So just as Jay said in this thread. Those CUSTOM_HYPERTHREADING_ON and >>> CUSTOM_HYPERTHREADING_OFF are configuration. Those >>> > configuration isn't used for scheduling. Actually, Traits is designed >>> for scheduling. >>> > >>> > So yes, there should be only one trait. CUSTOM_HYPERTHREADING, this >>> trait is used for indicating the host support HT. About whether enable it >>> in the instance is configuration info. >>> > >>> > That is also pain for change the configuration in the flavor. The >>> flavor is the spec of instance's virtual resource, not the configuration. >>> > >>> > So another way is we should store the configuration into another >>> place. Like the server's metadata. >>> > >>> > So for the HT case. We only fill the CUSTOM_HYPERTHREADING trait in >>> the flavor, and fill a server metadata 'hyperthreading_config=on' in server >>> metadata. The nova will find out a BM node support HT. And ironic based on >>> the server metadata 'hyperthreading_config=on' to enable the HT. >>> > >>> > When change the configuration of HT to off, the user can update the >>> server's metadata. Currently, the nova will send a rpc call to the compute >>> node and calling a virt driver interface when the server metadata is >>> updated. In the ironic virt driver, it can trigger a hyper-threading >>> configuration deploy step to turn the HT off, and do a reboot of the >>> instance. (The reboot is a step inside deploy-step, not part of ironic virt >>> driver flow) >>> > >>> > But yes, this changes some design to the original deploy-steps and >>> deploy-templates. And we fill something into the server's metadata which >>> I'm not sure nova people like it. >>> > >>> > Anyway, just put my idea at here. >>> >>> We did consider using metadata. The problem is that it is >>> user-defined, so there is no way for an operator to restrict what can >>> be done by a user. Flavors are operator-defined and so allow for >>> selection from a 'menu' of types and configurations. >>> >> >> The end user can change the BIOS config by the ipmi inside the guest OS, >> and do a reboot. It is already out of control for the operator. >> (Correct me if ironic doesn't allow the end user change the config inside >> the guest OS) >> > > It depends. Normally you can't configure BIOS via IPMI, but need to use a > vendor interface such as racadm or on hardware that supports it, Redfish. > Access to the management controller can and should be locked down though. > It's also usually possible to reconfigure via serial console, if this is > exposed to users. > It sounds that breaking the operator control partially. (Sorry for drop the mallist thread again...I will paste a note to the wall "click the "Reply All"...") > >> So Flavor should be thing to strict the resource( or resource's capable) >> which can be requested by the end user. For example, flavor will say I need >> a BM node has hyper-thread capable. But enable or disable can be controlled >> by the end user. >> >> >>> >>> What might be nice is if we could use a flavor extra spec like this: >>> >>> deploy-config:hyperthreading=enabled >>> >>> The nova ironic virt driver could pass this to ironic, like it does with >>> traits. >>> >>> Then in the ironic deploy template, have fields like this: >>> >>> name: Hyperthreading enabled >>> config-type: hyperthreading >>> config-value: enabled >>> steps: <deploy steps> >>> >>> Ironic would then match on the config-type and config-value to find a >>> suitable deploy template. >>> >>> As an extension, the deploy template could define a trait (or list of >>> traits) that must be supported by a node in order for the template to >>> be applied. Perhaps this would even be a standard relationship between >>> config-type and traits? >>> >>> Haven't thought this through completely, I'm sure it has holes. >>> >>> > >>> >> efried >>> >> . >>> >> >>> >>

4 10

RE: Openstack Glance Image custom properties issue
by Hemant Sonawane 26 Jun '19

26 Jun '19

Hello, My self Hemant Sonawane. Recently I am working on openstack and I am getting some strange error when I installed openstack dashboard and tried to edit the visibility of images and update their metadata it dosent work I get *Forbidden : Redirecting to login *and *Unable to edit the image custom properties *such errors. I also tried to see the glance-api logs and found this *"2019-06-26 13:11:51,705.705 1 WARNING glance.api.v2.images [-] Could not find schema properties file schema-image.json. Continuing without custom properties"* Is there any glance policy in horizon? that might cause this error I think. The detail log file attached for your ready reference. So please look into my concern and let me know if there is any solution to resolve my issue. Thanks and Regards,

1 0

[nova] Can we drop the kilo era ComputeNode host/service_id compat code now?
by Matt Riedemann 26 Jun '19

26 Jun '19

There are still quite a few TODOs in the code [1][2][3] from a kilo era blueprint [4]. At this point I'm pretty sure you can't startup the nova-compute service without having a ComputeNode record without a host and hypervisor_hostname field set (we don't set the ComputeNode.service_id anywhere anymore as far as I can tell, except in some ComputeNode RPC compat code [5]). I've stumbled across all of this code before, but was looking at it again today because I have a very simple change I need to make which is going from a ComputeNode object and getting the related nova-compute Service object for that node. Looking at the code one might think this is reasonable: service = objects.Service.get_by_id(ctxt, compute_node.service_id) But compute_node.service_id is likely None. Or how about: service = objects.Service.get_by_compute_host(ctxt, compute_node.host) But ComputeNode.host is also nullable (though likely should have a value as noted above). This is a long way of me saying this code is all gross and we should clean it up, which means making sure all of this Kilo era compat code for old records is no longer necessary, which means all of those records should be migrated by now but how should we check? I *think* this might just be as simple as a "nova-status upgrade check" check which scans the cells looking for (non-deleted) compute_nodes records where host is NULL and report an error if any are found. I believe the recovery action for an operator that hits this is to delete the busted compute_nodes record and restart the nova-compute service so a new compute node record is created. I would really think that anything this scan would find would be orphaned compute_nodes records that could just be deleted since another compute_nodes record probably already exists for the same hypervisor_hostname value. IOW, I don't think we need an online data migration routine for this. Hopefully at least one person (Sylvain) can agree with me here and the plan of action I've put forth. [1] https://github.com/openstack/nova/blob/91647a9b711a8102c79bb17c6b4dff24ad6f… [2] https://github.com/openstack/nova/blob/91647a9b711a8102c79bb17c6b4dff24ad6f… [3] https://github.com/openstack/nova/blob/91647a9b711a8102c79bb17c6b4dff24ad6f… [4] https://blueprints.launchpad.net/nova/+spec/detach-service-from-computenode [5] https://github.com/openstack/nova/blob/91647a9b711a8102c79bb17c6b4dff24ad6f… -- Thanks, Matt

3 2

OSA with cumulus linux
by Grant Morley 26 Jun '19

26 Jun '19

Hi All, Just wondered if anyone was using OSA with cumulus linux at all? - We are starting a new POC and have cumulus switches for a new environment. We currently use OSA and wondered if there was any support / playbooks for it using cumulus linux? Or if anyone is running cumulus, have you had to write your own playbooks with it alongside OSA? Happy to do the latter, but didn't know if anyone has already done this or can point me to any documentation that would be useful. I couldn't find anything when I looked here: https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/ Regards, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com <https://www.civo.com/>| Signup for an account! <https://www.civo.com/signup>

2 2

[mistral] No office hours meeting today
by Renat Akhmerov 26 Jun '19

26 Jun '19

Hello, Since a number of key people can’t attend today’s meeting we’re cancelling it. See you next week. Thanks Renat Akhmerov @Nokia

1 0

Installation of Cinder in multi compute environment
by Jyoti Dahiwele 25 Jun '19

25 Jun '19

Dear Team, What is the best practise of installation of Cinder service on a controller node or separate node in multi compute openstack installation?

1 0

[tc] July meeting host
by Alexandra Settle 25 Jun '19

25 Jun '19

Hi team, The next TC meeting is set to be on the 5th of July, 2019. As I have every intention of leaving my laptop in the UK whilst I vacay on the sunny shores of US, I need a volunteer to host the meeting. I am currently going through the process of updating the meeting agenda, so minimum effort required :) just turn up, and ensure your gif game is strong, and run the commands. Any takers? I'll return for the following meeting on the 1st of August. Cheers, Alex

11 18

[scientific-sig] IRC meeting shortly: Shanghai summit CFP and planning
by Stig Telfer 25 Jun '19

25 Jun '19

Hi all - We have a Scientific SIG IRC meeting at 2100 UTC (about 20 minutes) in channel #openstack-meeting. Everyone is welcome. This week with the Shanghai summit CFP approaching we’d like to kick off the planning for Scientific SIG activities for the summit. Cheers, Stig

1 0

[heat] Resource replacement terminates at DELETE_COMPLETE
by Erik McCormick 25 Jun '19

25 Jun '19

HI everyone! I have a situation with a heat stack where it has an Octavia Load Balancer resource which it thinks it's already replaced and so will not recreate it. Resource api_lb with id 3978 already replaced by 3999; not checking check /var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/check_resource.py:310 : It goes to a DELETE_COMPLETED state and just sits there. The stack stays UPDATE_IN_PROGRESS and nothing else moves. It doesn't even time out after 4 hours. Doing a stack check puts everytinng as CHECK_COMPLETE, even the non-existent load balancers. I can mark the LB and its components unhealthy and start another update, but this just repeats the cycle. This all started with some Octavia shenanigans which ended with all the load balancers being deleted manually. I have 2 similar stacks which recreated fine, but this one went through the cycle several other times as we were trying to fix the LB problem. This is a super edge case, but hopefully someone has another idea how to get out of it. Thanks! Erik

2 1

[devel] Fast flake8 testing
by Zane Bitter 25 Jun '19

25 Jun '19

Those of you who work on a fairly large project will have noticed that running flake8 over all of it takes some time, and that this slows down development. Nova (at least) has a solution to this, in the form of a "fast8" tox environment that runs flake8 only against the files that have changed in the latest patch + the working directory. This is *much* faster, but that approach has some limitations: the script is buggy, it only tests the top-most patch, it creates a second tox environment (which is slow) that can then get out of sync with your regular pep8 environment, and of course it requires the project to add it explicitly. If you're interested in a solution with none of those limitations, here is a script that I've been using: https://gist.github.com/zaneb/7a8c752bfd97dd8972756d296fc5e41f It tests all changes on the branch, using your existing pep8 tox environment, handles deleted files and changes to non-python files correctly, and should be usable for every OpenStack project. I hope this is helpful to someone. (Note that the pep8 environment on many projects includes other test commands in addition to flake8 - such as bandit - so you should still run the pep8 tox tests once before submitting a patch.) cheers, Zane.

3 2

[magnum] nodegroups
by Bharat Kunwar 25 Jun '19

25 Jun '19

Hi Theodoros, Just replying to your message on IRC here as you appear offline. > if you don't care for what you have in the db i would do this: > - stop both api and conductor > - drop magnum db > - checkout master > - run the migrations (from master) After getting this far, I tried running `magnum-db-manage upgrade` but I hit this: sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1049, u"Unknown database 'magnum'") (Background on this error at: http://sqlalche.me/e/2j85) > - start the services I can’t start the services either but that is probably due to lack of a database… > - create a cluster > - checkout magnum_nodegroups > - stop services > - run migrations > - start the services > I know it's not great… Would you mind letting me know where I’m going wrong? Thanks Bharat

2 1

How to reduce image size?
by Manuel Sopena Ballesteros 25 Jun '19

25 Jun '19

Dear Openstack community, I would like to reduce the size of an image, I wrote this command: # openstack image set --property size=700000 centos7.6-kudu-image Unable to set 'size' to '700000'. Reason: '700000' is not of type u'null', u'integer' Failed validating u'type' in schema[u'properties'][u'size']: {u'description': u'Size of image file in bytes', u'readOnly': True, u'type': [u'null', u'integer']} On instance[u'size']: '700000' I guess the real issue is u'readOnly': True How can I reduce the image size? Thank you very much Manuel NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

2 1

[kolla][octavia][openstack-ansible][tripleo] octavia in kolla ansible
by Mark Goddard 25 Jun '19

25 Jun '19

Hi, In the recent kolla meeting [1] we discussed the usability of octavia in kolla ansible. We had feedback at the Denver summit [2] that this service is difficult to deploy and requires a number of manual steps. Certificates are one of the main headaches. It was stated that OSA [3] may have some useful code we could look into. As a starting point to improving this support, I'd like to gather information from people who are using octavia in kolla ansible, and what they have had to do to make it work. Please respond to this email. I've also tagged openstack-ansible and Tripleo - if there is any useful information those teams have to share about this topic, it is most welcome. Alternatively if your support for octavia also falls short perhaps we could collaborate on improvements. Thanks, Mark [1] http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-06-19-15.00.l… [2] https://etherpad.openstack.org/p/DEN-train-kolla-feedback [3] https://opendev.org/openstack/openstack-ansible-os_octavia

7 11

[ironic] To not have meetings?
by Julia Kreger 25 Jun '19

25 Jun '19

Last week the discussion came up of splitting the ironic meeting to alternate time zones as we have increasing numbers of contributors in the Asia/Pacific areas of the world[0]. With that discussion, an additional interesting question came up posing the question of shifting to the mailing list instead of our present IRC meeting[1]? It is definitely an interesting idea, one that I'm personally keen on because of time zones and daylight savings time. I think before we do this, we should collect thoughts and also try to determine how we would pull this off so we don't forget the weekly checkpoint that the meeting serves. I think we need to do something, so I guess now is a good time to provide input into what everyone thinks would be best for the project and facilitating the weekly check-in. What I think might work: By EOD UTC Monday: * Listed primary effort participants will be expected to update the whiteboard[2] weekly before EOD Monday UTC * Contributors propose patches to the whiteboard that they believe would be important for reviewers to examine this coming week. * PTL or designee sends weekly email to the mailing list to start an update thread shortly after EOD Monday UTC or early Tuesday UTC. ** Additional updates, questions, and topical discussion (new features, RFEs) would ideally be wrapped up by EOD UTC Tuesday. With that, I think we would also need to go ahead and begin having "office hours" as during the week we generally know some ironic contributors will be in IRC and able to respond to questions. I think this would initially consist of our meeting time and perhaps the other time that seems to be most friendly to the contributors int he Asia/Pacific area[3]. Thoughts/ideas/suggestions welcome! -Julia [0]: http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-iro… [1]: http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-iro… [2]: https://etherpad.openstack.org/p/IronicWhiteBoard [3]: https://doodle.com/poll/bv9a4qyqy44wiq92

6 5

[neutron] Provider/External Networking
by Jan Marquardt 25 Jun '19

25 Jun '19

Hallo, we have/are planning an Openstack cluster which is supposed to look like this: http://paste.openstack.org/show/753337/ We are currently struggling with the external/provider networks. Is there any way/best practice to terminate the provider networks directly on the network nodes and announce the prefixes through BGP to the upstream routers? Any experience report or hint would really be appreciated. Best regards Jan -- Artfiles New Media GmbH | Zirkusweg 1 | 20359 Hamburg Tel: 040 - 32 02 72 90 | Fax: 040 - 32 02 72 95 E-Mail: support(a)artfiles.de | Web: http://www.artfiles.de Geschäftsführer: Harald Oltmanns | Tim Evers Eingetragen im Handelsregister Hamburg - HRB 81478

2 1

NetApp E-series infiniband support for Cinder
by Grant Morley 24 Jun '19

24 Jun '19

Hi All, Just a quick one to see if anybody knows if there is currently any infiniband support for cinder using a NetApp E-Series SAN. I have had a look at: https://docs.openstack.org/cinder/queens/configuration/block-storage/driver… and: https://docs.openstack.org/cinder/rocky/reference/support-matrix.html They seem to suggest that only iSCSI and FC are supported. I just want to make sure before I start trying to do a POC with the E-series and infiniband. Any advice would be much appreciated. Kind Regards, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com <https://www.civo.com/>| Signup for an account! <https://www.civo.com/signup>

4 5

[neutron] CI issue
by Slawomir Kaplonski 24 Jun '19

24 Jun '19

Hi neutrinos, Since about 2 days we have some problem with iptables_hybrid jobs in Neutron CI. See [1] for details. I’m now trying to find out what can be the problem there but if I will not find it anything soon I will propose temporary patch to skip those 3 failing tests for now. Please don’t recheck Your patches if it failed on those 2 jobs - this will fail again :/ [1] https://bugs.launchpad.net/neutron/+bug/1833902 — Slawek Kaplonski Senior software engineer Red Hat

2 6

Re: https://review.opendev.org/#/c/663089
by Goutham Pacha Ravi 24 Jun '19

24 Jun '19

Hi Helen, On Mon, Jun 24, 2019 at 3:18 AM Walsh, Helen <Helen.Walsh(a)dell.com> wrote: > Hi Ravi, > > A request for information if I may. I will need some direction on how to > generate manila-powermax.inc > > I am not finding anything online > > https://review.opendev.org/#/c/663089 > Thank you for checking. I'm assuming you want to update the file rendered to: https://docs.openstack.org/manila/latest/configuration/shared-file-systems/… Manila currently doesn't have a way to automatically generate these configuration files. Please see associated bug: https://bugs.launchpad.net/manila/+bug/1713062. So for now, you can manually modify those files; we'll look to prioritizing https://bugs.launchpad.net/manila/+bug/1713062. Contributions to add the config option automation would be welcome! Thanks, Goutham > > > Thank you, > > Helen >

1 0

[cinder] ceph multiattach details?
by Bernd Bausch 24 Jun '19

24 Jun '19

The Stein release notes mention that the RBD driver now supports multiattach, but i have not found any details. Are there limitations? Is there a need to configure anything? In the RBD driver <https://opendev.org/openstack/cinder/src/branch/master/cinder/volume/driver…>, I find this: def _enable_multiattach(self, volume): multipath_feature_exclusions = [ self.rbd.RBD_FEATURE_JOURNALING, self.rbd.RBD_FEATURE_FAST_DIFF, self.rbd.RBD_FEATURE_OBJECT_MAP, self.rbd.RBD_FEATURE_EXCLUSIVE_LOCK, ] This seems to mean that journaling and other features (to me, it's not quite clear what they are) will be automatically disabled when switching on multiattachment. Further down in the code I see that replication and multiattach are mutually exclusive. Is there some documentation about the Ceph multiattach feature, even an email thread? Thanks, Bernd ||

6 8

[all] Long overdue cleanups of Zuulv2 compatibility base configs
by Clark Boylan 24 Jun '19

24 Jun '19

As part of our transition to Zuulv3 a year and a half ago, we carried over some compatibility tooling that we would now like to clean up. Specifically, we add a zuul-cloner (which went away in zuulv3) shim and set a global bindep fallback file value in all jobs. Zuulv3 native jobs are expected to use the repos zuul has precloned for you (no zuul-cloner required) as well as supply an in repo bindep.txt (or specify a bindep.txt path or install packages via some other method). This means that we should be able to remove both of these items from the non legacy base job in OpenDev's zuul. The legacy base job will continue to carry these for you so that you can write new native jobs over time. We have two changes [0][1] ready to go for this; however, due to the potential for disruption we would like to give everyone some time to test and prepare for this change. Fungi has a change to base-test [2] which will remove the zuul-cloner shim. Once this is in you can push "Do Not Merge" changes to your zuul config that reparent your tests from "base" to "base-test" and that will run the jobs without the zuul-cloner shim. Testing the bindep fallback removal is a bit more difficult as we set that in zuul's server config globally. What you can do is check your jobs' job-output.txt log files for usage of "bindep-fallback.txt". Our current plan is to merge these changes on June 24, 2019. We will be around to help debug any unexpected issues that come up. Jobs can be updated to use the "legacy-base" base job instead of the "base" base job if they need to be reverted to the old behavior quickly. Finally, Fungi did some excellent spelunking through history to understand how we got here. If you are curious you can find more details at: http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006881.html. [0] https://review.opendev.org/656195 [1] https://review.opendev.org/663151 [2] https://review.opendev.org/663135 Clark

3 4

[Watcher] team meeting and agenda
by li.canwei2＠zte.com.cn 24 Jun '19

24 Jun '19

Hi, Watcher will have a meeting at 08:00 UTC today in the #openstack-meeting-alt channel. The agenda is available on https://wiki.openstack.org/wiki/Watcher_Meeting_Agenda feel free to add any additional items. Thanks! Canwei Li

2 1

[designate] DKIM TXT record problem
by Ionut Biru 24 Jun '19

24 Jun '19

Hello guys, I'm running Rocky and as backend for designate I have powerdns. Whenever I try to add a TXT record for DKIM, the API returns that the specified record is not a TXT record. https://paste.xinu.at/OOz7/ It seems that is due to the length of the record It has a maxim a 255 limit. How should I proceed in this case? -- Ionut Biru - https://fleio.com

2 2

[Watcher] about inactive core members
by li.canwei2＠zte.com.cn 23 Jun '19

23 Jun '19

Hi Watcher team, Last week in the Watcher IRC meeting we discussed to remove inactive core members. It's welcome if you can take some time to review, and I wish you can reply before the end of this week if you want to get back. Best Regards, Canwei Li

1 0

Openstack cinder HA
by Md. Farhad Hasan Khan 22 Jun '19

22 Jun '19

Hi, In my environment openstack volume service running on compute node. When the openstack-cinder-volume service goes down on, the Block Storage volumes which were created using the openstack-cinder-volume service cannot be managed until the service comes up again. I need help how to configure openstack cinder volume HA. Thanks & B'Rgds, Rony

2 4

[neutron] bug deputy report for the week of June 17th
by Takashi Yamamoto 22 Jun '19

22 Jun '19

hi, this week i have looked at the following bugs. a few of them still need triage. i will have a travel tomorrow and i'm not sure if i will have time to triage them. i don't think i can attend the team meeting next week. (sorry!) Still need triage: https://bugs.launchpad.net/neutron/+bug/1833175 pptp vpn doesn't work with openvswitch firewall https://bugs.launchpad.net/neutron/+bug/1833156 neutron fwaas v2 log function does not work https://bugs.launchpad.net/neutron/+bug/1832636 Error creating IPv6 subnet on routed network segment High: https://bugs.launchpad.net/neutron/+bug/1833257 By default Horizon sets Firewall group admin state to False when user trying to set it true Firewall always remains in DOWN state Medium: https://bugs.launchpad.net/neutron/+bug/1833279 TestNeutronServer: start function not called (or not logged in the temp file) https://bugs.launchpad.net/neutron/+bug/1833653 We should cleanup ipv4 address if keepalived is dead https://bugs.launchpad.net/neutron/+bug/1833589 neutron-dynamic-routing broken by introduction of agent_timestamp to _log_heartbeat() https://bugs.launchpad.net/neutron/+bug/1833721 ip_lib synchronized decorator should wrap the privileged one Low: https://bugs.launchpad.net/neutron/+bug/1833125 Remaining neutron-lbaas relevant code and documentation RFE: https://bugs.launchpad.net/neutron/+bug/1832758 [RFE] Allow/deny custom ethertypes in security groups https://bugs.launchpad.net/neutron/+bug/1833674 [RFE] Improve profiling of port binding and vif plugging

1 0

[keystone] Keystone Team Update - Week of 17 June 2019
by Colleen Murphy 21 Jun '19

21 Jun '19

# Keystone Team Update - Week of 17 June 2019 ## News ### Next steps for oslo.limit In order to make progress on having a usable interface for oslo.limit, we discussed[1][2] simplifying the current implementation. Lance has some patches[3] that remove the ideas of claims and context managers, which sacrifices the risk of race conditions and performance penalties in favor of simplicity for the time being. [1] http://eavesdrop.openstack.org/meetings/keystone/2019/keystone.2019-06-18-1… [2] http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-k… [3] https://review.opendev.org/665708 ## Open Specs Train specs: https://bit.ly/2uZ2tRl Ongoing specs: https://bit.ly/2OyDLTh Train specs need to be merged by July 26. Please help review these specs and be mindful of feedback. ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 20 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 44 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Bugs This week we opened 6 new bugs and closed 5. Bugs opened (6) Bug #1833085 (keystone:High) opened by Sebastian Riese https://bugs.launchpad.net/keystone/+bug/1833085 Bug #1833739 (keystone:High) opened by David Orman https://bugs.launchpad.net/keystone/+bug/1833739 Bug #1833311 (keystone:Undecided) opened by Michael Carpenter https://bugs.launchpad.net/keystone/+bug/1833311 Bug #1833340 (keystone:Undecided) opened by cnaik https://bugs.launchpad.net/keystone/+bug/1833340 Bug #1833554 (keystone:Undecided) opened by Chason Chan https://bugs.launchpad.net/keystone/+bug/1833554 Bug #1833207 (keystoneauth:Undecided) opened by Jagatjot Singh https://bugs.launchpad.net/keystoneauth/+bug/1833207 Bugs closed (2) Bug #1832005 (keystone:Undecided) https://bugs.launchpad.net/keystone/+bug/1832005 Bug #1833340 (keystone:Undecided) https://bugs.launchpad.net/keystone/+bug/1833340 Bugs fixed (3) Bug #1750676 (keystone:High) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1750676 Bug #1818844 (keystone:Medium) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1818844 Bug #1649735 (keystonemiddleware:Medium) fixed by Colleen Murphy https://bugs.launchpad.net/keystonemiddleware/+bug/1649735 ## Milestone Outlook https://releases.openstack.org/train/schedule.html Milestone 2 is in 5 weeks. Train specs need to be merged by then. Feature proposal freeze is in 8 weeks. ## Shout-outs Shout out to Nathan Oyler, whose first patch to OpenStack[4] fixed a major omission in our audit notifications. Thanks, Nathan! [4] https://review.opendev.org/664618 ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[goal][python3] Train unit tests weekly update (goal-12)
by Corey Bryant 21 Jun '19

21 Jun '19

This is the goal-12 weekly update for the "Update Python 3 test runtimes for Train" goal [1]. There are 12 weeks remaining for completion of Train community goals [2]. == What's the Goal? == To ensure (in the Train cycle) that all official OpenStack repositories with Python 3 unit tests are exclusively using the 'openstack-python3-train-jobs' Zuul template or one of its variants (e.g. 'openstack-python3-train-jobs-neutron') to run unit tests, and that tests are passing. This will ensure that all official projects are running py36 and py37 unit tests in Train. For complete details please see [1]. == Ongoing Work == I've submitted patch automation scripts for review: https://review.opendev.org/#/c/666934 And I've started submitting patches that were generated using the above scripts. Open patches needing reviews: https://review.openstack.org/#/q/topic:python3-train+is:open Failing patches: https://review.openstack.org/#/q/topic:python3-train+status:open+(+label:Ve… Some notes on 2 issues I came across this week: 1) Some projects that have a valid reason to keep old tox.ini py3 environments or setup.cfg classifiers. For example, the OpenStack Charms project is a deployment project that still supports deployments on xenial, and keeping the py35 environment in tox.ini makes sense. In scenarios like this I plan to continue with the Zuul switch to openstack-python3-train-jobs without removing py35 tox.ini or Python 3.5 classifier from setup.cfg. 2) Some projects are missing Zuul config. Not to pick on any projects in particular but as an example there are projects that I'd expect to have .zuul.yaml or equivalent, such as python-adjutant, however it doesn't. In scenarios like this I plan to skip the project entirely. == Completed Work == Merged patches: https://review.openstack.org/#/q/topic:python3-train+is:merged == How can you help? == Please take a look at the failing patches and help fix any failing unit tests for your project(s). Python 3.7 unit tests will be self-testing in Zuul. If you're interested in helping submit patches, please let me know. == Reference Material == [1] Goal description: https://governance.openstack.org/tc/goals /train/python3-updates.html [2] Train release schedule: https://releases.openstack.org/train/schedule.html (see R-5 for "Train Community Goals Completed") Storyboard: https://storyboard.openstack.org/#!/story/2005924 Porting to Python 3.7: https://docs.python.org/3/whatsnew/3.7.html#porting-to-python-3-7 Python Update Process: https://opendev.org/openstack/governance/src/branch/master/resolutions/2018… Train runtimes: https://opendev.org/openstack/governance/src/branch/master/reference/runtim… Thanks, Corey

1 0

[ThirdParty][CI] : CI-watch is not updating with latest patchset
by Sharma, Virendra Sharma 21 Jun '19

21 Jun '19

Hi Team, >From last few days unable to see latest patch set entry in CI-WATCH (http://ciwatch.mmedvede.net/project?project=cinder&time=7+days) However third party CI running on latest patch set and reporting its vote to Gerrit accordingly. Currently it is visible for last 7 days but soon it will also go off, if it won't reflect updated one. Please help me to understand if any update regarding this or else provide way to get it resolved from my end. Regards, Virendra

2 1

[openstack-operators][neutron][olso.db] connection pool configuration
by Arnaud Morin 21 Jun '19

21 Jun '19

Hey all, I am wondering what configuration the operators in this mailing list are using in [oslo.db] section. For example, the max_pool_size default value is 5, which seems very low to me (our deployment is having more than 1000 neutron agents). So, is there any recommendation somewhere for those settings? Thanks for any tip. -- Arnaud Morin

1 0

[placement] update 19-24
by Chris Dent 21 Jun '19

21 Jun '19

HTML: https://anticdent.org/placement-update-19-24.html Here's a 19-24 pupdate. Last week I said there wouldn't be one this week. I was wrong. There won't be one next week. I'm taking the week off to reset (I hope). I've tried to make sure that there's nothing floating about in Placement land that is blocking on me. # Most Important The [spec for nested magic](https://review.opendev.org/662191) is close. What it needs now is review from people outside the regulars to make sure it doesn't suffer from tunnel vision. The features discussed form the foundation for Placement being able to service queries for real world uses of nested providers. Placement can already model nested providers but asking for the right one has needed some work. # Editorial I read an article this morning which touches on the importance of considering [cognitive load in software design](https://techbeacon.com/app-dev-testing/forget-monoliths-vs-microser…. It's fully of glittering generalities, but it reminds me of some of the reasons why it was important to keep a solid boundary between Nova and Placement and why, now that Placement is extracted, the complexity of this nested magic is something we need to measure against the cognitive load it will induce in the people who come after us as developers and users. # What's Changed * Support for mappings in allocation candidates has merged as [microversion 1.34](https://docs.openstack.org/placement/latest/placement-api-microversio…. * Gibi made it so [OSProfiler](https://docs.openstack.org/osprofiler/latest/) works with placement again. # Specs/Features * <https://review.opendev.org/654799> Support Consumer Types. This has some open questions that need to be addressed, but we're still go on the general idea. * <https://review.opendev.org/662191> Spec for nested magic 1. See "Most Important" above. Some non-placement specs are listed in the Other section below. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 23 (3) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 4 (1) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 5 (-1) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 3 (1) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 16 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 5 (-1). [1832814: Placement API appears to have issues when compute host replaced](https://bugs.launchpad.net/nova/+bug/1832814) is an interesting bug. In a switch from RDO to OSA, resource providers are being duplicated because of a change in node name. # osc-placement osc-placement is currently behind by 11 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. # Main Themes ## Nested Magic The overview of the features encapsulated by the term "nested magic" are in a [story](https://storyboard.openstack.org/#!/story/2005575) and [spec](https://review.opendev.org/662191). Code related to this: * <https://review.opendev.org/663009> PoC: resourceless request, including some code from <https://review.opendev.org/657510> WIP: Allow RequestGroups without resources * <https://review.opendev.org/#/q/topic:story/2005575-33753> Support for the new root_required query parameter. Introduces the useful concepts of required wide params and a request wide search context. ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. A [spec](https://review.opendev.org/654799) has started. There are some questions about request and response details that need to be resolved, but the overall concept is sound. ## Cleanup We continue to do cleanup work to lay in reasonable foundations for the nested work above. As a nice bonus, we keep eking out additional performance gains too. * <https://review.opendev.org/#/q/topic:story/2005443> Add a nested-perfload job, using gabbi to create the trees. Unsurprisingly, nested topologies are slower than non; having this job will help us track that. # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are five [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And one [os-resource-classes change](https://review.opendev.org/#/q/project:openstack/os-resource-classe…. # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/601596> Nova: spec: support virtual persistent memory * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.openstack.org/650188> nova-spec: Allow compute nodes to use DISK_GB from shared storage RP * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/657801> rpm-packaging: placement service * <https://review.opendev.org/662229> helm: WIP: add placement chart * <https://review.opendev.org/662164> kolla-ansible: Add a explanatory note for "placement_api_port" * <https://review.opendev.org/656023> Nova: Use OpenStack SDK for placement * <https://review.opendev.org/612497> Nova: Spec: Provider config YAML file * <https://review.opendev.org/623558> Nova: single pass instance info fetch in host manager * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/665168> TripleO: Enable Request Filter for Image Types * <https://review.opendev.org/664689> Nova: get_ksa_adapter: nix by-service-type confgrp hack * <https://review.opendev.org/664862> OSA: Add nova placement to placement migration * <https://review.opendev.org/657796> Nova: Defaults missing group_policy to 'none' * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/666298> OSA: Fix aio_distro_metal jobs for openSUSE # End Go outside. Reflect a bit. Then do. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[Security SIG] Weekly Newsletter June 13th & June 20th
by Gage Hugo 20 Jun '19

20 Jun '19

So I wrote the newsletter for last week, but forgot to send out the actual email, so this week will have double the content! #Week of: 20 June 2019 - Security SIG Meeting Info: http://eavesdrop.openstack.org/#Security_SIG_meeting - Weekly on Thursday at 1500 UTC in #openstack-meeting - Agenda: https://etherpad.openstack.org/p/security-agenda - https://security.openstack.org/ - https://wiki.openstack.org/wiki/Security-SIG #Meeting Notes - Summary: http://eavesdrop.openstack.org/meetings/security/2019/security.2019-06-20-1… - This week we discussed cleaning up the security.openstack.org page - Overall there are many outdated sections, we came up with a current rough plan that is outlined in the security-agenda notes for this week's meeting - Retiring Syntribos - On the topic of cleaning up the security.openstack.org page, one section is security tools, which currently lists bandit and syntribos. - Looking at the Syntribos repo, it seems like lately there's only been changes related to doc fixes and overall zuul updates, with a couple actual updates to the project. - If there's anyone still with interest in updating/using Syntribos, please reach out to us. ## News - [nova] TPM thread: http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007258.html # VMT Reports - A full list of publicly marked security issues can be found here: https://bugs.launchpad.net/ossa/ - No new public security bugs this week ======================================================================================================== #Week of: 13 June 2019 - Security SIG Meeting Info: http://eavesdrop.openstack.org/#Security_SIG_meeting - Weekly on Thursday at 1500 UTC in #openstack-meeting - Agenda: https://etherpad.openstack.org/p/security-agenda - https://security.openstack.org/ - https://wiki.openstack.org/wiki/Security-SIG #Meeting Notes - Summary: http://eavesdrop.openstack.org/meetings/security/2019/security.2019-06-13-1… - This week we finalized the details and settings for the [openstack-security] mailing list. The list will be used to provide purely automated notifications about security-related changes and bug reports within OpenStack. Anyone looking to reach out to the security SIG can either use the [openstack-discuss] mailing list or use the #openstack-security channel on freenode IRC. ## News - Storyboard: The security team autoassignment feature landed - If a new story is marked as "security" upon creation, it will automatically become private, however it can be edited to become public after it is created. # VMT Reports - A full list of publicly marked security issues can be found here: https://bugs.launchpad.net/ossa/ - No new public security bugs this week

1 0

[nova]Bug #1829696: qemu-kvm process takes 100% CPU usage when running redhat/centos 7.6 guest OS
by 钱熙 20 Jun '19

20 Jun '19

Hi there, I am struggling with qemu-kvm 100% CPU usage problem. When running redhat or centos 7.6 guest os on vm, the cpu usage is very low on vm(almost 100% idle when no tasks run), but on the host, qemu-kvm reports 100% cpu busy usage. https://bugs.launchpad.net/nova/+bug/1829696 I opened the bug above, however it did not find some interest. After searching some related bugs report, I suspect that it is due to the clock settings in vm's domain xml. My settings are as follows: <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> And details about the bug, please see https://bugs.launchpad.net/nova/+bug/1829696 It shows that only the version 7.6 of redhat or centos affected by this bug behavior. In my cluster, it is OK for versions from redhat or centos 6.8 to 7.5. Any clue or suggestion? ------------------ Thanks a lot! Best regards QianXi

2 1

[nova] API updates week 19-25
by Ghanshyam Mann 20 Jun '19

20 Jun '19

Hi All, Please find the Nova API updates of this week. API Related BP : ============ Code Ready for Review: ------------------------------ 1. Support adding description while locking an instance: - Topic: https://review.opendev.org/#/q/topic:bp/add-locked-reason+(status:open+OR+s… - Weekly Progress: OSC patch has been updated by tssurya. mriedem is +1. 2. Add host and hypervisor_hostname flag to create server - Topic: https://review.opendev.org/#/q/topic:bp/add-host-and-hypervisor-hostname-fl… - Weekly Progress: patch is updated with review comment. ready for re-review. 3. Specifying az when restore shelved server - Spec: https://review.opendev.org/#/q/topic:bp/support-specifying-az-when-restore-… - Weekly Progress: Spec is merged and code is up for review. 4. Nova API cleanup - Spec: https://review.openstack.org/#/c/603969/ - Weekly Progress: working on code, seems like a lot of tests changes. Should be able to push code by next report. 5. Detach and attach boot volumes: - Topic: https://review.openstack.org/#/q/topic:bp/detach-boot-volume+(status:open+O… - Weekly Progress: No Progress Spec Ready for Review: ----------------------------- 1. Nova API policy improvement - Spec: https://review.openstack.org/#/c/547850/ - PoC: https://review.openstack.org/#/q/topic:bp/policy-default-refresh+(status:op… - Weekly Progress: Under review and updates. 2. Support for changing deleted_on_termination after boot -Spec: https://review.openstack.org/#/c/580336/ - Weekly Progress: No update this week. 3. Support delete_on_termination in volume attach api -Spec: https://review.openstack.org/#/c/612949/ - Weekly Progress: No updates this week. 4. Add API ref guideline for body text - ~8 api-ref are left to fix. Previously approved Spec needs to be re-proposed for Train: --------------------------------------------------------------------------- 1. Servers Ips non-unique network names : - https://blueprints.launchpad.net/nova/+spec/servers-ips-non-unique-network-… - https://review.openstack.org/#/q/topic:bp/servers-ips-non-unique-network-na… 2. Volume multiattach enhancements: - https://blueprints.launchpad.net/nova/+spec/volume-multiattach-enhancements - https://review.openstack.org/#/q/topic:bp/volume-multiattach-enhancements+(… Bugs: ==== No progress report in this week. NOTE- There might be some bug which is not tagged as 'api' or 'api-ref', those are not in the above list. Tag such bugs so that we can keep our eyes. -gmann

1 0

[sigs][publiccloud][publiccloud-wg][publiccloud-sig][billing] Cancellation todays meeting for the Public Cloud SIG
by Tobias Rydberg 20 Jun '19

20 Jun '19

Hi all, Unfortunate I need to cancel today meeting for the Public Cloud SIG since I can't manage to make it. Feel free to use the time if you will to continue our ongoing discussions regarding the billing initiative. I'm happy to schedule a meeting next week at the same time, please respond to this email or drop a message in the channel if that i suitable for you. I also want to bring the new User Survey to attention to you all if you have missed that and haven't taken it. Please also spread the word to fellow colleagues, customers and what not. URL to survey: https://www.openstack.org/user-survey/survey-2019/ Cheers, Tobias -- Tobias Rydberg Senior Developer Twitter & IRC: tobberydberg www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED

1 0

[placement][docs][packaging] rpm- and deb- related placement install docs
by Chris Dent 20 Jun '19

20 Jun '19

The placement install docs [1] have the following warning plastered on top: These installation documents are a work in progress. Some of the distribution packages mentioned are not yet available so the instructions will not work. It is likely that packages are now available, but the docs have not been updated to reflect that, nor have the docs been verified against those packages. None of the regular and active contributors to placement are involved with distro-related packaging. In addition they are all already fully booked. Are there members of the community who are involved with the distros, and who can update and verify these docs? If not, we will likely need to remove the distro related install documents before the end of the Train release and solely maintain the install-from-pypi docs [2] (which are up to date). This is because the existing docs are now misleading, and thus potentially dangerous and extra-work-inducing, as suggested by a recent bug [3]. If you are interested in helping out, please follow up. If you're unable to help out, but know people involved with packaging who should know about this concern, please let them know. Thanks for your help. [1] https://docs.openstack.org/placement/latest/install/index.htmlk [2] https://docs.openstack.org/placement/latest/install/from-pypi.html [3] https://storyboard.openstack.org/#!/story/2005910 -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

4 3

[ironic] cancelling the meeting for June 24th
by Julia Kreger 19 Jun '19

19 Jun '19

Greetings everyone! Next Monday, June 24th, I will be largely unavailable due to business travel. On top of that, The person who I would normally ask to run the meeting in my absence, will also be traveling that day. With that being said, I think it is best to cancel the meeting. Now, if there is someone willing to be the ad-hoc meeting organizer and run the meeting or if everyone were to just treat it like an office hours that would be a good thing. Thanks everyone! -Julia

1 0

Re: [nova][placement] Need for a nova-manage placement audit command
by Matt Riedemann 19 Jun '19

19 Jun '19

Re-posting this since it came up again in IRC today. On 9/20/2018 6:09 PM, Matt Riedemann wrote: > mnaser wrote a simple placement audit tool today and dumped his script > into a bug report [1]. It turns out several operators have already > written a variant of this same tool. The idea is to compare what's in > placement for allocations and (compute node) resource providers against > what's in nova for instances and compute nodes, look for orphans in > placement and then report them, or maybe also clean them up. > > I think this could go into a "nova-manage placement audit" command and > should be pretty easy to write for minimum functionality (start with > just the report). > > I'm advertising the need here in case someone wants to work on this. I'd > like to myself, but just can't justify the time right now. > > [1] https://bugs.launchpad.net/nova/+bug/1793569 > -- Thanks, Matt

2 1

[Thirdparty CI] CI Watch is stuck
by Rajini.Karthik＠Dell.com 19 Jun '19

19 Jun '19

Hi all, Is there any issue with CI-WATCH, it seems to be stuck from June 14th http://ciwatch.mmedvede.net/project?project=cinder&time=24+hours Thanks Rajini

1 0

[cinder] nfs in kolla
by Ignazio Cassano 19 Jun '19

19 Jun '19

Hello, I' just deployed ocata with kolla and my cinder backend is nfs. Volumes are created successfully but live migration does not work. While cinder_volume container mounts the cinder nfs backend, the cinder api not and during live migration the cinder api logs reports errors accessing volumes : Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/451bacc11bd88b51ce7bdf31aa97cf39/volume-4889a547-0a0d-440e-8b50-413285b5979c' Any help, please ? Regards Ignazio

2 2

[Kolla] ansible compatibility
by Ignazio Cassano 19 Jun '19

19 Jun '19

Hello All, I'd like to know if there is a Matrix for kolla-ansible and ansible version ...in other words, which ansible version must be used for a kolla-ansible version. For example ocata used kolla-ansible 4.0.5 but I do not know which version of ansible must be used. Installing kolla-ansible with pip it does not install ansible. Reverse Ignazio

3 5

[tripleo][tripleo-ansible] Reigniting tripleo-ansible
by Kevin Carter 19 Jun '19

19 Jun '19

Hello Stackers, I wanted to follow up on this post from last year, pick up from where it left off, and bring together a squad to get things moving. > http://lists.openstack.org/pipermail/openstack-dev/2018-August/133801.html The effort to convert tripleo Puppet and heat templates with embedded Ansible to a more consumable set of playbooks and roles is in full effect. As we're working through this effort we believe co-locating all of the Ansible tasks/roles/libraries/plugins throughout the code base into a single purpose-built repository will assist us in streamlining and simplifying. Structurally, at this time, most of tripleo will remain the same. However, the inclusion of tripleo-Ansible will allow us to create more focused solutions which are independently testable, much easier understand, and simple to include into the current heat template deployment methodologies. While a straight port of the existing Ansible tasks will not be entirely possible, the goal of this ongoing effort will be zero impact on our existing workflow and solutions. To reigniting this effort, I've put up a review to create a new "transformation" squad[0] geared toward building the structure around tripleo-ansible[1] and converting our current solutions into roles/playbooks/libraries/plugins. Initially, we'll be focused on our existing code base; however, long term, I believe it makes sense for this squad to work across projects to breakdown deployment barriers for folks using similar technologies. We're excited to get this effort rolling again and would love to work with anyone and everyone throughout the community. If folks are interested in this effort, please let us know. [0] - https://review.opendev.org/662763 [1] - https://opendev.org/openstack/tripleo-ansible -- Kevin Carter IRC: cloudnull

8 17

[tc][tripleo][charms][helm][kolla][ansible][puppet][chef] Deployment tools capabilities
by Thierry Carrez 19 Jun '19

19 Jun '19

Hi everyone, In the "software" section of the openstack.org website, the deployment tools page is not very helpful for users looking into picking a way to deploy OpenStack: https://www.openstack.org/software/project-navigator/deployment-tools Furthermore, each detailed page is a bit dry. We do not display deliverable tags as most are irrelevant: https://www.openstack.org/software/releases/rocky/components/openstack-helm This was discussed in a forum session in Denver[1], and the outcome was that we should develop a taxonomy of deployment tools capabilities and characteristics, that would help users understand the technologies the various tools are based on, their prerequisites, which services and versions they cover, etc. The web UI should allow users to search deployment tools based on those tags. [1] https://etherpad.openstack.org/p/DEN-deployment-tools-capabilities As a first step, volunteers from that session worked on a draft categorized list[2] of those tags. If you are interested, please review that list, add to it or comment: [2] https://etherpad.openstack.org/p/deployment-tools-tags The next steps are: - commit the detailed list of tags (action:ttx) - apply it to existing deployment tools (action:deploy tools teams) - implementation those tags and data in the openstack website (action:jimmymcarthur) - maybe expand to list 3rd-party installers in a separate tab (tbd) The first two next steps will be implemented as patches to the osf/openstack-map repository, which already contains the base YAML data used in the software pages. Thanks for your help, -- Thierry Carrez (ttx)

1 1

[nova][kolla][openstack-ansible][tripleo] Cells v2 upgrades
by Mark Goddard 19 Jun '19

19 Jun '19

Hi, At the recent kolla virtual PTG [1] we had a good discussion about adding support for multiple nova cells in kolla-ansible. We agreed a key requirement is to be able to perform operations on one or more cells without affecting the rest for damage limitation. This also seems like it would apply to upgrades. We're seeking input on ordering. Looking at the nova upgrade guide [2] I might propose something like this: 1. DB syncs 2. Upgrade API, super conductor For each cell: 3a. Upgrade cell conductor 3b. Upgrade cell computes 4. SIGHUP all services 5. Run online migrations At some point in here we also need to run the upgrade check. Presumably between steps 1 and 2? It would be great to get feedback both from the nova team and anyone running cells Thanks, Mark [1] https://etherpad.openstack.org/p/kolla-train-ptg [2] https://docs.openstack.org/nova/latest/user/upgrade.html

2 2

[all] "Popup teams" proposal about to merge
by Thierry Carrez 19 Jun '19

19 Jun '19

Hi everyone, Following the discussion at [1], a governance change formalizing "popup teams" (short-lived, objective-driven cross-project groups) is about to be approved by the TC at: https://review.opendev.org/#/c/661356/ Please have a look and comment if you have strong feelings about that. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006225.html -- Thierry Carrez (ttx)

1 0

[nova] Spec: Standardize CPU resource tracking
by Shewale, Bhagyashri 19 Jun '19

19 Jun '19

Hi All, Currently I am working on implementation of cpu pinning upgrade part as mentioned in the spec [1]. While implementing the scheduler pre-filter as mentioned in [1], I have encountered one big issue: Proposed change in spec: In scheduler pre-filter we are going to alias request_spec.flavor.extra_spec and request_spec.image.properties form ``hw:cpu_policy`` to ``resources=(V|P)CPU:${flavor.vcpus}`` of existing instances. So when user will create a new instance or execute instance actions like shelve, unshelve, resize, evacuate and migration post upgrade it will go through scheduler pre-filter which will set alias for `hw:cpu_policy` in request_spec flavor ``extra specs`` and image metadata properties. In below particular case, it won’t work:- For example: I have two compute nodes say A and B: On Stein: Compute node A configurations: vcpu_pin_set=0-3 (used as dedicated CPU, This host is added in aggregate which has “pinned” metadata) Compute node B Configuration: vcpu_pin_set=0-3 (used as dedicated CPU, This host is added in aggregate which has “pinned” metadata) On Train, two possible scenarios: Compute node A configurations: (Consider the new cpu pinning implementation is merged into Train) vcpu_pin_set=0-3 (Keep same settings as in Stein) Compute node B Configuration: (Consider the new cpu pinning implementation is merged into Train) cpu_dedicated_set=0-3 (change to the new config option) 1. Consider that one instance say `test ` is created using flavor having old extra specs (hw:cpu_policy=dedicated, "aggregate_instance_extra_specs:pinned": "true") in Stein release and now upgraded Nova to Train with the above configuration. 2. Now when user will perform instance action say shelve/unshelve scheduler pre-filter will change the request_spec flavor extra spec from ``hw:cpu_policy`` to ``resources=PCPU:$<no. of cpus>`` which ultimately will return only compute node B from placement service. Here, we expect it should have retuned both Compute A and Compute B. 3. If user creates a new instance using old extra specs (hw:cpu_policy=dedicated, "aggregate_instance_extra_specs:pinned": "true") on Train release with the above configuration then it will return only compute node B from placement service where as it should have returned both compute Node A and B. Problem: As Compute node A is still configured to be used to boot instances with dedicated CPUs same behavior as Stein, it will not be returned by placement service due to the changes in the scheduler pre-filter logic. Propose changes: Earlier in the spec [2]: The online data migration was proposed to change flavor extra specs and image metadata properties of request_spec and instance object. Based on the instance host, we can get the NumaTopology of the host which will contain the new configuration options set on the compute host. Based on the NumaTopology of host, we can change instance and request_spec flavor extra specs. 1. Remove cpu_policy from extra specs 2. Add “resources:PCPU=<count>” in extra specs We can also change the flavor extra specs and image metadata properties of instance and request_spec object using the reshape functionality. Please give us your feedback on the proposed solution so that we can update specs accordingly. [1]: https://review.opendev.org/#/c/555081/28/specs/train/approved/cpu-resources… [2]: https://review.opendev.org/#/c/555081/23..28/specs/train/approved/cpu-resou… Thanks and Regards, -Bhagyashri Shewale- Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.

4 19