July 2019 - openstack-discuss - lists.openstack.org

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[Syntribos] Does anyone still use this?
by Gage Hugo 07 May '20

07 May '20

The Security SIG has recently been looking into updating several sites/documentation and one part we discussed the last couple meetings was the security tools listings[0]. One of the projects listed there, Syntribos, doesn't appear to have much activity[1] outside of zuul/python/docs maintenance and the idea of retiring the project was mentioned. However, there does seem to be an occasional bug-fix submitted, so we are sending this email out to see if anyone is still utilizing Syntribos. If so, please either respond here, reach out to us in the #openstack-security irc channel, or fill out the section in the Security SIG Agenda etherpad[2]. Thanks! [0] https://security.openstack.org/#security-tool-development [1] https://review.opendev.org/#/q/project:openstack/syntribos [2] https://etherpad.openstack.org/p/security-agenda

2 2

[openstack-ansible] dropping suse support
by Mohammed Naser 05 May '20

05 May '20

Hi everyone, I've been trying to avoid writing this email for the longest time ever. It's really painful to have to reach this state, but I think we've hit a point where we can no longer maintain support for SUSE within OpenStack Ansible. Unfortunately, with the recent layoffs, the OSA team has taken a huge hit in terms of contributors which means that there is far less contributors within the project. This means that we have less resources to go around and it forces us to focus on a more functional, reliable and well tested set of scenarios. Over the past few cycles, there has been effort to add SUSE support to OpenStack Ansible, during the time that we had maintainers, it was great and SUSE issues were being fixed promptly. In addition, due to the larger team at the time, we found ourselves having some extra time where we can help unbreak other gates. Jesse used to call this a "labour of love", which I admired at the time and hoped we continue to do as much as we can of. However, the lack of a committed maintainer for OpenSUSE has resulted in constantly failing jobs[1][2] (which were moved to non-voting, wasting CI resources as no one fixed them). In addition, it's causing several gate blocks for a few times with no one really finding the time to clean them up. We are resource constrained at this point and we need the resource to go towards making the small subset of supported features functional (i.e. CentOS/Ubuntu). We struggle with that enough, and there seems to be no deployers that are running SUSE in real life at the moment based on bugs submitted. With that, I propose that we drop SUSE support this cycle. If anyone would like to volunteer to maintain it, we can review that option, but that would require a serious commitment as we've had maintainers step off and it hurts the velocity of the project as no one can merge code anymore. ..Really wish I didn't have to write this email Mohammed [1]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct… [2]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct…

6 9

[kolla-ansible] migration
by Ignazio Cassano 02 Oct '19

02 Oct '19

Hello, Anyone have tried to migrate an existing openstack installation to kolla containers? Thanks Ignazio

5 16

[nova] critical bug around reload/upgrades
by Mohammed Naser 24 Aug '19

24 Aug '19

Hello: I've discussed this for quite sometime with Dan over IRC and a bit with Zane as well, but basically, Nova thinks that when it gets a reload (aka SIGHUP), nothing else has occurred. However, oslo.service actually calls stop(), reload() then start() again, which potentially kills all RPC. This has caused a pretty big issue in our gates and it also means that the whole idea behind 'reload nova-compute while upgrading to refresh info' concept is fundamentally broken. I tried to do some work on this here, however, I wasn't really able to get to the bottom of it. There seems to be a decision that needs to be taken in terms of .. do we change what reload() actually means in oslo_service (it actually is more like a restart, not a reload) or does Nova (and other projects) change their implementation in assuming what reload() does? https://review.openstack.org/#/c/641907/ This seems to have been floating around for a really long time, so I'd be happy to work with someone to find the fix (and we can totally test it inside OpenStack Ansible by reloading instead of restarting). Thanks! Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

4 4

[nova][ops] What should the compute service delete behavior be wrt resource providers with allocations?
by Matt Riedemann 22 Aug '19

22 Aug '19

Before [1] when deleting a compute service in the API we did not check to see if the compute service was hosting any instances and just blindly deleted the service and related compute_node(s) records which orphaned the resource provider(s) for those nodes. With [2] we built on that and would cleanup the (first [3]) compute node resource provider by first deleting any allocations for instances still on that host - which because of the check in [1] should be none - and then deleted the resource provider itself. [2] forgot about ironic where a single compute service can be managing multiple (hundreds or even thousands) of baremetal compute nodes so I wrote [3] to delete *all* resource providers for compute nodes tied to the service - again barring there being any instances running on the service because of the check added in [1]. What we've failed to realize until recently is that there are cases where deleting the resource provider can still fail because there are allocations we haven't cleaned up, namely: 1. Residual allocations for evacuated instances from a source host. 2. Allocations held by a migration record for an unconfirmed (or not yet complete) migration. Because the delete_resource_provider method isn't checking for those, we can get ResourceProviderInUse errors which are then ignored [4]. Since that error is ignored, we continue on to delete the compute service record [5], effectively orphaning the providers (which is what [2] was meant to fix). I have recreated the evacuate scenario in a functional test here [6]. The question is what should we do about the fix? I'm getting lost thinking about this in a vacuum so trying to get some others to help think about it. Clearly with [1] we said you shouldn't be able to delete a compute service that has instances on it because that corrupts our resource tracking system. If we extend that to any allocations held against providers for that compute service, then the fix might be as simple as not ignoring the ResourceProviderInUse error and fail if we can't delete the provider(s). The question I'm struggling with is what does an operator do for the two cases mentioned above, not-yet-complete migrations and evacuated instances? For migrations, that seems pretty simple - wait for the migration to complete and confirm it (reverting a cold migration or resize would put the instance back on the compute service host you're trying to delete). The nastier thing is the allocations tied to an evacuated instance since those don't get cleaned up until the compute service is restarted [7]. If the operator never intends on restarting that compute service and just wants to clear the data, then they have to manually delete the allocations for the resource providers associated with that host before they can delete the compute service, which kind of sucks. What are our options? 1. Don't delete the compute service if we can't cleanup all resource providers - make sure to not orphan any providers. Manual cleanup may be necessary by the operator. 2. Change delete_resource_provider cascade=True logic to remove all allocations for the provider before deleting it, i.e. for not-yet-complete migrations and evacuated instances. For the evacuated instance allocations this is likely OK since restarting the source compute service is going to do that cleanup anyway. Also, if you delete the source compute service during a migration, confirming or reverting the resize later will likely fail since we'd be casting to something that is gone (and we'd orphan those allocations). Maybe we need a functional recreate test for the unconfirmed migration scenario before deciding on this? 3. Other things I'm not thinking of? Should we add a force parameter to the API to allow the operator to forcefully delete (#2 above) if #1 fails? Force parameters are hacky and usually seem to cause more problems than they solve, but it does put the control in the operators hands. If we did remove allocations for an instance when deleting it's compute service host, the operator should be able to get them back by running the "nova-manage placement heal_allocations" CLI - assuming they restart the compute service on that host. This would have to be tested of course. Help me Obi-Wan Kenobi. You're my only hope. [1] https://review.opendev.org/#/q/I0bd63b655ad3d3d39af8d15c781ce0a45efc8e3a [2] https://review.opendev.org/#/q/I7b8622b178d5043ed1556d7bdceaf60f47e5ac80 [3] https://review.opendev.org/#/c/657016/ [4] https://github.com/openstack/nova/blob/cb0cfc90e1e03e82c42187ec60f46fb8fd59… [5] https://github.com/openstack/nova/blob/cb0cfc90e1e03e82c42187ec60f46fb8fd59… [6] https://review.opendev.org/#/c/663737/ [7] https://github.com/openstack/nova/blob/cb0cfc90e1e03e82c42187ec60f46fb8fd59… -- Thanks, Matt

6 16

[storyboard] email notification on stories/tasks of subscribed projects
by Akihiro Motoki 22 Aug '19

22 Aug '19

Hi, Is there a way to get email notification on stories/tasks of subscribed projects in storyboard? In launchpad, I configure launchpad bug notifications of all bug changes in interested projects. After migration to storyboard (for example, openstackclient), I failed to find a similar way. Thanks, Akihiro Motoki (irc: amotoki)

5 8

[qa][openstackclient] Debugging devstack slowness
by Clark Boylan 15 Aug '19

15 Aug '19

Today I have been digging into devstack runtime costs to help Donny Davis understand why tempest jobs sometimes timeout on the FortNebula cloud. One thing I discovered was that the keystone user, group, project, role, and domain setup [0] can take many minutes [1][2] (in the examples here almost 5). I've rewritten create_keystone_accounts to be a python tool [3] and get the runtime for that subset of setup from ~100s to ~9s [4]. I imagine that if we applied this to the other create_X_accounts functions we would see similar results. I think this is so much faster because we avoid repeated costs in openstack client including: python process startup, pkg_resource disk scanning to find entrypoints, and needing to convert names to IDs via the API every time osc is run. Given my change shows this can be so much quicker is there any interest in modifying devstack to be faster here? And if so what do we think an appropriate approach would be? [0] https://opendev.org/openstack/devstack/src/commit/6aeaceb0c4ef078d028fb6605… [1] http://logs.openstack.org/05/672805/4/check/tempest-full/14f3211/job-output… [2] http://logs.openstack.org/05/672805/4/check/tempest-full/14f3211/job-output… [3] https://review.opendev.org/#/c/673108/ [4] http://logs.openstack.org/08/673108/6/check/devstack-xenial/a4107d0/job-out… Note the jobs compared above all ran on rax-dfw. Clark

11 23

[all] [tc] General questions on PDF community goal
by Akihiro Motoki 15 Aug '19

15 Aug '19

Hi, I have a couple of general questions on PDF community goal. What is the criteria of completing the PDF community goal? Is it okay if we publish a PDF deliverable anyway? During working on it, I hit the following questions. We already reached Train-2 milestone, so I think it is the time to clarify the detail criteria of completing the community goal. - Where should the generated PDF file to be located and What is the recommended PDF file name? <top directory>/<repository-name>.pdf? index.pdf? Any path and any file is okay? - Do we create a link to the PDF file from index.html (per-project top page)? If needed, perhaps this would be a thing covered by openstackdocstheme. Otherwise, how can normal consumers know PDF documents? - Which repositories need to provide PDF version of documents? My understanding (amotoki) is that repositories with 'publish-openstack-docs-pti' should publish PDF doc. right? - Do we need PDF version of release notes? - Do we need PDF version of API reference? I see no coordination efforts recently and am afraid that individual projects cannot decide whether patches to their repositories are okay to merge. Thanks, Akihiro Motoki (irc: amotoki)

3 2

[tripleo][openstack-ansible] Integrating ansible-role-collect-logs in OSA
by Chandan kumar 12 Aug '19

12 Aug '19

Hello, With os_tempest project, TripleO and Openstack Ansible projects started collaborating together to reuse the tools developed by each other to avoid duplicates and enable more collaboration. During Denver Train 2019 PTG, we decided to unifying the CI logs for both the projects by providing a unified experience to the developers while browsing the CI logs making sure we have a similar logs structure for both the projects, so that one can easily navigate through the logs without scratching the heads and also the logs tree structure should say what logs are present where. In TripleO, we have ansible-role-collect-logs[1.] role for the same and in OSA we have logs_collect.sh[2.] script for the same. But once the logs gets collected at each other projects, It is very hard to navigate and find out where is the respective files. A little about ansible-role-collect-logs roles: * A role for aggregating logs from different nodes * Provide a list for files to collect at one places in playbook [3] * Hyperlink the logs with description about log files * Once logs collection is done, it pushes to a particular log server For example, tempest.html or stestr_results.html is the common file for viewing tempest results. In TripleO, we keep it under logs folder but in OSA logs/openstack/aio1-utility/stestr_results.html. If a new user contributes to another project, he tries to follow the same pattern for find logs as he seen in current project. By Keeping the same structure at all places It would be easier. So moving ahead what we are going to do: * Refactor collect-logs role to pass defaults list of files at one place * Pass the list of different logs files based on deployment tools * Put system/containers related commands at one place * Replace the collect_logs.sh script with playbook in OSA and replace it. Thanks for reading, We are looking for the comments on the above suggestion. Links: [1.] https://opendev.org/openstack/ansible-role-collect-logs/ [2.] https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/l… [3.] https://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/… Thanks, Chandan Kumar

4 4

[nova-lxd] no one uses nova-lxd?
by Adrian Andreias 10 Aug '19

10 Aug '19

Hey, We were planing to migrate some thousand containers from OpenVZ 6 to Nova-LXD this fall and I know at least one company with the same plans. I read the message about current team retiring from the project. Unfortunately we don't have the manpower to invest heavily in the project development. We would however be able to allocate a few hours per month, at least for bug fixing. So I'm curios if there are organizations using or planning to use Nova-LXD in production and they have the know-how and time to contribute. It would be a pity if the the project dies. Cheers! - Adrian Andreias https://fleio.com

5 5

[ironic][ops] Taking ironic nodes out of production
by Arne Wiebalck 09 Aug '19

09 Aug '19

Dear all, One of the discussions at the PTG in Denver raised the need for a mechanism to take ironic nodes out of production (a task for which the currently available 'maintenance' flag does not seem appropriate [1]). The use case there is an unhealthy physical node in state 'active', i.e. associated with an instance. The request is then to enable an admin to mark such a node as 'faulty' or 'in quarantine' with the aim of not returning the node to the pool of available nodes once the hosted instance is deleted. A very similar use case which came up independently is node retirement: it should be possible to mark nodes ('active' or not) as being 'up for retirement' to prepare the eventual removal from ironic. As in the example above, ('active') nodes marked this way should not become eligible for instance scheduling again, but automatic cleaning, for instance, should still be possible. In an effort to cover these use cases by a more general "quarantine/retirement" feature: - are there additional use cases which could profit from such a "take a node out of service" mechanism? - would these use cases put additional constraints on how the feature should look like (e.g.: "should not prevent cleaning") - are there other characteristics such a feature should have (e.g.: "finding these nodes should be supported by the cli") Let me know if you have any thoughts on this. Cheers, Arne [1] https://etherpad.openstack.org/p/DEN-train-ironic-ptg, l. 360

5 7

[kolla][nova][cinder] Got Gateway-Timeout error on VM evacuation if it has volume attached.
by Eddie Yen 09 Aug '19

09 Aug '19

Hi everyone, I met an issue when try to evacuate host. The platform is stable/rocky and using kolla-ansible to deploy. And all storage backends are connected to Ceph. Before I try to evacuate host, the source host had about 24 VMs running. When I shutdown the node and execute evacuation, there're few VMs failed. The error code is 504. Strange is those VMs are all attach its own volume. Then I check nova-compute log, a detailed error has pasted at below link; https://pastebin.com/uaE7YrP1 Does anyone have any experience with this? I googled but no enough information about this. Thanks!

5 14

[tripleo] Proposing Kevin Carter (cloudnull) for tripleo-ansible core
by Alex Schultz 08 Aug '19

08 Aug '19

Hey folks, I'd like to propose Kevin as a core for the tripleo-ansible repo (e.g. tripleo-ansible-core). He has made excellent progress centralizing our ansible roles and improving the testing around them. Please reply with your approval/objections. If there are no objections, we'll add him to tripleo-ansible-core next Friday Aug 2, 2019. Thanks, -Alex

11 11

[neutron] OpenvSwitch firewall sctp getting dropped
by thuanlk＠viettel.com.vn 08 Aug '19

08 Aug '19

I have installed Openstack Queens on CentOs 7 with OvS and I recently used the native openvswitch firewall to implement SecusiryGroup. The native OvS firewall seems to work just fine with TCP/UDP traffic but it does not forward any SCTP traffic going to the VMs no matter how I change the security groups, But it run if i disable port security completely or use iptables_hybrid firewall driver. What do I have to do to allow SCTP packets to reach the VMs?

4 7

Slow instance launch times due to RabbitMQ
by Grant Morley 08 Aug '19

08 Aug '19

Hi all, We are randomly seeing slow instance launch / deletion times and it appears to be because of RabbitMQ. We are seeing a lot of these messages in the logs for Nova and Neutron: ERROR oslo.messaging._drivers.impl_rabbit [-] [f4ab3ca0-b837-4962-95ef-dfd7d60686b6] AMQP server on 10.6.2.212:5671 is unreachable: Too many heartbeats missed. Trying again in 1 seconds. Client port: 37098: ConnectionForced: Too many heartbeats missed The RabbitMQ cluster isn't under high load and I am not seeing any packets drop over the network when I do some tracing. We are only running 15 compute nodes currently and have >1000 instances so it isn't a large deployment. Are there any good configuration tweaks for RabbitMQ running on OpenStack Queens? Many Thanks, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com <https://www.civo.com/>| Signup for an account! <https://www.civo.com/signup>

7 9

[edge][all] Edge Hacking Days
by Ildiko Vancsa 06 Aug '19

06 Aug '19

Hi, I’m reaching out with an attempt to organize hacking days to work on edge related tasks. The idea is to get together remotely on IRC/Zoom or any other platform that supports remote communication and work on items like building and testing our reference architectures or work on some project specific items like in Keystone or Ironic. Here are Doodle polls for the next three months: August: https://doodle.com/poll/ucfc9w7iewe6gdp4 September: https://doodle.com/poll/3cyqxzr9vd82pwtr October: https://doodle.com/poll/6nzziuihs65hwt7b Please mark any day when you have some availability to dedicate to hack even if it’s not a full day. Please let me know if you have any questions. As a reminder you can find the edge computing group’s resources and information about latest activities here: https://wiki.openstack.org/wiki/Edge_Computing_Group Thanks and Best Regards, Ildikó (IRC: ildikov)

1 1

[ironic] Moving to office hours as opposed to weekly meetings for the next month
by Julia Kreger 06 Aug '19

06 Aug '19

Greetings Everyone! This week, during the weekly meeting, we seemed to reach consensus that we would try taking a break from meetings[0] and moving to orienting around using the mailing list[1] and our etherpad "whiteboard" [2]. With this, we're going to want to re-evaluate in about a month. I suspect it would be a good time for us to have a "mid-cycle" style set of topical calls. I've gone ahead and created a poll to try and identify a couple days that might be ideal for contributors[3]. But in the mean time, we want to ensure that we have some times for office hours. The suggestion was also made during this week's meeting that we may want to make the office hours window a little larger to enable more discussion. So when will we have office hours? ---------------------------------- Ideally we'll start with two time windows. One to provide coverage to US and Europe friendly time zones, and another for APAC contributors. * I think 2-4 PM UTC on Mondays would be ideal. This translates to 7-9 AM US-Pacific or 10 AM to 12 PM US-Eastern. * We need to determine a time window that would be ideal for APAC contributors. I've created a poll to help facilitate discussion[4]. So what is Office Hours? ------------------------ Office hours are a time window when we expect some contributors to be on IRC and able to partake in higher bandwidth discussions. These times are not absolute. They can change and evolve, and that is the most important thing for us to keep in mind. -- If there are any questions, Please let me know! Otherwise I'll send a summary email out on next Monday. -Julia [0]: http://eavesdrop.openstack.org/meetings/ironic/2019/ironic.2019-07-01-15.00… [1]: http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007038.html [2]: https://etherpad.openstack.org/p/IronicWhiteBoard [3]: https://doodle.com/poll/652gzta6svsda343 [4]: https://doodle.com/poll/2ta5vbskytpntmgv

2 4

Re: [telemetry] Team meeting agenda for tomorrow
by Joseph Davis 06 Aug '19

06 Aug '19

On 5/8/19 7:12 AM, openstack-discuss-request(a)lists.openstack.org wrote: > Hello Trinh, > Where does the meeting happen? Will it be via IRC Telemetry channel? Or, in > the Etherpad (https://etherpad.openstack.org/p/telemetry-meeting-agenda)? I > would like to discuss and understand a bit better the context behind > the Telemetry > events deprecation. Unfortunately, I have a conflict at that time and will not be able to attend. I do have a little bit of context on the Events deprecation to share. First, you will note the commit message from the commit [0] when Events were deprecated: " Deprecate event subsystem This subsystem has never been finished and is not maintained. Deprecate it for future removal. " I got the impression from jd at the time that there were a number of features in Telemetry, including Panko, that were not really "finished" and that the engineers who had worked on them had moved on to other things, so the features had become unsupported. In late 2018 there was an effort to clean up things that were not well maintained or didn't fit the direction of Telemetry. See also: https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/ Events is one feature that often gets requested, but the use cases and demand for it are not expressed strongly or well understood by most people. If the Telemetry project has demand to de-deprecate Event handling (including Panko), I'd suggest a review of the requirements for event handling and possibly choosing a champion for maintaining the Panko service. Also note: over in Monasca we have a spec [1] for handling Events ingestion which I hope we will be completing in Train. Contributions and comments welcome. :) joseph [0] https://github.com/openstack/ceilometer/commit/8a0245a5b3e1357d35ad6653be37… [1] https://github.com/openstack/monasca-specs/blob/master/specs/stein/approved… > > On Wed, May 8, 2019 at 12:19 AM Trinh Nguyen<dangtrinhnt(a)gmail.com> wrote: > >> Hi team, >> >> As planned, we will have a team meeting at 02:00 UTC, May 9th on >> #openstack-telemetry to discuss what we gonna do for the next milestone >> (Train-1) and continue what we left off from the last meeting. >> >> I put here [1] the agenda thinking that it should be fine for an hour >> meeting. If you have anything to talk about, please put it there too. >> >> [1]https://etherpad.openstack.org/p/telemetry-meeting-agenda >> >> >> Bests, >> >> -- >> *Trinh Nguyen* >> *www.edlab.xyz<https://www.edlab.xyz>* >> >>

14 25

stestr Python 2 Support
by Matthew Treinish 05 Aug '19

05 Aug '19

Hi Everyone, I just wanted to send a quick update about the state of python 2 support in stestr, since OpenStack is the largest user of the project. With the recent release of stestr 2.4.0 we've officially deprecated the python 2.7 support in stestr. It will emit a DeprecationWarning whenever it's run from the CLI with python 2.7 now. The plan (which is not set in stone) is that we will be pushing a 3.0.0 release that removes the python 2 support and compat code from stestr sometime in early 2020 (definitely after the Python 2.7 EoL on Jan. 1st). I don't believe this conflicts with the current Python version support plans in OpenStack [1] but just wanted to make sure people were aware so that there are no surprises when stestr stops working with Python 2 in 3.0.0. Thanks, -Matt Treinish [1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecatio…

3 3

[Airship-Seaworthy] Deployment of Airship-Seaworthy on Virtual Environment
by Anirudh Gupta 05 Aug '19

05 Aug '19

Hi Team, We want to test Production Ready Airship-Seaworthy in our virtual environment The link followed is https://airship-treasuremap.readthedocs.io/en/latest/seaworthy.html<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairship-t…> As per the document we need 6 DELL R720xd bare-metal servers: 3 control, and 3 compute nodes. But we need to deploy our setup on Virtual Environment. Does Airship-Seaworthy support Installation on Virtual Environment? We have 2 Rack Servers with Dual-CPU Intel® Xeon® E5 26xx with 16 cores each and 128 GB RAM. Is it possible that we can create Virtual Machines on them and set up the complete environment. In that case, what possible infrastructure do we require for setting up the complete setup. Looking forward for your response. Regards अनिरुद्ध गुप्ता (वरिष्ठ अभियंता) Hughes Systique Corporation D-23,24 Infocity II, Sector 33, Gurugram, Haryana 122001 DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus.

3 5

[manila] Enabling CephFS snapshot support by default
by Goutham Pacha Ravi 05 Aug '19

05 Aug '19

Hi Zorillas and interested parties, (Copying a couple of folks explicitly as they may not be part of openstack-discuss) Manila's CephFS driver has a configuration option "cephfs_enable_snapshots" that administrators can toggle in manila.conf to allow project users to take snapshots of their shares. [1][2] This option has defaulted to False, since the time the CephFS driver was introduced (Mitaka release of OpenStack Manila) IIUC, CephFS snapshots have been "stable" since the Mimic release of CephFS, which debuted in June 2018. [3] Since then, ceph deployers/administrators don't have to toggle anything on the backend to enable snapshots. So, can we consider changing the default value of the config opt in manila to "True" in the Train release? This will allow the driver to report the capability "snapshot_support" as enabled, and makes life easier for OpenStack deployers/administrators. I would also love to deprecate this option and remove it in a future release, since it isn't really necessary, read on: Users cannot arbitrarily take snapshots in manila, there are a couple of things that can be used to prevent them: Quotas and Share Types. Unless users create their shares using a share type that explicitly requests "snapshot_support", users can't snapshot their shares. So, we're covered by the service's API in case someone is afraid this has an impact on someone in the wild using newer versions of Manila with older versions of Ceph, where snapshots were unstable. Thoughts? [1] https://opendev.org/openstack/manila/src/commit/721bb70a81b80bf065705e8790b… [2] https://docs.openstack.org/manila/queens/admin/cephfs_driver.html#enabling-… [3] http://docs.ceph.com/docs/mimic/dev/cephfs-snapshots/

2 1

[openstack-dev][magnum] Project updates
by feilong 05 Aug '19

05 Aug '19

Hi all, Now we're very close to the milestone-2 of Train release. I'd like to take this opportunity to update the project status of Magnum. So far, we have done some great work in this cycle which make Magnum to achieve to a higher level. The first feature I'd like to highlight is the k8s version rolling upgrade. With this feature, user can upgrade their k8s version with zero downtime for their application. Another one is auto scaling, now Magnum can support k8s cluster auto scaling by leveraging the Cluster Autoscaler (there is a OpenStack Magnum driver contributed by CERN team). Besides, Magnum now can support auto repair, aka auto healing, by using a combination of Draino, Cluster Autoscaler and Node Problem Detector. Except above work we have done, there are still a lot of cool features coming: 1. The first one I'd like to mention is the boot volume support. With this feature, user can boot k8s cluster node from volume and user can even set the volume type which is super useful if the cloud provider can support high performance storage, e.g. NVME. And a new useful label named "etcd_volume_type" will be introduced by the way so that user can set the volume type for etcd volume to get a great performance for etcd cluster. 2. Private cluster. With this feature, user can set the network, subnet and floating IP when creating cluster with a public cluster template, which is very handy when user want to use public templates but still want to create the cluster in their existing network. 3. Another super fancy feature is introducing a new auto healing controller except the Draino+CA+NPD combination. Recently, we (Catalyst Cloud) contributed our Magnum auto healer[1] to Cloud-Provider-Openstack repo and we're trying to introduce it to Magnum as another auto healing controller, it would be also very useful for end user to get a zero downtime k8s cluster. As you can see, there are quite a lot of exciting features are being introduced into Magnum, however all of them are done by such a small team. What a cool team! As you know, Kubernetes is still evolving very fast, so we're very keen to get more help from the community. We're working hard to add some new core members to our existing team, but we do need more. In short, if you're interested in provisioning Kubernetes on OpenStack, please try Magnum and feel free to pop up on #openstack-containers for any question. Cheers. -- Cheers & Best regards, Feilong Wang (王飞龙) ------------------------------------------------------ Senior Cloud Software Engineer Tel: +64-48032246 Email: flwang(a)catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington ------------------------------------------------------

3 4

[release] (a bit belated) Release countdown for week R-11, July 29 - August 2
by Kendall Nelson 02 Aug '19

02 Aug '19

Hello Everyone! Development Focus ----------------- We are now past the Train-2 milestone, and entering the last development phase of the cycle. Teams should be focused on implementing planned work for the cycle. Now is a good time to review those plans and reprioritize anything if needed based on the what progress has been made and what looks realistic to complete in the next few weeks. General Information ------------------- The following cycle-with-intermediary deliverables have not done any intermediary release yet during this cycle. The cycle-with-rc release model is more suited for deliverables that plan to be released only once per cycle. As a result, we will be proposing to change the release model for the following deliverables: blazar-dashboard cloudkitty-dashboard ec2-api freezer-web-ui freezer heat-agents heat-dashboard ironic-ui karbor-dashboard karbor kuryr-kubernetes magnum-ui manila-ui masakari-dashboard monasca-agent monasca-api monasca-ceilometer monasca-events-api monasca-kibana-plugin monasca-log-api monasca-notification monasca-persister monasca-thresh monasca-transform monasca-ui murano-agent networking-baremetal networking-generic-switch networking-hyperv neutron-fwaas-dashboard neutron-vpnaas-dashboard requirements sahara-extra senlin-dashboard solum-dashboard tacker-horizon tricircle vitrage-dashboard vitrage watcher-dashboard PTLs and release liaisons for each of those deliverables can either +1 the release model change when we get them pushed, or propose an intermediary release for that deliverable. In absence of answer by the start of R-10 week we'll consider that the switch to cycle-with-rc is preferable. Upcoming Deadlines & Dates -------------------------- Non-client library freeze: September 05 (R-6 week) Client library freeze: September 12 (R-5 week) Train-3 milestone: September 12 (R-5 week) -Kendall (diablo_rojo) + the Release Management Team

6 7

[masakari] how to install masakari on centos 7
by Vu Tan 01 Aug '19

01 Aug '19

Vu Tan <vungoctan252(a)gmail.com> 10:30 AM (35 minutes ago) to openstack-discuss Sorry, I resend this email because I realized that I lacked of prefix on this email's subject Hi, I would like to use Masakari and I'm having trouble finding a step by step or other documentation to get started with. Which part should be installed on controller, which is should be on compute, and what is the prerequisite to install masakari, I have installed corosync and pacemaker on compute and controller nodes, , what else do I need to do ? step I have done so far: - installed corosync/pacemaker - install masakari on compute node on this github repo: https://github.com/openstack/masakari - add masakari in to mariadb here is my configuration file of masakari.conf, do you mind to take a look at it, if I have misconfigured anything? [DEFAULT] enabled_apis = masakari_api # Enable to specify listening IP other than default masakari_api_listen = controller # Enable to specify port other than default masakari_api_listen_port = 15868 debug = False auth_strategy=keystone [wsgi] # The paste configuration file path api_paste_config = /etc/masakari/api-paste.ini [keystone_authtoken] www_authenticate_uri = http://controller:5000 auth_url = http://controller:5000 auth_type = password project_domain_id = default user_domain_id = default project_name = service username = masakari password = P@ssword [database] connection = mysql+pymysql://masakari:P@ssword@controller/masakari

4 15

[rdo] [packstack] failing to setup Stein with Openvswitch/VXLAN networking
by Bernd Bausch 01 Aug '19

01 Aug '19

Trying to set up a Stein cloud with Packstack. I want the Openvswitch mech driver and VXLAN type driver. A few weeks ago, the following invocation was successful: |sudo packstack --debug --allinone --default-password pw \ --os-neutron-ovs-bridge-interfaces=br-ex:eth0 \ --os-neutron-ml2-tenant-network-types=vxlan \ --os-neutron-ml2-mechanism-drivers=openvswitch \ --os-neutron-ml2-type-drivers=vxlan,flat \ --os-neutron-l2-agent=openvswitch \ --provision-demo-floatrange=10.1.1.0/24\ --provision-demo-allocation-pools '["start=10.1.1.10,end=10.1.1.50"]'\ --os-heat-install=y --os-heat-cfn-install=y||| Now, it fails during network setup. My network connection to the Packstack server is severed, and it turns out that its only network interface /eth0 /has no IP address and is down. No bridge exists. In the /network.pp.finished /file, I find various /ovs-vsctl /commands including /add-br/, and a command /ifdown eth0 /which fails with exit code 1 (no error message from the /ifdown /command is logged). *Can somebody recommend the options required to successfully deploy a Stein cloud based on the Openvswitch and VXLAN drivers?* Thanks much, Bernd || ||

2 3

[dev][nova][glance][cinder] introducing 'compressed' image container_format
by Ma, Zheng 01 Aug '19

01 Aug '19

Hello Nova developers, Cinder is working on a feature that will allow it to compress volumes uploaded to Glance [0,1]. We are proposing a new container_format for Glance named 'compressed' [2]. The idea is that a consumer of a 'compressed' image would read the file to determine the kind of compression and take appropriate action (either successfully decompress to find a payload that matches the image's disk_format, or fail with 'unsupported container_format'). Our preliminary testing with a raw image with container_format == compressed, disk_format == raw is that Nova boots a VM from the image but the VM appears to be unusable. Nova seems taking it as a real RAW image as disk_format indicates. So we want to open a discussion about handling this. Thanks! [0] cinder spec patch https://review.opendev.org/#/c/652275/ [1] cinder code implementation https://review.opendev.org/#/c/668825 [2] glance spec patch https://review.opendev.org/#/c/670454/

2 1

[all] Pycharm Licences
by ʂʍɒρƞįł Ҟưȴķɒʁʉɨ 31 Jul '19

31 Jul '19

I have renewed the Pycharm licenses for community contributors until July 29, 2020. For everyone who is using it will be updated automatically. Please do not request again for renewal. Best Regards, Swapnil

2 1

[swift] Shanghai PTG attendance and etherpad
by tim＠swiftstack.com 31 Jul '19

31 Jul '19

你好! Most teams sent out emails two or three weeks ago with etherpads to start estimating headcount and coming up with discussion topics. I was a little slow, but here's Swift's: https://etherpad.openstack.org/p/swift-ptg-shanghai We're thinking of doing something a little different from our previous PTGs and hackathons and want to offer a kind of extended onboarding session. If you're new to Swift, this is a great opportunity to learn more about - how Swift works, - how to write applications against Swift, - how to deploy and operate your own Swift cluster, or - how to do upstream development for Swift. Interested? We'd love to hear from you and meet you! If there are any other aspects of Swift you'd like to learn about or dive deeper on, feel free to add them! I'll try to develop some materials based on people's level of interest so it won't all be off-the-cuff :-) Of course, there will also be more-traditional design discussions going on. (I don't think I could stop this if I tried -- it seems *any time* there are three or more Swift contributors around, there are going to be design discussions happening!) If you know you want to talk about some upcoming features or other work, please update the Development Topics at the bottom. 在上海见到您们! Tim

1 0

[openstack-ansible] office hours update
by Mohammed Naser 31 Jul '19

31 Jul '19

Hey everyone, Here is the update of what happened during the OpenStack Ansible office hours this week. We thought there was an issue with the logic that determines nova_cinder_rbd_inuse running on localhost and setting the fact only on localhost (and not the host that’s it’s running it on). So the variable ‘nova_cinder_rbd_inuse’ was always set to false on these hosts. The issue was a missing rbd configuration in nova.conf and that issue has been fixed. Thanks for tuning in and thanks for Tiffanie for taking care of the basic meeting logistics! Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

1 0

[tripleo][charms][helm][kolla][ansible][puppet][chef] Deployment tools capabilities v0.1.0
by Thierry Carrez 31 Jul '19

31 Jul '19

Hi, deployment tools teams, As mentioned here last month[1], we are working to improve the information present on the deployment tools pages on the OpenStack website, and we need your help! After the Forum session on this topic in Denver[2], a workgroup worked on producing a set of base capabilities that can be asserted by the various deployment tools we have. You can find version 0.1.0 of those capabilities here: https://opendev.org/osf/openstack-map/src/branch/master/deployment_tools_ca… As an example, I pushed a change that makes every deployment tool assert the capability to deploy keystone ("components:keystone" tag) at: https://review.opendev.org/#/c/669648/ Now it's your turn. Please have a look at the list of the capabilities above, and propose a change to add those that are relevant to your deployment tool in the following file: https://opendev.org/osf/openstack-map/src/branch/master/deployment_tools.ya… Capabilities are all of the form "category:tag" (components:keystone, starts-from:os-installed, technology:puppet...). Once all deployment projects have completed that task, we'll add the capabilities to the rendered page on the website and allow for basic searching for tools with matching capability. Now, capabilities go only so far in describing your deployment tool. I also encourage you to improve in the same file the "desc" field: that one is directly displayed on the site.Uuse it to describe in more details how your deployment tool actually works and what makes it unique, beyond basic capabilities tags. Please feel free to use this thread (or personal email) if you have questions on this. And thanks in advance for your help! [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006964.html [2] https://etherpad.openstack.org/p/DEN-deployment-tools-capabilities -- Thierry Carrez (ttx)

1 1

Re: [User-committee] [all][uc] Train UC Election Season
by Ian Y. Choi 31 Jul '19

31 Jul '19

+ openstack-discuss mailing list for broader announce of Train UC Election :) - Ed & Ian Ed Leafe wrote on 7/31/2019 11:20 PM: > Hello All! > > The User Committee (UC) election is coming! > > The OpenStack User Committee [1] is one of the governing bodies of the OpenStack project to serve the user community of OpenStack, representing them to the broader community. > > If you are not aware of UC election, please see the details first: > > Election details: > https://governance.openstack.org/uc/reference/uc-election-aug2019.html > > Please read the stipulations and timelines for candidates and electorate contained in this governance documentation. Note that the nomination period begins soon! (August 5). > > There will be further announcements posted to the mailing list as action is required from the electorate or candidates. This email is for information purposes only. > > If you have any questions which you feel affect others please reply to this email thread. > > If you have any questions that you which to discuss in private please email any of the election officials [2] so that we may address your concerns. > > Thank you, > > - Ed & Ian > > [1] https://governance.openstack.org/uc/reference/charter.html > [2] https://governance.openstack.org/uc/reference/uc-election-aug2019.html#offi… > > > _______________________________________________ > User-committee mailing list > User-committee(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee

1 0

[dev][keystone][keystonemiddleware] Keystone CADF notifications for rabbitmq
by Ondřej Duchoň 31 Jul '19

31 Jul '19

Hello, I am trying to make to work CADF notifying for nova_api and other modules into rabbitmq. I tried to use keystonemiddleware ( https://docs.openstack.org/keystonemiddleware/queens/audit.html) but no notifications appeared. The only thing that is sending CADF notifications into rabbitmq is keystone. I assume that is because keystone has its own notifier. (keystone/notifications.py) Do you have some idea about this problem? Which approach is correct in this case? Thank you for your help, Ondrej

2 1

[kolla] cancelling meeting today
by Michał Nasiadka 31 Jul '19

31 Jul '19

Hello, Since there are no hot topics and the PTL/core team are unavailable for the meeting - it will be cancelled. See you next week on #openstack-kolla -- Michał Nasiadka mnasiadka(a)gmail.com

1 0

Magnum Kubernetes openstack-cloud-controller-manager unable not resolve master node by DNS
by Pawel Konczalski 31 Jul '19

31 Jul '19

Hi, i try to deploy a Kubernetes cluster with OpenStack Magnum but the openstack-cloud-controller-manager pod fails to resolve the master node hostname. Does magnum require further parameter to configure the DNS names of the master and minions? DNS resolution in the VMs works fine. Currently there is no Designate installed in the OpenStack setup. openstack coe cluster template create kubernetes-cluster-template1 \ --image Fedora-AtomicHost-29-20190429.0.x86_64 \ --external-network public \ --dns-nameserver 8.8.8.8 \ --master-flavor m1.kubernetes \ --flavor m1.kubernetes \ --coe kubernetes \ --volume-driver cinder \ --network-driver flannel \ --docker-volume-size 25 openstack coe cluster create kubernetes-cluster1 \ --cluster-template kubernetes-cluster-template1 \ --master-count 1 \ --node-count 2 \ --keypair mykey # kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE kube-system coredns-78df4bf8ff-mjp2c 0/1 Pending 0 36m <none> <none> <none> kube-system heapster-74f98f6489-tgtzl 0/1 Pending 0 36m <none> <none> <none> kube-system kube-dns-autoscaler-986c49747-wrvz4 0/1 Pending 0 36m <none> <none> <none> kube-system kubernetes-dashboard-54cb7b5997-sk5pj 0/1 Pending 0 36m <none> <none> <none> kube-system openstack-cloud-controller-manager-dgk64 0/1 CrashLoopBackOff 11 36m <none> kubernetes-cluster1-vulg5fz6hg2n-master-0 <none> # kubectl -n kube-system logs openstack-cloud-controller-manager-dgk64 Error from server: Get https://kubernetes-cluster1-vulg5fz6hg2n-master-0:10250/containerLogs/kube-…: dial tcp: lookup kubernetes-cluster1-vulg5fz6hg2n-master-0 on 8.8.8.8:53: no such host BR Pawel

2 3

[loci][openstack-helm][tc] Proposal to fold Loci into OpenStack-Helm
by Chris Hoge 31 Jul '19

31 Jul '19

In the interest of consolidating workload, I'm proposing that we roll the Loci project under the governance of OpenStack Helm. Loci is a fairly low-bandwidth project, with the primary consumer of it being OpenStack Helm. By rolling Loci into OSH, we will be able focus shared efforts and increase review velocity for the Loci project. The architecture of OpenStack Helm allows it to be container build-system agnostic, and similarly the architecture of Loci allows it to be deployment tooling agnostic. Neither of these goals will change, as there are users of both projects that aren't using the other. However, as they are the largest consumers for each other's work, this is a natural pairing and both project stand to benefit from a tighter governance coupling. We already share a good proportion of the development team and frequently plan PTG sessions to allow for collaboration. If the community agrees with this approach, we would like to finalize the governance change before the next development cycle to allow for a graceful transition. Thanks, Chris Hoge Loci PTL

2 1

[scientific-sig][cloudkitty][monasca] SIG IRC meeting today - accounting using Monasca and CloudKitty
by Stig Telfer 31 Jul '19

31 Jul '19

Hi All - We have an IRC meeting today at 1100 UTC in channel #openstack-meeting. Everyone is welcome. Today’s agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_July_31st_2019 <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_July_31st_2019> Today we are looking to gather data on accounting, chargeback and billing, and we have a report on recent investigations using Monasca and CloudKitty together for this purpose. Cheers, Stig

1 0

[mistral] Office hour in ~30 min
by Renat Akhmerov 31 Jul '19

31 Jul '19

Hi, We’ll have an office hour at 8.00 UTC today (in ~30 min) so you’re welcome to join and chat with us. Thanks Renat Akhmerov @Nokia

1 0

[ops][keystone] Does anyone use external auth?
by Colleen Murphy 30 Jul '19

30 Jul '19

Currently, one of the default auth methods for keystone is 'external', meaning keystone offloads authentication to an HTTPD auth module like mod_ssl or mod_auth_kerb and gets the user's identity from the REMOTE_USER variable passed in by the web server: https://docs.openstack.org/keystone/latest/admin/external-authentication.ht… The 'external' auth method existed before federation. The biggest problem with external auth now is that it is effectively single-domain, there's no way to parse anything besides a user identifier from the REMOTE_USER variable, and keystone is barreling full steam ahead to a multidomain world. The 'external' auth method conflicts with the 'mapped' auth method as mentioned in the "Caution" notice in the above document for the same reason. Moreover, we should be able to achieve the same behavior with just federation, e.g. you can create a federated IdP representing your SSL CA, and continue to use mod_ssl with a mapping to properly parse all the attributes coming in from the auth module. We'd like to start discouraging, deprecating, and removing external auth in keystone. So our question to operators is: are you currently using external auth? If so, which HTTPD auth modules are you using? And is it a use case that we can't support with federated auth? Colleen (cmurphy)

2 2

[cinder] lvm_mirrors option seems to be broken
by Mark Lehrer 30 Jul '19

30 Jul '19

I have 4 fast nvme targets that I want to combine using RAID 10 and use with cinder-lvm. Unfortunately, lvm+mdadm is breaking all of my writes down to 4K -- without even increasing the queue depth to match. SSDs hate this workload. I tried bypassing mdadm by seting up LVM RAID10 by hand and got roughly a 10x increase in performance, so I definitely want to use this if possible. However, it looks like maybe the lvm_mirrors code hasn't been kept up to date. I ran into two main problems trying to use it: 1) The algorithm to calculate free space is broken and always returns 0. This means that the scheduler will never even try to use this pool 2) Thin provisioning is getting in the way... with lvm_mirrors enabled the code is apparently trying to create a new LV on top of the thin provisioned LV... and it is using an lvcreate option that appears to be deprecated (--mirrorlog mirrored) I expect I can fix #1, but what is the best way to handle #2? I was thinking that the easiest fix would be to use the mirror options when the thin LV is created, though I haven't tried it by hand yet to see if it works. The advantage here is that this is a one-time setting and the code path to create individual volumes wouldn't need to check lvm_mirrors at all. Thanks, Mark

2 4

[ops] NYC ops meetup sept 3,4 registration site live!
by Chris Morgan 30 Jul '19

30 Jul '19

Registration: https://go.bloomberg.com/attend/invite/open-stack-operators-quarterly-summi… planning etherpad : https://etherpad.openstack.org/p/NYC19-OPS-MEETUP Chris -- Chris Morgan <mihalis68(a)gmail.com>

1 0

Retiring tc-as-a-service
by Andreas Jaeger 30 Jul '19

30 Jul '19

This repo was imported and never used, I'll start the retirement process for it now. Ofer, one of the initial developer on the project confirmed that the repo is dead, Andreas -- Andreas Jaeger aj(a)suse.com Twitter: jaegerandi SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126

1 0

[release][infra] Supporting rget in our release process
by corvus＠inaugust.com 30 Jul '19

30 Jul '19

Hi, A colleague at Red Hat is working on an effort to record signatures of release artifacts. Essentially it's a way to help users verify release artifacts (or determine if they have been changed) independent of PGP signatures. You can read about it here: https://github.com/merklecounty/rget#rget It sounds like an interesting and useful effort, and I think we can support it at little cost. If we wanted to do so, I think we would need to do the following things: 1) Generate SHA256SUMS of our release artifacts. These could even include the GPG signature files. 2) Run "rget submit" on the resulting files after publication. That's it. Both of those would be changes to the release publication jobs, and wouldn't require any other changes to our processes. As mentioned in the README this is very early stages and the author, Brandon Philips, welcomes both further testing and feedback on the process in general. Thoughts? -Jim

5 4

Deprecate Aggregate[Core|Ram|Disk]Filters
by Sean Mooney 30 Jul '19

30 Jul '19

Hi all, back in january 2018 we annouced that the behavior of the Aggregate[Core|Ram|Disk]Filters http://lists.openstack.org/pipermail/openstack-dev/2018-January/126283.html form ocata on due to the use of placement you can no longer manage overcomit ratios using aggregate metadata as placement has no access to this data. as a result these filters if you use with placement then they only work if the aggregates enforce lower limits then placement default. if you also limit the responces from placemnet this also does not work as we may not recive any host that satisfy the lower limit in the aggregate metadata even if such host exist in the aggreate. we have a prominent warning https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#… advising operator of the impact of this change as well as this wont fix bug from last November https://bugs.launchpad.net/nova/+bug/1804125 11 months ago we deprecated the non aggreate [Core|Ram|Disk] Filters https://review.opendev.org/#/c/596502/ for several reasons noted in the commit and we furthermore removed the caching scheduler in stien. a few days ago we finally deleted the non aggreagate filters with https://github.com/openstack/nova/commit/78645e61c63bf042453d1f822ae8b3f1ee… but the Aggregate[Core|Ram|Disk]Filters are still present. we had intended to deprecate the Aggregate[Core|Ram|Disk]Filters in ocata when we sent the initial email about the behavior change however we never actully followed up and formally deprecated it in code. similarly when we formally deprecated the [Core|Ram|Disk] Filters we could have deprecated the Aggregate[Core|Ram|Disk]Filters too but we did not. As a result we cannot remove the Aggregate[Core|Ram|Disk]Filters in Train but i have proposed : https://review.opendev.org/673496 To formally deprecate them with the intent of removing them in the U cycle. In the 19.0.0 Stein release initial allocation ratios support was completed which allow managing of the allocation ratios via the placement api in addition to the existing capability to manage allocation ratios via the nova config. https://github.com/openstack/nova-specs/blob/master/specs/stein/implemented… As of the 18.0.0 rocky release nova automatically mirrors ``host-aggregates`` to placement aggregates. https://github.com/openstack/nova-specs/blob/master/specs/rocky/implemented… There is ongoing work in Train to enhance osc-placement to add a 'resource provider inventory update' command to make setting overcommit allocation ratios on a per-aggregate basis easier https://review.opendev.org/#/c/640898/ . These 3 features combined will allow operators to manage Aggreate Resource class allocation ratios in a placement native way. finally in the case the above was not sufficent motivation the AggregateCoreFilter it does not support handeling vCPU and pCPU resouce classes seperatly and will not work with the new cpu resouce tracking spec correctly. https://github.com/openstack/nova-specs/blob/master/specs/train/approved/cp… the AggregateRamFilter also does not understand the difference bewteen hugepage memory and standard memroy which means that even if we were to keep the filters they do not work correctly in all cases and will continue to becomre more broken over time, so i think the time has come to finally do this. regards sean

1 0

Error: python2-uritemplate conflicts with python2-uri-templates-0.6-5.el7.noarch
by Md. Farhad Hasan Khan 30 Jul '19

30 Jul '19

Hi, When I run # yum update and found below conflict: --> Processing Conflict: python2-uritemplate-3.0.0-1.el7.noarch conflicts python2-uri-templates --> Finished Dependency Resolution Error: python2-uritemplate conflicts with python2-uri-templates-0.6-5.el7.noarch You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles -nodigest If I remove python2-uri-templates-0.6-5.el7.noarch than below package ask to remove: Removing: python2-uri-templates noarch 0.6-5.el7 @centos-openstack-rocky 20 k Removing for dependencies: openstack-cinder noarch 1:13.0.5-1.el7 @centos-openstack-rocky 200 k python-cinder noarch 1:13.0.5-1.el7 @centos-openstack-rocky 19 M python2-google-api-client noarch 1.4.2-4.el7 @centos-openstack-rocky 305 k please let help me how can I update. Thanks/Rony

2 3

[qa]Proposing Attila Fazekas (afazekas) to Devstack Core list
by Ghanshyam Mann 30 Jul '19

30 Jul '19

Hello Everyone, I would like to propose to add Attila Fazekas (afazekas) in devstack core list. He has been contributing and reviewing code in devstack since a long time and has deep knowledge of code base. Please reply with your approval or objections. If no objection then till 5th Aug, I will add him in the list. -gmann

1 0

[I18n] project team status update regarding internationalization/translation as July 2019
by Ian Y. Choi 30 Jul '19

30 Jul '19

Hello all, I would like to share current project team status for better visibility into I18n team members, copying to openstack-discuss and user-committee thread: 1. Translation and team effort status There are still lots of translation contribution as you see through Stackalytics: https://www.stackalytics.com/?metric=translations&release=train . The number of translated words is not larger than the previous cycle, but I estimate that it will be similar or a little bit less than the previous cycle since I18n usually contributes translations a lot especially during Soft & Hard StringFreeze periods. Note that I18n team currently sets higher priority on OpenStack user survey translation and project document translations. Such priorities information can be found on Zanata dashboard: https://translate.openstack.org/ . Other progress such as migrating Zanata into other translation platform seems slow - me and Frank are planning to have a virtual sprint. 2. Shanghai Summit & PTG preparation : Frank submitted his presentation including me and Hoarce (China team). Let's wait for final approval on presentation submission. : There should be PTG at Shanghai - please add your name and detail thinking to Etherpad mentioned in http://lists.openstack.org/pipermail/openstack-i18n/2019-July/003436.html . I strongly think to more encourage participation since I expect more participation especially from Chinese translators & contributors. 3. Updating ATC & AUC : Frank previously submitted extra-ATC for Stein through https://review.opendev.org/#/c/633398/ , and this time I would like to submit the stat calculation from 2019-01-26 to 2019-07-25. : All contributors who had commits on openstack/i18n repository will be automatically regarded as Active Technical Contributors [1], and I18n team would like to nominate all translators who contributed translations on OpenStack official repositories (https://governance.openstack.org/tc/reference/projects/ ) >= 300 words as extra-ATCs assuming that their Zanata e-mail IDs are registered as Foundation members. : All translators who contributed >=300 word tranlations to the following projects will be nominated as Active User Contributors [2] assuming that their Zanata e-mail IDs are registered as Foundation members: "contributor-guide, edge-computing, leveraging-containers-openstack, openstack-user-survey, operations-guide". This is new nomination after the decision during Stein PTG with User Committee: Please see [3] and [4] if you would like to follow-up with more information. I will do this nomination for ATC & AUC as PTL just after July 25, although the due of extra ATC is Aug 26-30 [5] since I synced with Foundation that the price of Summit ticket will increase around Aug 7 : processing within the due will be fine, also I set period exactly for six months. 4. Team status : Although I regularly be online during I18n (+Docs) IRC team meetings, I couldn't find enough quorum, and I couldn't proceed more deep conversation with language teams (usually no reply from my e-mails unfortunately). : Current I18n team core members seem busy but when I communicate with them, all they are still very supportive to I18n team projects. I really appreciate their kind help. : Also, I find that some translators are very active comparing to some inactive language coordinators. In this case, would it be a good idea to nominate them as language coordinators? Please share your thoughts. : However, if such lack of team member conversations keep going, then I18n team might decide during upcoming PTG to make translation from an official team status into Special Interest Group [6]. This would mean that there will be no official PTL election and there will be no extra-ATC nomination, although I need to more figure out in details in future. With many thanks, /Ian [1] https://docs.openstack.org/i18n/latest/official_translator.html#atc-status-… [2]https://governance.openstack.org/uc/reference/charter.html#active-user-co… [3] http://lists.openstack.org/pipermail/openstack-i18n/2018-September/003316.h… [4] http://eavesdrop.openstack.org/meetings/uc/2018/uc.2018-10-01-18.02.html [5] https://releases.openstack.org/train/schedule.html [6] https://governance.openstack.org/sigs/

2 2

[TripleO] Transformation squad meeting time
by Kevin Carter 30 Jul '19

30 Jul '19

Hello, I'm writing to get feedback on what the best meeting time would be for all parties interested in participating in the TripleO Transformation Squad efforts. During this weeks meeting, it was raised that the current weekly time, Thursday 13:00 UTC, is less than ideal for a bunch of folks, and we should consider changing it [0]. To ensure we're getting everyone involved in the process, I've created a doodle that will run for one week [1]. In this Doodle, I'd like folks to vote on whatever time works best for them. During the next meeting, 25 Thursday 13:00 UTC, the results of the survey will be announced and the day and time with the highest number of votes will be used for all future meetings. Thank you, and please go vote! [0] http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-07-18-13.… [1] https://doodle.com/poll/nkkqnkarhbgp8cxr -- Kevin Carter IRC: kecarter

1 1

[openstack-dev] [neutron] [rally] [osprofiler]
by Miguel Lavalle 29 Jul '19

29 Jul '19

Hi, The Neutron team has been playing lately with Rally scenarios and osprofiler. One example is http://logs.openstack.org/80/671780/1/check/neutron-rally-task/6cc66e4/resu…. As can be seen in this example, we get a lot of trace points from SQLAlchemy with the corresponding SQL statement that was sent the DB. Is there a way we can add information to these trace points to help us relate them to the Neutron code that issued the statement? Cheers Miguel

1 0

[forum][sdk][nova] Closing compute API feature gaps in the openstack CLI - recap
by Matt Riedemann 29 Jul '19

29 Jul '19

I wanted to give a quick recap of this Forum session for those that couldn't attend and also find owners. Please reply to this thread if you'd like to sign up for any specific item. The etherpad [1] has the details. To restate the goal: "Identify the major issues and functional gaps (up through Mitaka 2.25) and prioritize which to work on and what the commands should do." We spent the majority of the time talking about existing issues with compute API functionality in openstack CLI, primarily boot-from-volume, live migration and lack of evacuate support (evacuate as in rebuild on a new target host because the source host is dead, not drain a host with live migrations [2]). We then talked through some of the microversion gaps and picked a few to focus on. Based on that, the agreements and priorities are: **High Priority** 1. Make the boot-from-volume experience better by: a) Support type=image for the --block-device-mapping option. b) Add a --boot-from-volume option which will translate to a root --block-device-mapping using the provided --image value (nova will create the root volume under the covers). Owner: TBD (on either) 2. Fix the "openstack server migrate" command We're going to deprecate the --live option and add a new --live-migration option and a --host option. The --host option can be used for requesting a target host for cold migration (omit the --live/--live-migration option for that). Then in a major release we'll drop the --live option and intentionally not add a --force option (since we don't want to support forcing a target host and bypassing the scheduler). Owner: TBD (I would split the 2.56 cold migration --host support from the new --live-migration option review-wise) **Medium Priority** Start modeling migration resources in the openstack CLI, specifically for microversions 2.22-2.24, but note that the GET /os-migrations API is available since 2.1 (so that's probably easiest to add first). The idea is to have a new command resource like: openstack compute migration (list|delete|set) [--server <server_id>] Owner: TBD (again this is a series of changes) **Low Priority** Closing other feature gaps can be done on an as-needed basis as we've been doing today. Sean Mooney is working on adding evacuate support, and there are patches in flight (see [3]) for other microversion-specific features. I would like to figure out how to highlight these to the OSC core team on a more regular basis, but we didn't really talk about that. I've been trying to be a type of liaison for these patches and go over them before the core team tries to review them to make sure they match the API properly and are well documented. Does the OSC core team have any suggestions on how I can better socialize what I think is ready for core team review? [1] https://etherpad.openstack.org/p/DEN-osc-compute-api-gaps [2] http://www.danplanet.com/blog/2016/03/03/evacuate-in-nova-one-command-to-co… [3] https://etherpad.openstack.org/p/compute-api-microversion-gap-in-osc -- Thanks, Matt

2 2

[ops][glance][nova] scheduling problem because of ImagePropertiesFilter
by Massimo Sgaravatto 29 Jul '19

29 Jul '19

We have just finished the update of our cloud to Rocky but we are seeing a strange issue with images with property: hypervisor_type='QEMU' The scheduling of instances created with such images fails because of the ImagePropertiesFilter [*] Removing that property from the image, the scheduling works. We also tried changing hypervisor_type='QEMU' --> hypervisor_type='qemu', but this didn't help. Any suggestions? Thanks, Massimo [*] 2019-07-17 13:52:58.148 13312 INFO nova.filters [req-1863bef0-0326-46d1-a836-436227e91eef 6e3b136d578f4292a5c03b16f443ab3d d27fe2becea94a3e980fb9f66e2f291a - default default] Filtering removed all hosts f\ or the request with instance ID '63810b60-76e4-4e76-a1c3-e4d3932c002e'. Filter results: ['AggregateMultiTenancyIsolation: (start: 49, end: 37)', 'AggregateInstanceExtraSpecsFilter: (start: 37, end: 34)', \ 'RetryFilter: (start: 34, end: 34)', 'AvailabilityZoneFilter: (start: 34, end: 34)', 'ComputeFilter: (start: 34, end: 32)', 'ComputeCapabilitiesFilter: (start: 32, end: 32)', 'ImagePropertiesFilter: (star\ t: 32, end: 0)']

5 21

[tc][all] seeking candidates to run in the next TC election
by Doug Hellmann 29 Jul '19

29 Jul '19

Nominations for the next Technical Committee term open on 27 August, just over 4 weeks from now. In the past, we have had people wait until the last minute to consider running, and then in some cases they had issues getting approval from their managers to serve in the role. This time, since I know there will be at least one open seat when I step down, I thought it would be a good idea to start that recruiting process a little earlier. Our community places a high value on democratically electing representative leadership — selecting members of the community to help steer the project. That system only works if we have people willing and able to serve in the elected roles like the Technical Committee. There is no fall-back option. We need community members to step up and run. So, if you are interested in serving on the TC, now is the time ask the necessary questions within your company. If you need advice about how to approach your manager, any current TC member, including me, would be happy to help you. Doug

1 0

[requirements] changes in requests library maintainership
by Doug Hellmann 29 Jul '19

29 Jul '19

It looks like the Kenneth Reitz, the primary maintainer of the requests library, is stepping down [1]. There are several offers to take over right now, so it's not really clear to me what's going to happen. I think we'll want to keep an eye on both requests and requests3 until the new maintainer is designated. And of course if anyone is interested in helping out, I encourage you to get involved in the discussion on github. [1] https://github.com/not-kennethreitz/team/issues/21 -- Doug

2 1

[nova][sfe] Nova part of Image Encryption
by Josephine Seifert 29 Jul '19

29 Jul '19

Hi, we would like to have a spec freeze exception for the spec for the Nova Part of Image Encryption [1] as this is part of a cross-project contribution with specs also in cinder (already merged) and glance. We will attend the next meeting on August 1st. Greetings Josephine (Luzi) [1] https://review.opendev.org/#/c/608696

1 0

[searchlight] Meeting today cancelled
by Trinh Nguyen 29 Jul '19

29 Jul '19

Hi team, I cannot hold the meeting today so please email or ping me on #openstack-searchlight channel. Bests, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

1 0

[ironic][edge] L3/DHCP-Less deployments
by Julia Kreger 29 Jul '19

29 Jul '19

Greetings everyone! Over the last few weeks, I've had numerous discussions with contributors who have expressed interest in the DHCP-less deployment method specification document[0]. It seems many parties are interested! I think the path forward at this time is to get everyone talking about where and how they can help push this feature forward. If those interested could propose time windows when they are available, I'll try to find a mutual time window where we can try to get everyone on IRC or on a call and try to map out the next steps together. Sound good? -Julia [0]: https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/L3…

2 2

[tc] weekly update
by Mohammed Naser 29 Jul '19

29 Jul '19

Hi everyone, This is the weekly update for what happened inside the Openstack TC last week. You can check for more information in the openstack/governance repository. We had one change only which was the retirement of puppet-crane (from TripleO). Thanks, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

1 0

[Watcher] No IRC meeting this week
by li.canwei2＠zte.com.cn 29 Jul '19

29 Jul '19

Hi team, I'll be on vacation until to August 5, so I can't host meeting this week. Thanks, licanwei

1 0

[neutron] Bug Deputy July 22 - July 29
by Boden Russell 29 Jul '19

29 Jul '19

Below is a list of neutron bugs that came in last week. To the best of my knowledge, the only bug still under discussion is #2 below (and the RFE in #3). Thanks to everyone on the team for their bug diligence. It looks like Brian H. is on deck for bug duty this week. 1) https://bugs.launchpad.net/neutron/+bug/1838068 "QoSTest:test_qos_basic_and_update" failing in DVR node scenario 2) https://bugs.launchpad.net/neutron/+bug/1837916 Trunk subports sometimes not becoming ACTIVE and trunk:subport 3) https://bugs.launchpad.net/neutron/+bug/1837847 [RFE] neutron-vpnaas OpenVPN driver 4) https://bugs.launchpad.net/neutron/+bug/1837635 HA router state change from "standby" to "master" should be delayed 5) https://bugs.launchpad.net/neutron/+bug/1837553 Neutron api-ref should mention that list of e.g. IDs is supported in GET requests 6) https://bugs.launchpad.net/neutron/+bug/1837552 neutron-tempest-with-uwsgi job finish with timeout very often 7) https://bugs.launchpad.net/neutron/+bug/1837529 Cannot use push-notification with custom objects

1 0

[nova][sfe] Use PCPU and VCPU in One Instance
by David Wang 29 Jul '19

29 Jul '19

Hi, We are proposing a Nova spec for the feature of creating an instance of mixed dedicated CPUs and shared CPUs and experience communities review. The spec link is https://review.opendev.org/#/c/668656/ and spec name is "Use PCPU and VCPU in One Instance". I recognized that Nova Sepc for Train release is frozen, I'd like to request a spec freeze exception process and let the spec to be merged in train release. I may have trouble in attending next week's team meeting, on Aug 1, so, please discuss in the email thread. Thanks Huaqiang

1 0

[osc] Very slow openstack commands in Stein Devstack running on VM
by Bernd Bausch 29 Jul '19

29 Jul '19

I run Stein Devstack in a virtual machine. Occasionally, openstack client commands take minutes to complete, which didn't happen at earlier releases, and which is rather annoying. The delay seems to take place at an early stage: Even when I forget to set the OS_* environment, it takes minutes until I am told that the request requires authentication. Interrupting the command, the stack trace confirms this (as a side remark, it also takes a long while until my ^C is processed): Traceback (most recent call last): File "/usr/local/bin/openstack", line 7, in <module> from openstackclient.shell import main File "/usr/local/lib/python2.7/dist-packages/openstackclient/shell.py", line 22, in <module> from osc_lib.api import auth File "/usr/local/lib/python2.7/dist-packages/osc_lib/api/auth.py", line 19, in <module> from keystoneauth1.loading import base File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/__init__.py", line 13, in <module> from keystoneauth1.loading import adapter File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/loading/adapter.py", line 13, in <module> from keystoneauth1 import adapter File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 17, in <module> from keystoneauth1 import session File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py", line 25, in <module> import requests File "/usr/local/lib/python2.7/dist-packages/requests/__init__.py", line 95, in <module> from urllib3.contrib import pyopenssl File "/usr/local/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 48, in <module> from cryptography.hazmat.backends.openssl import backend as openssl_backend File "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/__init__.py", line 7, in <module> from cryptography.hazmat.backends.openssl.backend import backend File "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 2419, in <module> backend = Backend() File "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 119, in __init__ self.activate_osrandom_engine() File "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 163, in activate_osrandom_engine with self._get_osurandom_engine() as e: File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/openssl/backend.py", line 146, in _get_osurandom_engine res = self._lib.ENGINE_init(e) KeyboardInterrupt My wild speculation is that the delay has something to do with obtaining a random number, which can be a problem on virtual machines due to their lack of entropy. My knowledge in this area is paper-thin, though. And I wonder what happens to all the Devstacks used in CI? Thus my questions: *Is the delay really caused by my running the cloud on a VM? What can I do to improve this?* Thanks, Bernd

4 4

[goals][IPv6-Only Deployments and Testing] Week R-12 Update
by Ghanshyam Mann 29 Jul '19

29 Jul '19

Hello Everyone, I am starting this status email for 'IPv6-Only Deployments and Testing ' community goal for Train. Storyboard: ========= - https://storyboard.openstack.org/#!/story/2005477 Current status: ============ 1. Base job 'devstack-tempest-ipv6' has been prepared on Tempest (patch yet to merge)[1] 2. 'tempest-ipv6-only' is defined in Tempest (patch yet to merge)[1] 3. 'tempest-ipv6-only' has been proposed to run on 5 services project date side [2]. 4. I have proposed few more projects jobs like congress etc and will keep doing for other projects also. 5. Neutron stadium projects jobs. I talked to slaweq on IRC [3]on what all neutron stadium falls under the IPv6 address communication criteria so that we can add ipv6-only job for those project only. He is going to prepare the list of those project and accordingly we will add jobs Everything related to this goal can be found under this topic: Topic: https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+(stat… How to define and run new IPv6 Job on project side: ======================================= - I prepared a wiki page to describe this section - https://wiki.openstack.org/wiki/Goal-IPv6-only-deployments-and-testing - I am adding this wiki info in goal doc also[4]. Review suggestion: ============== - Main goal of these jobs will be whether your service is able to listen on IPv6 and can communicate to any other services either OpenStack or DB or rabbitmq etc on IPv6 or not. So check your proposed job with that point of view. If anything missing, comment on patch. - One example was - I missed to configure novnc address to IPv6- https://review.opendev.org/#/c/672493/ - base script as part of 'devstack-tempest-ipv6' will do basic checks for endpoints on IPv6 and some devstack var setting. But if your project needs more specific varification then it can be added in project side job as post-run playbooks as described in wiki page[5]. [1] https://review.opendev.org/#/c/671231/ [2] https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+(stat… [3] http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-ne… [4] https://review.opendev.org/#/c/671898/ [5] https://wiki.openstack.org/wiki/Goal-IPv6-only-deployments-and-testing -gmann

2 2

[tripleo] Roadmap to simplify TripleO
by Emilien Macchi 29 Jul '19

29 Jul '19

Even though TripleO is well known for its maturity, it also has a reputation of being complex when it comes to the number of tools that it uses. Somewhat related to the efforts that James is leading with "Scaling TripleO" [1] [2], I would like to formalize our joint efforts to make TripleO simpler in the future. Some work has been done over the last releases already and yet we have seen net benefits; however we still have challenges ahead of us. - With no UI anymore, do we really need an API? - How can we reduce the number of languages in TripleO? ... and make Python + Ansible the main ones. - How can we reduce our dependencies? I created a document which explains the problems and propose some solutions: https://docs.google.com/document/d/1vY9rsccgp7NHFXpLtCFTHQm14F15Tij7lhn5X_P… For those who can't or don't want Google Doc, I've put together the notes into etherpad [3] and I'll take care of making sure it's updated at last at the beginning until we sort things out. Feel free to be involved: - comment or suggest edits if you have feedback / ideas - sign-up if you're interested to contribute For now I expect this document to be a clear place what our plan is but I wouldn't be surprised if in the future we break it down into specs / blueprints / etc for better tracking. Thanks, [1] https://docs.google.com/document/d/12tPc4NC5fo8ytGuFZ4DSZXXyzes1x3U7oYz9nea… [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007638.html [3] https://etherpad.openstack.org/p/tripleo-simplification -- Emilien Macchi

3 2

[requirements] docutils-0.15.1 pypi missing wheel
by Matthew Thode 28 Jul '19

28 Jul '19

Hi, In trying to update openstack past the broken 0.15 release to the 0.15.1 bugfix release I ran into a problem when trying to install it in python3 (we are currently testing on ppython 3.6). It seems like the missing wheel is the issue. When do you plan on publishing it? The following demonstrates the issue. The logs are not kept forever but it's a standard error. ERROR: No matching distribution found for docutils===0.15.1 http://logs.openstack.org/73/672873/3/check/requirements-tox-py36-check-uc/… It seems like other devs/users have reported the problem as well but there hasn't been any activity to address it. See the following messages: https://sourceforge.net/p/docutils/mailman/message/36726543/ https://sourceforge.net/p/docutils/mailman/message/36726888/ Please let me know if you need anything else from me. -- Matthew Thode

2 2

[keystone] Keystone Team Update - Week of 22 July 2019
by Colleen Murphy 27 Jul '19

27 Jul '19

# Keystone Team Update - Week of 22 July 2019 ## News ### Midcycle held this week We held our first midcycle since the PTGs became a thing, possibly also the team's first virtual midcycle (I'm not sure). It went quite well and it was great to sync up with everyone. Recap posted on the mailing list[1]. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007987.html ## Action Items As discussed during the midcycle retrospective, we'll start calling out action items from the last meeting (or last midcycle, or last PTG, or perhaps any other commitment worth mentioning in a weekly update). * Remove the keystone socket timeout option (IN PROGRESS: https://bugs.launchpad.net/keystone/+bug/1837407) * Seek feedback from operators on caching guide (guide draft IN PROGRESS: https://review.opendev.org/672120) ## Office Hours When there are topics to cover, the keystone team holds office hours on Tuesdays at 17:00 UTC. The topic for next week's office hour will be: bug triage! The location for next week's office hour will be: https://meet.jit.si/keystone-office-hours (fallback to #openstack-keystone if needed) Add topics you would like to see covered during office hours to the etherpad: https://etherpad.openstack.org/p/keystone-office-hours-topics ## Open Specs Ongoing specs: https://bit.ly/2OyDLTh ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 16 changes this week, which included progress made on system scope/default roles[2] and the last of the changes needed for the python3-train goal[3][4]. [2] https://review.opendev.org/670926 [3] https://review.opendev.org/667755 [4] https://review.opendev.org/667756 ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 35 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ### Priority Reviews * Train Roadmap Stories System scope/default roles (https://trello.com/c/ERo50T7r , https://trello.com/c/RlYyb4DU) - https://review.opendev.org/645968 (Implement domain reader support for grants) - https://review.opendev.org/667730 (Implement domain admin support for grants) Application credential access rules (https://trello.com/c/XyBGhKrE) - https://review.opendev.org/668238 (Expose access rules as its own API) - https://review.opendev.org/663440 (Add user_id, external_id to access rules table) Caching Guide (https://trello.com/c/UCFt3mfF) - https://review.opendev.org/672120 (Update the caching guide) Predictable IDs (https://trello.com/c/MVuu6DbU) - https://review.opendev.org/651655 (Predictable IDs for Roles) Federation Improvements (https://trello.com/c/GEaIpbpm) - https://review.opendev.org/637305 (Add new attribute to the federation protocol API) Oslo.limit (https://trello.com/c/KGGkNijR) - https://review.opendev.org/667242 (Add usage example) - https://review.opendev.org/666444 (Flush out basic enforcer and model relationship) - https://review.opendev.org/666085 (Add ksa connection logic) YAML Catalog (https://trello.com/c/Qv14G0xp) - https://review.opendev.org/483514 (Add yaml-loaded filesystem catalog backend) * Needs Discussion - https://review.opendev.org/669959 (discourage using X.509 with external auth) - https://review.opendev.org/655166 (Allows to use application credentials through group membership) * Oldest - https://review.opendev.org/448755 (Add federated support for creating a user) ## Bugs This week we opened 3 new bugs and closed 7 (yay!) Bugs opened (3) Bug #1837407 (keystone:Low) opened by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1837407 Bug #1837513 (keystone:Undecided) opened by ahmed elbendary https://bugs.launchpad.net/keystone/+bug/1837513 Bug #1837741 (oslo.policy:High) opened by Ben Nemec https://bugs.launchpad.net/oslo.policy/+bug/1837741 Bugs closed (1) Bug #1837513 (keystone:Undecided) https://bugs.launchpad.net/keystone/+bug/1837513 Bugs fixed (6) Bug #1750615 (keystone:High) fixed by Guang Yee https://bugs.launchpad.net/keystone/+bug/1750615 Bug #1782922 (keystone:Medium) fixed by Guang Yee https://bugs.launchpad.net/keystone/+bug/1782922 Bug #1818725 (keystone:Medium) fixed by Guang Yee https://bugs.launchpad.net/keystone/+bug/1818725 Bug #1828565 (keystone:Low) fixed by Jose Castro Leon https://bugs.launchpad.net/keystone/+bug/1828565 Bug #1818845 (keystone:Wishlist) fixed by Lance Bragstad https://bugs.launchpad.net/keystone/+bug/1818845 Bug #1829802 (keystonemiddleware:Undecided) fixed by norman shen https://bugs.launchpad.net/keystonemiddleware/+bug/1829802 ## Milestone Outlook https://releases.openstack.org/train/schedule.html This week marks the milestone 2 spec freeze. Feature proposal freeze is in three (3!) weeks. This means code implementing approved specs needs to be in a reviewable, non-WIP state. ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[nova] Train Cycle Themes - An Interactive Experience
by Eric Fried 26 Jul '19

26 Jul '19

Hi all. Now that we're past spec freeze, Matt suggested I write a report on how we're doing against our cycle themes. That sounded really tedious and boring and project-managementy, so I thought I would spread the joy. Bear with me, this is going to be a little different. I've created a null review in the nova-specs repository [1]. I'd like us to use this as a mechanism to collaboratively gather thoughts via review comments [2]. I've made a start on some bits. Have fun! efried [1] https://review.opendev.org/#/c/673095/ [2] I could have used an etherpad. But this will be more... permanent.

1 0

Forcing restart of a worker node with running guest
by Mauricio Tavares 26 Jul '19

26 Jul '19

So one of my worker nodes is, to put it mildly, rather unhappy: Message from syslogd@openstack-w1 at Jul 25 15:57:49 ... kernel:NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [vnc_worker:9141] Message from syslogd@openstack-w1 at Jul 25 15:57:57 ... kernel:NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [migration/10:60] Message from syslogd@openstack-w1 at Jul 25 15:58:05 ... kernel:NMI watchdog: BUG: soft lockup - CPU#8 stuck for 22s! [migration/8:49] I found out when it was taking 30 min to delete a guest. So, what I can do in a forceful way? 1. How to kill the guest? Can I kill it through virsh or openstack compute service will get sad? 2. What would happen if I stop the compute service? 3. What would happen if I get really annoyed and tell worker node to reboot?

2 3

[goal][python3] Train unit tests weekly update (goal-7)
by Corey Bryant 26 Jul '19

26 Jul '19

This is the goal-7 weekly update for the "Update Python 3 test runtimes for Train" goal [1]. There are 7 weeks remaining for completion of Train community goals [2]. == What's the Goal? == To ensure (in the Train cycle) that all official OpenStack repositories with Python 3 unit tests are exclusively using the 'openstack-python3-train-jobs' Zuul template or one of its variants (e.g. 'openstack-python3-train-jobs-neutron') to run unit tests, and that tests are passing. This will ensure that all official projects are running py36 and py37 unit tests in Train. For complete details please see [1]. == Ongoing Work == All patches have been submitted for all applicable projects. Open patches needing reviews: https://review.openstack.org/#/q/topic:python3-train+is:open Failing patches: https://review.openstack.org/#/q/topic:python3-train+status:open+(+label:Ve… Patch automation scripts needing review: https://review.opendev.org/#/c/666934 == Completed Work == Merged patches: https://review.openstack.org/#/q/topic:python3-train+is:merged == How can you help? == Please take a look at the failing patches and help fix any failing unit tests for your projects. Python 3.7 unit tests will be self-testing in Zuul. == Reference Material == [1] Goal description: https://governance.openstack.org/tc/goals /train/python3-updates.html [2] Train release schedule: https://releases.openstack.org/train/schedule.html (see R-5 for "Train Community Goals Completed") Storyboard: https://storyboard.openstack.org/#!/story/2005924 Porting to Python 3.7: https://docs.python.org/3/whatsnew/3.7.html#porting-to-python-3-7 Python Update Process: https://opendev.org/openstack/governance/src/branch/master/resolutions/2018… Train runtimes: https://opendev.org/openstack/governance/src/branch/master/reference/runtim… Thanks, Corey

1 0

[nova][sfe] Train Spec Freeze Exception Process
by Eric Fried 26 Jul '19

26 Jul '19

Hello Nova. Spec freeze happened, which means we won't be approving any more specs for Train. Unless we do. >From this point on, you will be required to run the following gauntlet if you want your spec to land in Train. - Put your name and a link to your spec on the agenda for the next Nova meeting [1][2]. - Send a note to openstack-discuss tagged with [nova][sfe] (like this one is) including the spec for which you are requesting an exception, and noting the date of the meeting. - Attend that meeting and state your case. Assuming a quorum of cores is present, we will attempt to render a decision, or at least indicate what further actions are necessary. Thanks, efried [1] https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting [2] If you can't attend a meeting, or "next Thursday" is just too long to wait, say so in your email and we'll endeavor to discuss it entirely on the mailing list.

1 0

[nova][oslo.log] global_request_id help
by Eric Fried 26 Jul '19

26 Jul '19

While doing something unrelated, I noticed this code [0] that shouldn't need to exist if global_request_id is working. I threw out some debug patches [1] to dump headers. Results: [2] (nova), [3] (placement). We're tracking global_request_id ``req-a51f...``. Nova is sending the global_request_id properly in the X-Openstack-Request-Id header, and Placement is receiving it correctly, because it's showing up in the expected spot in the Placement log line (after the opening bracket, per the logging_context_format_string in placement.conf [4]). Placement is sending back the request_id (``req-b3ce...``) in the x-openstack-request-id header in the response. (I thought this was a bug, but it turns out this is correct per the design [5].) So the problem is that the Nova log is printing ``None`` for the global_request_id. Based on logging_context_format_string in nova-cpu.conf [6], I expect it in the first field after the opening bracket, just like for Placement. If the global_request_id were showing up in the Nova log, we wouldn't need to correlate the Nova and Placement sides of this conversation via the Placement request_id, and we could get rid of [0] and its usages. I don't understand how LOG.xxx() is supposed to glean this value. Can someone help me find the disconnect? Thanks, efried [0] https://opendev.org/openstack/nova/src/commit/d358df32f7a2a1125962e27f24253… [1] https://review.opendev.org/#/q/topic:where-are-you-global_request_id [2] http://logs.openstack.org/86/672986/1/check/nova-live-migration/e426c8a/log… [3] http://logs.openstack.org/86/672986/1/check/nova-live-migration/e426c8a/log… [4] http://logs.openstack.org/86/672986/1/check/nova-live-migration/e426c8a/log… [5] https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.h… [6] http://logs.openstack.org/86/672986/1/check/nova-live-migration/e426c8a/log…

1 0

[nova] instence running, but status is Error
by Budai Laszlo 26 Jul '19

26 Jul '19

Dear all, I am testing different migration scenarios in our openstack, and I ended up with an instance that is running (I can access it by ssh), but its Status is Error. and its task_state is resize_migrating. I suppose that I got here due to the fact that the `allow_resize_to_same_host` is set to True, and the scheduler has tried to use the same compute node in the migration as described here: https://bugs.launchpad.net/nova/+bug/1811235. Is there any possibility to be able to manage again that instance using openstack (except to delete it)? I have tried the shut off (from horizon) and it gives me an error: fault: code: 400 created: '2019-07-26T14:46:05Z' message: Unavailable console type novnc. Thank you, Laszlo

2 3

[placement] update 19-29
by Chris Dent 26 Jul '19

26 Jul '19

HTML: https://anticdent.org/placement-update-19-29.html Welcome to a rushed pupdate 19-29. My morning was consumed by other things. A reminder: The Placement project holds office hours every Wednesday at 1500 UTC in the `#openstack-placement` IRC channel. If you have a topic that needs some synchronous discussion, then is an ideal time. Just start talking! # Most Important The two main things on the Placement radar are implementing Consumer Types and cleanups, performance analysis, and documentation related to nested resource providers. # What's Changed * The [api-ref](https://docs.openstack.org/api-ref/placement/) has moved to `docs.openstack.org` from `developer.openstack.org`. Redirects are in place. * Both traits and resource classes are [now cached](https://review.opendev.org/672298) per request, allowing for some name to id and id to name optimizations. * A new zuul template is [being used](https://review.opendev.org/671257) in placement that means fewer irrelevant tempest tests are run on placement changes. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 21 (-1) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 2 (0) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 5 (0) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 10 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 4 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 17 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 4 (0). # osc-placement osc-placement is currently behind by 12 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. There's been some useful discussion about how to achieve this, and a consensus has emerged on how to get the best results. # Main Themes ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. * <https://review.opendev.org/#/q/topic:bp/support-consumer-types> A WIP, as microversion 1.37, has started. ## Cleanup Cleanup is an overarching theme related to improving documentation, performance and the maintainability of the code. The changes we are making this cycle are fairly complex to use and are fairly complex to write, so it is good that we're going to have plenty of time to clean and clarify all these things. # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are two [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And zero [os-resource-classes changes](https://review.opendev.org/#/q/project:openstack/os-resource-class… (yay!). # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/662229> helm: add placement chart * <https://review.opendev.org/656023> Nova: Use OpenStack SDK for placement * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/664689> Nova: get_ksa_adapter: nix by-service-type confgrp hack * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/667952> nova: Support filtering of hosts by forbidden aggregates * <https://review.opendev.org/669079> blazar: Send global_request_id for tracing calls * <https://review.opendev.org/668252> Nova: Update HostState.\*\_allocation_ratio earlier * <https://review.opendev.org/670112> Nova: WIP: Add a placement audit command * <https://review.opendev.org/586960> zun: [WIP] Use placement for unified resource management * <https://review.opendev.org/671478> Kayobe: Build placement images by default * <https://review.opendev.org/671312> blazar: Fix placement operations in multi-region deployments * <https://review.opendev.org/672870> Watcher: Getting data from placement when updating datamodel * <https://review.opendev.org/671793> Nova: libvirt: Start reporting PCPU inventory to placement * <https://review.opendev.org/672649> chef: placement_api: create valid apache config before installing package * <https://review.opendev.org/670696> tempest: Add placement API methods for testing routed provider nets * <https://review.opendev.org/672678> openstack-helm: Build placement in OSH-images # End If we get the chance, it will be interesting to start working with placement with 1 million providers. Just to see. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[openstack-ansible] weekly update
by Mohammed Naser 26 Jul '19

26 Jul '19

Hi everyone, I'm going to try doing these more often to keep everyone updated on the current state of openstack-ansible (mostly around what is discussed during our office hours / meetings). - Dmitriy has been working on removing unused code in playbooks, mostly around logs as we move to 100% systemd-journald. - We're discussing the proposal to remove os-log-dir-setup.yml, as well as rsyslog_client and rsyslog_server roles as they aren’t used anymore with the move to systemd-journald - We're also discussing removing all the complex bind mounts, which would clean it and simplify the code base. - Our continuous integration has been stable but needs upgrade jobs and increased coverage. Our coverage would go up significantly by running aio_lxc for all supported systems. - Jonathan's work on binding to management IPs is making progress (and seems to be at the Galera stage). - CentOS CI has been very slow and nobody knows why. It seems to be something we've regressed at some point. - A unicast flood issue affecting deployments using linuxbridge that happened inside os-vif needs to be patched. It's still pending os-vif to patch. - Namrata mentioned an issue during the upgrade from Rocky to Stein (when upgrading WSREP SST method from xtrabackup-V2 to mariabackup) that will have to be fixed is Stein. That's something that's in progress now. Thanks everyone for reading! Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

2 1

[cinder][ceph][RBD mirroring] Error while creating a mirrored volume
by Ajitha Robert 26 Jul '19

26 Jul '19

Hi all Following picture shows my setup [image: image.png] I m facing following error when i issue a command to enable rbd mirroring on a particular image either manually or through cinder. *"rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x7fb364025310 finish: resending after timeout"* in rbd-mirroring.log and getting *"failed to get omap key: client_0921b3ce-716a-4462-b3cf-86b5102bfd9"* error in osd.log. . Should I enable anything else apart from timeout parametes, if the distance between the two ceph clusters is greater than 300km? when I create a bootable volume of 100 GB with a glance image.Image get downloaded and from cinder, volume log throws with "volume is busy deleting volume that has snapshot" . Image was enabled with exclusive lock, journaling, layering, object-map, fast-diff and deep-flatten Cinder volume is in error state but the rbd image is created in primary but not in secondary. pls check the logs 1)Log for manual rbd image creation http://paste.openstack.org/show/754766/ 2)Log for 16gb volume created from cinder, status in cinder volume is available http://paste.openstack.org/show/754767/ 3)Log for 100gb volume created from cinder, status in cinder volume is error http://paste.openstack.org/show/754769/ -- *Regards,Ajitha R*

1 0

[placement] update 19-27
by Chris Dent 26 Jul '19

26 Jul '19

HTML: https://anticdent.org/placement-update-19-27.html Pupdate 19-27 is here and now. # Most Important Of the features we planned to do this cycle, all are done save one: consumer types (in progress, see below). This means we have a good opportunity to focus on documentation, performance, and improving the codebase for maintainability. You do not need permission to work on these things. If you find a problem and know how to fix it, fix it. If you are not sure about the solution, please discuss it on this email list or in the `#openstack-placement` IRC channel. This also means we're in a good position to help review changes that use placement in other projects. The Foundation needs to know how much, if any, Placement time will be needed in Shanghai. I started a [thread](http://lists.openstack.org/pipermail/openstack-discuss/2019-July/00… and an [etherpad](https://etherpad.openstack.org/p/placement-shanghai-ptg). # What's Changed * The `same_subtree` query parameter has merged as [microversion 1.36](https://docs.openstack.org/placement/latest/placement-api-microversio…. This enables a form of nested provider affinity: "these providers must all share the same ancestor". * The placement projects have been updated (pending merge) to match the [Python 3 test runtimes for Train](https://governance.openstack.org/tc/goals/train/python3-updates.html) community wide goal. Since the projects have been Python3-enabled from the start, this was mostly a matter of aligning configurations with community-wide norms. When Python 3.8 becomes available we should get on that sooner than later to catch issues early. # Specs/Features All placement specs have merged. Thanks to everyone for the frequent reviews and quick followups. Some non-placement specs are listed in the Other section below. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 23 (0) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 3 (1) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 5 (0) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 4 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 16 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 4 (0). # osc-placement osc-placement is currently behind by 11 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. This is stuck and needs input from users of the tool. # Main Themes ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. * <https://review.opendev.org/#/q/topic:bp/support-consumer-types> This is currently the migration to add the necessary table and column. ## Cleanup Cleanup is an overarching theme related to improving documentation, performance and the maintainability of the code. The changes we are making this cycle are fairly complex to use and are fairly complex to write, so it is good that we're going to have plenty of time to clean and clarify all these things. As mentioned last week, one of the important cleanup tasks that is not yet in progress is updating the [gabbit](https://opendev.org/openstack/placement/src/branch/master/gate/gabb… that creates the nested topology that's used in nested performance testing. The topology there is simple, unrealistic, and doesn't sufficiently exercise the several features that may be used during a query that desires a nested response. This needs to be someone who is more closely related to real world use of nested than me. efried? gibi? Another cleanup that needs to start is satisfying the community wide goal of [PDF doc generation](https://storyboard.openstack.org/#!/story/2006110). Anyone know if there is a cookbook for this? # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are three [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And one [os-resource-classes change](https://review.opendev.org/#/q/project:openstack/os-resource-classe…. # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.openstack.org/650188> nova-spec: Allow compute nodes to use DISK_GB from shared storage RP * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/662229> helm: add placement chart * <https://review.opendev.org/656023> Nova: Use OpenStack SDK for placement * <https://review.opendev.org/612497> Nova: Spec: Provider config YAML file * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/664689> Nova: get_ksa_adapter: nix by-service-type confgrp hack * <https://review.opendev.org/664862> OSA: Add nova placement to placement migration * <https://review.opendev.org/657796> Nova: Defaults missing group_policy to 'none' * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/669309> tempest: Define the Integrated-gate-placement gate template * <https://review.opendev.org/668263> Nova: Restore RT.old_resources if ComputeNode.save() fails * <https://review.opendev.org/667952> nova: Support filtering of hosts by forbidden aggregates * <https://review.opendev.org/669079> blazar: Send global_request_id for tracing calls * <https://review.opendev.org/668252> Nova: Update HostState.\*\_allocation_ratio earlier * <https://review.opendev.org/670105> neutron: segments: fix rp inventory update * <https://review.opendev.org/670112> Nova: WIP: Add a placement audit command * <https://review.opendev.org/668526> OSA: Add placement client for neutron * <https://review.opendev.org/586960> zun: [WIP] Use placement for unified resource management # End A colleague suggested yesterday that the universe doesn't have an over subscription problem, rather there's localized contention, and what we really have is a placement problem. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

2 1

[keystone] Need clarification about Stein policies
by Bernd Bausch 25 Jul '19

25 Jul '19

The Keystone policy.json file I created with oslo-policy-generator contains lines I don't understand. For example /list_users/. The comment says: # DEPRECATED "identity:list_users":"rule:admin_required" has been # deprecated since S in favor of "identity:list_users":"(role:reader # and system_scope:all) or (role:reader and # domain_id:%(target.domain_id)s)". I do understand the expression starting with (role:reader .... , but contrarily to the comment, the policy is "identity:list_users": "rule:identity:list_users" This looks like a circular definition, and in any case, nowhere do I seerule:identity:list_users defined. Can someone in the know explain how this policy is processed? Thanks much, Bernd

2 1

[dev][nova][glance][cinder] introducing 'compressed' image container_format
by Ma, Zheng 25 Jul '19

25 Jul '19

Hello Nova developers, Cinder is working on a feature that will allow it to compress volumes uploaded to Glance [0,1]. We are proposing a new container_format for Glance named 'compressed' [2]. The idea is that a consumer of a 'compressed' image would read the file to determine the kind of compression and take appropriate action (either successfully decompress to find a payload that matches the image's disk_format, or fail with 'unsupported container_format'). Our preliminary testing with a raw image with container_format == compressed, disk_format == raw is that Nova boots a VM from the image but the VM appears to be unusable. Nova seems taking it as a real RAW image as disk_format indicates. So we want to open a discussion about handling this. Thanks! [0] cinder spec patch https://review.opendev.org/#/c/652275/ [1] cinder code implementation https://review.opendev.org/#/c/668825 [2] glance spec patch https://review.opendev.org/#/c/670454/

1 0

[Nova] migration issue
by Budai Laszlo 25 Jul '19

25 Jul '19

Dear all, we are testing the cold migratoin of instances that are using local ephemeral storage and it fails with the following error: 2019-07-24 13:47:13.115 58902 INFO nova.virt.libvirt.driver [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae 58-00d6-4317-b635-909fdf09ac49] Instance shutdown successfully after 3 seconds. 2019-07-24 13:47:13.126 58902 INFO nova.virt.libvirt.driver [-] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Instance destroyed successfully. 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae58- 00d6-4317-b635-909fdf09ac49] Setting instance vm_state to ERROR: ProcessExecutionError: Unexpected error while running command. Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph 0 Exit code: 1 Stdout: u'' Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured syst em and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. * \n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No suc h file or directory\n' 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Traceback (most recent call last): 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 75 18, in _error_out_instance_on_exception 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] yield 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 42 75, in _resize_instance 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] timeout, retry_interval) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", lin e 8200, in migrate_disk_and_power_off 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] shared_storage) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220 , in __exit__ 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] self.force_reraise() 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196 , in force_reraise 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", lin e 8185, in migrate_disk_and_power_off 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] compression=compression) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 226, in copy_image 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] compression=compression) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs .py", line 110, in copy_file 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] compression=compression) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs .py", line 196, in copy_file 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] on_execute=on_execute, on_completion=on_completion) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/utils.py", line 231, in exec ute 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] return processutils.execute(*cmd, **kwargs) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_concurrency/processutils.py" , line 424, in execute 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] cmd=sanitized_cmd) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] ProcessExecutionError: Unexpected error while running command. 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Exit code: 1 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Stdout: u'' 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured system and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. *\n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No such file or directory\n' 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] 2019-07-24 13:47:14.380 58902 INFO nova.compute.manager [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Swapping old allocation on 3da11b23-5ef8-4471-a3c9-5f743b0bbfd7 held by migration da66c416-2636-421b-99cb-470bd07dad4c for instance 2019-07-24 13:47:14.979 58902 INFO nova.compute.manager [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Successfully reverted task state from resize_migrating on failure for instance. 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] Exception during message handling: ProcessExecutionError: Unexpected error while running command. Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 Exit code: 1 Stdout: u'' Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured system and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. *\n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No such file or directory\n' 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server function_name, call_dict, binary) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 186, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 156, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/utils.py", line 977, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 214, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 202, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 4240, in resize_instance 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self._revert_allocation(context, instance, migration) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 4237, in resize_instance 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server instance_type, clean_shutdown) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 4275, in _resize_instance 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server timeout, retry_interval) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8200, in migrate_disk_and_power_off 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server shared_storage) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8185, in migrate_disk_and_power_off 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server compression=compression) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 226, in copy_image 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server compression=compression) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs.py", line 110, in copy_file 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server compression=compression) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs.py", line 196, in copy_file 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server on_execute=on_execute, on_completion=on_completion) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/utils.py", line 231, in execute 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return processutils.execute(*cmd, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 424, in execute 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server cmd=sanitized_cmd) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server ProcessExecutionError: Unexpected error while running command. 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Exit code: 1 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Stdout: u'' 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured system and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. *\n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No such file or directory\n' 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server 2019-07-24 13:47:17.065 58902 INFO nova.virt.libvirt.imagecache [req-e73474d0-6857-4c52-9c93-73f3bae247b5 - - - - -] image ca89475a-cf15-46d1-ab2d-c2b96c403248 at (/var/lib/nova/instances/_base/ab9f9dd0888a4950702b9c993c82f7ab36a3ed07): checking 2019-07-24 13:47:17.066 58902 INFO nova.virt.libvirt.imagecache [req-e73474d0-6857-4c52-9c93-73f3bae247b5 - - - - -] Active base files: /var/lib/nova/instances/_base/ab9f9dd0888a4950702b9c993c82f7ab36a3ed07 2019-07-24 13:47:27.768 58902 INFO nova.compute.manager [-] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] VM Stopped (Lifecycle Event) 2019-07-24 13:47:27.918 58902 INFO nova.compute.manager [req-e53e928d-81fe-4185-9e62-abc2113e838c - - - - -] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. The disk has its root FS n an RBD device (a volume created by cinder) and it has its ephemeral disk on a local LVM volume created by nova. It seems that the /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 is not there when scp is trying to copy it to the other side. Any ideas how to fix this? The nova config on the nodes looks like this: # Ansible managed [DEFAULT] allow_resize_to_same_host = True # Compute compute_driver = libvirt.LibvirtDriver # Scheduler cpu_allocation_ratio = 2.0 # Logs / State debug = False # Hypervisor default_ephemeral_format = ext4 disk_allocation_ratio = 1.0 # Api's enabled_apis = osapi_compute,metadata executor_thread_pool_size = 64 fatal_deprecations = False # Configdrive force_config_drive = False host = compute8.mgmt.lab.mydomain.com image_cache_manager_interval = 0 instance_name_template = instance-%08x # Ceilometer notification configurations instance_usage_audit = True instance_usage_audit_period = hour instances_path = /var/lib/nova/instances ## Vif libvirt_vif_type = ethernet log_dir = /var/log/nova # Metadata metadata_workers = 16 # Network my_ip = 192.168.56.47 notify_on_state_change = vm_and_task_state osapi_compute_workers = 16 ram_allocation_ratio = 1.0 reserved_host_disk_mb = 0 reserved_host_memory_mb = 2048 resume_guests_state_on_host_boot = False rootwrap_config = /etc/nova/rootwrap.conf rpc_response_timeout = 60 service_down_time = 120 state_path = /var/lib/nova # Rpc all transport_url = rabbit://nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.194:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.209:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e 8cf8bd7@192.168.56.179:5671//nova # Disable stderr logging use_stderr = False vif_plugging_is_fatal = True vif_plugging_timeout = 30 [api] auth_strategy = keystone enable_instance_password = True use_forwarded_for = False vendordata_jsonfile_path = /etc/nova/vendor_data.json # Cache [cache] backend = oslo_cache.memcache_pool enabled = true memcache_servers = 192.168.56.68:11211,192.168.56.85:11211,192.168.56.170:11211 # Cinder [cinder] cafile = /etc/ssl/certs/ca-certificates.crt catalog_info = volumev3:cinderv3:internalURL cross_az_attach = True os_region_name = A_Lab [conductor] workers = 16 [filter_scheduler] available_filters = nova.scheduler.filters.all_filters enabled_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, AggregateRamFilter, ComputeFilter, AggregateCoreFilter, DiskFilter, AggregateDiskFilter, AggregateNumInstancesFilter, AggregateIoOpsFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, NUMATopologyFilter, SameHostFilter, DifferentHostFilter host_subset_size = 10 max_instances_per_host = 50 max_io_ops_per_host = 10 ram_weight_multiplier = 5.0 tracks_instance_changes = True weight_classes = nova.scheduler.weights.all_weighers # Glance [glance] api_servers = https://vip.mgmt.lab.mydomain.com:9292 cafile = /etc/ssl/certs/ca-certificates.crt [key_manager] fixed_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [keystone_authtoken] auth_type = password auth_uri = https://vip.mgmt.lab.mydomain.com:5000 auth_url = https://vip.mgmt.lab.mydomain.com:35357 cafile = /etc/ssl/certs/ca-certificates.crt insecure = False memcache_secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx # if your memcached server is shared, use these settings to avoid cache poisoning memcache_security_strategy = ENCRYPT memcached_servers = 192.168.56.68:11211,192.168.56.85:11211,192.168.56.170:11211 password = xxxxxxxxxxxxxxxxxxxxxxx project_domain_id = default project_name = service region_name = A_Lab token_cache_time = 300 user_domain_id = default username = nova [libvirt] disk_cachemodes = network=writeback hw_disk_discard = unmap images_rbd_ceph_conf = /etc/ceph/ceph.conf images_rbd_pool = vms images_type = lvm images_volume_group = ssd_nova inject_key = False inject_partition = -2 inject_password = False live_migration_tunnelled = True live_migration_uri = "qemu+ssh://nova@%s/system?no_verify=1&keyfile=/var/lib/nova/.ssh/id_rsa" rbd_secret_uuid = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx # ceph rbd support rbd_user = cinder remove_unused_resized_minimum_age_seconds = 3600 use_virtio_for_bridges = True virt_type = kvm # Neutron [neutron] auth_type = password # Keystone client plugin authentication URL option auth_url = https://vip.mgmt.lab.mydomain.com:35357/v3 cafile = /etc/ssl/certs/ca-certificates.crt default_floating_pool = public insecure = False metadata_proxy_shared_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxx # Keystone client plugin password option password = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx project_domain_name = Default project_name = service region_name = A_Lab service_metadata_proxy = True url = https://vip.mgmt.lab.mydomain.com:9696 user_domain_name = Default # Keystone client plugin username option username = neutron [oslo_concurrency] lock_path = /var/lock/nova # Notifications [oslo_messaging_notifications] driver = messagingv2 notification_topics = notifications transport_url = rabbit://nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.194:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.209:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.179:5671//nova [oslo_messaging_rabbit] rpc_conn_pool_size = 30 ssl = True # Placement [placement] auth_type = "password" auth_url = https://vip.mgmt.lab.mydomain.com:35357/v3 cafile = /etc/ssl/certs/ca-certificates.crt insecure = False os_interface = internal os_region_name = A_Lab password = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx project_domain_name = Default project_name = service user_domain_name = Default username = placement [quota] cores = 20 injected_file_content_bytes = 10240 injected_file_path_length = 255 injected_files = 5 instances = 10 key_pairs = 100 max_age = 0 metadata_items = 128 ram = 32768 server_group_members = 10 server_groups = 10 [scheduler] discover_hosts_in_cells_interval = 60 host_manager = host_manager max_attempts = 5 periodic_task_interval = 60 scheduler_driver = filter_scheduler [spice] agent_enabled = True enabled = True # Console Url and binds html5proxy_base_url = https://dashboard.iaas.lab.mydomain.com:6080/spice_auto.html server_listen = 192.168.56.47 server_proxyclient_address = 192.168.56.47 [upgrade_levels] compute = auto [vnc] enabled = False [wsgi] api_paste_config = /etc/nova/api-paste.ini secure_proxy_ssl_header = HTTP_X_FORWARDED_PROTO Thank you in advance for any suggestions. Kind regards, Laszlo

5 10

[nova] Train Spec Freeze
by Eric Fried 25 Jul '19

25 Jul '19

Hi Nova contributors and maintainers. Spec freeze for Nova is Thursday, July 25th. This means that any specs not approved by EOD [1] become drastically less likely to land in Train. For your convenience, here is a search that's intended to catch open Train specs while filtering out things like updates to existing specs: https://review.opendev.org/#/q/project:openstack/nova-specs+status:open+pat… At the time of this writing, there are 15 on that list. Of those, IMO two are "close": Add Unified Limits Spec - https://review.opendev.org/602201 - Two +1s, needs review from "Nova API subteam" Spec for the Nova part of Image Encryption - https://review.opendev.org/608696 - +3, needs a second core If you own a spec you think should be able to land [2], please: - Check it to make sure any comments have been addressed (even if there's no -1). - Explicitly ask for reviews, here on the mailing list and/or in IRC in #openstack-nova. Reviewers, please make an effort to ensure that all the specs on that list have at least one vote. (Currently there are six with no vote.) I'll send an update Friday or Monday with a final tally. Thanks, efried [1] i.e. by the time cores have signed off for the day, wherever they are [2] meaning it has already been through some reviews and seems to have support. If you just proposed your spec this week, it's not reasonable to expect it to land.

2 1

Heat resource types - aodh and ceilometer
by Dhairyasheel Ghadge 25 Jul '19

25 Jul '19

Hello all, I have installed heat engine along with heat dashboard for orchestration in openstack cloud. I am creating a template to autoscale a web server pool and spin a new VM when load on other one increases. For the autoscaling purpose, I require to have an alarming system which will monitor the cp meter of the instance, which is provided by ceilometer and aodh services of openstack. I cannot find resource types of aodh and ceilometer on the heat dashboard. These same resource types are present in the heat openstack resources directory. Are there any steps to be carried out to make it available on the dashboard? Thanks & Regards Dhairyasheel

1 0

[nova] [horizon] Horizon and nova-cli inconsistency in live migration parameters
by Mary Atakova 25 Jul '19

25 Jul '19

Hi everyone, I think I have found an inconsistency between nova cli and Horizon in the way they send live migration commands to nova API. I'm not sure where this needs to be fixed, so please advise on that. The situation: I try to live-migrate a VM, and I do it through nova cli and through Horizon, and nova-api logs contain different data depending on the way I do it. 1. nova cli I run the command: nova live-migration 17359460-d23c-4acc-a9b1-5cf117b54430 and it shows up in nova-api logs as: {"os-migrateLive": {"block_migration": "auto", "host": null}} 2. horizon I run live migration via Horizon with unchecked Disk Overcommit and unchecked Block Migration. It shows up in nova-api logs as: {"os-migrateLive": {"disk_over_commit": false, "block_migration": false, "host": null}} This difference can lead to different parameters provided for live migration when block_migration: auto is translated into block_migration: true. My environment: Openstack Stein CentOS Linux release 7.6.1810 nova versions: # yum list installed | grep nova openstack-nova-api.noarch 1:19.0.1-1.el7 @centos-openstack-stein openstack-nova-common.noarch 1:19.0.1-1.el7 @centos-openstack-stein openstack-nova-conductor.noarch 1:19.0.1-1.el7 @centos-openstack-stein openstack-nova-console.noarch 1:19.0.1-1.el7 @centos-openstack-stein openstack-nova-novncproxy.noarch openstack-nova-placement-api.noarch openstack-nova-scheduler.noarch 1:19.0.1-1.el7 @centos-openstack-stein python2-nova.noarch 1:19.0.1-1.el7 @centos-openstack-stein python2-novaclient.noarch 1:13.0.0-1.el7 @centos-openstack-stein horizon version: # yum list installed | grep dashboard openstack-dashboard.noarch 1:15.1.0-1.el7 @centos-openstack-stein openstack-dashboard-theme.noarch 1:15.1.0-1.el7 @centos-openstack-stein Thank you in advance for your attention Best regards, Mary

2 2

DuplicateMessageError after restart of a control node
by Pawel Konczalski 25 Jul '19

25 Jul '19

Hello everybody, after restart of one of the control nodes i see this error message every few seconds int the logs: @timestamp July 23rd 2019, 16:44:08.927 t Hostname ctlX t Logger openstack.nova t Payload Failed to process message ... skipping it.: DuplicateMessageError: Found duplicate message(f2178cffb5d249f3a1d2df1c0322f18c). Skipping it. 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit Traceback (most recent call last): 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 361, in _callback 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit self.callback(RabbitMessage(message)) 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 251, in __call__ 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit unique_id = self.msg_id_cache.check_duplicate_message(message) 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py", line 122, in check_duplicate_message 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit raise rpc_common.DuplicateMessageError(msg_id=msg_id) 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit DuplicateMessageError: Found duplicate message(f2178cffb5d249f3a1d2df1c0322f18c). Skipping it. 2019-07-23 16:44:08.927 82 ERROR oslo.messaging._drivers.impl_rabbit t Pid 82 t Timestamp 2019-07-23 16:44:08.927 t _id AWwfSjEtsndUu9YeotwF t _index flog-2019.07.23 # _score - t _type fluentd t log_level ERROR t programname nova-scheduler t python_module oslo.messaging._drivers.impl_rabbit As soon as the restarted node is taken out of the cluster, the errors disappear. Any idea how to fix this? Thank you Pawel

2 2

[release] (Belated) Release countdown for week R-12, July 22-26
by Kendall Nelson 24 Jul '19

24 Jul '19

Hello Everyone, Development Focus ----------------- The Train-2 milestone is July 25! Train-related specs should now be finalized so that teams can move to implementation ASAP. Some teams observe specific deadlines on the second milestone (mostly spec freezes): please refer to https://releases.openstack.org/train/schedule.html for details. General Information ------------------- Libraries need to be released at least once per milestone period. This week, the release team will propose releases for any library that has not been otherwise released since milestone 1. See list at: http://paste.openstack.org/show/754817/. PTL's and release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well. A +1 would be appreciated, but if we do not hear anything at all by the end of the week, we will assume things are OK to proceed. Remember that non-library deliverables that follow the cycle-with-intermediary release model should have an intermediary release before milestone-2. Those who haven't will be proposed to switch to the cycle-with-rc model, which is more suited to deliverables that are released only once per cycle. This week is also the deadline to freeze the contents of the final release. All new 'Train' deliverables need to have a deliverable file ( https://opendev.org/openstack/releases/src/branch/master/deliverables/train) and need to have done a release by milestone-2. The following new deliverables have not had a release yet (or not been already discussed in the etherpad[1]), and will not be included in Train unless a release is requested for them in the coming week: python-adjutantclient, cyborg-tempest-plugin, neutron-interconnection, openstack-helm, openstack-helm-addons, openstack-help-docs, openstack-helm-images, openstack-helm-infra, and compute-hyperv. The full list of currently deliverable-less deliverables can be found here[2]. Upcoming Deadlines & Dates -------------------------- Train-2 Milestone: July 25 (R-12 week) Thanks! -Kendall Nelson(diablo_rojo) + the Release Management Team [1]https://etherpad.openstack.org/p/train-membership-freeze [2] http://paste.openstack.org/show/754816/

1 0

[oslo] PTO next week
by Ben Nemec 24 Jul '19

24 Jul '19

Just a heads up that I will be on PTO next week and incommunicado pretty much the whole time. Moises (moguimar) volunteered to run the Oslo meeting so that will still be happening. Herve (hberaud) and Doug (dhellmann) are our release liaisons so if you need a release of any Oslo libraries next week, they're the ones to talk to. Otherwise, things have been pretty quiet in Oslo-land (which would be Norway, I guess), so I expect all the things to break next week while I'm out. Good luck. ;-) Thanks. -Ben

1 0

[ops] [nova] How to change/remove a property from an instance ?
by Massimo Sgaravatto 24 Jul '19

24 Jul '19

In my OpenStack Rocky cloud I have some instances that were created (before updating to Rocky) using images with the property hypervisor_type='qemu'. Since in my compute nodes [libvirt]/virt_type is "kvm", after having updated to Rocky I had to change these images removing that property of changing its value (qemu --> kvm): this is discussed in the thread: http://lists.openstack.org/pipermail/openstack-discuss/2019-July/thread.htm… Now I am not able to migrate anymore the instances that were created using those images (ImagePropertiesFilter filters out all compute nodes) before fixing the property. As suggested by Matt I tried to play with the instance_system_metadata table of the nova database. I first tried to change the value of that property (qemu --> kvm) [*] I also tried to remove the property [**] but the nova migrate keeps failing. In the request I keep seeing that Instance contains properties ImageMetaProps...img_hv_type='qemu' What else needs to be modified? Thanks, Massimo [*] mysql> select * from instance_system_metadata where instance_uuid='57dcdc2f-ef30-4512-beff-128c1621c5c6'; +---------------------+------------+------------+---------+--------------------------------------+------------------------+--------------------------------------+---------+ | created_at | updated_at | deleted_at | id | instance_uuid | key | value | deleted | +---------------------+------------+------------+---------+--------------------------------------+------------------------+--------------------------------------+---------+ | 2017-05-15 12:32:57 | NULL | NULL | 9868217 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_hypervisor_type | kvm | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868220 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_disk_format | qcow2 | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868223 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_container_format | bare | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868226 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_min_ram | 512 | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868229 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_min_disk | 20 | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868232 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_base_image_ref | c2d3e37c-b14f-4472-9bc2-d75f26ab900d | 0 | +---------------------+------------+------------+---------+--------------------------------------+------------------------+--------------------------------------+---------+ [**] mysql> delete from instance_system_metadata where id='9868217'; Query OK, 1 row affected (0.01 sec) mysql> select * from instance_system_metadata where instance_uuid='57dcdc2f-ef30-4512-beff-128c1621c5c6'; +---------------------+------------+------------+---------+--------------------------------------+------------------------+--------------------------------------+---------+ | created_at | updated_at | deleted_at | id | instance_uuid | key | value | deleted | +---------------------+------------+------------+---------+--------------------------------------+------------------------+--------------------------------------+---------+ | 2017-05-15 12:32:57 | NULL | NULL | 9868220 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_disk_format | qcow2 | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868223 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_container_format | bare | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868226 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_min_ram | 512 | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868229 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_min_disk | 20 | 0 | | 2017-05-15 12:32:57 | NULL | NULL | 9868232 | 57dcdc2f-ef30-4512-beff-128c1621c5c6 | image_base_image_ref | c2d3e37c-b14f-4472-9bc2-d75f26ab900d | 0 | +---------------------+------------+------------+---------+--------------------------------------+------------------------+--------------------------------------+---------+ 5 rows in set (0.00 sec)

2 1

[opendev] git clone fails: "The TLS connection was non-properly terminated"
by Bernd Bausch 24 Jul '19

24 Jul '19

There seems to be something wrong with the opendev server. Is this a glitch, a daily occurrence (backup?), or a deeper problem? This happens around midnight between July 22 and 23 UTC. $ git clone https://opendev.org/openstack/devstack.git -b stable/stein Cloning into 'devstack'... fatal: unable to access 'https://opendev.org/openstack/devstack.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated. Same result when I try to clone any other random repo. I can get Devstack from Github, but when setting up the cloud, stack.sh wants to clone everything from git.openstack.org, which fails again. I could replace all the hardcoded /git.openstack.org/ strings with the equivalent /github.com/ URL, but I am not sure if I can trust that all Github repos are up to date. Bernd.

4 7

[devstack] Devstack fails with requirement pyparsing===2.4.1
by Bernd Bausch 24 Jul '19

24 Jul '19

Trying to install Devstack Stein on freshly installed Ubuntu 18.04. stack.sh fails with this: Downloading https://files.pythonhosted.org/packages/6f/7c/ebba93a2d3e09fd0220e1adfdb823… (131kB) Collecting pyparsing===2.4.1 (from -c https://releases.openstack.org/constraints/upper/master (line 492)) Could not find a version that satisfies the requirement pyparsing===2.4.1 (from -c https://releases.openstack.org/constraints/upper/master (line 492)) (from versions: 1.4.6, 1.4.7, 1.4.8, 1.4.11, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 1.5.7, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6, 2.1.7, 2.1.8, 2.1.9, 2.1.10, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0) No matching distribution found for *pyparsing===2.4.1* (from -c https://releases.openstack.org/constraints/upper/master (line 492)) (red highlight is mine) The most recent version of pyparsing <https://pypi.org/project/pyparsing/>is 2.4.0. I would file a bug, but I don't know against which project. Who owns https://releases.openstack.org/constraints/upper/master? Or is there a misunderstanding somewhere? My local.conf is below - it's based on a Rocky local.conf (which does work on Rocky) and might be incorrect. Bernd. [[local|localrc]] ########################################################################### ### Passwords ADMIN_PASSWORD=MYPASSWORD DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD ########################################################################### ### Logging LOG_COLOR=False LOGFILE=$DEST/logs/stack.sh.log ########################################################################### ### Networking IP_VERSION=4 HOST_IP=192.168.1.200 ########################################################################### ### Software sources # use https since the vlabs proxy seems to block the git:// scheme GIT_BASE=https://git.openstack.org # Ensure the latest software is installed PIP_UPGRADE=True ########################################################################### ### Glance DOWNLOAD_DEFAULT_IMAGES=False IMAGE_URLS="http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img" # Glance's image import doesn't work when Glance runs as a UWSGI server # For this setting to take effect, devstack/lib/glance must be patched GLANCE_USE_UWSGI=False ########################################################################### ### Swift # by default, Devstack interactively asks for a hash value. Avoid this. SWIFT_HASH=123 enable_service swift ########################################################################### ### Heat enable_plugin heat https://git.openstack.org/openstack/heat stable/stein enable_service h-eng h-api h-api-cfn h-api-cw enable_plugin heat-dashboard https://git.openstack.org/openstack/heat-dashboard stable/stein enable_service heat-dashboard ########################################################################### ### Cinder: Enable backup enable_service c-bak ########################################################################### ### Telemetry: Use Gnocchi as sample repository ### and configure medium archive policy CEILOMETER_BACKEND=gnocchi GNOCCHI_ARCHIVE_POLICY=medium enable_plugin ceilometer https://git.openstack.org/openstack/ceilometer stable/stein enable_plugin aodh https://git.openstack.org/openstack/aodh stable/stein ########################################################################### ### Neutron: Enable internal DNS resolution in ml2_conf.ini Q_ML2_PLUGIN_EXT_DRIVERS=port_security,dns_domain_ports ########################################################################### ### Post config for Neutron (DNS) and Nova (allow KVM in Vmware machine) [[post-config|$NEUTRON_CONF]] [DEFAULT] dns_domain = devstack.org. [[post-config|/etc/nova/nova-cpu.conf]] [libvirt] # set machine type to work around bug # https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1691109 hw_machine_type = x86_64=pc-i440fx-xenial

5 4

[mistral] Office hour in ~5 min
by Renat Akhmerov 24 Jul '19

24 Jul '19

Hi, We’ll have an office hour in about 5 mins. Join us at #openstack-mistral if you have anything to discuss. Thanks Renat Akhmerov @Nokia

1 0

[Kolla][Keystone] No "Domains" found in identity area.
by Eddie Yen 24 Jul '19

24 Jul '19

Hi, I'm using kolla-ansible stable/rocky release to deploy Openstack Try to test with multi-domains requirements. I'm do nothing but add "horizon_keystone_multidomain: True" into globals.yml After deployment, it works because I have to enter the domain name at login screen. Logged in with default domain and admin account. But I cannot found any "Domains" section in identity block. Is there something I miss?

1 2

[keystone] Keystone Train Midcycle Report
by Colleen Murphy 24 Jul '19

24 Jul '19

This week the keystone team conducted a two-day virtual midcycle. Thanks to everyone who participated and contributed to making this a successful and productive meeting! This email will be a short summary of the key takeaways from the last two days. The full agenda and meeting notes can be found on this etherpad: https://etherpad.openstack.org/p/keystone-train-midcycle-topics # Caching Guide We've long been proposing to document best practices for configuring caching for keystone. We began the meeting with a discussion about what exactly should be contained in such a guide, starting with anecdotes from the team about our own operational experience with configuring memcached for keystone. We quickly realized the guide needed at least two parts, one for keystone itself and one for keystonemiddleware. We also made two technical observations, first that the default value for the memcache_socket_timeout option in oslo.cache could be reduced[1] and second that the memcache_socket_timeout option exposed in keystone is not actually used by keystone and needs to be removed[2]. We talked about security strategies for memcached: oslo.cache exposes encryption capabilities which is required for organizations that consider the data stored in memcached to be "at rest", but it also hurts performance. An alternative strategy that could be recommended in the guide is to set up TLS tunneling to secure in-flight data without impeding fetching performance. The initial caching guide update was proposed here[3]. This guide will be a living document and will continue to grow as we gather real-life data from operators. After this initial update, we'll start taking steps to ensure that keystone requires caching be set up in order to operate. [1] https://review.opendev.org/672063 [2] https://bugs.launchpad.net/keystone/+bug/1837407 [3] https://review.opendev.org/672120 # Community Goals We then had a short status update session on the various community goals we need to meet. The mutable config goal from a few cycles ago is not yet complete but has an implementation proposed[4] which needs more eyes and testing. The PDF generation goal implementation is underway[5] but there are some issues already being tracked by the community[6]. We will need to work to ensure the issues we're seeing are noted and addressed. The last of the Python3 Train job changes for keystone repos were merged, so I think we've completed our end of that goal. We also already have a patch proposed to add the ipv6-only job[7] which is just waiting on the base job to merge in tempest. [4] https://review.opendev.org/585417 [5] https://review.opendev.org/669982 [6] https://etherpad.openstack.org/p/pdf-goal-train-common-problems [7] https://review.opendev.org/671903 # Milestone 2 Train Retrospective We closed out the first day with a mid-cycle retrospective[8]. Some key areas for improvement from this were: * Do more video conferencing - The real-time face-to-face communication is really effective for getting to the heart of discussions and making decisions quickly - Since it is looking likely that most of us will not be at the next PTG in Shanghai, we can set up a pre- and post- PTG video meeting to plan the cycle and catch people up after the IRL PTG - Milestone-ly video meetings are working for helping us keep pace and sync up - We can keep a queue of potential topics and make sure we have a meeting when there are enough topics to discuss - We'll try using video conferencing for office hours - We'll try Jitsi again even though there were some technical issues last time since it is OSS and supports recording * Improve office hours - Office hours still don't feel useful and haven't happened regularly, but we still think they have potential - We'll use the weekly newsletter to commit to the topic the week before, which gives the team a better opportunity to prepare for it and can also attract interest from people outside the keystone team - Use video conferencing as mentioned above * Improve the weekly newsletter - We'll use the weekly newsletter to call out action items from the last meeting - In addition to providing statistics about the number of open reviews, we'll highlight certain reviews that need attention based on various criteria such as age, priority with regard to cycle commitments, community goals, and specific review requests. - We'll announce the following week's office hours topic in the newsletter [8] https://etherpad.openstack.org/p/keystone-train-midcycle-retrospective # System Scope and Default Roles The second day of the midcycle was used to go through our list of open system scope[9] and default roles[10] bugs and work through what scopes and roles each API should support. The notes in the etherpad largely capture every decision fairly well, but a few issues worth highlighting are: * Implied roles[11]: the regular roles API is currently system-scope only, even though it's possible to have domain-specific roles, because for the most part for a role to be useful the role's creator must also have the ability to configure policy, which domain admins do not. Implied roles still require access to policy configuration except in the case where you are only using implied roles for aliasing, in which case it may make sense for a domain admin to have the ability to create an implied role relationship. We decided to start with making the implied roles API system-scope only for now and to open a separate bug to discuss how to deal with domain-specific roles and implied roles. * Revocation list[12]: we're currently not enforcing policy on this API at all because it can only ever return a 410 Gone or a 403 Forbidden, never a success, so we will just deprecate the policy entirely rather than updating it to work with system scope or default roles. We reevaluated the priority on some of the bugs in the list. However, we noted that while some APIs are more heavily used than others, we can't change enforce_scope to True until all of them are completed - picking and choosing the most important ones and letting the others slide is not really an option. [9] https://bugs.launchpad.net/keystone/+bugs?field.tag=system-scope [10] https://bugs.launchpad.net/keystone/+bugs?field.tag=default-roles [11] https://bugs.launchpad.net/keystone/+bug/1805371 [12] https://bugs.launchpad.net/keystone/+bug/1818845 If I've left out any important details, don't hesitate to add them to this thread. Thanks again to everyone who was able to join the meeting! Thanks also to Julia Kreger for her advice on how to hold one of these virtual midcycles, it proved to be very useful and this first-timer really appreciated it :) Colleen

1 0

[tripleo] CI scenario001-standalone broken
by John Fulton 23 Jul '19

23 Jul '19

As per $subject [0]. scenario010 is affected too. This correlates with the promotion of the ceph-ansible 4rc11 RPM [1]. 001 jobs using rc9 do not have this problem. I'm looking into a fix, e.g. removing the storage7-ceph-nautilus-release tag from the rc11 package as TripleO CI uses the version of ceph-ansible which is tagged for release [2]. I'll keep this thread updated with the status and when it's fixed. John [0] http://logs.openstack.org/74/670574/4/check/tripleo-ci-centos-7-scenario001… [1] https://cbs.centos.org/koji/buildinfo?buildID=26343 [2] https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/r…

3 4

[Diversity] Grace Hopper Lead Co-Mentor(s) needed
by Amy Marrich 23 Jul '19

23 Jul '19

OpenStack has participated in Open Source Day at the Grace Hopper Conference for the last 5 years and this year it is in Orlando on October 3rd. Due to unforeseen circumstances we are one lead mentor short to participate this year and are looking for someone willing to help lead the session. The Anita Borg Institute will provide a ticket for the entire conference, the downside is you or your company will be responsible for your travel, lodging and food. This year CityNetwork is supplying the VM infrastructure for us to use. The session submitted is as follows: Participants will sign up for the necessary accounts needed to contribute to OpenStack and install Git and Gerrit. They will configure their systems and learn how to file bugs and review code within the Open Dev infrastructure. Due to time and resource requirements the groups will be given a virtual machine running Devstack in which to create their own theme for the Horizon Dashboard. The group will decide on which humanitarian effort they wish to support but suggestions will be provided for those groups that can not decide. The group will also work as a team to locate the necessary documentation on the OpenStack site if they have not done so in preparation. The selected Project Manager will lead these discussions as well as making sure efforts are staying on track to meet the deadline of the demonstrations. The Developers in the group will work to locate the necessary files and directories on the VM in order to complete their task. They will be in charge of creating/changing the necessary files to create a new theme utilizing SCSS and restarting any services needed to implement. The Graphic Designer if present will create any necessary graphics for the theme and will either scp them themselves to the virtual machine or provide to the developer. If there is no designated graphic designer the PM and devs will divide this duty up. After the themes are designed relevant files will be scped back down to the participants machines and they will be able to commit their code for review in a sandbox environment for other groups to review. If you are interested in participating please let me know by July 20th as we need to send additional information to the OSD folks on July 21st. Any questions please reach out on the lists, IRC, or privately! Thanks, Amy (spotz)

1 1

[Octavia]-Seeking performance numbers on Octavia
by Singh, Prabhjit 23 Jul '19

23 Jul '19

Hi I have been trying to test Octavia with some traffic generators and my tests are inconclusive. Appreciate your inputs on the following * It would be really nice to have some performance numbers that you guys have been able to achieve for this to be termed as carrier grade. * Would also appreciate if you could share any inputs on performance tuning Octavia * Any recommended flavor sizes for spinning up Amphorae, the default size of 1 core, 2 Gb disk and 1 Gig RAM does not seem enough. * Also I noticed when the Amphorae are spun up, at one time only one master is talking to the backend servers and has one IP that its using, it has to run out of ports after 64000 TCP concurrent sessions, id there a way to add more IPs or is this the limitation * If I needed some help with Octavia and some guidance around performance tuning can someone from the community help Thanks & Regards Prabhjit Singh

3 5

[ironic] Using ironic in a devstack environment to deploy OS on bare-metal host.
by Guannan GN2 Sun 23 Jul '19

23 Jul '19

Hi team, I met some problem when using ironic in a devstack environment to deploy OS on bare-metal host. By now I can successfully boot into the tiny os and set ip, also I can ping the ip from my devstack server. However, when the tiny os running some configuration, it will failed after about 5 minutes. I found some log information in "/var/log/ironic-python-agent.log" in the tiny os that may relate to the problem: "DEBUG oslo_concurrency.processutils [-] u'iscsistart -f' failed. Not Retrying. execute /usr/local/lib/python2.7/site-packages/oslo_concurrency/processutils.py:457" "DEBUG root [-] No iscsi connection detected. Skipping iscsi. Error: [Errno 2] No such file or directory _check_for_iscsi /usr/local/lib/python2.7/site-packages/ironic_python_agent/hardware.py:107" "WARNING root [-] The root device was not detected in 27 seconds: DeviceNotFound: Error finding the disk or partition device to deploy the image onto: No suitable device was found for deployment - root device hints were not provided and all found block devices are smaller than 4294967296B." "WARNING root [-] Can't find field vendor for device usb0 in device class net: IOError: [Errno 2] No such file or directory: '/sys/class/net/usb0/device/vendor'" "WARNING root [-] Can't find field vendor for device tunl0 in device class net: IOError: [Errno 2] No such file or directory: '/sys/class/net/tunl0/device/vendor'" "DEBUG root [-] No Mellanox devices found evaluate_hardware_support /usr/local/lib/python2.7/site-packages/ironic_python_agent/hardware_managers/mlnx.py:84" And when it deploy faild and I run command "openstack baremetal node show server-51", it will show following information on 'last_error' field: "Asynchronous exception: Node failed to deploy. Exception: Connection to agent failed: Failed to connect to the agent running on node f8794658-bc69-40cb-9673-be75f78ced21 for invoking command iscsi.start_iscsi_target. Error: ('Connection aborted.', BadStatusLine("''",)) for node" I'm trying to debug, but have not find the root cause, please give me your advice if someone have experience on it. Thank you! Best Regards, Guannan

2 1

[dev][magnum] Magnum event notifications not having relevant content related to clusters
by Emnauel Andrecut 23 Jul '19

23 Jul '19

I'm writing about Magnum event notifications that do not contain any relevant information about the related Cluster (missing cluster id, other attributes..) As I've seen the PyCADF library receives no targetId and so the id (and mostly any other id) is randomly generated. Do you plan to add more information about the cluster on create success/pending events. Or would you accept a patch that does that?

3 2

[all][meta-sig][containers-sig] Creation of Containers SIG
by Jean-Philippe Evrard 23 Jul '19

23 Jul '19

Hello everyone, I have discussed with some of you about the fact OpenStack is lacking a a group interested in sharing the practices for the building/management of OpenStack application containers. Every container related project is doing its own way (and it's probably fine), but very few experiences are shared (and it's sad). I started an etherpad [1] to see areas where container projects/people could collaborate. I already received significant feedback on how to improve some of my practices, so I think it's already a positive experience for me to have this group started. Anyway, long story short, I believe we should create a SIG for this, so I proposed the addition of the containers-sig into governance [2]. We have enough work to started this group nowadays (with documentation, bindep shared practices, and improvement of projects). I suppose technologies will evolve over time, so I expect a SIG is the best tool to do this job of *for example* gathering experience, as it's not really time bound, and it _is_ a quite special interest. If you're interested by learning what is going on in the container building world for OpenStack, please add [containers-sig] to your openstack-discuss topics watch list. Regards, Jean-Philippe Evrard (evrardjp) [1]: https://etherpad.openstack.org/p/containers-sig [2]: https://review.opendev.org/#/c/666001/

1 0

[kolla] Stein release now available
by Mark Goddard 23 Jul '19

23 Jul '19

Hi, I'm pleased to announce that the first official Stein cycle releases for Kolla and Kolla Ansible are now available - both 8.0.0. A lot of work has gone into this release, including improving the CI tests, making it the most well tested release yet. Thanks to everyone who contributed. Release notes: https://docs.openstack.org/releasenotes/kolla/stein.html https://docs.openstack.org/releasenotes/kolla-ansible/stein.html Join us on #openstack-kolla to help make the Train cycle release even better. Cheers, Mark

1 0

[telemetry] Looking for new meeting time
by Trinh Nguyen 23 Jul '19

23 Jul '19

Hi team, I've changed job and could not afford doing the meeting at 02:00 UTC on Thursday anymore. My possible time slot is from 13:00-15:00 UTC Tuesday-Thursday. Please propose a better time frame for our team meeting. Sincerely, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

2 3

[nova] FYI to out-of-tree drivers: adding a positional argument to finish_revert_migration()
by Artom Lifshitz 22 Jul '19

22 Jul '19

[1] will most likely merge in its current shape or something very close to it. It adds a `migration` positional argument to the finish_revert_migration() virt driver method. `migration` is the, err, migration that's being reverted. If you have out of tree virt drivers, please update them to match this new method signature. Thanks! [1] https://review.opendev.org/#/c/668631/

1 0

[tripleo] TripleO CI Summary: Sprint 33
by Rafael Folco 22 Jul '19

22 Jul '19

Greetings, The TripleO CI team has just completed Sprint 33 / Unified Sprint 12 (Jun 27 thru Jul 17). The following is a summary of completed work during this sprint cycle: - Tested check jobs for periodic image/container builds on RHEL8 and resolved its dependencies on rhui repositories. - Enabled tripleo-repos to support RHEL8. - Addressed changes in kolla to support RHEL8. - Created a nodepool image on RDO for RHEL8 w/ elements and dependencies installed. - Improved container build script to consolidate kolla build logs and ease troubleshooting. - Manually pushed rhel containers to RDO registry and identified gaps in authentication workflow for the container build jobs using buildah tooling. - Modified scripts and post playbooks to upload RHEL8 overcloud images. - Created RHEL8 jobs to build a periodic pipeline in the RDO Software Factory and provide feedback for CentOS8 coverage. - Resumed scenario007 and featureset039 job updates upstream. - Promotion status: green on all branches at most of the sprint. The planned work for the next sprint [1] are: - Get RDO on RHEL8 build jobs working in periodic pipeline and bootstrap standalone and featureset001 jobs to consume RHEL8 images/containers. - Get featureset039 job to run novajoin tests and merge scenario007 job changes. - Review templates for branched and non-branched projects to have more control on what jobs are being run and reduce queue lengths. - Resume the design work for a staging environment to test changes in the promoter server for the multi-arch builds. The Ruck and Rover for this sprint are Marios Andreou (marios) and Rafael Folco (rfolco). Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes are being tracked in etherpad [2]. Thanks, rfolco [1] https://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-13 [2] https://etherpad.openstack.org/p/ruckroversprint13

1 0