April 2019 - openstack-discuss - lists.openstack.org

[docs] Documenting CLI references (openstack-auto-commands)
by Sean McGinnis 19 Apr '19

19 Apr '19

Hey everyone, Hoping someone has discussed this and has a recommended course of action. Back when we had the awesome support of the openstack-docs team, there was a command called openstack-auto-commands that would load up client commands and generate help output for all commands. With the resource constraints that hit the team, there were less folks to support tools like this. Docs moved into each team's repos and, for the most part, I think that has worked out well. Some of the automatic syncing stopped happening with this move to in-repo docs. One of them was CLI docs. But it would appear there are at least a few teams that now have not updated these docs since that move happened several releases ago: http://codesearch.openstack.org/?q=This%20file%20is%20tool-generated&i=nope… There are others that appear to have used the last automatically generated version and have been (varying degrees of) successful at just manually updating the doc as things change. When the tool was removed (https://review.openstack.org/#/c/509402/) it was stated that it wasn't needed because cliff has sphinx integration to do the equivalent automatically. That's all well and good... if you are using cliff. But I think many of our legacy clients are not. So looking for recommendations on ways to best keep these docs up to date now. Migrate to cliff? Other in-repo automation tools? (and yeah, I know, "drop the CLI and move to openstackclient) Thanks for any suggestions! Sean

2 2

shelve server taking too long
by Manuel Sopena Ballesteros 19 Apr '19

19 Apr '19

Dear Openstack community, I am trying to shelve a server but the process seems to be stuck in shelving_image_pending_upload state. is it normal for the shelve process to take several hours? thank you NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

2 1

[searchlight] Searchlight planning for Train
by Trinh Nguyen 18 Apr '19

18 Apr '19

Hi team, We just released Searchlight for Stein last week successfully. Congratulations to all of you who have contributed to the project. We did it!!!! So now the Train cycle started [1]. I would like to plan for the things that we would want to accomplish in this new cycle. Please have a look at the etherpad [2] and put your ideas or comment on existing ones. [1] https://releases.openstack.org/train/schedule.html [2] https://etherpad.openstack.org/p/searchlight-train Yours, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

1 0

[tripleo] Scale up/down Ansible tasks
by Emilien Macchi 18 Apr '19

18 Apr '19

Hi folks, Today I spent a bit of time on: https://blueprints.launchpad.net/tripleo/+spec/scale-down-tasks Which is basically adding the capability of running Ansible tasks before a node is removed during a scale down or after a scale-up. I'm focusing on the scale-down right now, as I know it's something people have been waiting for (e.g. RHSM unsubscribe, Ceph OSD tear down, Nova Compute, etc). I need inputs from folks now, on what kind of tasks would be needed, I will test them and make sure the interface we provide is enough. John, Olie, and Martin in copy have maybe some ideas, please let me know some examples of Ansible tasks that you folks want to run before a node is deleted in Ironic. Prototype: https://review.openstack.org/#/q/topic:bp/scale-down-tasks+(status:open+OR+… Thanks a lot, -- Emilien Macchi

4 7

[ops] offer to host next OpenStack Ops meetup 2019-9-3, 2019-9-4 NYC
by Chris Morgan 18 Apr '19

18 Apr '19

Hello All, My employer Bloomberg LP would like to offer to host the next OpenStack Ops meetup here in NYC in early September this year. Please see https://etherpad.openstack.org/p/ops-meetup-2nd-2019 for some early details on this. All hosting offers are subject to scrutiny by the ops meetups team and the overal community for content, location, timing etc and then approved via a mix of consensus gathering and (if there are multiple offers) voting, see https://wiki.openstack.org/wiki/Ops_Meetups_Team and https://wiki.openstack.org/wiki/Operations/Meetups Thanks, Chris -- Chris Morgan <mihalis68(a)gmail.com>

7 6

[nova][ptg] Stein retrospective
by Eric Fried 18 Apr '19

18 Apr '19

All- I have seeded an etherpad for Nova Stein nostalgia [1]. Please populate it. Following tradition, I'm putting this first on the schedule. Unless this etherpad gets just huge, I'm going to time box the discussion to 30 minutes. Make it count. Thanks, efried [1] https://etherpad.openstack.org/p/nova-ptg-train-retrospective

1 0

Re: [openstack-dev] [nova][xenapi][ptg] can we deprecate the xenapi-specific 'nova-console' service?
by Eric Fried 18 Apr '19

18 Apr '19

Scrubbing the Nova PTG agenda (hence addition of [ptg] subject tag), and this is on it: ================ Can we remove the nova-console service? (mriedem) The nova-console service is xenapi-specific and was deprecated in stein: https://review.openstack.org/#/c/610075/ There are, however, REST APIs for it: https://developer.openstack.org/api-ref/compute/#server-consoles-servers-os… So if we drop the nova-console service, the APIs would no longer work. It seems our options are: Delete the nova-console service and obsolete the REST APIs (410 response on all microversions like what we're doingwith nova-cells and nova-network APIs) Deprecate the REST APIs on a new microversion but continue to support older microversions - this means the nova-console service would live forever. Are people still using the nova-console service? Are there alternatives/replacements for xen users? BobBall seemed to suggest there was http://lists.openstack.org/pipermail/openstack-dev/2018-October/135422.html but it's not clear to me. ================ I'm looking for any excuse to remove things from the agenda, and this seems like a good candidate, because we don't really stand to gain anything by bringing it up at the PTG. Does anyone a) object to removing the nova-console service; or b) think we need to discuss it at the PTG? Thanks, efried .

2 1

[neutron] Cancelling drivers meeting
by Miguel Lavalle 18 Apr '19

18 Apr '19

Dear Neutrinos, We don't have RFEs lined up for discussion for tomorrow's meeting. So we are cancelling it this week Best regards Miguel

1 0

[nova] Privsep is not giving us any security
by Matthew Booth 18 Apr '19

18 Apr '19

Privsep is intended to provide some security in depth. The idea is based on the principal of least privilege. We assume that an attacker has exploited some bug in Nova and is able to execute arbitrary code. If privsep is doing its job, the attacker should at least not be able to perform any privileged operations which Nova does not normally perform. For developers, from the spec[1]: "The intention is to allow slightly more code into the privileged portion - enough that we now have sufficient “context” to make better security decisions. For example move from “run chown” to “take ownership of VM output file”." Unfortunately, I suspect out of expediency in the initial forklift from rootwrap, we've lost this critical principal of moving security-critical logic into privsep itself. Consequently, a brief and non-exhaustive audit reveals we have privsep functions to do the following: fs.py: * mount anything of any type, including remote storage, anywhere. * unmount anything * shred any file or block device, without restriction path.py: * read any file or block device, without restriction * overwrite any existing file or block device utils.py: * send any signal to any process Just the existence of these methods in nova's privsep module essentially gives the unprivileged nova process unfettered root access, which entirely defeats the purpose of privsep. As it stands today, privsep gives us literally nothing. I suspect that it's worse than rootwrap, in fact. If privsep protection is something we want, we need to do 2 things: 1. Immediately stop adding or using generic privsep functions. 2. Audit privsep properly for problematic methods, and remove them. For example, in libvirt.driver.get_console_output() we do nova.privsep.path.writefile(console_log, 'a+', data). Instead, we should have, eg, nova.privsep.libvirt.ensure_console_log_for_instance(instance). This would determine the instance path from its own secure config (see [1] again), and touch that file. We should probably also compile a list of privsep anti-patterns if there isn't one already to make life simpler for reviewers. For example, I suggest that no privsep function should take as an argument a path, or any other reference to a system resource; it should determine these itself. I view this as essentially a continuation of the privsep forklift. As with all such large mechanical changes I expect this to be disruptive and unloved by reviewers. Is it something we'd be prepared to prioritise? Matt [1] https://specs.openstack.org/openstack/oslo-specs/specs/liberty/privsep.html -- Matthew Booth Red Hat OpenStack Engineer, Compute DFG Phone: +442070094448 (UK)

6 14

[OpenStack-sigs][publiccloud-wg][Action Required] help most needed for SIGs and WGs
by Rico Lin 18 Apr '19

18 Apr '19

Dear all SIG and WG members I would like to ask for three favors from each group. Thank you for your kindly help. tl;dr: 1. As an action from [3], we will have a forum `Help most needed for SIGs and WGs` [4]. And would like to ask each group kindly provide me the following help most needed information, including - What's your team's name - Who's leading or active in this team - What's this team recently plan or doing - How people can join this team? - What's your most needed help interns of actions or technical features? - Which team (or who) you think that have a part in above help request (high potentially is fine)? - Anything else you wish to share to attendees? 2. update help most needed section in [1]. So anyone can understand what help your group needs. Also, TCs and UCs will have a better view of how group effort been working in the community and how community priority should be if regarding SIGs and WGs. 3. Also, consider your needs as a community goal? Add it in [2] now. -------------------------------------------longer version--------------------------------------------------- ( don't worry, not like I can do a super long readable article, but under the power of line-counting this is longer and still readable for sure!) 1. Back in the last summit, we have a forum [3] to talk about how can we help to expose SIGs and WGs. As an action, We will host a forum in coming Denver summit, to talk about `Help most needed for SIGs and WGs` [4]. This forum will present the most needed help from SIGs and WGs. By plan to also invite everyone (will try to invite people from each team and group) in the community to join this forum. We might get attention from people and potentially helpers:). Therefore, I do wish to have participants from every group. Please kindly provide me the following help most needed information, including - What's your team's name - Who's leading or active in this team - What's this team recently plan or doing - How people can join this team? - What's your most needed help interns of actions or technical features? - Which team (or who) you think that have a part in above help request (high potentially is fine)? - Anything else you wish to share to attendees? I will try to reach out to each team too, but if you already have answers to the above questions, that will be super! Since we might get around 16 groups, will try to get to the ` most needed help` part for everyone. But hopefully, we can still give some time if any attendee is really interested in helping. We gonna get more focus on what's the needs and how we can help to make progress of it, so it won't be like another BoF or Project update for you. Also, it's a forum so I won't give my hope to have a video recorder, which makes the second part more essential. 2. Update help most needed section in [1]. So anyone can understand what help you need. Also, TCs and UCs will have a better view of how group effort been working in the community and how community priority should be. Possibly and hopefully organizations/users/people might find matching their inner plan and sending people to help. Now if your group are willing to help with the forum, this action should be easy. And as I mention we're going to have a forum [4], It will be benefit to have such a document ready for anyone who's not coming to summit. 3. Is this needed suitable as a community goal [2]? Since we're also discussing community goal in #openstack-tc and ML, I think this might be a good chance to raise your needs into suggestion as our goal [2] if you find it fit with community goal definition. As one of the Meta SIG chairs, I will try hard to make all SIG's life easier. So let me know if above action doesn't fit with your group or if you find it hard with making progress with your SIG mission. [1] https://github.com/openstack/governance/blob/master/reference/help-most-nee… [2] https://etherpad.openstack.org/p/community-goals [3] https://etherpad.openstack.org/p/expose-sigs-and-wgs [4] https://www.openstack.org/summit/denver-2019/summit-schedule/events/23612/h… -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

3 2

[tripleo] Stein RC1 is out (with stable/stein branch)
by Emilien Macchi 18 Apr '19

18 Apr '19

We decided to branch stable/stein and release our first Stein release candidate this week. The main action that is being asked to our dear community is to think about backporting patches landing in master that you want to see in Stein. As usual some of us will watch things which land on master but really any help on that is very welcome. Also I've produced some stats about the evolution of TripleO since Newton. https://docs.google.com/presentation/d/1RV30OVxmXv1y_z33LuXMVB56TA54Urp7oHI… A few highlights: - Bug fixing is pretty consistent since Pike, no major change - Stein is the release with the less implemented blueprints since before Newton - Our core contributors number keep growing - Number of LOC & reviews keep growing since Queens Thanks everyone for your hard work during this cycle, I'm looking forward to seeing you in Denver and work on "Train". -- Emilien Macchi

1 0

[Neutron][QoS] No meeting on April 23
by Rodolfo Alonso 18 Apr '19

18 Apr '19

Hello folks: Because of an agenda problem, I won't chair the QoS meeting on April 23. I talked to Miguel and there is no problem to postpone this meeting until May 7, after the PTG. Regards.

1 0

[tripleo] [Release-job-failures] Release of openstack/puppet-tripleo failed
by Sean McGinnis 18 Apr '19

18 Apr '19

There was a failure in the post-release job for puppet-tripleo that I think we will need someone from TripleO and or someone with Puppet knowledge to address. The error received was: Forge API failed to upload tarball with code: 400 errors: The dependency 'sensu/sensu' in metadata.json is empty. All dependencies must have a 'version_requirement'.,The dependency 'yelp/uchiwa' in metadata.json is empty. All dependencies must have a 'version_requirement'. http://logs.openstack.org/75/75c09955854b142fd08c843bebcb8ee90d951b4d/relea… I'm guessing that's something in the puppet-tripleo repo, but really have no idea how this part works. If someone from the team could look into this, we can then do a follow up release if needed to correct any issues. ----- Forwarded message from zuul(a)openstack.org ----- Date: Thu, 18 Apr 2019 15:04:39 +0000 From: zuul(a)openstack.org To: release-job-failures(a)lists.openstack.org Subject: [Release-job-failures] Release of openstack/puppet-tripleo failed Reply-To: openstack-discuss(a)lists.openstack.org Build failed. - release-openstack-puppet http://logs.openstack.org/75/75c09955854b142fd08c843bebcb8ee90d951b4d/relea… : POST_FAILURE in 3m 18s - announce-release announce-release : SKIPPED _______________________________________________ Release-job-failures mailing list Release-job-failures(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures ----- End forwarded message -----

2 1

[neutron] RFE Decouple placement reporting service plugin from ML2
by Bence Romsics 18 Apr '19

18 Apr '19

Hi Neutrinos, I'm wondering if we have a use case for the following RFE: https://bugs.launchpad.net/neutron/+bug/1818479 Decouple placement reporting service plugin from ML2 Very close to the Stein rc1 deadline we merged a change with a review comment we agreed to tackle in a followup RFE - the one above. Here's the original comment from Slawek: https://review.openstack.org/#/c/580672/30/neutron/services/placement_repor… I see why Slawek left the comment, the placement reporter service plugin's dependency on ML2 does hurt my aestethic sense too. But now starting to work out the details of the followup work I'm not sure if we have a use case here. I would like to understand if we have a use case or not, because the change we talked about would increase code complexity. So it needs justification. If we don't have a use case we better do nothing and leave the complexity as it is. But if we do have a use case I'm still happy to work on that. In short: What concrete problems does it mean if the placement reporter service plugin depends on the ML2 core plugin? When eliminating that dependency what use case(s) are we generalizing the code for? I see the following options: 1) There may be an agent reporting its resource information in a different format - unlike ovs-agent and sriov-agent. 2) There may be an agentless mechanism driver needing to sync to Placement. 3) There may be a non-ML2 core plugin needing to sync to Placement. For (1) we standardized the format included in the agent's configurations attribute. An agent newly implementing support for reporting resource information should only deviate from that format if that format cannot express the resource information. Do we have an agent like that? The format is basically a 2-layer mapping from physnets to the list of their backing physical devices (whatever that is for an agent: bridges, sriov PFs, etc) and further mapping to the backing devices' inventory numbers. For the exact format please see this piece of the docs: https://docs.openstack.org/neutron/stein/admin/config-qos-min-bw.html#neutr… Case (2) may be a realistic use case, but I'm not much familiar with agentless mechanism drivers. IIUC some agentless mechanism drivers actually maintain fake agent entries in the neutron db. Those drivers can utilize the current service plugin via the fake agent entries as if they were real. For really agentless mechanism drivers (not even fake agent records) and for (3) I'm inclined to say: Don't load this placement reporter service plugin but write your own. To summarize on a high level: Slawek's original comment on the implicit dependency detects a real design smell, but I don't see how refactoring the service plugin to a driver architecture would improve the situation. What could nicely make this plugin more generic is subscribing to another interface: create/update_host_resources() instead of create/update_agent() because agents today are an implementation detail of the ML2 drivers. But the host (instead of an agent) is an abstraction we never had in neutron. What do you think? Sorry for the long mail, Bence irc: rubasov

1 0

[ironic] Stepping down from core
by shiina.hironori＠fujitsu.com 18 Apr '19

18 Apr '19

Hi ironic folks, I'm sorry for my long absence. My job responsibility changed last November. Since it, I have been trying to spend some time on the core activity. But, I couldn't. So, I'm stepping down from the ironic core team. Thank you for accepting me to the team even though I didn't often appear in IRC due to the time difference. I hope I will work with you again someday. Best regards, Hironori

4 3

[placement][ptg] pre-ptg checkpoint
by Chris Dent 18 Apr '19

18 Apr '19

Since I'm going to be out of touch next week, I thought it best to provide a checkpoint summarizing where things are with the virtual pre-PTG process so that we go into the PTG in reasonable shape. For reference, the pre-PTG has been a series of email threads discussion topics raised in the two placement-related PTG etherpads: * https://etherpad.openstack.org/p/placement-ptg-train * https://etherpad.openstack.org/p/ptg-train-xproj-nova-placement Each thread is linked from its topic in the etherpad. Any thread which reached some resolution is marked with a boldface "ACTION", often with a link to a story or review associated with the next step. Overall I think the process has been useful. We've figured out some plans of action for resource provider partitioning and consumer types, worked out better testing for osc-placement, and have some ideas on better container manager and performance testing. These things are reflected in worklists on storyboard [1, 2]. We've also managed to uncover some more ideas, challenges, and benefits with nested resource providers. We have a planned Saturday morning session at the PTG with Ironic, Blazar and anyone else who is thinking about using placement outside of nova. Where we've made less progress is getting owners on the new features [3] and reaching some resolution or plan with the several nested provider related threads (aggregate spanning, subtree "affinity", json body, NUMA in placement, request group mapping, resource trait filters). We've shared some ideas and managed to dismiss some things that won't work, but at least so far a solid plan is yet to emerge. Part of the latter may be my fault. I'm compelled to fully understand what's going on and for things to make as much sense to as many people as possible and I believed we had the time and space to have a wide-open discussion in that direction in the weeks prior to the PTG. There's still next week, so perhaps that's still the case, but maybe a different approach is required. If you've got ideas, please share. Please carry on with the threads if there's more to add. I'll incorporate them early in the week before the PTG and extract some kind of schedule for our time at the PTG. As I've said before there may not be much opportunity, we've only got Friday and Saturday and it is mostly consumed. Friday will mostly be taken up with Nova, Saturday morning will be talking with Ironic and Blazar. By Saturday afternoon we will probably be exhausted and I suspect the best use of our time will be talking without agenda about making sure working on Placement is pleasant and useful for everyone [4]. We don't need to actively seek more work: We've already defined got a lot, especially when we account for work in other projects. Thanks. [1] Cleanups: https://storyboard.openstack.org/#!/worklist/575 [2] Features: https://storyboard.openstack.org/#!/worklist/594 [3] However, thanks very much to Surya for stepping up to make a spec for the consumer type feature: https://storyboard.openstack.org/#!/story/2005473 [4] To that end, please make sure you have added your thoughts to the retrospective etherpad: https://etherpad.openstack.org/p/placement-ptg-train-retrospective -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

1 0

[neutron] neutron-metering-agent track router and floating IP traffic
by Florian Engelmann 18 Apr '19

18 Apr '19

Hi, I configured our neutron meters like: labels: openstack network meter create --description "shared bandwidth-public-out" --share bandwidth-public-out openstack network meter create --description "shared bandwidth-public-in" --share bandwidth-public-in rules: openstack network meter rule create --egress --remote-ip-prefix 0.0.0.0/0 bandwidth-public-out openstack network meter rule create --ingress --remote-ip-prefix 0.0.0.0/0 bandwidth-public-in This created the iptables rules in all namespaces and collects data: openstack metric measures show bc6c670f-cbb5-4fe1-974c-10455448fa27 +---------------------------+-------------+---------------+ | timestamp | granularity | value | +---------------------------+-------------+---------------+ | 2019-04-15T16:00:00+02:00 | 3600.0 | 581854.954545 | | 2019-04-15T17:00:00+02:00 | 3600.0 | 23169398.5625 | | 2019-04-15T18:00:00+02:00 | 3600.0 | 586508.9 | | 2019-04-15T19:00:00+02:00 | 3600.0 | 1276372.84375 | | 2019-04-15T20:00:00+02:00 | 3600.0 | 507482.911765 | [...] But how to know which project caused which amount of traffic? It looks like only one metric is created for each label? openstack metric list | grep bandwidth | 0f9f20a4-1c60-442f-81cb-48285e3f70da | ceilometer-ew-low | bandwidth | B | efae8040-85d7-455a-958d-213a81ef98ef | | 1a058a79-965f-4a0a-a9ee-d482f57892ab | ceilometer-ew-low | bandwidth | B | 837ed34a-65d2-42ef-924e-fd3c7e5aaed6 | What does "shared" mean? I was not able to find any relevant documentation. Do I have to create a meter for each project or a rule or both? Is it possible to define a "default" meter and rule for every existing project and all future projects? All the best, Flo

1 1

[tripleo] Nominate Cédric Jeanneret (Tengu) for tripleo-validations core
by Gaël Chamoulaud 18 Apr '19

18 Apr '19

Hi TripleO devs, The new Validation Framework is a big step further for the tripleo-validations project. In many ways, it improves the way of detecting & reporting potential issues during a TripleO deployment. As the mastermind of this new framework, Cédric brought a new lease of life to the tripleo-validations project. That's why we would highly benefit from his addition to the core reviewer team. Assuming that there are no objections, we will add Cédric to the core team next week. Thanks, Cédric, for your excellent work! =Gaël

2 1

[sahara] Sahara meeting April 18 - volunteer chair? (or cancel?)
by Jeremy Freudberg 18 Apr '19

18 Apr '19

I have a personal conflict and will not be around to run the Sahara meeting on April 18 (tomorrow). There isn't too much to discuss, so I'm comfortable letting it be cancelled. But if there is any volunteer chair, the meeting can still run. Thanks, Jeremy

2 1

[murano] Galera Table package_to_tag has no PRIMARY_KEY
by Marc Vorwerk 18 Apr '19

18 Apr '19

Hello, uploading an application to Murano fails after upgrading "percona-xtradb-cluster-common-5.6" to "percona-xtradb-cluster-common-5.7": Importing package io.murano Error {"explanation": "The server has either erred or is incapable of performing the requested operation.", "code": 500, "error": {"message": "(pymysql.err.InternalError) (1105, u'Percona-XtraDB-Cluster prohibits use of DML command on a table (murano.package_to_tag) without an explicit primary key with pxc_strict_mode = ENFORCING or MASTER') [SQL: u'INSERT INTO package_to_tag (package_id, tag_id) VALUES (%(package_id)s, %(tag_id)s)'] [parameters: {'package_id': 'a345ab1fe4924aaf8ed32f954a6fe32d', 'tag_id': '0cb3765d7fe1486ba9741f4eb9b2a437'}]", "traceback": null, "type": "DBError"}, "title": "Internal Server Error"} (HTTP 500) occurred while installing package io.murano If i looks on the galera table, i get the following information: $ mysql -u murano -p mysql> Use murano; mysql> INSERT INTO package_to_tag (package_id, tag_id) VALUES (a345ab1fe4924aaf8ed32f954a6fe32d, 0cb3765d7fe1486ba9741f4eb9b2a437); ERROR 1105 (HY000): Percona-XtraDB-Cluster prohibits use of DML command on a table (murano.package_to_tag) without an explicit primary key with pxc_strict_mode = ENFORCING or MASTER mysql> SHOW KEYS FROM package_to_tag WHERE Key_name = 'PRIMARY'; Empty set (0.00 sec) My package list: $pkg -l | grep -e mysql -e percona root@control02ve:~# dpkg -l | grep -e mysql -e percona ii libdbd-mysql-perl 4.033-1ubuntu0.1 amd64 Perl5 database interface to the MySQL database ii libmysqlclient20:amd64 5.7.25-0ubuntu0.16.04.2 amd64 MySQL database client library ii mysql-common 5.7.25-0ubuntu0.16.04.2 all MySQL database common files, e.g. /etc/mysql/my.cnf ii percona-release 1.0-11.generic all Package to install Percona gpg key and APT repos ii percona-xtrabackup-24 2.4.13-1.xenial amd64 Open source backup tool for InnoDB and XtraDB ii percona-xtradb-cluster-client-5.7 5.7.25-31.35-1.xenial amd64 Percona XtraDB Cluster database client binaries ii percona-xtradb-cluster-common-5.7 5.7.25-31.35-1.xenial amd64 Percona XtraDB Cluster database common files (e.g. /etc/mysql/my.cnf) ii percona-xtradb-cluster-galera-3.x 3.32-1.xenial amd64 Galera components of Percona XtraDB Cluster ii percona-xtradb-cluster-server-5.7 5.7.25-31.35-1.xenial amd64 Percona XtraDB Cluster database server binaries ii python-mysqldb 1.3.7-1build2 amd64 Python interface to MySQL ii python-pymysql 0.7.11-1~cloud0 all Pure-Python MySQL driver - Python 2.x Murano is installed in the latest version of queens (https://github.com/openstack/murano/tree/stable/queens) Has anyone an idea? Best Regards, Marc

2 2

[fenix][forum][ptg]
by Juvonen, Tomi (Nokia - FI/Espoo) 17 Apr '19

17 Apr '19

Hi, If you did not yet know Fenix: Fenix - rolling infrastructure maintenance and upgrade with VNF interaction to guarantee zero impact. https://wiki.openstack.org/wiki/Fenix Now there has been the rolling OpenStack upgrade with zero impact to VNF showcased in ONS America two weeks ago and it is time to look the Train and the way forward. Most importantly to harmonize with ETSI NFV and prioritize the work to have a solid base after this cycle. Here is etherpad for the forum: Fenix project update, future, discuss Tue 30th April, 9:00am - 9:40am The Colorado Convention Center - Ballroom Level - 4B https://etherpad.openstack.org/p/DEN-fenix-forum-brainstorming Here are etherpads for the PTG: Fenix and ETSI NFV synch: Sat 4th May, 10:00am - 11:00am https://etherpad.openstack.org/p/DEN2019-fenix-ETSI-NFV-PTG Fenix Train PTG, What to do next, prioritizing Sat 4th May, 11:10am - 12:10am https://etherpad.openstack.org/p/DEN2019-fenix-PTG br, Tomi

1 0

[telemetry] Nominating Rong Zhu for Telemetry core
by Trinh Nguyen 17 Apr '19

17 Apr '19

Hi team, I'd like to nominate Rong Zhu (zhurong) to become a Telemetry core. Rong Zhu has been contributing to OpenStack for a while now [1] and sometimes jump in to help Telemetry projects fix bugs. While being the PTL of Murano and Solum projects [2], his experience and network of developers working on ceilometer/aodh projects would be a huge benefit for the team. [1] https://review.openstack.org/#/q/owner:%22zhurong+%253Caaronzhu1121%2540gma… [2] https://governance.openstack.org/election/results/train/ptl.html Let give +1 for your agreement. Yours, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

3 3

[telemetry] Update cores team and team lead recruitment
by Trinh Nguyen 17 Apr '19

17 Apr '19

Hi team, As you may know, existing core members of Telemetry cannot work on the projects anymore. So, I would like to set up a new core team for Telemetry projects in the Train cycle. At the moment, only Lixian Kong and I have the core privileges over Ceilometer, AODH, and Panko projects. I expect to have 1 more core reviewer to help maintain Telemetry. So *please send us your proposal if you're interested*. This is the current structure of Telemetry: - AODH: Lixian Kong is the team lead - Ceilometer and Panko: myself. This is just tempory until we can find someone that has enough experience and willing to take the role. As discussed in yesterday meeting [1], you may argue that having experience does not necessarily qualify for code contributions but with the current situation, I would propose "*the willing to help*" as the number 1 criteria. We will move forward with it and fix processes when we see issues. Thank all the people that want to keep Telemetry alive. Yours, [1] https://etherpad.openstack.org/p/telemetry-train -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

4 8

[charms] OpenStack Charms 19.04 release is available with Stein enablement
by David Ames 17 Apr '19

17 Apr '19

The OpenStack Charms development team is happy to announce the 19.04 charms release, introducing support for OpenStack Stein with Ceph Mimic on Ubuntu 18.04 (LTS) and Ubuntu 19.04. This charm release also brings several new and valuable features to the existing OpenStack Charms deployments for Queens, Rocky, Stein, and many other stable combinations of Ubuntu + OpenStack. For the full release notes, please reference: https://docs.openstack.org/charm-guide/latest/1904.html Some highlights: * Masakari instance high-availability * Keystone SSO/SAML integration * Nova GPU and general device pass-through * Multi-site replication between Ceph RADOS Gateways * Mirroring of RBD images across Ceph Clusters The charms dev team is available on Freenode #openstack-charms, and will be at the Open Infrastructure Summit and PTG events in Denver, CO (May 2019). Huge thanks to the 71 charm contributors who worked together to squash 247 bugs, to enable an entirely new charmed version of OpenStack, and to move the line forward on several key features! Ryan Beisner Frode Nordahl Liam Young James Page Doug Hellmann David Ames Corey Bryant Chris MacNaughton Paul Goins Pete Vander Giessen Alex Kavanagh Edward Hope-Morley melissaml Chris MacNaughton Sahid Orentino Ferdjaoui zhouxinyong Andreas Jaeger lijunjie Felipe Reyes caoyuan zhulingjie Vladimir Grevtsev Dmitrii Shcherbakov inspurericzhang Andrew McLeod Syed Mohammad Adnan Karim Andre Ruiz Tiago Pasqualini Billy Olsen Drew Freiberger Seyeong Kim Hua Zhang Vu Cong Tuan wanghui Trent Lloyd Przemyslaw Hausman xulei Andrea Ieri Xav Paice Alvaro Uria Tytus Kurek Nikolay Vinogradov lvxianguo Nobuto Murata Nicolas Pochet Aymen Frikha Marian Gasparovic lei zhang Niels van Adrichem andy wang Shane Peters Jonathan Herlin Andrey Grebennikov wangyayan zhangbailin Diko Parvanov Dimos Alevizos Mike Wilson David Coronel Tilman Baumann MaoyangLiu gaobin Jesper M Gábor Mészáros Junien Fridrick Pedro Guimarães Wouter van Bommel Abhijeet Patra Peter Zhang Barry Price Dongdong Tao -- OpenStack Charm Team

1 0

[cinder][dev] support-revert-to-any-snapshot
by Walsh, Helen 17 Apr '19

17 Apr '19

To cinder driver developers: * https://blueprints.launchpad.net/cinder/+spec/support-revert-to-any-snapshot In relation to the blueprint above. I am wondering if there are any drivers that whose later snapshots will not be valid after a revert to an snapshot that is NOT the last in the series Vol1 \ ----------------------------------------------------- Mon Tue Wed Thu Fri \ \ \ \ \ S1 -------- S2 ---------S3 -------- S4 ---------S5 If on Fri, I was to revert back to Tuesday's Snapshot S2, would S3, S4 and S5 still be usable Snapshots? Vol1 \ ----------------------------------------------------- Mon Tue Wed Thu Fri \ \ \ \ \ S1 -------- S2 ---------S3 -------- S4 ---------S5 Thank you, Helen

2 1

[PTG] FINAL SCHEDULE..NO SERIOUSLY.
by Kendall Nelson 17 Apr '19

17 Apr '19

Since I published the first 'Final Schedule' we have had a number of small changes that we've had to make so I wanted to update everyone to make sure we are all on the same page. Seriously though, this is the last round of changes as this version of the schedule has already gone to the design team and is going to print sometime tomorrow/Friday. To textually summarize the changes: - Tacker will no longer be meeting - Converted the Interop WG into a dynamically scheduled team - Horizon will be all day Thursday instead of a half day See you all soon! -Kendall (diablo_rojo)

1 0

Does anyone care about reviewstats?
by Matt Riedemann 17 Apr '19

17 Apr '19

There are open reviewstats changes [1] but the non-infra core team is entirely made up of people that no longer work on openstack [2]. I had a change that was open so long some of the projects are now owned by placement [3] so I just abandoned it. So the question is, does anyone use reviewstats anymore? If so, we should probably just start adding new people to the core team that actually use it. Otherwise we should probably just put it in the attic and rely on stackalytics (which is probably what most people are already doing). [1] https://review.openstack.org/#/q/project:openstack-infra/reviewstats+status… [2] https://review.openstack.org/#/admin/groups/893,members [3] https://review.openstack.org/#/c/499197/ -- Thanks, Matt

11 15

[all] Gerrit will be offline April 19, 2019
by Anita Kuno 17 Apr '19

17 Apr '19

Pursuant to this email thread: http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.ht… , which discusses the rationale for moving git repo namespaces, Gerrit will be offline on April 19, 2019. Specific start time for the outage will be posted to this thread when it is decided next week. Outage may be sporadic as needed by the infra team, you can follow the progress in the #openstack-infra irc channel on freenode. Please review the above linked email and all the subsequent posts, and please post any concerns about the rationale or details to the above thread. If you have concerns about the Gerrit outage scheduled to begin April 19, 2019, please reply to this thread and/or join us in #openstack-infra on freenode to share your questions or concerns. Thank you, Anita

1 2

[placement][ptg] JSON payload for GET /a_c
by Chris Dent 17 Apr '19

17 Apr '19

>From the etherpad [1] * would make things like "resource provider affinity" above easier. much easier. and let us express granular-ness less awkwardly, and let us add new and richer group_policy concepts (which is basically the affinity thing). * Brought up briefly in the above spec at https://review.openstack.org/#/c/650476/1/doc/source/specs/train/approved/2… As clients making GET /allocation_candidates queries desire to express more and more complexity [2], query parameters start to feel cumbersome. At various times people have suggest that a JSON body might be easier to manage. The first time was during the discussion that resulted in the creation of the allocation_candidates resource. My personal feeling on this is that if we have a way to satisfy a use case without changing the API, even if it is cumbersome, we should write in that mode first and then decide if adding the other way is appropriate. This isn't simply because of some over-adherence to YAGNI, but because we need to deliver on some of placement's promise sooner than later and there don't seem to be a lot of people around. There may, however, be plenty of reasons that not everyone knows for why this needs to happen. If so, please share them. Only once we know those reasons should we start considering formats. What do we need to express (and why do we need to express it) that we can't express now? [1] https://etherpad.openstack.org/p/placement-ptg-train [2] https://review.openstack.org/#/c/555081/22/specs/train/approved/cpu-resourc… -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

4 4

[placement][ptg] Resource Provider Partitioning
by Chris Dent 17 Apr '19

17 Apr '19

>From the etherpad [1]: * do we need this? * what is it? * who is going to drive it? As as I recall, resource provider partitioning (distinct from allocation partitioning) is a way of declaring that a set of resource providers are in a thing. This would allow, for example, one placement to service multiple OpenStack clouds or for a placement to be a part of a single pane of glass system in a FOG or edge setup. This was mentioned during Stein nova discussions [2] but since then I've not personally heard a lot of discussion on this topic so it's unclear if it is a pressing issue. Do we want to be build it so they come, or wait until they come and then build it? The discussion at [2] mentions the possibility of an 'openstack-shard' header (instead of query parameter) that would be sent with any request to placement. There is, however, no substantive discussion on the internal implementation. Options: * Do nothing (see above) * Internally manipulate aggregates (all these resource providers below to shard X). * Add a 1:1 or 1:N relation between an RP and a shard uuid in the DB. * Use a trait! [3] But before we get into implementation details we should discuss the use cases for this (if any), the need to do it (if any), and the people who will do it (if any). All three of those are thin at this point. [1] https://etherpad.openstack.org/p/placement-ptg-train [2] around lines 243 on https://etherpad.openstack.org/p/nova-ptg-stein where both types (allocation/rp) of partitioning are discussed. [3] Not for the trait strict constructionists. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

4 13

[placement][ptg] Allocation Partitioning
by Chris Dent 17 Apr '19

17 Apr '19

>From the etherpad [1]: * do we need this? * what is it? * who is going to drive it? As as I recall, allocation partitioning (distinct from resource provider partitioning) is a way of declaring that a set of allocations below to a specific service. This is useful as a way, for example, of counting instances spawned via nova. Right now it is possible to count vcpus, ram and disk but if something besides nova is making allocations using those resource classes how do we distinguish? Of course it's important to also ask "should we distinguish?". If there's a concept of unified limits does it matter whether a VCPU is consume by _this_ nova or something else if they are consumed by the same user? This functionality is closely tied to resource provider partitioning. In a complex placement scenario, where placement is managing multiple instance spawning tools, in multiple cloud-like things, it seems like both would be needed. The ongoing work to implement quota counting in placement [2] has a workaround for instances not being counted in placement, but the "more than one nova per placement" limitation has to be documented. How urgent is this? Is there anyone available to do the work this cycle? How damaging is it to punt to U? What details are missing in the above description? [1] https://etherpad.openstack.org/p/placement-ptg-train [2] https://review.openstack.org/#/q/topic:bp/count-quota-usage-from-placement -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

7 26

[tripleo][openstack-ansible] collaboration on os_tempest role update XIX - Apr 16, 2019
by Chandan kumar 17 Apr '19

17 Apr '19

Hello, Here is the 19th update (Apr 10 to Apr 06, 2019) on collaboration on os_tempest[1] role between TripleO and OpenStack-Ansible projects. Highlights of the week: * os_tempest user guide is now published here: https://docs.openstack.org/openstack-ansible-os_tempest/stein/user/index.ht… * os_tempest role now have debian support. * In tripleo side, all standalone and standalone scenario [1-4] periodics jobs are consuming os_tempest to run tempest in CI Thanks to mnaser for debian support, mkopec, noonedeadpunk for user guide & Arx for porting all standalone periodic jobs to os_tempest Things got merged: os_tempest: * os_tempest user guide: https://review.openstack.org/644570 * debian: add ci and update meta : https://review.openstack.org/652346 tripleo: * Port base standalone job to os_tempest - https://review.rdoproject.org/r/20061 * Port standalone scenario01 to os_tempest - https://review.rdoproject.org/r/19916 * Port scenario002-standalone-master to os_tempest - https://review.rdoproject.org/r/20059 * Port scenario003-standalone-master to os_tempest - https://review.rdoproject.org/r/20037 * Port scenario004-standalone-master to os_tempest - https://review.rdoproject.org/r/20140 * Port standalone full tempest job to os_tempest - https://review.rdoproject.org/r/20203 Things in Progress: os_tempest: * Make smoke tests as a default whitelist tests - https://review.openstack.org/652060 * Enable support for novajoin tempest plugin - https://review.openstack.org/652986 tripleo: * Enable os_tempest in baremetal-full-overcloud-validate playbook - https://review.openstack.org/652983 * Run novajoin tempest tests in all TLS job using os_tempest - https://review.openstack.org/639324 What's coming? Once we have tripleo master promotion, we will update the tripleo ci upstream job config to os_tempest We will be continue working on current os_tempest patches. Here is the 18th update [2]. Have queries, Feel free to ping us on #tripleo or #openstack-ansible channel. Links: [1.] http://git.openstack.org/cgit/openstack/openstack-ansible-os_tempest [2.] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004878.ht… Thanks, Chandan Kumar

1 1

[vitrage][ptg] Vitrage PTG etherpad
by Ifat Afek 17 Apr '19

17 Apr '19

Hi, I created an etherpad for discussion topics for the Vitrage PTG, which will be held on Friday morning, May 3rd. Feel free to comment or add other topics. https://etherpad.openstack.org/p/vitrage-train-ptg Thanks, Ifat

1 0

Routed networks and FIP gateway assignments
by Syd Logan 16 Apr '19

16 Apr '19

I'm currently learning about routed networks and have a lab that is implementing a 3 tier leaf spine and have a question. I know that I can create a subnet from which FIP gateway IP addresses are consumed, e.g., openstack subnet create demo-floating-ip-agent-gateway-subnet \ --subnet-range 198.51.100.0/24 --no-dhcp \ --service-type 'network:floatingip_agent_gateway' \ --network demo-ext-net In videos from past OpenStack summits such as this https://youtu.be/XnXSwCSHc4U drawings show that the subnet is used within a single segment's compute nodes, and I can believe that would work. What isn't clear is whether the same subnet should be used across multiple segments, or the intent is for each segment defined in the routed network to have a unique subnet from which these FIP gateway IPs are assigned, e.g., some other segment would be using, say 198.52.100.0/24. In the non-routed network setup where I leanred about DVR it was pretty clear only one subnet was allowed, but what is the best practice/intent for routed nets? Thanks. syd

1 0

[Zun] Openstack Rocky Appcontainer start error
by Navdeep Uniyal 16 Apr '19

16 Apr '19

Dear All, I am getting following error while trying to start the ubuntu container Command: openstack appcontainer run --name containerA --net network=5guknet ubuntu ping 8.8.8.8 Error: status_reason | Docker internal error: 500 Server Error: Internal Server Error ("legacy plugin: Post http://127.0.0.1:23750/Plugin.Activate: dial tcp 127.0.0.1:23750: connect: connection refused"). Secondly, with Heat Template: heat_template_version: 2015-04-30 resources: the_resource: type: OS::Zun::Container properties: cpu: 4 image: ubuntu image_pull_policy: "ifnotpresent" interactive: true memory: 2048 name: "ubuntu16" command: "ping 8.8.8.8" networks: [{"network": 5guknet}] restart_policy: "no" The command I can see on the appcontainer show is : command | [u'p', u'i', u'n', u'g', u' ', u'8', u'.', u'8', u'.', u'8', u'.', u'8'] Please advise how to resolve the two issues. Kind Regards, Navdeep

2 1

[neutron] Cleaning in Neutron
by Slawomir Kaplonski 16 Apr '19

16 Apr '19

Hi, I just run abandon_old_reviews.sh [1] script again for neutron, neutron-lib and neutron-tempest-plugin repositories. So if You have Your patch abandoned but You want to continue to work on it, please simple restore it or if You can’t restore it by self, please ping me on irc (slaweq) and I will restore it for You. [1] https://github.com/openstack/neutron/blob/master/tools/abandon_old_reviews.… — Slawek Kaplonski Senior software engineer Red Hat

3 3

[stackalytics] Reported numbers seem inaccurate
by Sergey Nikitin 16 Apr '19

16 Apr '19

Hello everyone, As you mentioned there are some inaccuracies in stats on stackalytics.com. It happened because of outdated Prod env. At this moment we are in a process of migration to the new env. All data on this env is fresh and accurate. So the problem will be solved soon. -- Best Regards, Sergey Nikitin

2 1

[gentoo] Looking for members to help maintain OpenStack packages on Gentoo
by Matthew Thode 16 Apr '19

16 Apr '19

I'm considering disbanding the Gentoo OpenStack project and switching to sole maintainership of the project. So far I'm the only active maintianer in the project that I can see. I'm looking for others that are interested in helping maintain the openstack ebuilds. It's fairly easy as an ongoing thing with two major releases a year (which starts an update of 160 or so packages). It's python based and I've found maintaining the ebuilds to be very easy individually if that helps. There are other actions that'd be helpful as well. Some arm64 stage work (catalyst builds these) to then do some arm64 diskimage-builder work to work on building VM images for other arches (openstack or 'vanilla'). Please respond if you are intrested in joining. I've also emailed the Gentoo developer list separately with the same request. Thanks, -- Matthew Thode

1 0

[tripleo][ptg] Initial PTG schedule
by Alex Schultz 16 Apr '19

16 Apr '19

Hey folks, I've taken the topics from the previously announced etherpad[0] and put together a basic schedule for Thursday-Saturday. Please take a moment to review https://etherpad.openstack.org/p/tripleo-ptg-train and let me know if there are any issues with the timing. We have some extra time most days if we need to shift things around. Thanks, -Alex [0] https://etherpad.openstack.org/p/tripleo-train-topics

2 2

[placement][ptg] Keeping osc-placement healthy
by Chris Dent 16 Apr '19

16 Apr '19

>From the etherpad [1]: * It is behind (in microversions) and sometimes hard to remember * It's functional tests require a working devstack, which is a bit meh * Maybe something like https://vcrpy.readthedocs.io/en/latest/ could help (reasonable given microversions) * (edleafe) It might be useful to have an osc-placement lead (czar? princess? mayor?) who can devote some of their time to this and help coordinate others. I might be able to rustle up some cycles. Not much to add to the above. What additional ideas to people have. Who (in addition to Ed) has some time to care about osc-placement? What do people think of vcrpy? What else? [1] https://etherpad.openstack.org/p/placement-ptg-train -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

3 8

[tripleo] Canceling IRC meeting on April 30, 2019
by Alex Schultz 16 Apr '19

16 Apr '19

Due to the summit and PTG, most folks won't be around so we'll cancel the meeting for that week. We will resume the weekly IRC meeting on May 7, 2019. Thanks, -Alex

1 0

[ironic] Cancelling meetings April 22nd through May 6th, resuming May 13th
by Julia Kreger 16 Apr '19

16 Apr '19

Greeting fellow practitioners of irony! I meant to raise this question during our weekly meeting yesterday, but simply forgot. Some discussion on irc[0] today has reached a consensus that we should just cancel the next few meetings. * This next week, April 22nd, out meeting conflicts with holidays for a number of contributors. * The week after, April 29th, will be during the Open Infrastructure Summit. Historically we cancel the meeting on the week following such a gathering to give contributors time to re-group. We will be resuming meetings on May 13th. As always you can find ironic contributors in #openstack-ironic. Review priorities should be sourced from the whiteboard[1] using the "This Week's Priorities" section.. Thanks everyone! -Julia [0]: http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-iro… [1]: https://etherpad.openstack.org/p/IronicWhiteBoard

1 0

Re: [stackalytics] Reported numbers seem inaccurate
by Jeremy Stanley 16 Apr '19

16 Apr '19

On 2019-04-11 18:21:13 UTC (+0000), Mark Goddard wrote: > I've heard a couple of people say they don't feel the numbers > reported by Stackalytics accurately reflect reality. I've been > trying to gather a few stats for my Kolla update session in > Denver, and am finding the same. [...] I'm not going to attempt to debate the value of the Stackalytics site, but given the maintainers seem to have set their sights beyond OpenStack lately they're probably not following discussions on the openstack-discuss mailing list (their lack of replies here after nearly a week would indicate this as well). According to the readme[*] for the project, they seem to prefer bug reports via Launchpad so that may be a more effective place to follow up on this concern if there is a desire for continued use of Stackalytics by the OpenStack community. That said, someone already seems to have reported[**] the problem back in January and the bug hasn't been triaged yet, but maybe additional mee-toos will up their urgency? [*] https://opendev.org/openstack/stackalytics/src/branch/master/README.rst#pro… [**] https://launchpad.net/bugs/1811028 -- Jeremy Stanley

1 0

[nova][scheduler] scheduler spawns to the same compute node only
by Nicolas Ghirlanda 16 Apr '19

16 Apr '19

Hi all, we have a kolla-ansible deployed "Queens" Release of openstack with 8 compute nodes and an external Percona XtraDB Cluster (with read-write split with haproxy). New VMs are just currently always scheduled to the same compute node, even though a manual live-migration is working fine to other compute nodes. We're not sure, what the issue is, but perhaps someone may spot it from our config: # nova.conf scheduler config default_availability_zone = az1 ... [filter_scheduler] available_filters = nova.scheduler.filters.all_filters enabled_filters = RetryFilter, AvailabilityZoneFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, AggregateInstanceExtraSpecsFilter, AggregateMultiTenancyIsolation, DifferentHostFilter, RamFilter, SameHostFilter, NUMATopologyFilter Database is an external Percona XtraDB Cluster (Version 5.7.24) with haproxy for read-write-splitting (currently only one write node). We do see mysql errors in the nova-scheduler.log on the write DB node when an instance is created. 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db [-] Unexpected error while reporting service status: OperationalError: (pymysql.err.OperationalError) (1213, u'WSREP detected deadlock/conflict and aborted the transaction. Try restarting the transaction') (Background on this error at: http://sqlalche.me/e/e3q8) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db Traceback (most recent call last): 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/servicegroup/drivers/db.py", line 91, in _report_state 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db service.service_ref.save() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 226, in wrapper 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db return fn(self, *args, **kwargs) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/objects/service.py", line 397, in save 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db db_service = db.service_update(self._context, self.id, updates) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/db/api.py", line 183, in service_update 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db return IMPL.service_update(context, service_id, values) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_db/api.py", line 154, in wrapper 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db ectxt.value = e.inner_exc 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.force_reraise() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db six.reraise(self.type_, self.value, self.tb) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_db/api.py", line 142, in wrapper 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db return f(*args, **kwargs) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 227, in wrapped 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db return f(context, *args, **kwargs) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.gen.next() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1043, in _transaction_scope 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db yield resource 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.gen.next() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 653, in _session 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.session.rollback() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.force_reraise() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db six.reraise(self.type_, self.value, self.tb) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 650, in _session 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self._end_session_transaction(self.session) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 678, in _end_session_transaction 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db session.commit() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 943, in commit 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.transaction.commit() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 471, in commit 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db t[1].commit() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1643, in commit 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self._do_commit() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1674, in _do_commit 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.connection._commit_impl() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 726, in _commit_impl 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self._handle_dbapi_exception(e, None, None, None, None) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1409, in _handle_dbapi_exception 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db util.raise_from_cause(newraise, exc_info) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db reraise(type(exception), exception, tb=exc_tb, cause=cause) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self.engine.dialect.do_commit(self.connection) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 1765, in do_commit 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db dbapi_connection.commit() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pymysql/connections.py", line 422, in commit 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db self._read_ok_packet() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pymysql/connections.py", line 396, in _read_ok_packet 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db pkt = self._read_packet() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pymysql/connections.py", line 683, in _read_packet 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db packet.check_error() 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pymysql/protocol.py", line 220, in check_error 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db err.raise_mysql_exception(self._data) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db raise errorclass(errno, errval) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db OperationalError: (pymysql.err.OperationalError) (1213, u'WSREP detected deadlock/conflict and aborted the transaction. Try restarting the transaction') (Background on this error at: http://sqlalche.me/e/e3q8) 2019-04-15 16:52:10.016 24 ERROR nova.servicegroup.drivers.db 2019-04-15 16:52:20.020 24 INFO nova.servicegroup.drivers.db [-] Recovered from being unable to report status. The deadlock message is quite strange, as we have haproxy configured so all write requests are handled by one node. There are NO errors in the mysqld.log WHILE creating an instance, but we see from time to time aborted connections from nova. 2019-04-15T14:22:36.232108Z 30616972 [Note] Aborted connection 30616972 to db: 'nova' user: 'nova' host: '10.x.y.z' (Got an error reading communication packets) As I said, all instances are allocated to the same compute node. nova-compute.log doesn't show an error while creating the instance. Beside that, we also see messages from nova.scheduler.host_manager on all other nodes like (but those messages are _not_ triggered, when an instance is spawned.!) 2019-04-15 16:28:47.771 22 INFO nova.scheduler.host_manager [req-f92e340e-a88a-44a0-8cad-588390c25bc2 - - - - -] The instance sync for host 'xxx' did not match. Re-created its InstanceList. Don't know if that may be relevant, but somehow our (currently single) AZ is listed several times. # openstack availability zone list +------------+-------------+ | Zone Name | Zone Status | +------------+-------------+ | internal | available | | az1 | available | | az1 | available | | az1 | available | | az1 | available | +------------+-------------+ May that be related somehow? Thanks for any consideration and support! kind regards Nicolas --

4 7

[taas] stein
by Takashi Yamamoto 16 Apr '19

16 Apr '19

hi, i'll branch and release it sooner or later. probably in this week, or maybe the next one. wrt dashboard, i'm not sure if the repo is in a ready-to-release state. have anyone tested it recently?

1 1

[ceilometer] Missing some notifications
by Florian Engelmann 16 Apr '19

16 Apr '19

Hi, I am not sure if this problem is related to ceilometer or rabbitmq but it looks like our ceilometer notification agent is missing some notifications. Creating, eg. 10 cinder volumes in a row results in some of those volumes are not stored in gnocchi. "openstack metric resource show <ID>" results in a 404 error. Attaching the volume and detaching it again sometimes helps to "create" it in gnocchi. We do use gnocchi for billing. So missing volumes is very bad. Any recommendation on how to debug this problem? Thank you very much, flo

1 0

[scientific-sig] Meeting tonight: Ironic and Data Center Inventory, Denver Summit
by Stig Telfer 16 Apr '19

16 Apr '19

Hi All - We have a Scientific SIG IRC meeting today at 2100 UTC in channel #openstack-meeting. Everyone is welcome. Today's agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_16th_2019 <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_16th_2019> We’d like to continue the discussion started last week (in the other timezone) on surveying data center inventory management systems used in research computing, and how those might be better integrated with Ironic. We also have preparations for the Denver summit to discuss. Cheers, Stig

1 0

[tc][all] Dropping py35 testing
by Ghanshyam Mann 15 Apr '19

15 Apr '19

Hello Everyone, All the integration testing has been moved to Bionic[1] now and as discussed in the TC meeting[2], we are good to drop the py35 testing. Train: Good to drop. I am going to push the patches to remove the py35 jobs from the master (which is Train now). Stable/Stein: Ok to drop if the project wants. Stable/Stein gate is also moved to Bionic so we are good to drop py35 from stable/stein too. But backporting all the patches will be too many patches for projects to merge on stable/stein so we discussed not to propose all of them to stable/stein. If the project really wants to drop it from stable/stein then, they can backport. NOTE: Till stable/rocky, we should keep testing py35. If your project is branchless like Tempest and gate the same jobs for stable/rocky or older then, you should keep testing the py35 until stable/rocky EOL. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.ht… [2] http://eavesdrop.openstack.org/meetings/tc_python3/2019/tc_python3.2019-03-… http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003610.ht… -gmann

7 12

[telemetry] Vote for team meeting datetime
by Trinh Nguyen 15 Apr '19

15 Apr '19

Hi team, There're many things we need to discuss for the Train cycle so I propose we have a bi-weekly meeting and the first one would be on May 2 (next two weeks will be the vacation of most of the people). Please cast your vote for an appropriate time: https://doodle.com/poll/8akawam8yk38q2hh Yours, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

2 6

[kuryr] [kuryr-kubernetes] Unable to use kubernetes-related things after rebooting host
by Jaewook Oh 15 Apr '19

15 Apr '19

Hello kuryr-kubernetes team, currently, I deployed openstack with devstack, and I added "kuryr-kubernetes" to use kubernetes vim. At first it worked fine and no error was occurred, so I made some c-vnfs with kuryr-kubernetes. But after rebooting the host, the systems that worked well before are not working normally at all. First, I deployed an example one ('redis') with kubectl command, (Not using Tacker, but only using kubectl cli) it showed below error, Events: │obeTime":null,"lastTransitionTime":"2019-04-15T02:28:31Z"}],"hostIP":"192.168.1.20","startTime":"2019-04-15T02:28:31Z" Type Reason Age From Message │,"containerStatuses":[{"name":"web-server","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":f ---- ------ ---- ---- ------- │alse,"restartCount":0,"image":"kyleoh95/iistrc-k8s-demo2:0.4","imageID":""}],"qosClass":"Guaranteed"}}]} Normal Scheduled 2m25s default-scheduler Successfully assigned default/redis-master-6fbbc44567-ms8n│Apr 15 11:36:52 master tacker-server[1881]: from (pid=1881) request /usr/local/lib/python2.7/dist-packages/kubernetes k to master │/client/rest.py:219 Warning MissingClusterDNS 2m25s kubelet, master pod: "redis-master-6fbbc44567-ms8nk_default(8a6f9079-5f27-│Apr 15 11:36:52 master tacker-server[1881]: 2019-04-15 11:36:52.633 DEBUG tacker.vnfm.infra_drivers.kubernetes.kuberne 11e9-9b42-a4bf01550f1a)". kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" pol│tes_driver [-] status: Pending from (pid=1881) create_wait /opt/stack/tacker/tacker/vnfm/infra_drivers/kubernetes/kube icy. Falling back to "Default" policy. │rnetes_driver.py:137 Warning FailedCreatePodSandBox 19s kubelet, master Failed create pod sandbox: rpc error: code = Unknown desc │Apr 15 11:36:52 master tacker-server[1881]: 2019-04-15 11:36:52.634 DEBUG tacker.vnfm.infra_drivers.kubernetes.kuberne = failed to set up sandbox container "310e18b859e118ad6abcaa60d29a9e332da516ce57b8cdf47456ea4882ca66f5" network for pod│tes_driver [-] VNF initializing status: ['default', 'svc-vdu1-b9e851'] Pending from (pid=1881) create_wait /opt/stack/ "redis-master-6fbbc44567-ms8nk": NetworkPlugin cni failed to set up pod "redis-master-6fbbc44567-ms8nk_default" networ│tacker/tacker/vnfm/infra_drivers/kubernetes/kubernetes_driver.py:141 k: Got invalid status code from CNI daemon.; Traceback (most recent call last): │Apr 15 11:36:52 master tacker-server[1881]: 2019-04-15 11:36:52.634 WARNING tacker.vnfm.infra_drivers.kubernetes.kuber File "/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py", line 80, in run │netes_driver [-] VNF Creation failed: Resource creation is not completed within 500 seconds as creation of stack defau vif = self._add(params) Are there any reason for this error? And how can I resolve this problem? Best Regards, Jaewook. ================================================ *Jaewook Oh* (오재욱) IISTRC - Internet Infra System Technology Research Center 369 Sangdo-ro, Dongjak-gu, 06978, Seoul, Republic of Korea Tel : +82-2-820-0841 | Mobile : +82-10-9924-2618 E-mail : jwoh95(a)dcn.ssu.ac.kr ================================================

5 6