January 2019 - openstack-discuss - lists.openstack.org

Scuba Diving at the Denver Summit
by Monty Taylor 23 Apr '19

23 Apr '19

Hey everybody! tl;dr - Let me know if you'd like to go diving in the Rocky Mountains. Of all of the places we've had a Summit where one might reasonably expect there to be a possibility of Scuba Diving, the Denver Summit is probably REALLY low on the list. But it turns out that there is a Dive Shop that does dives at the Denver Aquarium [0] (with all the fish and sharks and whatnot) The Denver Aquarium is also very close to the Convention Center. I'm going to see about putting together a private group dive event for those of us who think that sounds like a fun idea. Due to cross-contamination concerns for the health and safety of the fish, the dive shop provides all equipment - so you don't have to bring gear. They will allow bringing your own mask - and I'm assuming if a well-scrubbed mask is ok then a well-scrubbed dive computer would be ok - but I'll be checking with them when I call to arrange things. If you are Open Water certified and would like to join, please send me a response back (reply directly to me rather than the list is fine) so I can judge general interest level when talking to the lovely humans. If you are **NOT** Open Water certified but are thinking to yourself "wow, that sounds like fun" - the dive shop does certification dives at the aquarium on the weekends. That's a bit out of scope for me to arrange, but if you want to show up in Denver early and do an Open Water class then join us, that's awesome. You can also get certified at home before coming ... but you'll need an Open Water certification to join us. BUT WAIT - THERE'S MORE! There is another interesting diving opportunity in the general Mountain West area - although not immediately **close** to Denver - that Sandy and I are going to be going to before the Summit - Homestead Crater. We're arranging to meet a Scuba instructor there to do our second Altitude cert dive, and also to do a 2-dive "Hot Springs Diver" specialty. We'd be happy to have folks join us. Homestead Crater [1] is outside of Salt Lake City. It's a hot spring - the water is 96F/35.5C and it's also an altitude dive. If people are interested, we can meet at the crater on Thursday morning pre-summit, dive the crater, then drive to Denver on Friday/Saturday. ** If only a small number of people are interested in any of this - we'll likely just do the Denver dives the weekend before during one of the normally scheduled dive times instead of arranging a group outing, and shift Homestead, if anyone is interested, a few days earlier ** I will not have space enough in my car to get you from the crater to Denver, so if you want to fly in to SLC ahead of time and meet there, awesome. Unlike Denver, the crater will require you have equipment, including scuba tanks. For anyone interested in coming, there are dive shops in Salt Lake where you can rent gear. There is a dive shop at the crater to do air fills. Anyway - if anyone is interested in some or all of these things, let me know this week so I can get a headcount and then we can talk more off-list about logistics and figure out how to deal with things for people coming from far off (carpools, equipment, etc) Monty PS. If any of you aren't already using Subsurface for dive logging, I'll be happy to help you get set up. [0] https://divedowntown.com/ [1] https://homesteadresort.com/utah-resort-things-to-do/homestead-crater/

4 5

Proposal to move cinder backup tests out of the integrated gate
by Matt Riedemann 11 Apr '19

11 Apr '19

I wanted to send this separate from the latest gate status update [1] since it's primarily about latent cinder bugs causing failures in the gate for which no one is really investigating. Running down our tracked gate bugs [2] there are several related to cinder-backup testing: * http://status.openstack.org/elastic-recheck/#1483434 * http://status.openstack.org/elastic-recheck/#1745168 * http://status.openstack.org/elastic-recheck/#1739482 * http://status.openstack.org/elastic-recheck/#1635643 All of those bugs were reported a long time ago. I've done some investigation into them (at least at the time of reporting) and some are simply due to cinder-api using synchronous RPC calls to cinder-volume (or cinder-backup) and that doesn't scale. This bug isn't a backup issue, but it's definitely related to using RPC call rather than cast: http://status.openstack.org/elastic-recheck/#1763712 Regarding the backup tests specifically, I don't see a reason why they need to be run in the integrated gate jobs, e.g. tempest-full(-py3). They don't involve other services, so in my opinion we should move the backup tests to a separate job which only runs on cinder changes to alleviate these latent bugs failing jobs for unrelated changes and resetting the entire gate. I would need someone from the cinder team that is more involved in knowing what their job setup looks like to identify a candidate job for these tests if this is something everyone can agree on doing. [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000867… [2] http://status.openstack.org/elastic-recheck/ -- Thanks, Matt

6 12

[horizon] Unused xstatic-* projects retirement
by Ivan Kolodyazhny 03 Apr '19

03 Apr '19

Hi team, There are some xstatic packages which I didn't start to use in Horizon or plugins. We didn't do any release of them. During the last meeting [1] we agreed to mark them as retired. I'll start retired procedure [2] today. If you're going to use them, please let me know. The list of the projects to be retired: - xstatic-angular-ui-router - xstatic-bootstrap-datepicker - xstatic-hogan - xstatic-jquery-migrate - xstatic-jquery.quicksearch - xstatic-jquery.tablesorter - xstatic-rickshaw - xstatic-spin - xstatic-vis [1] http://eavesdrop.openstack.org/meetings/horizon/2019/horizon.2019-01-09-15.… Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

3 4

[stable][kolla] EOL kolla ocata branch
by Eduardo Gonzalez 03 Apr '19

03 Apr '19

Hi stable team, Kolla project is ready to tag EOL the stable/ocata branch. We would like to tag the following repositories as EOL: - openstack/kolla - openstack/kolla-ansible Let me know if need anything else. Regards

4 5

[ironic][neutron] nf_conntrack_helper now disabled by default
by Derek Higgins 18 Mar '19

18 Mar '19

Hi All, Shortly before the holidays CI jobs moved from xenial to bionic, for Ironic this meant a bunch failures[1], all have now been dealt with, with the exception of the UEFI job. It turns out that during this job our (virtual) baremetal nodes use tftp to download a ipxe image. In order to track these tftp connections we have been making use of the fact that nf_conntrack_helper has been enabled by default. In newer kernel versions[2] this is no longer the case and I'm now trying to figure out the best way to deal with the new behaviour. I've put together some possible solutions along with some details on why they are not ideal and would appreciate some opinions 1. Why not enable the conntrack helper with echo 1 > /proc/sys/net/netfilter/nf_conntrack_helper The router namespace is still created with nf_conntrack_helper==0 as it follows the default the nf_conntrack module was loaded with 2. Enable it in modprobe.d # cat /etc/modprobe.d/conntrack.conf options nf_conntrack nf_conntrack_helper=1 This works but requires the nf_conntrack module to be unloaded if it has already been loaded, for devstack and I guess in the majority of cases (including CI nodes) this means a reboot stage or a potentially error prone sequence of stopping the firewall and unloading nf_conntrack modules. This also globally turns on the helper on the host reintroducing the security concerns it comes with 3. Enable the contrack helper in the router network namespace when it is created[3] This works for ironic CI, but there may be better solutions that can be worked within neutron that I'm not aware of. Of the 3 options above this would be most transparent to other operators as the original behaviour would be maintained. thoughts on any of the above? or better solutions? 1 - https://storyboard.openstack.org/#!/story/2004604 2 - https://kernel.googlesource.com/pub/scm/linux/kernel/git/horms/ipvs-next/+/… 3 - https://review.openstack.org/#/c/628493/1

6 10

[Interop-wg] [dev] [cinder] [qa] Strict Validation for Volume API using JSON Schema
by Ghanshyam Mann 08 Mar '19

08 Mar '19

Hello everyone, Tempest is planning to add the strict API validation using JSON schema for Volume test [1]. We do the same for all the compute test. With *Strict* JSON schema validation, all the API response will be validated with predefined schema with additionalProperties=False. additionalProperties=False will not allow any additional attribute in API response than what upstream APIs has in their current version. For example: If any vendor has modified the API response and returning additional attributes then Tempest tests going to fail. This will help: - To improve the OpenStack interoperability. Strict validation of API response is always helpful to maintain the interoperability. - To improve the volume API testing to avoid the backward compatible changes. Sometime we accidentally change the API in backward incompatible way and strict validation with JSON schema help to block those. We want to hear from cinder and interop team about any impact of this change to them. [1] https://blueprints.launchpad.net/tempest/+spec/volume-response-schema-valid… -gmann

4 6

[baremetal-sig][ironic] Proposing Formation of Bare Metal SIG
by Chris Hoge 25 Feb '19

25 Feb '19

This morning at the OpenStack Foundation board meeting we announced that we will focusing more our open infrastructure messaging efforts on bare-metal, and the promotion of Ironic in particular. To support these efforts, I'd like to form a Bare Metal SIG to bring together community members to collaborate on this work. The purpose of the Bare Metal SIG will be to promote the development and use of Ironic and other OpenStack bare-metal software. This will include marketing efforts like case studies of Ironic clusters in industry and academia, supporting integration of Ironic with projects like Airship and the Kubernetes Cluster API, coordinating presentations for industry events, developing documentation and tutorials, gathering feedback from the community on usage and feature gaps, and other broader community-facing efforts to encourage the adoption of Ironic as a bare-metal management tool. If you would like to participate, please indicate your interest in the linked planning etherpad. Ideally we would like to have broad engagement from across the community, from developers and practitioners alike. We'd like to highlight all of the efforts and usage across our community communicate how powerful Ironic is for hardware management. https://etherpad.openstack.org/p/bare-metal-sig Thanks in advance to everyone. I’ve been using Ironic to manage my home cluster for quite a while now, and I’m really excited to be working on the SIG and in supporting the efforts of the Ironic team and the users who are running Ironic in production. Chris Hoge Strategic Program Manager OpenStack Foundation

4 5

[tc] The future of the "Help most needed" list
by Thierry Carrez 19 Feb '19

19 Feb '19

Hi everyone, The "Help most needed" list[1] was created by the Technical Committee to clearly describe areas of the OpenStack open source project which were in the most need of urgent help. This was done partly to facilitate communications with corporate sponsors and engineering managers, and be able to point them to an official statement of need from "the project". [1] https://governance.openstack.org/tc/reference/help-most-needed.html This list encounters two issues. First it's hard to limit entries: a lot of projects teams, SIGs and other forms of working groups could use extra help. But more importantly, this list has had a very limited impact -- new contributors did not exactly magically show up in the areas we designated as in most need of help. When we raised that topic (again) at a Board+TC meeting, a suggestion was made that we should turn the list more into a "job description" style that would make it more palatable to the corporate world. I fear that would not really solve the underlying issue (which is that at our stage of the hype curve, no organization really has spare contributors to throw at random hard problems). So I wonder if we should not reframe the list and make it less "this team needs help" and more "I offer peer-mentoring in this team". A list of contributor internships offers, rather than a call for corporate help in the dark. I feel like that would be more of a win-win offer, and more likely to appeal to students, or OpenStack users trying to contribute back. Proper 1:1 mentoring takes a lot of time, and I'm not underestimating that. Only people that are ready to dedicate mentoring time should show up on this new "list"... which is why it should really list identified individuals rather than anonymous teams. It should also probably be one-off offers -- once taken, the offer should probably go off the list. Thoughts on that? Do you think reframing help-needed as mentoring-offered could help? Do you have alternate suggestions? -- Thierry Carrez (ttx)

13 35

[openstack-helm] would like to discuss review turnaround time
by Jaesuk Ahn 19 Feb '19

19 Feb '19

Dear all, There has been several patch sets getting sparse reviews. Since some of authors wrote these patch sets are difficult to join IRC meeting due to time and language constraints, I would like to pass some of their voice, and get more detail feedback from core reviewers and other devs via ML. I fully understand core reviewers are quite busy and believe they are doing their best efforts. period! However, I sometimes feel that turnaround time for some of patch sets are really long. I would like to hear opinion from others and suggestions on how to improve this. It can be either/both something each patch set owner need to do more, or/and it could be something we as a openstack-helm project can improve. For instance, it could be influenced by time differences, lack of irc presence, or anything else. etc. I really would like to find out there are anything we can improve together. I would like to get any kind of advise on the following. - sometimes, it is really difficult to get core reviewers' comments or reviews. I routinely put the list of patch sets on irc meeting agenda, however, there still be a long turnaround time between comments. As a result, it usually takes a long time to process a patch set, does sometimes cause rebase as well. - Having said that, I would like to have any advise on what we need to do more, for instance, do we need to be in irc directly asking each patch set to core reviewers? do we need to put core reviewers' name when we push patch set? etc. - Some of patch sets are being reviewed and merged quickly, and some of patch sets are not. I would like to know what makes this difference so that I can tell my developers how to do better job writing and communicating patch sets. There are just some example patch sets currently under review stage. 1. https://review.openstack.org/#/c/603971/ >> this ps has been discussed for its contents and scope. Cloud you please add if there is anything else we need to do other than wrapping some of commit message? 2. https://review.openstack.org/#/c/633456/ >> this is simple fix. how can we make core reviewer notice this patch set so that they can quickly view? 3. https://review.openstack.org/#/c/625803/ >> we have been getting feedbacks and questions on this patch set, that has been good. but round-trip time for the recent comments takes a week or more. because of that delay (?), the owner of this patch set needed to rebase this one often. Will this kind of case be improved if author engages more on irc channel or via mailing list to get feedback rather than relying on gerrit reviews? Frankly speaking, I don't know if this is a real issue or just way it is. I just want to pass some of voice from our developers, and really would like to hear what others think and find a better way to communicate. Thanks you. -- *Jaesuk Ahn*, Ph.D. Software Labs, SK Telecom

2 3

[all][tc] Formalizing cross-project pop-up teams
by Thierry Carrez 16 Feb '19

16 Feb '19

TL;DR: Maybe to help with cross-project work we should formalize temporary teams with clear objective and disband criteria, under the model of Kubernetes "working groups". Long version: Work in OpenStack is organized around project teams, who each own a set of git repositories. One well-known drawback of this organization is that it makes cross-project work harder, as someone has to coordinate activities that ultimately affects multiple project teams. We tried various ways to facilitate cross-project work in the past. It started with a top-level repository of cross-project specs, a formal effort which failed due to a disconnect between the spec approvers (TC), the people signed up to push the work, and the teams that would need to approve the independent work items. This was replaced by more informal "champions", doing project-management and other heavy lifting to get things done cross-project. This proved successful, but champions are often facing an up-hill battle and often suffer from lack of visibility / blessing / validation. SIGs are another construct that helps holding discussions and coordinating work around OpenStack problem spaces, beyond specific project teams. Those are great as a permanent structure, but sometimes struggle to translate into specific development work, and are a bit heavy-weight just to coordinate a given set of work items. Community goals fill the gap between champions and SIGs by blessing a given set of cross-community goals for a given release. However, given their nature (being blessed by the TC at every cycle), they are a better fit for small, cycle-long objectives that affect most of the OpenStack project teams, and great to push consistency across all projects. It feels like we are missing a way to formally describe a short-term, cross-project objective that only affects a number of teams, is not tied to a specific cycle, and organize work around a temporary team specifically formed to reach that objective. A team that would get support from the various affected project teams, increasing chances of success. Kubernetes encountered the same problem, with work organized around owners and permanent SIGs. They created the concept of a "working group"[1] with a clear limited objective, and a clear disband criteria. I feel like adopting something like it in OpenStack could help with work that affects multiple projects. We would not name it "working group" since that's already overloaded in OpenStack, but maybe "pop-up team" to stress the temporary nature of it. We've been sort-of informally using those in the past, but maybe formalizing and listing them could help getting extra visibility and prioritization. Thoughts? Alternate solutions? [1] https://github.com/kubernetes/community/blob/master/committee-steering/gove… -- Thierry Carrez (ttx)

11 23

Re: Why COA exam is being retired?
by Lauren Sell 15 Feb '19

15 Feb '19

Thanks very much for the feedback. When we launched the COA, the commercial market for OpenStack was much more crowded (read: fragmented), and the availability of individuals with OpenStack experience was more scarce. That indicated a need for a vendor neutral certification to test baseline OpenStack proficiency, and to help provide a target for training curriculum being developed by companies in the ecosystem. Three years on, the commercial ecosystem has become easier to navigate, and there are a few thousand professionals who have taken the COA and had on-the-job experience. As those conditions have changed, we've been trying to evaluate the best ways to use the Foundation's resources and time to support the current needs for education and certification. The COA in its current form is pretty resource intensive, because it’s a hands-on exam that runs in a virtual OpenStack environment. To maintain the exam (including keeping it current to OpenStack releases) would require a pretty significant investment in terms of time and money this year. From the data and demand we’re seeing, the COA did not seem to be a top priority compared to our investments in programs that push knowledge and training into the ecosystem like Upstream Institute, supporting OpenStack training partners, mentoring, and sponsoring internship programs like Outreachy and Google Summer of Code. That said, we’ve honestly been surprised by the response from training partners and the community as plans have been trickling out these past few weeks, and are open to discussing it. If there are people and companies who are willing to invest time and resources into a neutral certification exam, we could investigate alternative paths. It's very helpful to hear which education activities you find most valuable, and if you'd like to have a deeper discussion or volunteer to help, let me know and we can schedule a community call next week. Regardless of the future of the COA exam, we will of course continue to maintain the training marketplace at openstack.org to promote commercial training partners and certifications. There are also some great books and resources developed by community members listed alongside the community training. > From: Jay Bryant jungleboyj(a)gmail.com > Date: January 25, 2019 07:42:55 > Subject: Re: Why COA exam is being retired? > To: openstack-discuss(a)lists.openstack.org > >> On 1/25/2019 9:27 AM, Jonathan Proulx wrote: >>> On Fri, Jan 25, 2019 at 10:09:04AM -0500, Jay Pipes wrote: >>> :On 01/25/2019 09:09 AM, Erik McCormick wrote: >>> :> On Fri, Jan 25, 2019, 8:58 AM Jay Bryant <jungleboyj(a)gmail.com >>> >>> :> That's sad. I really appreciated having a non-vendory, ubiased, >>> :> community-driven option. >>> : >>> :+10 >>> : >>> :> If a vendor folds or moves on from Openstack, your certification >>> :> becomes worthless. Presumably, so long as there is Openstack, there >>> :> will be the foundation at its core. I hope they might reconsider. >>> : >>> :+100 >>> >>> So to clarify is the COA certifiaction going away or is the Foundation >>> just no longer administerign the exam? >>> >>> It would be a shame to loose a standard unbiased certification, but if >>> this is a transition away from directly providing the training and >>> only providing the exam specification that may be reasonable. >>> >>> -Jon >> >> When Allison e-mailed me last week they said they were having meetings >> to figure out how to go forward with the COA. The foundations partners >> were going to be offering the exam through September and they were >> working on communicating the status of things to the community. >> >> So, probably best to not jump to conclusions and wait for the official >> word from the community. >> >> - Jay > > >

1 2

[tc][all] Train Community Goals
by Lance Bragstad 15 Feb '19

15 Feb '19

Hi all, The purpose of this thread is to have a more focused discussion about what we'd like to target for Train community goals, bootstrapped with the outcomes from the session in Berlin [0]. During the session, we went through each item as a group and let the person who added it share why they thought it would be a good community goal candidate for the next release. Most goals have feedback captured in etherpad describing next steps, but the following stuck out as top contenders from the session (rated by upvotes): 1. Moving legacy clients to python-openstackclient 2. Cleaning up resources when deleting a project 3. Service-side health checks I don't think I missed any goals from the session, but if I did, please let me know and I'll add it to the list so that we can discuss it here. Does anyone have strong opinions either way about the goals listed above? [0] https://etherpad.openstack.org/p/BER-t-series-goals

15 26

Open Infrastructure Summit Denver - Community Voting Open
by Ashlee Ferguson 14 Feb '19

14 Feb '19

Hi everyone, Community voting for the Open Infrastructure Summit Denver sessions is open! You can VOTE HERE <https://www.openstack.org/summit/denver-2019/vote-for-speakers>, but what does that mean? Now that the Call for Presentations has closed, all submissions are available for community vote and input. After community voting closes, the volunteer Programming Committee members will receive the presentations to review and determine the final selections for Summit schedule. While community votes are meant to help inform the decision, Programming Committee members are expected to exercise judgment in their area of expertise and help ensure diversity of sessions and speakers. View full details of the session selection process here <https://www.openstack.org/summit/denver-2019/call-for-presentations/selecti…>. In order to vote, you need an OSF community membership. If you do not have an account, please create one by going to openstack.org/join <http://openstack.org/join>. If you need to reset your password, you can do that here <https://www.openstack.org/Security/lostpassword>. Hurry, voting closes Monday, February 4 at 11:59pm Pacific Time (Tuesday, February 5 at 7:59 UTC). Continue to visit https://www.openstack.org/summit/denver-2019 <https://www.openstack.org/summit/denver-2019> for all Summit-related information. REGISTER Register for the Summit <https://openinfrasummitandptgdenver.eventbrite.com/> before prices increase in late February! VISA APPLICATION PROCESS Make sure to secure your Visa soon. More information <https://www.openstack.org/summit/denver-2019/travel/#travel-support> about the Visa application process. TRAVEL SUPPORT PROGRAM February 27 is the last day to submit applications. Please submit your applications <https://www.openstack.org/summit/denver-2019/travel/#travel-support> by 11:59pm Pacific Time (February 28 at 7:59am UTC). If you have any questions, please email summit(a)openstack.org. Cheers, Ashlee Ashlee Ferguson OpenStack Foundation ashlee(a)openstack.org

4 5

[tc][all][self-healing-sig] Service-side health checks community goal for Train cycle
by Jean-Philippe Evrard 14 Feb '19

14 Feb '19

Hello everyone, As you might have seen on the ML, two of the 3 top contenders for the Train cycle community got some traction. Let's here talk about the last one: The service-side health checks. While people were interested in this goal previously, nobody really came forward on the pre-work. Last week, I met a few of my colleagues to see what we can do together. Matt (irc: mattoliverau), Adam (irc: aspiers), and I discussed about the different ways to implement this new API, with the help of many in #openstack-sdk. Long story short, the current framework might be "good enough" for extension already, as we could have extra "backends" (basically "tests"), to increase the coverage of this healthcheck endpoint. While the immediate next step would be to work on the v2 prototype that Graham started (see link [1], anyone is welcome to help there!), the next step would be far easier if it was crowd sourced: We need to know which service is already using that oslo middleware, which service doesn't want to use it, and which service is already ready for healtchecks. When we'll have a lay of the land, we'll know where the energy will be spent in this community goal: Would that be bringing oslo.middleware to <insert amount here> services or bringing common "backends" that can be used by each service (like DB/MQ/cache checks). I would be very happy if you could have a look at this ethercal [2], and add/edit your project capabilities there. Thank you in advance. Jean-Philippe Evrard (evrardjp) [1]: https://review.openstack.org/#/c/617924/ [2]: https://ethercalc.openstack.org/di0mxkiepll8

4 5

[manila][glusterfs] on queens error
by Ignazio Cassano 13 Feb '19

13 Feb '19

Hello All, I installed manila on my queens openstack based on centos 7. I configured two servers with glusterfs replocation and ganesha nfs. I configured my controllers octavia,conf but when I try to create a share the manila scheduler logs reports: Failed to schedule create_share: No valid host was found. Failed to find a weighted host, the last executed filter was CapabilitiesFilter.: NoValidHost: No valid host was found. Failed to find a weighted host, the last executed filter was CapabilitiesFilter. 2019-01-31 16:07:32.614 159380 INFO manila.message.api [req-241d66b3-8004-410b-b000-c6d2d3536e4a 89f76bc5de5545f381da2c10c7df7f15 59f1f232ce28409593d66d8f6495e434 - - -] Creating message record for request_id = req-241d66b3-8004-410b-b000-c6d2d3536e4a I did not understand if controllers node must be connected to the network where shares must be exported for virtual machines, so my glusterfs are connected on the management network where openstack controllers are conencted and to the network where virtual machine are connected. My manila.conf section for glusterfs section is the following [gluster-manila565] driver_handles_share_servers = False share_driver = manila.share.drivers.glusterfs.GlusterfsShareDriver glusterfs_target = root@10.102.184.229:/manila565 glusterfs_path_to_private_key = /etc/manila/id_rsa glusterfs_ganesha_server_username = root glusterfs_nfs_server_type = Ganesha glusterfs_ganesha_server_ip = 10.102.184.229 #glusterfs_servers = root(a)10.102.185.19 ganesha_config_dir = /etc/ganesha PS 10.102.184.0/24 is the network where controlelrs expose endpoint 10.102.189.0/24 is the shared network inside openstack where virtual machines are connected. The gluster servers are connected on both. Any help, please ? Ignazio

3 14

[keystone] x509 authentication
by Lance Bragstad 12 Feb '19

12 Feb '19

Hi all, We've been going over keystone gaps that need to be addressed for edge use cases every Tuesday. Since Berlin, Oath has open-sourced some of their custom authentication plugins for keystone that help them address these gaps. The basic idea is that users authenticate to some external identity provider (Athenz in Oath's case), and then present an Athenz token to keystone. The custom plugins decode the token from Athenz to determine the user, project, roles assignments, and other useful bits of information. After that, it creates any resources that don't exist in keystone already. Ultimately, a user can authenticate against a keystone node and have specific resources provisioned automatically. In Berlin, engineers from Oath were saying they'd like to move away from Athenz tokens altogether and use x509 certificates issued by Athenz instead. The auto-provisioning approach is very similar to a feature we have in keystone already. In Berlin, and shortly after, there was general agreement that if we could support x509 authentication with auto-provisioning via keystone federation, that would pretty much solve Oath's use case without having to maintain custom keystone plugins. Last week, Colleen started digging into keystone's existing x509 authentication support. I'll start with the good news, which is x509 authentication works, for the most part. It's been a feature in keystone for a long time, and it landed after we implemented federation support around the Kilo release. Chances are there won't be a need for a keystone specification like we were initially thinking in the edge meetings. Unfortunately, the implementation for x509 authentication has outdated documentation, is extremely fragile, hard to set up, and hasn't been updated with improvements we've made to the federation API since the original implementation (like shadow users or auto-provisioning, which work with other federated protocols like OpenID Connect and SAML). We've started tracking the gaps with bugs [0] so that we have things written down. I think the good thing is that once we get this cleaned up, we'll be able to re-use some of the newer federation features with x509 authentication/federation. These updates would make x509 a first-class federated protocol. The approach, pending the bug fixes, would remove the need for Oath's custom authentication plugins. It could be useful for edge deployments, or even deployments with many regions, by allowing users to be auto-provisioned in each region. Although, it doesn't necessarily solve the network partition issue. Now that we have an idea of where to start and some bug reports [0], I'm wondering if anyone is interested in helping with the update or refactor. Because this won't require a specification, we can get started on it sooner, instead of having to wait for Train development and a new specification. I'm also curious if anyone has comments or questions about the approach. Thanks, Lance [0] https://bugs.launchpad.net/keystone/+bugs?field.tag=x509

4 6

[openstack-ansible] bug squash day!
by Mohammed Naser 12 Feb '19

12 Feb '19

Hi team, As you may have noticed, bug triage during our meetings has been something that has kinda killed attendance (really, no one seems to enjoy it, believe it or not!) I wanted to propose for us to take a day to go through as much bugs as possible, triaging and fixing as much as we can. It'd be a fun day and we can also hop on a more higher bandwidth way to talk about this stuff while we grind through it all. Is this something that people are interested in, if so, is there any times/days that work better in the week to organize? Thanks! Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

4 9

[neutron] CI issue related to pyroute2 and latest oslo.privsep
by Slawomir Kaplonski 12 Feb '19

12 Feb '19

Hi, Recently we had one more issue related to oslo.privsep and pyroute2. This caused many failures in Neutron CI. See [1] for details. Now fix (more like a workaround) for this issue is merged [2]. So if You saw in Your patch failing tempest/scenario jobs and in failed tests there were issues with SSH to instance through floating IP, please now rebase Your patch. It should be better :) [1] https://bugs.launchpad.net/neutron/+bug/1811515 [2] https://review.openstack.org/#/c/631275/ — Slawek Kaplonski Senior software engineer Red Hat

2 2

Re: Subject: Re: [Trove] State of the Trove service tenant deployment model
by Darek Król 12 Feb '19

12 Feb '19

On Wed, Jan 23, 2019 at 9:27 AM Fox, Kevin M <Kevin.Fox(a)pnnl.gov<mailto:Kevin.Fox@pnnl.gov>> wrote: > > I'd recommend at this point to maybe just run kubernetes across the vms and push the guest agents/workload to them. > This sounds like an overkill to me. Currently, different projects in openstack are solving this issue > in different ways, e.g. Octavia is using two-way SSL authentication API between the controller service and amphora(which is the vm running HTTP server inside), Magnum is using heat-container-agent that is communicating with Heat via API, etc. However, Trove chooses another option which has brought a lot of discussions over a long time. > In the current situation, I don't think it's doable for each project heading to one common solution, but Trove can learn from other projects to solve its own problem. > Cheers, > Lingxian Kong The Octavia way of communication was discussed by Trove several times in the context of secuirty. However, the security threat has been eliminated by encryption. I'm wondering if the Octavia way prevents DDOS attacks also ? Implementation of two-way SSL authentication API could be included in the Trove priority list IMHO if it solves all issues with security/DDOS attacks. This could also creates some share code between both projects and help other services as well. Best, Darek

4 5

Horizon extremely slow with 400 instances
by Satish Patel 10 Feb '19

10 Feb '19

folks, we have mid size openstack cloud running 400 instances, and day by day its getting slower, i can understand it render every single machine during loading instance page but it seems it's design issue, why not it load page from MySQL instead of running bunch of API calls behind then page? is this just me or someone else also having this issue? i am surprised why there is no good and robust Web GUI for very popular openstack? I am curious how people running openstack in large environment using Horizon. I have tired all kind of setting and tuning like memcache etc.. ~S

3 2

[TripleO] containers logging to stdout
by Juan Antonio Osorio Robles 08 Feb '19

08 Feb '19

Hello! In Queens, the a spec to provide the option to make containers log to standard output was proposed [1] [2]. Some work was done on that side, but due to the lack of traction, it wasn't completed. With the Train release coming, I think it would be a good idea to revive this effort, but make logging to stdout the default in that release. This would allow several benefits: * All logging from the containers would en up in journald; this would make it easier for us to forward the logs, instead of having to keep track of the different directories in /var/log/containers * The journald driver would add metadata to the logs about the container (we would automatically get what container ID issued the logs). * This wouldo also simplify the stacks (removing the Logging nested stack which is present in several templates). * Finally... if at some point we move towards kubernetes (or something in between), managing our containers, it would work with their logging tooling as well. Any thoughts? [1] https://specs.openstack.org/openstack/tripleo-specs/specs/queens/logging-st… [2] https://blueprints.launchpad.net/tripleo/+spec/logging-stdout-rsyslog

7 11

[ironic] Hardware leasing with Ironic
by Lars Kellogg-Stedman 07 Feb '19

07 Feb '19

Howdy. I'm working with a group of people who are interested in enabling some form of baremetal leasing/reservations using Ironic. There are three key features we're looking for that aren't (maybe?) available right now: - multi-tenancy: in addition to the ironic administrator, we need to be able to define a node "owner" (someone who controls a specific node) and a node "consumer" (someone who has been granted temporary access to a specific node). An "owner" always has the ability to control node power or access the console, can mark a node as available or not, and can set lease policies (such as a maximum lease lifetime) for a node. A "consumer" is granted access to power control and console only when they hold an active lease, and otherwise has no control over the node. - leasing: a mechanism for marking nodes as available, requesting nodes for a specific length of time, and returning those nodes to the available pool when a lease has expired. - hardware only: we'd like the ability to leave os provisioning up to the "consumer". For example, after someone acquires a node via the leasing mechanism, they can use Foreman to provisioning an os onto the node. For example, a workflow might look something like this: - The owner of a baremetal node makes the node part of a pool of available hardware. They set a maximum lease lifetime of 5 days. - A consumer issues a lease request for "3 nodes with >= 48GB of memory and >= 1 GPU" and "1 node with >= 16GB of memory and >= 1TB of local disk", with a required lease time of 3 days. - The leasing system finds available nodes matching the hardware requirements and with owner-set lease policies matching the lease lifetime requirements. - The baremetal nodes are assigned to the consumer, who can then attach them to networks and make use of their own provisioning tools (which may be another Ironic instance?) to manage the hardware. The consumer is able to control power on these nodes and access the serial console. - At the end of the lease, the nodes are wiped and returned to the pool of available hardware. The previous consumer no longer has any access to the nodes. Our initial thought is to implement this as a service that sits in front of Ironic and provides the multi-tenancy and policy logic, while using Ironic to actually control the hardware. Does this seem like a reasonable path forward? On paper there's a lot of overlap here between what we want and features provided by things like the Nova schedulers or the Placement api, but it's not clear we can leverage those at the baremetal layer. Thanks for your thoughts, -- Lars Kellogg-Stedman <lars(a)redhat.com> | larsks @ {irc,twitter,github} http://blog.oddbit.com/ |

7 21

[tc] agenda for upcoming TC meeting on 7 Feb
by Doug Hellmann 07 Feb '19

07 Feb '19

TC Members, Our next meeting will be on Thursday, 7 Feb at 1400 UTC in #openstack-tc. This email contains the agenda for the meeting. If you will not be able to attend, please include your name in the "Apologies for Absence" section of the wiki page [0]. * corrections to TC member election section of bylaws are completed (fungi, dhellmann) * status update for project team evaluations based on technical vision (cdent, TheJulia) * defining the role of the TC (cdent, ttx) * keeping up with python 3 releases (dhellmann, gmann) * status update of Train cycle goals selection update (lbragstad, evrardjp) * TC governance resolution voting procedures (dhellmann) * upcoming TC election (dhellmann) * review proposed OIP acceptance criteria (dhellmann, wendar) * TC goals for Stein (dhellmann) [0] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee -- Doug

1 1

[openstack-ansible][magnum]
by Alfredo De Luca 07 Feb '19

07 Feb '19

Hi all. I finally instaledl successufully openstack ansible (queens) but, after creating a cluster template I create k8s cluster, it stuck on kube_masters <https://10.1.8.113/project/stacks/stack/6221608c-e7f1-4d76-b694-cdd7ec22c38…> b7204f0c-b9d8-4ef2-8f0b-afe4c077d039 <https://10.1.8.113/project/stacks/stack/b7204f0c-b9d8-4ef2-8f0b-afe4c077d03…> OS::Heat::ResourceGroup 16 minutes Create In Progress state changedcreate in progress....and after around an hour it says...time out. k8s master seems to be up.....at least as VM. any idea? *Alfredo*

5 43

[openstack-helm] How to specify nova override for multiple pci alias
by Gauld, James 06 Feb '19

06 Feb '19

How can I specify a helm override to configure nova PCI alias when there are multiple aliases? I haven't been able to come up with a YAML compliant specification for this. Are there other alternatives to be able to specify this as an override? I assume that a nova Chart change would be required to support this custom one-alias-entry-per-line formatting. Any insights on how to achieve this in helm are welcomed. Background: There is a limitation in the nova.conf specification of PCI alias in that it does not allow multiple PCI aliases as a list. The code says "Supports multiple aliases by repeating the option (not by specifying a list value)". Basically nova currently only supports one-alias-entry-per-line format. Ideally I would specify global pci alias in a format similar to what can be achieved with PCI passthrough_whitelist, which can takes JSON list of dictionaries. This is what I am trying to specify in nova.conf (i.e., for nova-api-osapi and nova-compute): [pci] alias = {dict 1} alias = {dict 2} . . . The following nova configuration format is desired, but not as yet supported by nova: [pci] alias = [{dict 1}, {dict 2}] The following snippet of YAML works for PCI passthrough_whitelist, where the value encoded is a JSON string: conf: nova: overrides: nova_compute: hosts: - conf: nova: pci: passthrough_whitelist: '[{"class_id": "030000", "address": "0000:00:02.0"}]' Jim Gauld

3 5

[nova] [placement] [packaging] placement extraction check in meeting
by Chris Dent 06 Feb '19

06 Feb '19

As discussed in the recent pupdate [1] there will be a meeting this Wednesday at 1700 UTC to discuss the current state of the placement extraction and get some idea on the critical items that need to be addressed to feel comfy. If you're interested in this topic, meet near that time in the #openstack-placement IRC channel and someone will produce links for a hangout, etherpad, whatever is required. Thanks. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001666.… -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

6 13

[openstack-dev] [stackalytics] Stackalytics Facelift
by Boris Renski 06 Feb '19

06 Feb '19

Folks, Happy New Year! We wanted to start the year by giving a facelift to stackalytics.com (based on stackalytics openstack project). Brief summary of updates: - We have new look and feel at stackalytics.com - We did away with DriverLog <http://stackalytics.com/report/driverlog> and Member Directory <http://stackalytics.com/report/members>, which were not very actively used or maintained. Those are still available via direct links, but not in the men on the top - BIGGEST CHANGE: You can now track some of the CNCF and Unaffiliated project commits via a separate subsection accessible at the top nav. Before this was all bunched up in Project Type -> Complimentary Happy to hear comments or feedback or answer questions. -Boris

6 10

[tc][all] Project deletion community goal for Train cycle
by Adrian Turjak 06 Feb '19

06 Feb '19

Hello OpenStackers! As discussed at the Berlin Summit, one of the proposed community goals was project deletion and resource clean-up. Essentially the problem here is that for almost any company that is running OpenStack we run into the issue of how to delete a project and all the resources associated with that project. What we need is an OpenStack wide solution that every project supports which allows operators of OpenStack to delete everything related to a given project. Before we can choose this as a goal, we need to define what the actual proposed solution is, and what each service is either implementing or contributing to. I've started an Etherpad here: https://etherpad.openstack.org/p/community-goal-project-deletion Please add to it if I've missed anything about the problem description, or to flesh out the proposed solutions, but try to mostly keep any discussion here on the mailing list, so that the Etherpad can hopefully be more of a summary of where the discussions have led. This is mostly a starting point, and I expect there to be a lot of opinions and probably some push back from doing anything too big. That said, this is a major issue in OpenStack, and something we really do need because OpenStack is too big and too complicated for this not to exist in a smart cross-project manner. Let's solve this the best we can! Cheers, Adrian Turjak

7 15

[neutron] OVS OpenFlow L3 DVR / dvr_bridge agent_mode
by Duarte Cardoso, Igor 05 Feb '19

05 Feb '19

Hi Neutron, I've been internally collaborating on the ``dvr_bridge`` L3 agent mode [1][2][3] work (David Shaughnessy, Xubo Zhang), which allows the L3 agent to make use of Open vSwitch / OpenFlow to implement ``distributed`` IPv4 Routers thus bypassing kernel namespaces and iptables and opening the door for higher performance by keeping packets in OVS for longer. I want to share a few questions in order to gather feedback from you. I understand parts of these questions may have been answered in the past before my involvement, but I believe it's still important to revisit and clarify them. This can impact how long it's going to take to complete the work and whether it can make it to stein-3. 1. Should OVS support also be added to the legacy router? And if so, would it make more sense to have a new variable (not ``agent_mode``) to specify what backend to use (OVS or kernel) instead of creating more combinations? 2. What is expected in terms of CI for this? Regarding testing, what should this first patch include apart from the unit tests? (since the l3_agent.ini needs to be configured differently). 3. What problems can be anticipated by having the same agent managing both kernel and OVS powered routers (depending on whether they were created as ``distributed``)? We are experimenting with different ways of decoupling RouterInfo (mainly as part of the L3 agent refactor patch) and haven't been able to find the right balance yet. On one end we have an agent that is still coupled with kernel-based RouterInfo, and on the other end we have an agent that either only accepts OVS-based RouterInfos or only kernel-based RouterInfos depending on the ``agent_mode``. We'd also appreciate reviews on the 2 patches [4][5]. The L3 refactor one should be able to pass Zuul after a recheck. [1] Spec: https://blueprints.launchpad.net/neutron/+spec/openflow-based-dvr [2] RFE: https://bugs.launchpad.net/neutron/+bug/1705536 [3] Gerrit topic: https://review.openstack.org/#/q/topic:dvr_bridge+(status:open+OR+status:me… [4] L3 agent refactor patch: https://review.openstack.org/#/c/528336/29 [5] dvr_bridge patch: https://review.openstack.org/#/c/472289/17 Thank you! Best regards, Igor D.C.

5 6

[openstack-dev] [Neutron] Propose Liu Yulong for Neutron core
by Miguel Lavalle 05 Feb '19

05 Feb '19

Hi Stackers, I want to nominate Liu Yulong (irc: liuyulong) as a member of the Neutron core team. Liu started contributing to Neutron back in Mitaka, fixing bugs in HA routers. Since then, he has specialized in L3 networking, developing a deep knowledge of DVR. More recently, he single handedly implemented QoS for floating IPs with this series of patches: https://review.openstack.org/#/q/topic:bp/floating-ip-rate-limit+(status:op…. He has also been very busy helping to improve the implementation of port forwardings and adding QoS to them. He also works for a large operator in China, which allows him to bring an important operational perspective from that part of the world to our project. The quality and number of his code reviews during the Stein cycle is on par with the leading members of the core team: https://www.stackalytics.com/?module=neutron-group. I will keep this nomination open for a week as customary. Best regards Miguel

7 7

[publiccloud] New Contributor Joining
by Sindisiwe Chuma 05 Feb '19

05 Feb '19

Hi All, I am Sindi, a new member. I am interested in participating in the Pubic Cloud Operators Working Group. Are there current projects or initiatives running and documentation available to familiarize myself with the work done and currently being done? Could you please refer me to resources containing information. Kind Regards, Sindi +27 72 572 7757

2 1

[ironic] [thirdparty-ci] BaremetalBasicOps test
by Michael Turek 04 Feb '19

04 Feb '19

Hello all, Our ironic job has been broken and it seems to be due to a lack of IPs. We allocate two IPs to our job, one for the dhcp server, and one for the target node. This had been working for as long as the job has existed but recently (since about early December 2018), we've been broken. The job is able to clean the node during devstack, successfully deploy to the node during the tempest run, and is successfully validated via ssh. The node then moves to clean failed with a network error [1], and the job subsequently fails. Sometime between the validation and attempting to clean, the neutron port associated with the ironic port is deleted and a new port comes into existence. Where I'm having trouble is finding out what this port is. Based on it's MAC address It's a virtual port, and its MAC is not the same as the ironic port. We could add an IP to the job to fix it, but I'd rather not do that needlessly. Any insight or advice would be appreciated here! Thanks, Mike Turek <mjturek> [1] http://paste.openstack.org/show/743191/

3 3

[infra][zuul]Run only my 3rd party CI on my environment
by Rikimaru Honjo 03 Feb '19

03 Feb '19

Hello, I have a question about Zuulv3. I'm preparing third party CI for openstack/masakari PJ. I'd like to run my CI by my Zuulv3 instance on my environment. In my understand, I should add my pipeline to the project of the following .zuul.yaml for my purpose. https://github.com/openstack/masakari/blob/master/.zuul.yaml But, as a result, my Zuulv3 instance also run existed pipelines(check & gate). I want to run only my pipeline on my environment. (And, existed piplines will be run on openstack-infra environment.) How can I make my Zuulv3 instance ignore other pipeline? Best regards, -- _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/ Rikimaru Honjo E-mail:honjo.rikimaru@po.ntt-tx.co.jp

2 2

[nova] Per-instance serial number implementation question
by Matt Riedemann 31 Jan '19

31 Jan '19

The proposal from the spec for this feature was to add an image property (hw_unique_serial), flavor extra spec (hw:unique_serial) and new "unique" choice to the [libvirt]/sysinfo_serial config option. The image property and extra spec would be booleans but really only True values make sense and False would be more or less ignored. There were no plans to enforce strict checking of a boolean value, e.g. if the image property was True but the flavor extra spec was False, we would not raise an exception for incompatible values, we would just use OR logic and take the image property True value. The boolean usage proposed is a bit confusing, as can be seen from comments in the spec [1] and the proposed code change [2]. After thinking about this a bit, I'm now thinking maybe we should just use a single-value enum for the image property and flavor extra spec: image: hw_guest_serial=unique flavor: hw:guest_serial=unique If either are set, then we use a unique serial number for the guest. If neither are set, then the serial number is based on the host configuration as it is today. I think that's more clear usage, do others agree? Alex does. I can't think of any cases where users would want hw_unique_serial=False, so this removes that ability and confusion over whether or not to enforce mismatching booleans. [1] https://review.openstack.org/#/c/612531/2/specs/stein/approved/per-instance… [2] https://review.openstack.org/#/c/619953/7/nova/virt/libvirt/driver.py@4894 -- Thanks, Matt

9 19

[dev] How to develop changes in a series
by Eric Fried 31 Jan '19

31 Jan '19

This started off as a response to some questions Sundar asked, but I thought it might be interesting/useful for new[er] OpenStack developers at large, so broadcasting. I'm sure there's a document somewhere that covers this, but here's a quick run-down on how to develop multiple changes in a series. (Or at least, how I do it.) Start on a freshly-pull'd master branch: efried@efried-ThinkPad-W520:~/Neo/nova$ git checkout master Switched to branch 'master' Your branch is up-to-date with 'origin/master'. efried@efried-ThinkPad-W520:~/Neo/nova$ git pull --all <snip> When you're working on a blueprint, you want to name your local branch after the blueprint. So in this case, bp/nova-cyborg-interaction. efried@efried-ThinkPad-W520:~/Neo/nova$ git checkout -b bp/nova-cyborg-interaction Switched to a new branch 'bp/nova-cyborg-interaction' efried@efried-ThinkPad-W520:~/Neo/nova$ git log --oneline -1 --decorate 5bf6f63 (HEAD, origin/master, origin/HEAD, gerrit/master, master, bp/nova-cyborg-interaction) Merge "Deprecate the nova-xvpvncproxy service" When you `git commit` (without `--amend`), you're creating a new commit on top of whatever commit you started at. If you started with a clean, freshly pull'd master branch, that'll be whatever the most recently merged commit in the master branch was. In this example, that's commit 5bf6f63. So let's say I make an edit for my first patch and commit it: efried@efried-ThinkPad-W520:~/Neo/nova$ echo 'python-cyborgclient>=1.0' >> requirements.txt efried@efried-ThinkPad-W520:~/Neo/nova$ echo 'python-cyborgclient==1.1' >> lower-constraints.txt efried@efried-ThinkPad-W520:~/Neo/nova$ git commit -a -m "Add cyborg client to requirements" [bp/nova-cyborg-interaction 1b2c453] Add cyborg client to requirements 2 files changed, 2 insertions(+) efried@efried-ThinkPad-W520:~/Neo/nova$ git log --oneline -2 --decorate 1b2c453 (HEAD, bp/nova-cyborg-interaction) Add cyborg client to requirements 5bf6f63 (origin/master, origin/HEAD, gerrit/master, master) Merge "Deprecate the nova-xvpvncproxy service" I just made commit 1b2c453 on top of 5bf6f63. You'll notice my branch name (bp/nova-cyborg-interaction) came along with me. Now I'm going to make another change, but just part of it, a work-in-progress commit: efried@efried-ThinkPad-W520:~/Neo/nova$ mkdir nova/pci/cyborg efried@efried-ThinkPad-W520:~/Neo/nova$ touch nova/pci/cyborg/__init__.py efried@efried-ThinkPad-W520:~/Neo/nova$ git add nova/pci/cyborg/__init__.py efried@efried-ThinkPad-W520:~/Neo/nova$ git commit -m "WIP: Cyborg PCI handling" [bp/nova-cyborg-interaction ebb3505] WIP: Cyborg PCI handling 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 nova/pci/cyborg/__init__.py efried@efried-ThinkPad-W520:~/Neo/nova$ git log --oneline -3 --decorate ebb3505 (HEAD, bp/nova-cyborg-interaction) WIP: Cyborg PCI handling 1b2c453 Add cyborg client to requirements 5bf6f63 (origin/master, origin/HEAD, gerrit/master, master) Merge "Deprecate the nova-xvpvncproxy service" Now commit ebb3505 is on top of 1b2c453, which is still on top of 5bf6f63 (the master). Note that my branch name came with me again. At this point, I push my series up to gerrit. Note that it makes me confirm that I really want to push two commits at once. efried@efried-ThinkPad-W520:~/Neo/nova$ git review You are about to submit multiple commits. This is expected if you are submitting a commit that is dependent on one or more in-review commits, or if you are submitting multiple self-contained but dependent changes. Otherwise you should consider squashing your changes into one commit before submitting (for indivisible changes) or submitting from separate branches (for independent changes). The outstanding commits are: ebb3505 (HEAD, bp/nova-cyborg-interaction) WIP: Cyborg PCI handling 1b2c453 Add cyborg client to requirements Do you really want to submit the above commits? Type 'yes' to confirm, other to cancel: yes remote: remote: Processing changes: new: 2, refs: 2 (\) remote: Processing changes: new: 2, refs: 2 (\) remote: Processing changes: new: 2, refs: 2 (\) remote: Processing changes: new: 2, refs: 2, done remote: remote: New Changes: remote: https://review.openstack.org/623026 Add cyborg client to requirements remote: https://review.openstack.org/623027 WIP: Cyborg PCI handling remote: To ssh://efried@review.openstack.org:29418/openstack/nova.git * [new branch] HEAD -> refs/publish/master/bp/nova-cyborg-interaction Now if you go to either of those links - e.g. https://review.openstack.org/#/c/623026/ - you'll see that the patches are stacked up in series on the top right. But oops, I made a mistake in my first commit. My lower constraint can't be higher than my minimum in requirements.txt. If I still had my branch locally, I could skip this next step, but as a matter of rigor to avoid some common pratfalls, I will pull the whole series afresh from gerrit by asking git review to grab the *top* change: efried@efried-ThinkPad-W520:~/Neo/nova$ git review -d 623027 Downloading refs/changes/27/623027/1 from gerrit Switched to branch "review/eric_fried/bp/nova-cyborg-interaction" Now I'm sitting on the top change (which you'll notice happens to be exactly the same as before I pushed it - again, meaning I could technically have just worked from where I was, but see above): efried@efried-ThinkPad-W520:~/Neo/nova$ git log --oneline -3 --decorate ebb3505 (HEAD, review/eric_fried/bp/nova-cyborg-interaction, bp/nova-cyborg-interaction) WIP: Cyborg PCI handling 1b2c453 Add cyborg client to requirements 5bf6f63 (origin/master, origin/HEAD, gerrit/master, master) Merge "Deprecate the nova-xvpvncproxy service" But I want to edit 1b2c453, while leaving ebb3505 properly stacked on top of it. Here I use a tool called `git restack` (run `pip install git-restack` to install it). efried@efried-ThinkPad-W520:~/Neo/nova$ git restack This pops me into an editor showing me all the commits between wherever I am and the main branch (now they're in top-first order): pick 1b2c453 Add cyborg client to requirements pick ebb3505 WIP: Cyborg PCI handling <snip> I want to fix the first one, so I change to: edit 1b2c453 Add cyborg client to requirements pick ebb3505 WIP: Cyborg PCI handling <snip> Save and quit the editor, and I see: Stopped at 1b2c453453242a3fa57f2d4fdc80c837b02b804f... Add cyborg client to requirements You can amend the commit now, with git commit --amend Once you are satisfied with your changes, run git rebase --continue I fix lower-constraints: efried@efried-ThinkPad-W520:~/Neo/nova$ sed -i 's/cyborgclient==1.1/cyborgclient==1.0/' lower-constraints.txt ...and *amend* the current commit efried@efried-ThinkPad-W520:~/Neo/nova$ git commit -a --amend --no-edit [detached HEAD 6b3455f] Add cyborg client to requirements Date: Wed Dec 5 09:43:15 2018 -0600 2 files changed, 2 insertions(+) ...and tell `git restack` to proceed efried@efried-ThinkPad-W520:~/Neo/nova$ git rebase --continue Successfully rebased and updated refs/heads/review/eric_fried/bp/nova-cyborg-interaction. If I had a taller series, and I had changed 'pick' to 'edit' for more than one commit, I would now be sitting on the next one I needed to edit. As it is, that was the only thing I needed to do, so I'm done and sitting on the top of my series again. 124b612 (HEAD, review/eric_fried/bp/nova-cyborg-interaction) WIP: Cyborg PCI handling 6b3455f Add cyborg client to requirements 5bf6f63 (origin/master, origin/HEAD, gerrit/master, master) Merge "Deprecate the nova-xvpvncproxy service" Notice that the commit hashes have changed for *both* commits (but not the master). The top one changed because it got rebased onto the new version of the middle one. Now if I push the series back up to gerrit, I get the same confirmation prompt, and both changes get a new patch set. If you look at the top patch in gerrit, you'll see that PS2 shows up as just a rebase. ==================== That ought to be enough for now. There's a couple of gotchas when restacking and the automatic rebase results in merge conflicts, but we can save that for another time. To answer your specific question: > * If you have a patch sequence A followed by B, where patch B depends on patch A, > how do you communicate that in the submission? If they're in the same commit series, like the example above, It Just Happens. Zuul and the CIs know to pull the whole series when they run; gerrit won't merge N+1 until N is merged; etc. If they're *not* in the same commit series, or if they're not in the same repository, you can use Depends-On in the commit message, to the same effect. (But I'm not sure if/how Depends-On works for feature branches.) HTH, efried .

7 8

[release] Release countdown for week R-9, February 4-8
by Sean McGinnis 31 Jan '19

31 Jan '19

Welcome back to the countdown emails. Some important deadlines are starting to come up quick, so please take a look at what might affect you. Development Focus ----------------- Teams should be focused on implementing planned work for the cycle. It is also a good time to review those plans and reprioritize anything if needed based on the what progress has been made and what looks realistic to complete in the next few weeks. General Information ------------------- We have a few deadlines coming up as we get closer to the end of the cycle: * Non-client libraries (generally, any library that is not python-${PROJECT}client) must have a final release by February 28. Only critical bugfixes will be allowed past this point. Please make sure any important feature works has required library changes by this time. * Client libraries must have a final release by March 7. Quick reminder for teams with cycle-with-intermediary deliverables that are not libraries - we had mentioned earlier in the cycle that you may want to consider moving to the new cycle-with-rc model if you are not actually doing intermediary releases: http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000465… Next week we will be looking at what cycle-with-intermediary deliverables have not done a release yet in Stein. We can still give a little time if teams want to quick get one in before Stein-3, but we will look at switching over those deliverables to cycle-with-rc if that appears to be a more appropriate release model for the way they are being released. Upcoming Deadlines & Dates -------------------------- Non-client library freeze: February 28 Stein-3 milestone: March 7 -- Sean McGinnis (smcginnis)

1 0

[3rd party CI] [ironic] BaremetalBasicOps test
by Michael Turek 31 Jan '19

31 Jan '19

Hello all, Our ironic job has been broken and it seems to be due to a lack of IPs. We allocate two IPs to our job, one for the dhcp server, and one for the target node. This had been working for as long as the job has existed but recently (since about early December 2018), we've been broken. The job is able to clean the node during devstack, successfully deploy to the node during the tempest run, and is successfully validated via ssh. The node then moves to clean failed with a network error [1], and the job subsequently fails. Sometime between the validation and attempting to clean, the neutron port associated with the ironic port is deleted and a new port comes into existence. Where I'm having trouble is finding out what this port is. Based on it's MAC address It's a virtual port, and its MAC is not the same as the ironic port. We could add an IP to the job to fix it, but I'd rather not do that needlessly. Any insight or advice would be appreciated here! Thanks, Mike Turek <mjturek> [1] http://paste.openstack.org/show/743191/

1 0

[Openstack-sigs] [publiccloud-wg] Late reminder weekly meeting Public Cloud WG
by Tobias Rydberg 31 Jan '19

31 Jan '19

Hi everyone, Time for a new meeting for PCWG - today (31st) 1400 UTC in #openstack-publiccloud! Agenda found at https://etherpad.openstack.org/p/publiccloud-wg Sorry for the late reminder! Talk to you later today! Cheers, Tobias -- Tobias Rydberg Senior Developer Twitter & IRC: tobberydberg www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED

1 0

Issue with launching instance with OVS-DPDK
by David Lake 31 Jan '19

31 Jan '19

Hello I've built an Openstack all-in-one using OVS-DPDK via Devstack. I can launch instances which use the "m1.small" flavour (which I have modified to include the hw:mem_size large as per the DPDK instructions) but as soon as I try to launch anything more than m1.small, I get this error: Jan 28 12:56:52 localhost nova-conductor: #033[01;31mERROR nova.scheduler.utils [#033[01;36mNone req-917cd3b9-8ce6-41af-8d44-045002512c91 #033[00;36madmin admin#033[01;31m] #033[01;35m[instance: 25cfee28-08e9-419c-afdb-4d0fe515fb2a] #033[01;31mError from last host: localhost (node localhost): [u'Traceback (most recent call last):\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1935, in _do_build_and_run_instance\n filter_properties, request_spec)\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 2215, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 25cfee28-08e9-419c-afdb-4d0fe515fb2a was re-scheduled: internal error: qemu unexpectedly closed the monitor: 2019-01-28T12:56:48.127594Z qemu-kvm: -chardev socket,id=charnet0,path=/var/run/openvswitch/vhu46b3c508-f8,server: info: QEMU waiting for connection on: disconnected:unix:/var/run/openvswitch/vhu46b3c508-f8,server\n2019-01-28T12:56:49.251071Z qemu-kvm: -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/4-instance-00000005,share=yes,size=4294967296,host-nodes=0,policy=bind: os_mem_prealloc: Insufficient free host memory pages available to allocate guest RAM\n']#033[00m#033[00m My Hypervisor is reporting 510.7GB of RAM and 61 vCPUs. Build is the latest git clone of Devstack. Thanks David

3 9

[infra] fedora-29 node failures
by Paul Belanger 30 Jan '19

30 Jan '19

Morning, Jobs depending on fedora-latest (fedora-29 in this case) look to be in a broken state currently. Since this past weekend, we seems to be hitting an issue with network manager and dbus. From what I can see, we are only currently launching fedora-29 nodes in limestone (resulting it long wait times), and when they come online, seem to fail quikcly with DNS related issues. If you are interested in these jobs, please join #openstack-infra and aid in debugging them. Thanks, Paul

2 1

[openstack-ansible] adding James Denton to os_neutron core
by Mohammed Naser 30 Jan '19

30 Jan '19

Hi team, I'm happy to announce the addition of James Denton to the os_neutron Ansible role core team. James is one of the very knowledgeable OpenStack Networking experts (having authored books about it as well!) and we're happy to have him working on our team. James has contributed a lot of work to add and maintain SDNs support in OpenStack Ansible, with many useful reviews across the os_neutron role (his most recent work, iterating on adding VPP support), he's also always available to help new users of our project in IRC. Thanks & welcome James! Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

3 2

Open Infrastructure Summit Shanghai - Dates and Location
by Erin Disney 30 Jan '19

30 Jan '19

Hi everyone- We are excited to announce that after Denver, the Open Infrastructure Summit will be traveling to Shanghai, China the week of November 4, 2019. The Summit will be held at the Shanghai Expo Centre. Registration and sponsorship opportunities will be available soon. We’ll follow up on the mailing list, and you can check out the website for updates[1] and reach out to summit(a)openstack.org <mailto:summit@openstack.org> with any questions. Cheers, Erin [1] https://www.openstack.org/summit/shanghai-2019 <https://www.openstack.org/summit/shanghai-2019> Erin Disney OpenStack Marketing erin(a)openstack.org <mailto:erin@openstack.org>

1 0

RE: [openstack] [Nova] Instance automatically got shutdown
by Md. Farhad Hasan Khan 30 Jan '19

30 Jan '19

Hi, Sorry for mistake. I corrected the subject line. Thanks Rony -----Original Message----- From: Slawomir Kaplonski [mailto:skaplons@redhat.com] Sent: Wednesday, January 30, 2019 2:55 PM To: rony.khan(a)brilliant.com.bd Cc: openstack-discuss(a)lists.openstack.org Subject: Re: [openstack] [Neutron] Instance automatically got shutdown Hi, Why You tagged „Neutron” in topic? IMO it’s some issue related to Nova rather than Neutron. > Wiadomość napisana przez Md. Farhad Hasan Khan <rony.khan(a)brilliant.com.bd> w dniu 30.01.2019, o godz. 05:07: > > Hi, > we are getting Openstack instance automatically shut down. Instance not shut down from horizon/cli & from inside instance. This is happening randomly in our environment. Please help us to solve this. > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > cat /etc/nova/nova.conf > > [DEFAULT] > sync_power_state_interval=-1 > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > Here is log: > > [root@compute2 ~]# cat /var/log/nova/nova-compute.log |grep 4e143f50-49d4-40c9-b80e-c99dfe45d9d4 > 2019-01-30 08:06:18.750 417615 INFO nova.compute.manager [-] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] VM Stopped (Lifecycle Event) > 2019-01-30 08:06:18.855 417615 INFO nova.compute.manager [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. > 2019-01-30 08:06:19.027 417615 WARNING nova.compute.manager [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4 > 2019-01-30 08:06:19.376 417615 INFO nova.compute.manager [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance is already powered off in the hypervisor when stop is called. > 2019-01-30 08:06:19.441 417615 INFO nova.virt.libvirt.driver [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance already shutdown. > 2019-01-30 08:06:19.445 417615 INFO nova.virt.libvirt.driver [-] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance destroyed successfully. > > > Thanks & B’Rgds, > Rony — Slawek Kaplonski Senior software engineer Red Hat

3 2

[infra][tc] Container images in openstack/ on Docker Hub
by corvus＠inaugust.com 30 Jan '19

30 Jan '19

Hi, As part of the recent infrastructure work described in http://lists.openstack.org/pipermail/openstack-discuss/2019-January/002026.… we now have the ability to fairly easily support uploading of container images to the "openstack/" namespace on Docker Hub. The Infrastructure team does have an account on Docker Hub with ownership rights to this space. It is now fairly simple for us to allow any OpenStack project to upload to openstack/$short_name. As a (perhaps unlikely, but simple) example, Nova could upload images to "openstack/nova", including suffixed images, such as "openstack/nova-scheduler". The system that would enable this is described in this proposed change: https://review.openstack.org/632818 I believe it's within the TC's purview to decide whether this should happen, and if so, what policies should govern it (i.e., what projects are entitled to upload to openstack/). It's possible that the status quo where deployment projects upload to their own namespaces (e.g., loci/) while openstack/ remains empty is desirable. However, since we recently gained the technical ability to handle this, I thought it worth bringing up. Personally, I don't presently advocate one way or the other. -Jim

9 18

[oslo] oslo.utils EventletEvent changes
by Ben Nemec 30 Jan '19

30 Jan '19

Hey, We have a bit of an issue in the EventletEvent class in oslo.utils. We had to blacklist the versions that had the race fix because it turned out that it reintroduced the double-set bug. So the current version is subtly broken, and we released the oslo.service change that started to use it (which is how we found the double-set bug). There are actually two patches proposed right now related to this.[1][2] The first simply fixes the bug, the second optimizes it so we aren't doing unnecessary event replacement. What I would like to do is merge the first patch, release the library with just that, then merge the second patch and release again. That way if we're wrong about the optimization patch being correct (I don't think we are, but this is concurrency, so...) we have the safer fix available in a release. The reason I'm emailing instead of just doing this is that I'm going to be unavailable after today until the 11th, and it would be nice to get at least the first release done before then. I can maybe get that done today, but in case I don't I wanted to get this on record. Thanks. -Ben 1: https://review.openstack.org/#/c/632758 2: https://review.openstack.org/#/c/633053

1 0

[scientific-sig] IRC meeting today 1100 UTC: Baremetal SIG, GPFS+Manila, Open Infra Summit
by Stig Telfer 30 Jan '19

30 Jan '19

Hi All - We have a Scientific SIG IRC meeting at 1100 UTC (about 40 minutes time) in channel #openstack-meeting. Everyone is welcome. The agenda is online here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_January_30th_2019 Today we’d like to discuss the formation of the new baremetal SIG, do a little planning on participation at the Denver Summit, and share experiences on GPFS with Manila. Cheers, Stig

1 0

[oslo] Proposing Zane Bitter as general Oslo core
by Ben Nemec 30 Jan '19

30 Jan '19

Hi all, Zane is already core on oslo.service, but he's been doing good stuff in adjacent projects as well. We could keep playing whack-a-mole with giving him +2 on more repos, but I trust his judgment so I'm proposing we just add him to the oslo-core group. If there are no objections in the next week I'll proceed with the addition. Thanks. -Ben

7 8

[openstack] [Neutron] Instance automatically got shutdown
by Md. Farhad Hasan Khan 30 Jan '19

30 Jan '19

Hi, we are getting Openstack instance automatically shut down. Instance not shut down from horizon/cli & from inside instance. This is happening randomly in our environment. Please help us to solve this. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ cat /etc/nova/nova.conf [DEFAULT] sync_power_state_interval=-1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here is log: [root@compute2 ~]# cat /var/log/nova/nova-compute.log |grep 4e143f50-49d4-40c9-b80e-c99dfe45d9d4 2019-01-30 08:06:18.750 417615 INFO nova.compute.manager [-] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] VM Stopped (Lifecycle Event) 2019-01-30 08:06:18.855 417615 INFO nova.compute.manager [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. 2019-01-30 08:06:19.027 417615 WARNING nova.compute.manager [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state: None, original DB power_state: 1, current VM power_state: 4 2019-01-30 08:06:19.376 417615 INFO nova.compute.manager [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance is already powered off in the hypervisor when stop is called. 2019-01-30 08:06:19.441 417615 INFO nova.virt.libvirt.driver [req-c2b7b111-e124-4aac-8971-59c3f840bacc - - - - -] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance already shutdown. 2019-01-30 08:06:19.445 417615 INFO nova.virt.libvirt.driver [-] [instance: 4e143f50-49d4-40c9-b80e-c99dfe45d9d4] Instance destroyed successfully. Thanks & B’Rgds, Rony

2 1

Re: [Release-job-failures] Release of openstack/ovsdbapp failed
by Doug Hellmann 29 Jan '19

29 Jan '19

zuul(a)openstack.org writes: > Build failed. > > - release-openstack-python http://logs.openstack.org/76/767f3ea3efcedb4c8df1dae1a6b40d0b0a230d3d/relea… : POST_FAILURE in 3m 17s > - announce-release announce-release : SKIPPED > - propose-update-constraints propose-update-constraints : SKIPPED > > _______________________________________________ > Release-job-failures mailing list > Release-job-failures(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures This release failed due to a known issue for which Sean has already proposed a fix. https://review.openstack.org/633829 -- Doug

1 0

[tripleo][ptl]What is the future of availability monitoring in TripleO?
by David M Noriega 29 Jan '19

29 Jan '19

I've noticed that the availability monitoring integration in TripleO using Sensu is marked for deprecation post Rocky, but I do not see any blueprints detailing a replacement plan like there is for fluentd to rsyslog for the logging integration. Where can I find information on what the roadmap is? Is this a feature that will be dropped and left to operators to implement?

3 2