February 2019 - openstack-discuss

[oslo][operators] oslo.messaging and RabbitMQ SSL
by Ben Nemec 22 Feb '19

22 Feb '19

For the past few months, we've been investigating a significant bug when enabling SSL for oslo.messaging connections to RabbitMQ.[0] Thanks to some patient and excellent investigation, it was tracked down to an issue in the amqp library that we use in oslo.messaging. The fix has now been released as 2.4.1, and we've updated the requirements on master to reflect that, but we can't backport requirements changes to the stable branches. Since this affects releases going back to Pike, that's potentially a lot of affected users. We're planning to release note[1] all of the stable branches to communicate the need to use a newer version of the library, but I also wanted to send an email to the list in order to help get the word out. Basically, since we can't fix this in the library itself I'm running a publicity campaign to let everyone know what the fix is. :-) If you have any questions, feel free to reach out here or on IRC in #openstack-oslo. Thanks. -Ben 0: https://bugs.launchpad.net/oslo.messaging/+bug/1800957 1: https://review.openstack.org/#/c/638461

3 3

[neutron] openvswitch switch connection timeout?
by Lars Kellogg-Stedman 22 Feb '19

22 Feb '19

I was trying to track down some connectivity issues with some baremetal nodes booting from iSCSI LUNs provided by Cinder. It turns out that openvswitch is going belly-up. We see in openvswitch-agent.log a "Switch connection timeout" error [1]. Just before that, in /var/log/openvswitch/ovs-vswitchd.log, we see: 2019-02-21T00:32:38.696Z|00795|bridge|INFO|bridge br-tun: deleted interface patch-int on port 1 2019-02-21T00:32:38.696Z|00796|bridge|INFO|bridge br-tun: deleted interface br-tun on port 65534 2019-02-21T00:32:38.823Z|00797|bridge|INFO|bridge br-int: deleted interface int-br-ctlplane on port 1 2019-02-21T00:32:38.823Z|00798|bridge|INFO|bridge br-int: deleted interface br-int on port 65534 2019-02-21T00:32:38.823Z|00799|bridge|INFO|bridge br-int: deleted interface tapb0101920-b9 on port 4 2019-02-21T00:32:38.824Z|00800|bridge|INFO|bridge br-int: deleted interface patch-tun on port 3 2019-02-21T00:32:38.954Z|00801|bridge|INFO|bridge br-ctlplane: deleted interface phy-br-ctlplane on port 4 2019-02-21T00:32:38.954Z|00802|bridge|INFO|bridge br-ctlplane: deleted interface br-ctlplane on port 65534 2019-02-21T00:32:38.954Z|00803|bridge|INFO|bridge br-ctlplane: deleted interface em2 on port 3 At this point, while the output from 'ovs-vsctl' looks fine, other tools like ovs-ofctl report: ovs-ofctl: br-int is not a bridge or a socket Has anyone seen this before? [1] https://termbin.com/ipil -- Lars Kellogg-Stedman <lars(a)redhat.com> | larsks @ {irc,twitter,github} http://blog.oddbit.com/ |

1 2

Fwd: Renaissance Denver Hotel: Quiet Zone (no more train horns!) is OFFICIAL for the A line Light Rail!
by David Medberry 22 Feb '19

22 Feb '19

Just a note from the hotel used at both Denver PTGs... There are no horns at the old Stapleton Renaissance now. They have probably raised their rates as a result. Of course, this is too late for most/all of us to appreciate, but sharing nonetheless. -dave ---------- Forwarded message --------- From: Mioduchoski, Lauren <Lauren.Mioduchoski(a)renaissancehotels.com> Date: Wed, Feb 20, 2019 at 12:48 PM Subject: Renaissance Denver Hotel: Quiet Zone (no more train horns!) is OFFICIAL for the A line Light Rail! To: Good afternoon, We are very excited to announce that as of March 1st all intersections along Denver’s A Line light rail train will be Quiet Zones! This means the A Line trains will no longer use their horns when passing through intersections (unless there is an unusual, emergent situation). I know during your stay here, you shared in your feedback that the train noise was disruptive so I wanted to personally reach out and share this exciting news with you! We hope to welcome you back to our hotel in the future so that you can enjoy our Quiet Zone and our newly renovated rooms! Sincerely, LAUREN MIODUCHOSKI FRONT OFFICE MANAGER Renaissance Denver Hotel 3801 Quebec St, Denver, CO 80207 T 303.399.7500 F 303.321.1966 Renaissance Hotels Renhotels.com l facebook.com/renhotels l twitter.com/renhotels Notice: This e-mail message and or fax is confidential, intended only for the named recipient(s) above and may contain information that is privileged, attorney work product or exempt from disclosure under applicable law. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender at 303.399.7500 and delete/destroy this e-mail message/fax. Thank you.

11 12

[nova][dev] forum brainstorming etherpad available
by melanie witt 22 Feb '19

22 Feb '19

Hey all, We have an etherpad for brainstorming forum topics for the Denver Summit which is open for adding topics: https://etherpad.openstack.org/p/DEN-train-nova-brainstorming There will be an online tool for submitting abstracts opening tomorrow Feb 22, that we'll use for proposing the most popular topics. Cheers, -melanie

1 0

[tc][election] TC Candidacy
by Graham Hayes 22 Feb '19

22 Feb '19

Hi All, I realised I missed this important part of the election process - sending it to the list. > Hello Everyone, > > I have been a member of the TC for a a year now, and I would like to > renominate myself for another term on the TC. I have been working on OpenStack > (mainly in Desigate - the DNS as a Service project) since Havana, which is when > I started to get involved in the work of Technical Committee, for the Designate > Incubation application. > > I have been PTL for Designate for Mitaka, Newton, Ocata, Queens, Rocky, Stein > and Train cycles, and a core for a longer period. I believe my experience > working in a younger, smaller project within OpenStack is still a benefit, > especially as we add more projects to the OpenStack Foundation (OSF) along > side the original OpenStack project. > > I have spent time recently working on a very large OpenStack cloud in a day to > operations role, and I think that this experience is important to have on the > Technical Committee. The experience that a lot of our users have is very > different to what we may assume, and knowing how end users deal with bugs, > deployment life cycles and vendors should guide us. > > I have been involved in the discussion around the future goals this cycle, and > I would like to try and keep driving this forward. I see goals as a real, > tangible change that the TC can drive that helps the community as a whole. > > I think the experience I have had (from both an operator perspective, and as > the PTL of a small project that is struggling with contributor levels) is > valuable. > > I do think cycling out members of the TC is important, but I think that with > the turn over we have is at a good level, and that I can still provide a fresh > out look on the TC. > > Thanks, > > Graham Additionally, I did want to address something that a few people have been asking about - my new employment :) Yes - I am no longer working for a company that has ties (direct or otherwise) to OpenStack. However they do have a strong open source culture, and are completely supportive of me spending some of my time on OpenStack and the TC. I really enjoy working with this community, and don't see myself going anywhere in the near or medium term. My new role has components in the same area (infrastructure as a service) , and I think the insight I will be gaining from this work will definitely benifit OpenStack, combined with my history of involvement and experience running production OpenStack clouds. I am happy to talk to anyone who has any concerns about this, ping me on IRC (mugsie), gr(a)ham.ie or reply to this thread. Thanks for your attention, and please remember to vote! - Graham

1 0

[nova][dev] Any VMware resource pool and shares kind of feature available in openstack nova?
by Sanjay K 22 Feb '19

22 Feb '19

Hi All, Is there any VMware resource pools and shares kind of features available in nova compute scheduler which can do run time scheduling based on available resources on compute host for a launched instance with KVM/Qemu hypervisors based? Basically I want to prioritize some vm instances over others so that higher priority VM instances get better resource allocated at runtime and chance to run than lower priority VMs? Any future proposal on this kind of feature in scope? Please let me know if there is any work around or have to manage with the current default scheduling functions inside nova compute? thanks and regards

2 1

[release] Release countdown for week R-6, February 25 - March 1
by Sean McGinnis 22 Feb '19

22 Feb '19

Some important deadlines in this week's edition as we get closer to the end of the cycle. PTLs and release liaisons, please read through and make sure you are tracking things as we start to get close to the end of Stein. Development Focus ----------------- Non-client library work should be wrapping up for this weeks freeze. Any changes that will require client library changes should get some focus to make sure anything needed there is merged by the client library freeze on March 7. Feature freeze is also coming up with the Stein-3 milestone on March 7. General Information ------------------- We are now getting close to the end of the cycle. The non-client library (typically any lib other than the "python-$PROJECTclient" deliverables) deadline is February 28, followed quickly the next Thursday with the final client library release. Releases for critical fixes will be allowed after this point, but we will be much more restrictive about what is allowed if there are more lib release requests after this point. Please keep this in mind. When requesting these library releases, you should also include the stable branching request with the review (as an example, see the "branches" section here: http://git.openstack.org/cgit/openstack/releases/tree/deliverables/pike/os-…) As we are getting to the point of creating stable/stein branches, this would be a good point for teams to review membership in their $project-stable-maint groups. Once the stable/stein branches are cut for a repo, the ability to approve any necessary backports into those branches for stein will be limited to the members of that stable team. If there are any questions about stable policy or stable team membership, please each out in the #openstack-stable channel. The following cycle-with-intermediary release model deliverables have not had a release for Stein yet: ansible-role-container-registry ansible-role-openstack-operations ansible-role-tripleo-modify-image automaton blazar-nova ceilometermiddleware debtcollector heat-agents instack-undercloud instack kuryr-libnetwork mistral-lib monasca-statsd monasca-thresh mox3 murano-agent networking-baremetal networking-hyperv os-client-config patrole sahara-extra tripleo-ansible tripleo-ipsec If no release is requested by the teams in the week after the deadline, the release team will need to force a release from HEAD in order to have a good point to create stable branches. Upcoming Deadlines & Dates -------------------------- Non-client library freeze: February 28 Stein-3 milestone: March 7 RC1 deadline: March 21 -- Sean McGinnis (smcginnis)

1 0

[nova][numa]Question regarding numa affinity balancer/weigher per host
by Li Liu 22 Feb '19

22 Feb '19

HI Nova folks, I am trying to find out how Numa balance/weighs per host is taking care of in Nova. I know how weighers work in general, but it's weighing between hosts. I am not so clear when it comes to a single host with multiple sockets, how does nova weigh them? For instance, a host has 4 sockets, A, B, C, D. When scheduling request comes in asking for 4 cores on 2 sockets(2 cores per socket), scheduler realized that A+B, A+C, and C+D combination can all fit the request. In this case, how does nova make the decision on which combination to choose from? -- Thank you Regards Li

2 2

Re: [tc] Questions for TC Candidates
by Sean Mooney 21 Feb '19

21 Feb '19

On Thu, 2019-02-21 at 08:12 -0500, Jim Rollenhagen wrote: > On Thu, Feb 21, 2019 at 7:37 AM Sean Mooney <smooney(a)redhat.com> wrote: > > On Wed, 2019-02-20 at 10:24 -0500, Mohammed Naser wrote: > > > Hi Chris, > > > > > > Thanks for kicking this off. I've added my replies in-line. > > > > > > Thank you for your past term as well. > > > > > > Regards, > > > Mohammed > > > > > > On Wed, Feb 20, 2019 at 9:49 AM Chris Dent <cdent+os(a)anticdent.org> wrote: > > > > <snip> > > > > * If you had a magic wand and could inspire and make a single > > > > sweeping architectural or software change across the services, > > > > what would it be? For now, ignore legacy or upgrade concerns. > > > > What role should the TC have in inspiring and driving such > > > > changes? > > > > > > Oh. > > > > > > - Stop using RabbitMQ as an RPC, it's the worst most painful component > > > to run in an entire OpenStack deployment. It's always broken. Switch > > > into something that uses HTTP + service registration to find endpoints. > > as an on looker i have mixed feeling about this statement. > > RabbitMQ can have issue at scale but it morstly works when its not on fire. > > Would you be advocating building a openstack specific RPC layer perhaps > > using keystone as the service registry and a custom http mechanism or adopting > > an existing technology like grpc? > > > > investigating an alternative RPC backend has come up in the past (zeromq and qupid) > > and i think it has merit but im not sure as a comunity creating a new RPC framework is > > a project wide effort that openstack need to solve. that said zaqar is a thing > > https://docs.openstack.org/zaqar/latest/ if it is good enough for our endusers to consume > > perhaps it would be up to the task of being openstack rpc transport layer. > > > > anyway my main question was would you advocate adoption of an exisiting technology > > or creating our own solution if we were to work on this goal as a community. > > > > I'll also chime in here since I agree with Mohammed. > > We're certainly going to have to write software to make this happen. Maybe > that's a new oslo.messaging driver, maybe it's a new equivalent of that layer. > > But we've re-invented enough software already. This isn't a place where we > should do it. We should use build on or glue together existing tools to build > something scalable and relatively simple to operate. > > Are your mixed feelings about this statement a concern about re-inventing > wheels, or about changing an underlying architectural thing at all? with re-inventing wheels. a new oslo.messaging driver i think would be perfectly fine. that said we dont want 50 of them as it will be impossible to test and maintain them all. my concern would be that even if we deveploped the perfect rpc layer for openstack ourselves it would be a lot of effort that could have been spent elsewhere. developing glue logic to use an alternitiv is much less invasive and achievable. the other thing i woudl say is if im felling pedantinc the fact that we have an rpc bus is an architecutal question as may in some cases be the perfromce or feature set the final system provide. but personally i dont see rabbitmq vs grps as an architectual question at all. the message bus is just an io device. we should be able to change that io device for another without it martially effecting our architecutre. if it does we are too tightly coupled to the implementation. you can sub out rabbitmq today with qpid or activmq and in the past you could use zeromq too but rabbit more or less one out over that set of message queue. there is also limited support for kafka. kafka is pretty heavy weight as is itsio from what i have heard so im not sure they would be a good replacement for small cloud but i belive they are ment to scale well. grpc and nats are proably the two rabitmq alternative is would personally consider but i know some projects hack etcd to act as a psudo rpc bus. this type of change is something i could see as a comunity goal eventually but would like to see done with one project first before it got to that point. there is value in useing 1 or 2 rpc buses istead of support many and this is the type of change i hope would be guided by measurement and community feedback. thanks for following up. > > // jim

3 2

[Searchlight] TC vision reflection
by Trinh Nguyen 21 Feb '19

21 Feb '19

Hi team, Follow by the call of the TC [1] for each project to self-evaluate against the OpenStack Cloud Vision [2], the Searchlight team would like to produce a short bullet point style document comparing itself with the vision. The purpose is to find the gaps between Searchlight and the TC vision and it is a good practice to align our work with the rest. I created a new pad [3] and welcome all of your opinions. Then, after about 3 weeks, I will submit a patch set to add the vision reflection document to our doc source. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001417.… [2] https://governance.openstack.org/tc/reference/technical-vision.html [3] https://etherpad.openstack.org/p/-tc-vision-self-eval Ping me on the channel #openstack-searchlight Bests, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

2 2