March 2019 - openstack-discuss - lists.openstack.org

[ptl][swift][election] Tim Burke's candidacy for Swift PTL
by Tim Burke 06 Mar '19

06 Mar '19

I'd like to announce my candidacy for Swift PTL. Within the Swift community, we have established traditions that ensure we move carefully, and with purpose. Somewhere out there, there is an out-of-date-node with data scribbled down in a legacy format that we must still make durable, or an old client written against a years-old version of Swift that continues to perform its business-critical function. By keeping these realities in mind, we have (out of necessity) featured rolling-upgrades since our first release. This is not to say that we must therefore move slowly (though we often do, particularly now that the hype wave has crested, crashed, and begun to ebb). Despite our conservatism, there are few signs of ossification: new use-cases and new workloads bring new demands, and Swift evolves to satisfy them with new features. We now approach a new transition: John, our long-serving PTL, is stepping down. I have no worries, however; the Swift community is accustomed to dealing with (and thinking in terms of) an array of mostly-independent actors, working to improve the state of the system. The Swift developers are among the best I've had the pleasure to work alongside, and I have no doubt that Swift will continue to improve. I believe the best way I can further that improvement is to serve as PTL; to listen to users; to engage with operators; to enable developers. We are working on great, ambitious projects: * Pete from Red Hat is driving us supporting Python 3. We've known for a while that this would be necessary; it's good to see it finally happening. * Alex and Romain from OVH are upstreaming their (already running in production!) alternate diskfile format to support small objects. * Clay at SwiftStack continues to make replication and reconstruction better. * Kazuhiro at NTT is reworking the object-expirer queue, turning it into a general task queue. There is a lot going on in Swift. I can't wait to see what we build. Tim Burke

1 0

Re: [scientific-sig] IRC meeting 1100 UTC: Edge computing and Scientific use cases
by Bogdan Dobrelya 06 Mar '19

06 Mar '19

[added edge-computing(a)lists.openstack.org] JFYI: some background for ICFC 2019 event: > ================================================================================= > IEEE International Conference on Fog Computing (ICFC 2019) > June 24-26, 2019 > Prague, Czech Republic > http://conferences.computer.org/ICFC/2019/ > Co-located with the IEEE International Conference on Cloud Engineering > (IC2E 2019) > ================================================================================== > > Important Dates > > -------------------- > > Paper registration and abstract: Nov 1st, 2018 > Full paper submission due: Nov 8th, 2018 > Notification of paper acceptance: Jan. 20th, 2019 > Workshop and tutorial proposals due: Nov 11, 2018 > Notification of proposal acceptance: Nov 18, 2018 So I have a draft of a positioning paper [0]... what missed that event :) tl;dr: It is known that causal consistency is the best suitable for high-latency, high-scale and highly dynamic nature of membership in clusters... which is exactly what multiple autonomous control planes represent in edge clouds for post-MVP phases. Even though the paper didn't make it for ICFC 2019 acceptance as I naively planned (not), hopefully we as community could find some use of it in future. Like collaboratively improving its contents, or making it readable by humans, then pushing it for other places for wider audience. It's just free for any use, please go for it! :) Or maybe we could prototyping something for OpenStack edge post-MVP cases out of SwiftCloud [1] just like the folks did for CockroachDB [2]? Note, while that github repo does not look super alive and driven by many contributions, that work looks really promising, you can read more for the theoretical fundamentals of it in "SwiftCloud: Fault-Tolerant Geo-Replication Integrated all the Way to the Client Machine" and "SwiftCloud: Write fast, read in the past: Causal consistency for client-side applications". That really buys to me to put it all client-sided at the edge and make it doing all geo-replication dances. Ofc there is more for prototyping. For example, building some Bolt-on'ish shim middleware on top of any popular eventual consistent storage, like Riak (not sure there is opensource implementations for Bolt-on yet). Or trying something at api-to-api level for high-level synchronization with "delayed replication" of queued operations. PS. @Pierre and @Adrien I gave it a quick glimpse into some of the sources you linked for the related work section of the positioning paper, thank you a lot for that and x1000 for authoring such things. So the draft now almost hits the recommended limit of a 6 pages, so I think there should be pretty much it for a positioning paper. [0] https://github.com/bogdando/papers-ieee/blob/master/ICFC-2019/LaTeX/positio… [1] https://github.com/SyncFree/SwiftCloud [2] http://beyondtheclouds.github.io/blog/openstack/cockroachdb/2018/06/04/eval… On 27.02.2019 14:33, Adrien Lebre wrote: > Hi, > > Getting a look to this post could also make sense: > http://beyondtheclouds.github.io/blog/openstack/cockroachdb/2018/06/04/eval… > > Best regards, > Adrien > PS: Thanks Pierre. > -- > Prof. IMT Atlantique / Inria / LS2N > Head of the STACK Research Group > www.emn.fr/x-info/alebre08 <http://www.emn.fr/x-info/alebre08> > >> On 27 Feb 2019, at 14:18, Pierre Riteau <pierre(a)stackhpc.com >> <mailto:pierre@stackhpc.com>> wrote: >> >> Hello Bogdan, >> >> In the context of data management in edge computing, you may be >> interested by research done by the Discovery Initiative: >> http://beyondtheclouds.github.io/publications.html >> I am adding Adrien Lèbre to the thread, he may be able to point you to >> the publications most relevant to your whitepaper draft. >> >> Best wishes, >> Pierre Riteau >> >> On Wed, 27 Feb 2019 at 10:21, Bogdan Dobrelya <bdobreli(a)redhat.com> wrote: >>> >>> On 27.02.2019 10:51, Stig Telfer wrote: >>>> Hi All - >>>> >>>> We have an IRC meeting at 1100 UTC (about an hour’s time) in channel >>>> #openstack-meeting. Everyone is welcome. >>>> >>>> Today’s agenda is here: >>>> >>>> https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_February_27th_20… >>>> >>>> Today’s main event is that we have Ildikó Vancsa joining us to talk >>>> about edge computing use cases and find overlap with Scientific SIG >>>> use cases. >>>> >>>> Cheers, >>>> Stig >>>> >>>> >>> >>> Edge computing is a *great* topic for lot of future research work indeed. >>> Thank you for this useful announcement, appreciated! >>> I'll attend at least the first half of it, and in case I'll miss the >>> agenda item that I placed at the end of the list, please be kind to >>> review that research request with my absence as well. Thanks! >>> >>> >>> -- >>> Best regards, >>> Bogdan Dobrelya, >>> Irc #bogdando >>> > -- Best regards, Bogdan Dobrelya, Irc #bogdando

1 1

[nova][qa] Should we make legacy-grenade-dsvm-neutron-multinode-live-migration voting?
by Matt Riedemann 06 Mar '19

06 Mar '19

The legacy-grenade-dsvm-neutron-multinode-live-migration job was recently fixed [1][2] but had been broken for awhile because it's non-voting so no one really noticed. The unique thing about this job is it runs live migration tests in a grenade setting where one of the computes is n-1 and it does the live migration back and forth between the nodes so we have old->new and new->old coverage, and it does this across local disk and shared storage with ceph. Now that it's working again, should we make it voting? It came up while discussing regression test coverage for Artom's NUMA live migration series. It seems like a no-brainer to me at least that we should be gating on this. Before doing that, I'd like to move the job config from openstack-zuul-jobs to nova so we "own" any changes to the job and then might also consider converting it to zuul v3, but that's not a blocker to make it voting. [1] https://review.openstack.org/#/c/634962/ [2] https://review.openstack.org/#/c/637231/ -- Thanks, Matt

5 5

Adding overcloud server entries to /etc/hosts on undercloud
by James LaBarre 06 Mar '19

06 Mar '19

I have been doing this manually at present, but I had wondered if there is a switch/setting for the overcloud deploy to add the overcloud servers to the /etc/hosts file as part of the deploy?

2 2

[tc] Train Technical Committee kickoff
by Doug Hellmann 06 Mar '19

06 Mar '19

Now that the most recent TC election results are in [1], it is time to start organizing the committee for the Train cycle [2]. First, thank you to all of our election officials for helping us to run a smooth and transparent election process, yet again. I have been involved with other communities where elections were fraught and tense operations. You all make it look easy. I would also like to thank our outgoing members, Chris Dent (cdnet), Davanum Srinivas (dims), and Sean McGinnis (smcginnis) for their service as they rotate off of the committee. Each of them has worked on the TC for several years, and in community leadership roles before then. They are excellent examples of open source leaders. And of course, welcome to our new members: Alexandra Settle (asettle), Jim Rollenhagen (jroll), and Rico Lin (ricolin). We're excited to have you join the team! The next step for us is to approve the patch prepared by the election officials to update the TC roster. Please take a few minutes to vote on https://review.openstack.org/#/c/641171/ now. After the TC is seated, we need to select a chair. As I announced at our February meeting, based on some other commitments that I have I think it is best if I not serve as chair this term. If you are interested in being chair, please propose a patch on top of the election patch to add the '{chair}' designation next to your name. As a TC member, we have two primary ways of communicating. Please take a moment to ensure that your email filters are configured to ensure you see any messages on the mailing list with the '[tc]' topic tag. Please also join the '#openstack-tc' channel on IRC, and set a notification for the string 'tc-members'. Thank you, and I look forward to continuing to work with all of you this cycle. -- Doug [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003503.ht… [2] http://git.openstack.org/cgit/openstack/governance/tree/CHAIR.rst#n67

1 2

[tc] Train Technical Committee kickoff
by Doug Hellmann 06 Mar '19

06 Mar '19

Now that the most recent TC election results are in [1], it is time to start organizing the committee for the Train cycle [2]. First, thank you to all of our election officials for helping us to run a smooth and transparent election process, yet again. I have been involved with other communities where elections were fraught and tense operations. You all make it look easy. I would also like to thank our outgoing members, Chris Dent (cdnet), Davanum Srinivas (dims), and Sean McGinnis (smcginnis) for their service as they rotate off of the committee. Each of them has worked on the TC for several years, and in community leadership roles before then. They are excellent examples of open source leaders. And of course, welcome to our new members: Alexandra Settle (asettle), Jim Rollenhagen (jroll), and Rico Lin (ricolin). We're excited to have you join the team! The next step for us is to approve the patch prepared by the election officials to update the TC roster. Please take a few minutes to vote on https://review.openstack.org/#/c/641171/ now. After the TC is seated, we need to select a chair. As I announced at our February meeting, based on some other commitments that I have I think it is best if I not serve as chair this term. If you are interested in being chair, please propose a patch on top of the election patch to add the '{chair}' designation next to your name. As a TC member, we have two primary ways of communicating. Please take a moment to ensure that your email filters are configured to ensure you see any messages on the mailing list with the '[tc]' topic tag. Please also join the '#openstack-tc' channel on IRC, and set a notification for the string 'tc-members'. Thank you, and I look forward to continuing to work with all of you this cycle. -- Doug [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003503.ht… [2] http://git.openstack.org/cgit/openstack/governance/tree/CHAIR.rst#n67

1 0

[nova] failing OVS offload on mellanox connectx-4 (RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7)
by Manuel Sopena Ballesteros 06 Mar '19

06 Mar '19

Dear Openstack community, first let me apologise for my ignorance in this topic, this is the first time I try to setup an environment like this and I don't know anyone who has done it before. I am trying to setup OVS offload following this document https://docs.openstack.org/neutron/rocky/admin/config-ovs-offload.html My Openstack is Rocky deployed through kolla-ansible. I have a feeling my OVS offload is not setup properly as vm creation fails and the error "RepresentorNotFound: Failed getting representor port for PF" can be see in nova-compute logs. # Nova compute logs: 2019-03-05 11:24:46.392 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Attempting claim on node zeus-54.localdomain: memory 8192 MB, disk 80 GB, vcpus 4 CPU 2019-03-05 11:24:46.393 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Total memory: 524173 MB, used: 8704.00 MB 2019-03-05 11:24:46.394 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] memory limit not specified, defaulting to unlimited 2019-03-05 11:24:46.394 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Total disk: 18630 GB, used: 80.00 GB 2019-03-05 11:24:46.395 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] disk limit not specified, defaulting to unlimited 2019-03-05 11:24:46.396 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Total vcpu: 56 VCPU, used: 4.00 VCPU 2019-03-05 11:24:46.396 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] vcpu limit not specified, defaulting to unlimited 2019-03-05 11:24:46.408 7 INFO nova.compute.claims [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Claim successful on node zeus-54.localdomain 2019-03-05 11:24:47.144 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Creating image 2019-03-05 11:24:48.957 7 ERROR os_vif [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Failed to plug vif VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.957 7 ERROR os_vif Traceback (most recent call last): 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 77, in plug 2019-03-05 11:24:48.957 7 ERROR os_vif plugin.plug(vif, instance_info) 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 215, in plug 2019-03-05 11:24:48.957 7 ERROR os_vif self._plug_vf_passthrough(vif, instance_info) 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 189, in _plug_vf_passthrough 2019-03-05 11:24:48.957 7 ERROR os_vif representor = linux_net.get_representor_port(pf_ifname, vf_num) 2019-03-05 11:24:48.957 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py", line 381, in get_representor_port 2019-03-05 11:24:48.957 7 ERROR os_vif raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num) 2019-03-05 11:24:48.957 7 ERROR os_vif RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.957 7 ERROR os_vif 2019-03-05 11:24:48.960 7 ERROR nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Failed to start libvirt guest: InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.969 7 ERROR os_vif [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Failed to unplug vif VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.969 7 ERROR os_vif Traceback (most recent call last): 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 110, in unplug 2019-03-05 11:24:48.969 7 ERROR os_vif plugin.unplug(vif, instance_info) 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 277, in unplug 2019-03-05 11:24:48.969 7 ERROR os_vif self._unplug_vf_passthrough(vif, instance_info) 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 250, in _unplug_vf_passthrough 2019-03-05 11:24:48.969 7 ERROR os_vif representor = linux_net.get_representor_port(pf_ifname, vf_num) 2019-03-05 11:24:48.969 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py", line 381, in get_representor_port 2019-03-05 11:24:48.969 7 ERROR os_vif raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num) 2019-03-05 11:24:48.969 7 ERROR os_vif RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:48.969 7 ERROR os_vif 2019-03-05 11:24:48.971 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Deleting instance files /var/lib/nova/instances/1b03f2e7-0c44-46c2-87f5-d68f93654c58_del 2019-03-05 11:24:48.972 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Deletion of /var/lib/nova/instances/1b03f2e7-0c44-46c2-87f5-d68f93654c58_del complete 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Instance failed to spawn: InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Traceback (most recent call last): 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2368, in _build_resources 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] yield resources 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2132, in _build_and_run_instance 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] block_device_info=block_device_info) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3080, in spawn 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] destroy_disks_on_failure=True) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5646, in _create_domain_and_network 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] destroy_disks_on_failure) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self.force_reraise() 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] six.reraise(self.type_, self.value, self.tb) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5605, in _create_domain_and_network 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self.plug_vifs(instance, network_info) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 819, in plug_vifs 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self.vif_driver.plug(instance, vif) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 807, in plug 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] self._plug_os_vif(instance, vif_obj) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 787, in _plug_os_vif 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] raise exception.InternalError(msg) 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] InternalError: Failure running os_vif plugin plug method: Failed to plug VIF VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True). Got error: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.083 7 ERROR nova.compute.manager [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] 2019-03-05 11:24:49.085 7 INFO nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Terminating instance 2019-03-05 11:24:49.091 7 INFO nova.virt.libvirt.driver [-] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Instance destroyed successfully. 2019-03-05 11:24:49.099 7 ERROR os_vif [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Failed to unplug vif VIFHostDevice(active=False,address=fa:16:3e:b8:41:ff,dev_address=0000:88:02.1,dev_type='ethernet',has_traffic_filtering=True,id=3162e3d2-08ea-4674-8b48-9ce2916faf35,network=Network(f630b54b-2b07-44a3-8196-5a0ea9cc7969),plugin='ovs',port_profile=VIFPortProfileOVSRepresentor,preserve_on_delete=True): RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.099 7 ERROR os_vif Traceback (most recent call last): 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/os_vif/__init__.py", line 110, in unplug 2019-03-05 11:24:49.099 7 ERROR os_vif plugin.unplug(vif, instance_info) 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 277, in unplug 2019-03-05 11:24:49.099 7 ERROR os_vif self._unplug_vf_passthrough(vif, instance_info) 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/ovs.py", line 250, in _unplug_vf_passthrough 2019-03-05 11:24:49.099 7 ERROR os_vif representor = linux_net.get_representor_port(pf_ifname, vf_num) 2019-03-05 11:24:49.099 7 ERROR os_vif File "/usr/lib/python2.7/site-packages/vif_plug_ovs/linux_net.py", line 381, in get_representor_port 2019-03-05 11:24:49.099 7 ERROR os_vif raise exception.RepresentorNotFound(ifname=pf_ifname, vf_num=vf_num) 2019-03-05 11:24:49.099 7 ERROR os_vif RepresentorNotFound: Failed getting representor port for PF ens2f1 with 7 2019-03-05 11:24:49.099 7 ERROR os_vif 2019-03-05 11:24:49.101 7 INFO nova.virt.libvirt.driver [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Deletion of /var/lib/nova/instances/1b03f2e7-0c44-46c2-87f5-d68f93654c58_del complete 2019-03-05 11:24:49.198 7 INFO nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Took 0.11 seconds to destroy the instance on the hypervisor. 2019-03-05 11:24:50.384 7 INFO nova.compute.manager [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] [instance: 1b03f2e7-0c44-46c2-87f5-d68f93654c58] Took 0.77 seconds to deallocate network for instance. 2019-03-05 11:24:50.624 7 INFO nova.scheduler.client.report [req-8744317f-1a06-48a1-8c04-f495935867d8 91e83343e9834c8ba0172ff369c8acac b91520cff5bd45c59a8de07c38641582 - default default] Deleted allocation for instance 1b03f2e7-0c44-46c2-87f5-d68f93654c58 2019-03-05 11:25:07.884 7 INFO nova.compute.resource_tracker [req-1b32a794-0bf0-4abe-a024-699920f791d2 - - - - -] Final resource view: name=zeus-54.localdomain phys_ram=524173MB used_ram=8704MB phys_disk=18630GB used_disk=80GB total_vcpus=56 used_vcpus=4 pci_stats=[PciDevicePool(count=15,numa_node=1,product_id='1016',tags={dev_type='type-VF',physical_network='physnet1'},vendor_id='15b3')] ### This is my environment/configuration # OVS version: docker exec -it openvswitch_vswitchd ovs-vswitchd --version ovs-vswitchd (Open vSwitch) 2.10.1 # Mellanox firmware version: [root@zeus-54 ~]# ibv_devinfo hca_id: mlx5_9 transport: InfiniBand (0) fw_ver: 14.24.1000 # Kernel version: [root@zeus-54 ~]# uname -r 3.10.0-862.el7.x86_64 # OS distro: [root@zeus-54 ~]# cat /etc/*release CentOS Linux release 7.6.1810 (Core) NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" CentOS Linux release 7.6.1810 (Core) CentOS Linux release 7.6.1810 (Core) # SR-IOV devices: [root@zeus-54 ~]# lspci -D -nn | grep -i mel 0000:88:00.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] [15b3:1015] 0000:88:00.1 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] [15b3:1015] 0000:88:00.2 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.3 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.4 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.5 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.6 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:00.7 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.1 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.2 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.3 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.4 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.5 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.6 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:01.7 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:02.0 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] 0000:88:02.1 Ethernet controller [0200]: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] [15b3:1016] [root@zeus-54 ~]# ibdev2netdev -v 0000:88:00.0 mlx5_0 (MT4117 - MT1611X09936) CX4121A - ConnectX-4 LX SFP28 fw 14.24.1000 port 1 (ACTIVE) ==> ens2f0 (Up) 0000:88:00.1 mlx5_1 (MT4117 - MT1611X09936) CX4121A - ConnectX-4 LX SFP28 fw 14.24.1000 port 1 (ACTIVE) ==> ens2f1 (Up) 0000:88:01.2 mlx5_10 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f2 (Up) 0000:88:01.3 mlx5_11 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f3 (Up) 0000:88:01.4 mlx5_12 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f4 (Up) 0000:88:01.5 mlx5_13 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f5 (Up) 0000:88:01.6 mlx5_14 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f6 (Up) 0000:88:01.7 mlx5_15 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f7 (Up) 0000:88:02.0 mlx5_16 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s2 (Up) 0000:88:02.1 mlx5_17 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s2f1 (Up) 0000:88:00.2 mlx5_2 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f2 (Up) 0000:88:00.3 mlx5_3 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f3 (Up) 0000:88:00.4 mlx5_4 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f4 (Up) 0000:88:00.5 mlx5_5 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f5 (Up) 0000:88:00.6 mlx5_6 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f6 (Up) 0000:88:00.7 mlx5_7 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> ens2f7 (Up) 0000:88:01.0 mlx5_8 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1 (Up) 0000:88:01.1 mlx5_9 (MT4118 - NA) fw 14.24.1000 port 1 (ACTIVE) ==> enp136s1f1 (Up) # Embeded switch configuration: [root@zeus-54 ~]# devlink dev eswitch show pci/0000:88:00.0 pci/0000:88:00.0: mode legacy inline-mode none encap enable [root@zeus-54 ~]# devlink dev eswitch show pci/0000:88:00.1 pci/0000:88:00.1: mode legacy inline-mode none encap enable [root@zeus-54 ~]# sudo ethtool -k ens2f0 | grep hw-tc-offload hw-tc-offload: on [root@zeus-54 ~]# sudo ethtool -k ens2f1 | grep hw-tc-offload hw-tc-offload: on [root@zeus-54 ~]# docker exec -it openvswitch_vswitchd sudo ovs-vsctl get Open_vSwitch . other_config {hw-offload="true", max-idle="30000"} [root@zeus-54 ~]# docker exec -it openvswitch_vswitchd sudo ovs-vsctl get Open_vSwitch . other_config {hw-offload="true", max-idle="30000"} # Check ovs offload [root@zeus-54 ~]# docker exec -it openvswitch_vswitchd sudo ovs-dpctl show 2019-03-05T00:50:42Z|00001|dpif_netlink|INFO|The kernel module does not support meters. system@ovs-system: lookups: hit:0 missed:0 lost:0 flows: 0 masks: hit:0 total:0 hit/pkt:0.00 port 0: ovs-system (internal) port 1: br-ex (internal) port 2: br-int (internal) port 3: br-tun (internal) QUESTIONS: Is my kernel version right? Am I supposed to see the PFs attached to a port in the ovs-dpctl show output? What else can I check? Thank you very much NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

4 5

[ptl][placement][election] Chris Dent's Candidacy for Placement PTL
by Chris Dent 06 Mar '19

06 Mar '19

Hi, I'm announcing my candidacy [1] for Placement PTL for Train. I've been working on the Placement service from the start. Helping it become a straightforward HTTP API for managing resources and their inventories, qualities, and use. I believe the primary focus of the project should be to fulfill the promise of the service — to make any variety of workload placement more reasonable-to-manage and correct-to-use — while keeping the straightforwardness. Just as important is making sure the project, being newly independent, settles in well. We will need time, attention and patience to find our way. What we do as a group should be decided by the group. The role of the PTL should be to help the group reach decisions smoothly and make sure the information needed to make decisions is shared broadly. To help understand how I think about Placement, here are some ideas on areas we may wish to focus (if we agree): * Get the house in order. Get bug and feature tracking systems in place and properly documented. Prepare the Forum and PTG. * Help other projects use Placement. Adding features to Placement is often easier than taking advantage of those new features. A lot of features exist that are not yet used to their full potential. * Attend to maintenance. Bug fixing, documentation, performance, refactoring, gate management, and keeping clients up to date need to have equal or greater attention than feature development. * Do what matters. Only add features that have clear customers and use cases, and have committed owners responsible for long-term maintenance. * Experiment. Contrary to the above, some of our time should be spent discovering what may matter in the future and sharing those discoveries: "Placement can help with that." Edge and multi-cloud related use cases have plenty of potential. * Document. Fill in the gaps in documentation for users and contributors. There's a lot of power in Placement and using it most effectively is not as well-documented as it could be. * Be welcoming, fun, nice, and encouraging to everyone. I hope my history with the project is solid evidence for my commitment to its success. I'm extremely proud of the work we've done and optimistic about the positive impact we will have in the future. Thanks. [1] https://review.openstack.org/641339 -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

1 0

[kuryr] Daniel Mellado's Candidacy for Kuryr PTL
by Daniel Mellado 06 Mar '19

06 Mar '19

Hi Kuryrs, I'm writing this in order to announce my candidacy for Kuryr PTL during the Train cycle, althought I expect no Train horns this time at Denver! It has been my pleaseure and nonor to act as PTL for Kuryr the past two cycles and I'd like like to run for this for another release. I'd like to thank you all on helping me deliver and run this project during all this time I think we have been doing an awesome job so far within the OpenStack community. During Stein, we had an great cycle, delivering most of the goals we set for it, and movign forward, these are some of the goals that I propose for the team during the Train cycle: * Network Policies: Although this is a feature we landed in last cycle, I'd like for it to be enhanced so it gets faster, so optimizing its performance would be one of the cycle goals. * Neutron pooling resource speedupts: Tied to the last topic, it'll still need to further improve the resource handling speed. * Release model: So far we've been catching up with two release models, OpenStack and Kubernetes, as this is quite an effort and could lead to an endless testing matrix, one of the goals that I'd like to settle in the cycle would be on how to deal with such circumstances. * SRIOV/DPDK upstream gate testing. * Further improve debugging and instrospection tools usign Kubernetes plugins. * Expand the Kuryr community by adding more contributors. Also, I'd like to coordinate integrating some features that we started last cycle, such as Native OVN Layer Load balancing and Multi device/net support. Outside of this key areas, my priority is also helping the community by acting as an interface for the cross-project sessions and further improve our presence in initiatives such as Openlab, OPNFV and Edge. Besides that, I've also sent an email with etherpad links for PTG planning[1] and forum sessions[2] at the Summit. [1] https://etherpad.openstack.org/p/kuryr-denver-train-ptg-planning [2] https://etherpad.openstack.org/p/DEN-train-forum-kuryr-brainstorming Thanks a lot! Daniel Mellado (dmellado)

1 0

[horizon] Horizon PTG planning
by Ivan Kolodyazhny 06 Mar '19

06 Mar '19

Hi team, I created etherpad to collect topics for PTG. Feel free to add any topic you would like to discuss face to face. [1] https://etherpad.openstack.org/p/horizon-train-ptg Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

1 0