January 2019 - openstack-discuss - lists.openstack.org

[neutron][qos][dpdk] is QoS supported with DPDK
by cristian.calin＠orange.com 16 Jan '19

16 Jan '19

Hi all, Is QoS supported when DPDK is leveraged on the openvswitch? >From what I can tell the two are incompatible but I'm hoping someone has some information otherwise. Thanks, Cristian Calin _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.

2 1

[self-healing-sig] best practices for haproxy health checking
by Dirk Müller 15 Jan '19

15 Jan '19

Hi, Does anyone have a good pointer for good healthchecks to be used by the frontend api haproxy loadbalancer? in one case that I am looking at right now, the entry haproxy loadbalancer was not able to detect a particular backend being not responding to api requests, so it flipped up and down repeatedly, causing intermittend spurious 503 errors. The backend was able to respond to connections and to basic HTTP GET requests (e.g. / or even /v3 as path), but when it got a "real" query it hung. the reason for that was, as it turned out, the configured caching backend memcached on that machine being locked up (due to some other bug). I wonder if there is a better way to check if a backend is "working" and what the best practices around this are. A potential thought I had was to do the backend check via some other healthcheck specific port that runs a custom daemon that does more sophisticated checks like checking for system wide errors (like memcache, database, rabbitmq) being unavailable on that node, and hence not accepting any api traffic until that is being resolved. Any pointers to read upon / best practices appreciated. Thanks, Dirk

3 2

Training
by Alfredo De Luca 15 Jan '19

15 Jan '19

Hi all. I am looking for openstack training both basic and advanced here in Italy or Europe. Possibly instructor led on site. Any suggestions? -- *Alfredo*

2 2

[horizon][keystone][dev] Cross-domain administrators and context-switching
by Robert Donovan 15 Jan '19

15 Jan '19

Hello, We run a cloud service with multiple domains (one per tenant) and offer services on top which can, amongst other things, involve administrators creating instances, snapshots etc. on behalf to those tenants. My understanding is that, in order to achieve this with Horizon, we currently have to create a separate admin user in each domain with a role that allows those abilities. The administrator then needs to log into that domain using the new user to perform the required actions. Firstly, is that assumption correct? Or is it possible use the same user credentials across domain boundaries? Secondly, have there ever been discussions around the “Set Domain Context” function having a wider effect to scope the whole dashboard to that particular domain, including the project panels? Are there potential issues with this as a proposal? Many thanks, Rob

2 1

[ops] last weeks meetups team minutes
by Chris Morgan 15 Jan '19

15 Jan '19

openstack ops meetups team meeting minutes from last week: Meeting ended Tue Jan 8 15:53:43 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 10:53 AM Minutes: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… 10:53 AM Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… 10:53 AM Log: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… This week's meeting will be in approximately 40 minutes as I write. Our main priority now is to get the planning for the Berlin meetup underway Chris -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[nova][dev] vCPU Pinning for L1/L2 cache side-channel vulnerability mitigation
by Robert Donovan 15 Jan '19

15 Jan '19

Hello Nova folks, I spoke to some of you very briefly about this in Berlin (thanks again for your time), and we were resigned to turning off SMT to fully protect against future CPU cache side-channel attacks as I know many others have done. However, we have stubbornly done a bit of last-resort research and testing into using vCPU pinning on a per-tenant basis as an alternative and I’d like to lay it out in more detail for you to make sure there are no legs in the idea before abandoning it completely. The idea is to use libvirt’s vcpupin ability to ensure that two different tenants never share the same physical CPU core, so they cannot theoretically steal each other’s data via an L1 or L2 cache side-channel. The pinning would be optimised to make use of as many logical cores as possible for any given tenant. We would also isolate other key system processes to a separate range of physical cores. After discussions in Berlin, we ran some tests with live migration, as this is key to our maintenance activities and would be a show-stopped if it didn’t work. We found that removing any pinning restrictions immediately prior to migration resulted in them being completely reset on the target host, which could then be optimised accordingly post-migration. Unfortunately, there would be a small window of time where we couldn’t prevent tenants from sharing a physical core on the target host after a migration, but we think this is an acceptable risk given the nature of these attacks. Obviously, this approach may not be appropriate in many circumstances, such as if you have many tenants who just run single VMs with one vCPU, or if over-allocation is in use. We have also only looked at KVM and libvirt. I would love to know what people think of this approach however. Are there any other clear issues that you can think of which we may not have considered? If it seems like a reasonable idea, is it something that could fit into Nova and, if so, where in the architecture is the best place for it to sit? I know you can currently specify per-instance CPU pinning via flavor parameters, so a similar approach could be taken for this strategy. Alternatively, we can look at implementing it as an external plugin of some kind for use by those with a similar setup. Many thanks, Rob

4 4

RE: [Edge-computing] Use cases mapping to MVP architectures - FEEDBACK NEEDED
by Csatari, Gergely (Nokia - HU/Budapest) 15 Jan '19

15 Jan '19

Hi, Thanks for the comment. Do you see any other use of the capability to create images on an edge site except creating snapshots? Thanks, Gerg0 From: Ildiko Vancsa <ildiko(a)openstack.org> Sent: Tuesday, January 15, 2019 2:59 AM To: Csatari, Gergely (Nokia - HU/Budapest) <gergely.csatari(a)nokia.com> Subject: Fwd: [Edge-computing] Use cases mapping to MVP architectures - FEEDBACK NEEDED Begin forwarded message: From: Bogdan Dobrelya <bdobreli(a)redhat.com<mailto:bdobreli@redhat.com>> Subject: Re: [Edge-computing] Use cases mapping to MVP architectures - FEEDBACK NEEDED Date: 2019. January 10. 8:34:45 GMT-7 To: edge-computing(a)lists.openstack.org<mailto:edge-computing@lists.openstack.org>, "openstack-discuss(a)lists.openstack.org<mailto:openstack-discuss@lists.openstack.org>" <openstack-discuss(a)lists.openstack.org<mailto:openstack-discuss@lists.openstack.org>> On 10.01.2019 14:43, Ildiko Vancsa wrote: Hi, We are reaching out to you about the use cases for edge cloud infrastructure that the Edge Computing Group is working on to collect. They are recorded in our wiki [1] and they describe high level scenarios when an edge cloud infrastructure would be needed. Hello. Verifying the mappings created for the "Elementary operations on a site" [18] feature against the distributed glance specification [19], I can see a vital feature is missing for "Advanced operations on a site", like creating an image locally, when the parent control plane is not available. And consequences coming off that, like availability of create snapshots for Nova as well. All that boils down to a) better identifying the underlying requirement/limitations for CRUD operations available for middle edge sites in the Distributed Control Plane case. And b) the requirement of data replication and conflicts resolving tooling, which comes out, if we assume we want all CRUDs being always available for middle edge sites disregard of the parent edge's control plane state. So that is the missing and important thing to have socialised and noted for the mappings. [18] https://wiki.openstack.org/wiki/MappingOfUseCasesFeaturesRequirementsAndUse… [19] https://review.openstack.org/619638 During the second Denver PTG discussions we drafted two MVP architectures what we could build from the current functionality of OpenStack with some slight modifications [2]. These are based on the work of James and his team from Oath. We differentiate between a distributed [3] and a centralized [4] control plane architecture scenarios. In one of the Berlin Forum sessions we were asked to map the MVP architecture scenarios to the use cases so I made an initial mapping and now we are looking for feedback. This mapping only means, that the listed use case can be implemented using the MVP architecture scenarios. It should be noted, that none of the MVP architecture scenarios provide solution for edge cloud infrastructure upgrade or centralized management. Please comment on the wiki or in a reply to this mail in case you have questions or disagree with the initial mapping we put together. Please let us know if you have any questions. Here is the use cases and the mapped architecture scenarios: Mobile service provider 5G/4G virtual RAN deployment and Edge Cloud B2B2X [5] Both distributed [3] and centralized [4] Universal customer premise equipment (uCPE) for Enterprise Network Services[6] Both distributed [3] and centralized [4] Unmanned Aircraft Systems (Drones) [7] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Cloud Storage Gateway - Storage at the Edge [8] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Open Caching - stream/store data at the edge [9] Both distributed [3] and centralized [4] Smart City as Software-Defined closed-loop system [10] The use case is not complete enough to figure out Augmented Reality -- Sony Gaming Network [11] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Analytics/control at the edge [12] The use case is not complete enough to figure out Manage retail chains - chick-fil-a [13] The use case is not complete enough to figure out At this moment chick-fil-a uses a different Kubernetes cluster in every edge location and they manage them using Git [14] Smart Home [15] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Data Collection - Smart cooler/cold chain tracking [16] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event VPN Gateway Service Delivery [17] The use case is not complete enough to figure out [1]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases [2]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… [3]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… [4]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… [5]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Mobile_servi…. [6]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Universal_cu… [7]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Unmanned_Air… [8]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Cloud_Storag… [9]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Open_Caching… [10]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Smart_City_a… [11]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Augmented_Re… [12]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Analytics.2F… [13]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Manage_retai… [14]: https://schd.ws/hosted_files/kccna18/34/GitOps.pdf [15]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Smart_Home [16]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Data_Collect… [17]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#VPN_Gateway_… Thanks and Best Regards, Gergely and Ildikó _______________________________________________ Edge-computing mailing list Edge-computing(a)lists.openstack.org<mailto:Edge-computing@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/edge-computing -- Best regards, Bogdan Dobrelya, Irc #bogdando _______________________________________________ Edge-computing mailing list Edge-computing(a)lists.openstack.org<mailto:Edge-computing@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/edge-computing

1 0

[infra] NetworkManager on infra Fedora 29 and CentOS nodes
by Ian Wienand 15 Jan '19

15 Jan '19

Hello, Just a heads-up; with Fedora 29 the legacy networking setup was moved into a separate, not-installed-by-default network-scripts package. This has prompted us to finally move to managing interfaces on our Fedora and CentOS CI hosts with NetworkManager (see [1]) Support for this is enabled with features added in glean 1.13.0 and diskimage-builder 1.19.0. The newly created Fedora 29 nodes [2] will have it enabled, and [3] will switch CentOS nodes shortly. This is tested by our nodepool jobs which build images, upload them into devstack and boot them, and then check the networking [4]. I don't really expect any problems, but be aware NetworkManager packages will appear on the CentOS 7 and Fedora base images with these changes. Thanks -i [1] https://bugzilla.redhat.com/show_bug.cgi?id=1643763#c2 [2] https://review.openstack.org/618672 [3] https://review.openstack.org/619960 [4] https://review.openstack.org/618671

2 5

Re: [congress][infra] override-checkout problem
by Eric K 15 Jan '19

15 Jan '19

Hi Ghanshyam, On 1/11/19, 4:57 AM, "Ghanshyam Mann" <gmann(a)ghanshyammann.com> wrote: >Hi Eric, > >This seems the same issue happening on congress-tempest-plugin gate where >'congress-devstack-py35-api-mysql-queens' is failing [1]. >python-congressclient was >not able to install and openstack client trow error for congress command. > >The issue is stable branch jobs on congress-tempest-plugin does checkout >the master version for all repo >instead of what mentioned in override-checkout var. > >If you see congress's rocky patch, congress is checkout out with rocky >version[2] but >congress-tempest-plugin patch's rocky job checkout the master version of >congress instead of rocky version [3]. >That is why your test expectedly fail on congress patch but pass on >congress-tempest-plugin. > >Root cause is that override-checkout var does not work on the legacy job >(it is only zuulv3 job var, if I am not wrong), >you need to use BRANCH_OVERRIDE for legacy jobs. Myself, amotoki and >akhil was trying lot other workarounds >to debug the root cause but at the end we just notice that congress jobs >are legacy jobs and using override-checkout :). Gosh thanks so much for the investigation. Yes it's a legacy-dsvm job. So sorry for the run around! I'm thinking of taking the opportunity to migrate to devstack-tempest job. I've taken a first stab here: https://review.openstack.org/#/c/630414/ > >I have submitted the testing patch with BRANCH_OVERRIDE for >congress-tempest-plugin queens job[4]. >Which seems working fine, I can make those patches more formal for merge. And thanks so much for putting together those patches using BRANCH_OVERRIDE! Merging sounds good unless we can quickly migrate To non-legacy jobs. Realistically it'll probably end up take a while to get everything migrated and working. > > >Another thing I was discussing with Akhil that new tests of builins >feature need another feature flag >(different than congressz3.enabled) as that feature of z3 is in stein >onwards only. Yup. I was going to do that but wanted to first figure out why it wasn't failing on tempest plugin. I've now submitted a patch to do that. > > >[1] https://review.openstack.org/#/c/618951/ >[2] >http://logs.openstack.org/70/629070/4/check/congress-devstack-api-mysql/87 >474d7/logs/pip2-freeze.txt.gz >[3] >http://logs.openstack.org/51/618951/3/check/congress-devstack-api-mysql-ro >cky/23c0214/logs/pip2-freeze.txt.gz >[4] >https://review.openstack.org/#/q/topic:fix-stable-branch-testing+(status:o >pen+OR+status:merged) > >-gmann > > ---- On Fri, 11 Jan 2019 10:40:39 +0900 Eric K ><ekcs.openstack(a)gmail.com> wrote ---- > > The congress-tempest-plugin zuul jobs against stable branches appear > > to be working incorrectly. Tests that should fail on stable/rocky (and > > indeed fails when triggered by congress patch [1]) are passing when > > triggered by congress-tempest-plugin patch [2]. > > > > I'd assume it's some kind of zuul misconfiguration in > > congress-tempest-plugin [3], but I've so far failed to figure out > > what's wrong. Particularly strange is that the job-output appears to > > show it checking out the right thing [4]. > > > > Any thoughts or suggestions? Thanks so much! > > > > [1] > > https://review.openstack.org/#/c/629070/ > > >http://logs.openstack.org/70/629070/4/check/congress-devstack-api-mysql/87 >474d7/logs/testr_results.html.gz > > The two failing z3 tests should indeed fail because the feature was > > not available in rocky. The tests were introduced because for some > > reason they pass in the job triggered by a patch in > > congress-tempest-plugin. > > > > [2] > > https://review.openstack.org/#/c/618951/ > > >http://logs.openstack.org/51/618951/3/check/congress-devstack-api-mysql-ro >cky/23c0214/logs/testr_results.html.gz > > > > [3] >https://github.com/openstack/congress-tempest-plugin/blob/master/.zuul.yam >l#L4 > > > > [4] >http://logs.openstack.org/51/618951/3/check/congress-devstack-api-mysql-ro >cky/23c0214/job-output.txt.gz#_2019-01-09_05_18_08_183562 > > shows congress is checked out to the correct commit at the top of the > > stable/rocky branch. > > > > > > >

2 1

[heat][octavia][barbican] TLS-terminated HTTPS load balancer
by Ignazio Cassano 15 Jan '19

15 Jan '19

Hello everyone, I found a lot of exampe for creating TLS-terminated HTTPS load balancer by command line using octavia and barbican. I did not find any example to do it with heat. Anyone could post an example, please ? Ignazio

1 0