January 2019 - openstack-discuss - lists.openstack.org

[release] Release countdown for week R-12, Jan 14-18
by Sean McGinnis 10 Jan '19

10 Jan '19

Development Focus ----------------- Focus should be on wrapping up any design specs, then moving on to implementation as we head into the last stretch of Stein. General Information ------------------- Stein-2 is the membership freeze for deliverables to be included in the Stein coordinated release. We've reached out to a few folks, but if your project has any new deliverables that have not been released yet, please let us know ASAP if you hope to have them included in Stein. Following the changes we had proposed at the beginning of the release cycle, the release team will be proposing releases for any libraries that have significant changes merged that have not been released. PTL's and release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well. A +1 would be appreciated, but if we do not hear anything at all by the end of the week, we will assume things are OK to proceed. Upcoming Deadlines & Dates -------------------------- Individual OpenStack Foundation Board election: Jan 14-18 Non-client library freeze: February 28 -- Sean McGinnis (smcginnis)

1 0

[oslo][migrator] RFE Configuration mapping tool for upgrade - coordinate teams
by Herve Beraud 10 Jan '19

10 Jan '19

Hey teams, ## Summary The main goal of this thread is to coordinate teams to adopt the oslo realease migrator tools and raise prior on this topics. ## History During the openstack submit in 2017 at Sydney a team from the Fujitsu company propose a talk about how to make fast forward upgrade from an openstack release to an another, and present the state of art of the configuration migrator inside openstack and the tools availables to do that. This team describe that for now, we have oslo.config generator tool to show deprecated options in newer release but they are still lacking some mapping configurations. After that the same team propose oslo.config specifications to improve migrators. Where specifications was designed by the following reviews: - https://review.openstack.org/#/c/520043/ - https://review.openstack.org/#/c/526314/ - https://review.openstack.org/#/c/526261/ The main goal of the proposed changes in these specs is to handle configuration changes over releases: - https://specs.openstack.org/openstack/oslo-specs/specs/rocky/handle-config-… These specifications describe that there should be some helper in oslo.config for automatically adopt new changes and help users manage their configuration files easily. Scenario: Below is the proposed workflow that users can perform on old system to generate new configuration for preparing upgrading to new release: +--------------+ Old configuration +--------> | (N-1 release) | Oslo | | Config +-------> New configuration Namespaces +--------> Migrator | (N release) | | +--------------+ Running on new environment Since these specifications was validated, some patches was submitted by the Fujitsu team to implement them: https://review.openstack.org/#/q/status:open+project:openstack/oslo.config+… Details: - handle config mapping changes ( https://review.openstack.org/526314 ) - update valid value in choice list for the opt ( https://review.openstack.org/603060 ) - update 'sample_default' instead of 'default' when migrating ( https://review.openstack.org/606211 ) - use difflib to report mismatches in migrator tests ( https://review.openstack.org/606210 ) - replace loop with dictionary lookup ( https://review.openstack.org/606209 ) - replace 'upgrade_value' with 'convert_on_upgrade' ( https://review.openstack.org/606207 ) Since these changes was proposed some peoples from the Fujitsu seems to left the company and also seems to abandon this topic. During the last oslo meeting I had added this topic to the meeting agenda to bring up this topic and to help us to move forward on it. http://eavesdrop.openstack.org/meetings/oslo/2018/oslo.2018-12-17-15.00.log… http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000856… ## Current state Some reviews was validated but some others need to be recovered due to the lack of response about the original authors. I've submit some patches on reviews to help us to move on it. I've made some reviews based on the last comments and based on the asked changes but now we need someone with an deeply knowledge on oslo.config to validate the works. ## Others aspects Some tools like puppet/ansible/chef etc... already in use in openstack can already handle migrators/upgrades and manage config changes. - Do we need to introduce an another tool like this one to do this job? - How many people are manually writing OpenStack configs in the first place? - Does it make sense to implement these changes? ## List of similar features on others projects - Tripleo provide a major release upgrade mechanisme ( https://docs.openstack.org/tripleo-docs/latest/upgrade/developer/upgrades/m… ) - Openstack-Ansible also provide release upgrade mechanisme ( https://docs.openstack.org/openstack-ansible/latest/admin/upgrades/major-up… ) - OpenStack fuel also propose specifications to upgrade configuration and release ( https://specs.openstack.org/openstack/fuel-specs/specs/7.0/upgrade-major-op… ) This wiki can be useful to obtain the big picture of the release upgrade management in openstack ( https://wiki.openstack.org/wiki/Upgrade-with-minimal-downtime ) I surely forgot some tools and alternatives, if you see something that can be added here do not hesitate to reply to add it. ## Conclusion I bring up this topic to open a debate so do not hesitate react on this topic by responding at this thread to help us to have a better big picture to make the best choice. Thanks folks. Best regards. -- Hervé Beraud Senior Software Engineer Red Hat - Openstack Oslo irc: hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

4 9

[kolla] Stepping down from core
by Paul Bourke 10 Jan '19

10 Jan '19

Hi all, Due to a change of direction for me I'll be stepping down from the Kolla core group. It's been a blast, thanks to everyone I've worked/interacted with over the past few years. Thanks in particular to Eduardo who's done a stellar job of PTL since taking the reins. I hope we'll cross paths again in the future :) All the best! -Paul

4 3

[nova] Mempage fun
by Stephen Finucane 10 Jan '19

10 Jan '19

We've been looking at a patch that landed some months ago and have spotted some issues: https://review.openstack.org/#/c/532168 In summary, that patch is intended to make the memory check for instances memory pagesize aware. The logic it introduces looks something like this: If the instance requests a specific pagesize (#1) Check if each host cell can provide enough memory of the pagesize requested for each instance cell Otherwise If the host has hugepages (#2) Check if each host cell can provide enough memory of the smallest pagesize available on the host for each instance cell Otherwise (#3) Check if each host cell can provide enough memory for each instance cell, ignoring pagesizes This also has the side-effect of allowing instances with hugepages and instances with a NUMA topology but no hugepages to co-exist on the same host, because the latter will now be aware of hugepages and won't consume them. However, there are a couple of issues with this: 1. It breaks overcommit for instances without pagesize request running on hosts with different pagesizes. This is because we don't allow overcommit for hugepages, but case (#2) above means we are now reusing the same functions previously used for actual hugepage checks to check for regular 4k pages 2. It doesn't fix the issue when non-NUMA instances exist on the same host as NUMA instances with hugepages. The non-NUMA instances don't run through any of the code above, meaning they're still not pagesize aware We could probably fix issue (1) by modifying those hugepage functions we're using to allow overcommit via a flag that we pass for case (#2). We can mitigate issue (2) by advising operators to split hosts into aggregates for 'hw:mem_page_size' set or unset (in addition to 'hw:cpu_policy' set to dedicated or shared/unset). I need to check but I think this may be the case in some docs (sean-k-mooney said Intel used to do this. I don't know about Red Hat's docs or upstream). In addition, we did actually called that out in the original spec: https://specs.openstack.org/openstack/nova-specs/specs/juno/approved/virt-d… However, if we're doing that for non-NUMA instances, one would have to question why the patch is necessary/acceptable for NUMA instances. For what it's worth, a longer fix would be to start tracking hugepages in a non-NUMA aware way too but that's a lot more work and doesn't fix the issue now. As such, my question is this: should be look at fixing issue (1) and documenting issue (2), or should we revert the thing wholesale until we work on a solution that could e.g. let us track hugepages via placement and resolve issue (2) too. Thoughts? Stephen

5 12

Re: [Edge-computing] Use cases mapping to MVP architectures - FEEDBACK NEEDED
by Bogdan Dobrelya 10 Jan '19

10 Jan '19

On 10.01.2019 14:43, Ildiko Vancsa wrote: > Hi, > > We are reaching out to you about the use cases for edge cloud infrastructure that the Edge Computing Group is working on to collect. They are recorded in our wiki [1] and they describe high level scenarios when an edge cloud infrastructure would be needed. Hello. Verifying the mappings created for the "Elementary operations on a site" [18] feature against the distributed glance specification [19], I can see a vital feature is missing for "Advanced operations on a site", like creating an image locally, when the parent control plane is not available. And consequences coming off that, like availability of create snapshots for Nova as well. All that boils down to a) better identifying the underlying requirement/limitations for CRUD operations available for middle edge sites in the Distributed Control Plane case. And b) the requirement of data replication and conflicts resolving tooling, which comes out, if we assume we want all CRUDs being always available for middle edge sites disregard of the parent edge's control plane state. So that is the missing and important thing to have socialised and noted for the mappings. [18] https://wiki.openstack.org/wiki/MappingOfUseCasesFeaturesRequirementsAndUse… [19] https://review.openstack.org/619638 > > During the second Denver PTG discussions we drafted two MVP architectures what we could build from the current functionality of OpenStack with some slight modifications [2]. These are based on the work of James and his team from Oath. We differentiate between a distributed [3] and a centralized [4] control plane architecture scenarios. > > In one of the Berlin Forum sessions we were asked to map the MVP architecture scenarios to the use cases so I made an initial mapping and now we are looking for feedback. > > This mapping only means, that the listed use case can be implemented using the MVP architecture scenarios. It should be noted, that none of the MVP architecture scenarios provide solution for edge cloud infrastructure upgrade or centralized management. > > Please comment on the wiki or in a reply to this mail in case you have questions or disagree with the initial mapping we put together. > > Please let us know if you have any questions. > > > Here is the use cases and the mapped architecture scenarios: > > Mobile service provider 5G/4G virtual RAN deployment and Edge Cloud B2B2X [5] > Both distributed [3] and centralized [4] > Universal customer premise equipment (uCPE) for Enterprise Network Services[6] > Both distributed [3] and centralized [4] > Unmanned Aircraft Systems (Drones) [7] > None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event > Cloud Storage Gateway - Storage at the Edge [8] > None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event > Open Caching - stream/store data at the edge [9] > Both distributed [3] and centralized [4] > Smart City as Software-Defined closed-loop system [10] > The use case is not complete enough to figure out > Augmented Reality -- Sony Gaming Network [11] > None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event > Analytics/control at the edge [12] > The use case is not complete enough to figure out > Manage retail chains - chick-fil-a [13] > The use case is not complete enough to figure out > At this moment chick-fil-a uses a different Kubernetes cluster in every edge location and they manage them using Git [14] > Smart Home [15] > None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event > Data Collection - Smart cooler/cold chain tracking [16] > None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event > VPN Gateway Service Delivery [17] > The use case is not complete enough to figure out > > [1]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases > [2]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… > [3]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… > [4]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… > [5]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Mobile_servi…. > [6]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Universal_cu… > [7]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Unmanned_Air… > [8]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Cloud_Storag… > [9]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Open_Caching… > [10]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Smart_City_a… > [11]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Augmented_Re… > [12]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Analytics.2F… > [13]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Manage_retai… > [14]: https://schd.ws/hosted_files/kccna18/34/GitOps.pdf > [15]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Smart_Home > [16]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Data_Collect… > [17]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#VPN_Gateway_… > > > Thanks and Best Regards, > Gergely and Ildikó > > > > _______________________________________________ > Edge-computing mailing list > Edge-computing(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/edge-computing > -- Best regards, Bogdan Dobrelya, Irc #bogdando

1 0

[Searchlight] We reached Stein-2 milestone
by Trinh Nguyen 10 Jan '19

10 Jan '19

Hi team, Just so you know, we reached Stein-2 milestone and were able to release Searchlight yesterday :) Yay! I put a document here [1] to summarize what we covered in this release. Hope that it will get you excited and understand our vision. [1] https://www.dangtrinh.com/2019/01/searchlight-at-stein-2-r-14-r-13.html Bests, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

1 0

[edge] Use cases mapping to MVP architectures - FEEDBACK NEEDED
by Ildiko Vancsa 10 Jan '19

10 Jan '19

Hi, We are reaching out to you about the use cases for edge cloud infrastructure that the Edge Computing Group is working on to collect. They are recorded in our wiki [1] and they describe high level scenarios when an edge cloud infrastructure would be needed. During the second Denver PTG discussions we drafted two MVP architectures what we could build from the current functionality of OpenStack with some slight modifications [2]. These are based on the work of James and his team from Oath. We differentiate between a distributed [3] and a centralized [4] control plane architecture scenarios. In one of the Berlin Forum sessions we were asked to map the MVP architecture scenarios to the use cases so I made an initial mapping and now we are looking for feedback. This mapping only means, that the listed use case can be implemented using the MVP architecture scenarios. It should be noted, that none of the MVP architecture scenarios provide solution for edge cloud infrastructure upgrade or centralized management. Please comment on the wiki or in a reply to this mail in case you have questions or disagree with the initial mapping we put together. Please let us know if you have any questions. Here is the use cases and the mapped architecture scenarios: Mobile service provider 5G/4G virtual RAN deployment and Edge Cloud B2B2X [5] Both distributed [3] and centralized [4] Universal customer premise equipment (uCPE) for Enterprise Network Services[6] Both distributed [3] and centralized [4] Unmanned Aircraft Systems (Drones) [7] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Cloud Storage Gateway - Storage at the Edge [8] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Open Caching - stream/store data at the edge [9] Both distributed [3] and centralized [4] Smart City as Software-Defined closed-loop system [10] The use case is not complete enough to figure out Augmented Reality -- Sony Gaming Network [11] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Analytics/control at the edge [12] The use case is not complete enough to figure out Manage retail chains - chick-fil-a [13] The use case is not complete enough to figure out At this moment chick-fil-a uses a different Kubernetes cluster in every edge location and they manage them using Git [14] Smart Home [15] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event Data Collection - Smart cooler/cold chain tracking [16] None - assuming that this Use Case requires a Small Edge instance which can work in case of a network partitioning event VPN Gateway Service Delivery [17] The use case is not complete enough to figure out [1]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases [2]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… [3]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… [4]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Edge_Reference_Archite… [5]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Mobile_servi…. [6]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Universal_cu… [7]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Unmanned_Air… [8]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Cloud_Storag… [9]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Open_Caching… [10]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Smart_City_a… [11]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Augmented_Re… [12]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Analytics.2F… [13]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Manage_retai… [14]: https://schd.ws/hosted_files/kccna18/34/GitOps.pdf [15]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Smart_Home [16]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#Data_Collect… [17]: https://wiki.openstack.org/wiki/Edge_Computing_Group/Use_Cases#VPN_Gateway_… Thanks and Best Regards, Gergely and Ildikó

1 0

queens [magnum] patches
by Ignazio Cassano 10 Jan '19

10 Jan '19

Hello, last week I talked on #openstack-containers IRC about important patches for magnum reported here: https://review.openstack.org/#/c/577477/ I'd like to know when the above will be backported on queens and if centos7 and ubuntu packages will be upgraded with them. Any roadmap ? I would go on with magnum testing on queens because I am going to upgrade from ocata to pike and from pike to queens. At this time I have aproduction environment on ocata and a testing environment on queens. Best Regards Ignazio

3 4

[rpm-packaging] Proposing new core member
by Thomas Bechtold 10 Jan '19

10 Jan '19

Hi, I would like to nominate Colleen Murphy for rpm-packaging core. Colleen has be active in doing very valuable reviews since some time so I feel she would be a great addition to the team. Please give your +1/-1 in the next days. Cheers, Tom

3 2

[vitrage] Nominating Ivan Kolodyazhny for Vitrage core
by Ifat Afek 10 Jan '19

10 Jan '19

Hi, I would like to nominate Ivan Kolodyazhny for Vitrage core. Ivan has been contributing to Vitrage for a while now. He has focused on upgrade support, vitrage-dashboard and vitrage-tempest-plugin enhancements, and during this time gained a lot of knowledge and experience with Vitrage code base. I believe he would make a great addition to our team. Thanks, Ifat.

2 2