Open Stack

Hi Josh - I'm able to view all of them, but I probably have special
google powers ;)

Which links are broken for you?

thanks,

On Thu, Mar 8, 2018 at 3:53 PM, Joshua Harlow <harlowja at fastmail.com> wrote:
>
> Can we get some of those doc links opened.
>
> 'You need permission to access this published document.' I am getting for a
> few of them :(
>
>
> Ben Nemec wrote:
>>
>> Hi,
>>
>> Here's my summary of the discussions we had in the Oslo room at the PTG.
>> Please feel free to reply with any additions if I missed something or
>> correct anything I've misrepresented.
>>
>> oslo.config drivers for secret management
>> -----------------------------------------
>>
>> The oslo.config implementation is in progress, while the Castellan
>> driver still needs to be written. We want to land this early in Rocky as
>> it is a significant change in architecture for oslo.config and we want
>> it to be well-exercised before release.
>>
>> There are discussions with the TripleO team around adding support for
>> this feature to its deployment tooling and there will be a functional
>> test job for the Castellan driver with Custodia.
>>
>> There is a weekly meeting in #openstack-meeting-3 on Tuesdays at 1600
>> UTC for discussion of this feature.
>>
>> oslo.config driver implementation: https://review.openstack.org/#/c/513844
>> spec:
>>
>> https://specs.openstack.org/openstack/oslo-specs/specs/queens/oslo-config-drivers.html
>>
>> Custodia key management support for Castellan:
>> https://review.openstack.org/#/c/515190/
>>
>> "stable" libraries
>> ------------------
>>
>> Some of the Oslo libraries are in a mature state where there are very
>> few, if any, meaningful changes to them. With the removal of the
>> requirements sync process in Rocky, we may need to change the release
>> process for these libraries. My understanding was that there were no
>> immediate action items for this, but it was something we need to be
>> aware of.
>>
>> dropping support for mox3
>> -------------------------
>>
>> There was some concern that no one from the Oslo team is actually in a
>> position to support mox3 if something were to break (such as happened in
>> some libraries with Python 3.6). Since there is a community goal to
>> remove mox from all OpenStack projects in Rocky this will hopefully not
>> be a long-term problem, but there was some discussion that if projects
>> needed to keep mox for some reason that they would be asked to provide a
>> maintainer for mox3. This topic is kind of on hold pending the outcome
>> of the community goal this cycle.
>>
>> automatic configuration migration on upgrade
>> --------------------------------------------
>>
>> There is a desire for oslo.config to provide a mechanism to
>> automatically migrate deprecated options to their new location on
>> version upgrades. This is a fairly complex topic that I can't cover
>> adequately in a summary email, but there is a spec proposed at
>> https://review.openstack.org/#/c/520043/ and POC changes at
>> https://review.openstack.org/#/c/526314/ and
>> https://review.openstack.org/#/c/526261/
>>
>> One outcome of the discussion was that in the initial version we would
>> not try to handle complex migrations, such as the one that happened when
>> we combined all of the separate rabbit connection opts into a single
>> connection string. To start with we will just raise a warning to the
>> user that they need to handle those manually, but a templated or
>> hook-based method of automating those migrations could be added as a
>> follow-up if there is sufficient demand.
>>
>> oslo.messaging plans
>> --------------------
>>
>> There was quite a bit discussed under this topic. I'm going to break it
>> down into sub-topics for clarity.
>>
>> oslo.messaging heartbeats
>> =========================
>>
>> Everyone seemed to be in favor of this feature, so we anticipate
>> development moving forward in Rocky. There is an initial patch proposed
>> at https://review.openstack.org/546763
>>
>> We felt that it should be possible to opt in and out of the feature, and
>> that the configuration should be done at the application level. This
>> should _not_ be an operator decision as they do not have the knowledge
>> to make it sanely.
>>
>> There was also a desire to have a TTL for messages.
>>
>> bug cleanup
>> ===========
>>
>> There are quite a few launchpad bugs open against oslo.messaging that
>> were reported against old, now unsupported versions. Since we have the
>> launchpad bug expirer enabled in Oslo the action item proposed for such
>> bugs was to mark them incomplete and ask the reporter to confirm that
>> they still occur against a supported version. This way bugs that don't
>> reproduce or where the reporter has lost interest will eventually be
>> closed automatically, but bugs that do still exist can be updated with
>> more current information.
>>
>> deprecations
>> ============
>>
>> The Pika driver will be deprecated in Rocky. To our knowledge, no one
>> has ever used it and there are no known benefits over the existing
>> Rabbit driver.
>>
>> Once again, the ZeroMQ driver was proposed for deprecation as well. The
>> CI jobs for ZMQ have been broken for a while, and there doesn't seem to
>> be much interest in maintaining them. Furthermore, the breakage seems to
>> be a fundamental problem with the driver that would require non-trivial
>> work to fix.
>>
>> Given that ZMQ has been a consistent pain point in oslo.messaging over
>> the past few years, it was proposed that if someone does step forward
>> and want to maintain it going forward then we should split the driver
>> off into its own library which could then have its own core team and
>> iterate independently of oslo.messaging. However, at this time the plan
>> is to propose the deprecation and start that discussion first.
>>
>> CI
>> ==
>>
>> Need to migrate oslo.messaging to zuulv3 native jobs. The
>> openstackclient library was proposed as a good example of how to do so.
>>
>> We also want to have voting hybrid messaging jobs (where the
>> notification and rpc messages are sent via different backends). We will
>> define a devstack job variant that other projects can turn on if desired.
>>
>> We also want to add amqp1 support to pifpaf for functional testing.
>>
>> Low level messaging API
>> =======================
>>
>> A proposal for a new oslo.messaging API to expose lower level messaging
>> functionality was proposed. There is a presentation at
>>
>> https://docs.google.com/presentation/d/1mCOGwROmpJvsBgCTFKo4PnK6s8DkDVCp1qnRnoKL_Yo/edit?usp=sharing
>>
>>
>> This seemed to generally be well-received by the room, and dragonflow
>> and neutron reviewers were suggested for the spec.
>>
>> Kafka
>> =====
>>
>> Andy Smith gave an update on the status of the Kafka driver. Currently
>> it is still experimental, and intended to be used for notifications
>> only. There is a presentation with more details in
>>
>> https://docs.google.com/presentation/d/e/2PACX-1vQpaSSm7Amk9q4sBEAUi_IpyJ4l07qd3t5T_BPZkdLWfYbtSpSmF7obSB1qRGA65wjiiq2Sb7H2ylJo/pub?start=false&loop=false&delayms=3000&slide=id.p
>>
>>
>> testing for Edge/FEMDC use cases
>> ================================
>>
>> Matthieu Simonin gave a presentation about the testing he has done
>> related to messaging in the Edge/FEMDC scenario where messaging targets
>> might be widely distributed. The slides can be found at
>>
>> https://docs.google.com/presentation/d/1LcF8WcihRDOGmOPIU1aUlkFd1XkHXEnaxIoLmRN4iXw/edit#slide=id.p3
>>
>>
>> In short, there is a desire to build clouds that have widely distributed
>> nodes such that content can be delivered to users from a location as
>> close as possible. This puts a lot of pressure on the messaging layer as
>> compute nodes (for example) could be halfway around the world from the
>> control nodes, which is problematic for a broker-based system such as
>> Rabbit. There is some very interesting data comparing Rabbit with a more
>> distributed AMQP1 system based on qpid-dispatch-router. In short, the
>> distributed system performed much better for this use case, although
>> there was still some concern raised about the memory usage on the client
>> side with both drivers. Some followup is needed on the oslo.messaging
>> side to make sure we aren't leaking/wasting resources in some messaging
>> scenarios.
>>
>> For further details I suggest taking a look at the presentation.
>>
>> mutable configuration
>> ---------------------
>>
>> This is also a community goal for Rocky, and Chang Bo is driving its
>> adoption. There was some discussion of how to test it, and also that we
>> should provide an example of turning on mutability for the debug option
>> since that is the target of the community goal. The cinder patch can be
>> found here: https://review.openstack.org/#/c/464028/ Turns out it's
>> really simple!
>>
>> Nova is also using this functionality for more complex options related
>> to upgrades, so that would be a good place to look for more advanced use
>> cases.
>>
>> Full documentation for the mutable config options is at
>> https://docs.openstack.org/oslo.config/latest/reference/mutable.html
>>
>> The goal status is being tracked in
>> https://storyboard.openstack.org/#!/story/2001545
>>
>> Chang Bo was also going to talk to Lance about possibly coming up with a
>> burndown chart like the one he had for the policy in code work.
>>
>> oslo healthcheck middleware
>> ---------------------------
>>
>> As this ended up being the only major topic for the afternoon, the
>> session was unfortunately lightly attended. However, the self-healing
>> SIG was talking about related topics at the same time so we ended up
>> moving to that room and had a good discussion.
>>
>> Overall the feature seemed to be well-received. There is some security
>> concern with exposing service information over an un-authenticated
>> endpoint, but because there is no authentication supported by the health
>> checking functionality in things like Kubernetes or HAProxy this is
>> unavoidable. The feature won't be mandatory, so if this exposure is
>> unacceptable it can be turned off (with a corresponding loss of
>> functionality, of course).
>>
>> There was also some discussion of dropping the asynchronous nature of
>> the checks in the initial version in order to keep the complexity to a
>> minimum. Asynchronous testing can always be added later if it proves
>> necessary.
>>
>> The full spec is at https://review.openstack.org/#/c/531456
>>
>> oslo.config strict validation
>> -----------------------------
>>
>> I actually had discussions with multiple people about this during the
>> week. In both cases, they were just looking for a minimal amount of
>> validation that would catch an error such at "devug=True". Such a
>> validation might be fairly simple to write now that we have the
>> YAML-based sample config with (ideally) information about all the
>> options available to set in a project. It should be possible to compare
>> the options set in the config file with the ones listed in the sample
>> config and raise warnings for any that don't exist.
>>
>> There is also a more complete validation spec at
>>
>> http://specs.openstack.org/openstack/oslo-specs/specs/ocata/oslo-validator.html
>> and a patch proposed at https://review.openstack.org/#/c/384559/
>>
>> Unfortunately there has been little movement on that as of late, so it
>> might be worthwhile to implement something more minimalist initially and
>> then build from there. The existing patch is quite significant and
>> difficult to review.
>>
>> Conclusion
>> ----------
>>
>> I feel like there were a lot of good discussions at the PTG and we have
>> plenty of work to keep the small Oslo team busy for the Rocky cycle. :-)
>>
>> Thanks to everyone who participated and I look forward to seeing how
>> much progress we've made at the next Summit and PTG.
>>
>> -Ben
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Ken Giusti  (kgiusti at gmail.com)