Open Stack

----- Original Message -----
> From: "Jamie Lennox" <jamielennox at redhat.com>
> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
> Sent: Monday, August 10, 2015 12:36:14 PM
> Subject: Re: [openstack-dev] [oslo][keystone] oslo_config and wsgi middlewares
> 
> 
> 
> ----- Original Message -----
> > From: "Mehdi Abaakouk" <sileht at sileht.net>
> > To: openstack-dev at lists.openstack.org
> > Sent: Friday, August 7, 2015 1:57:54 AM
> > Subject: [openstack-dev] [oslo][keystone] oslo_config and wsgi middlewares
> > 
> > Hi,
> > 
> > I want to share with you some problems I have recently encountered with
> > openstack middlewares and oslo.config.
> > 
> > The issues
> > ----------
> > 
> > In project Gnocchi, I would use oslo.middleware.cors, I have expected to
> > just put the name of the middleware to the wsgi pipeline, but I can't.
> > The middlewares only works if you pass the oslo_config.cfg.ConfigOpts()
> > object or via 'paste-deploy'... Gnocchi doesn't use paste-deploy, so
> > I have to modify the code to load it...
> > (For the keystonemiddleware, Gnocchi already have a special
> > handling/hack to load it [1] and [2]).
> > I don't want to write the same hack for each openstack middlewares.
> > 
> > 
> > In project Aodh (ceilometer-alarm), we recently got an issue with
> > keystonemiddleware since we remove the usage of the global object
> > oslo_config.cfg.CONF. The middleware doesn't load its options from the
> > config file of aodh anymore. Our authentication is broken.
> > We can still pass them through paste-deploy configuration but this looks
> > a method of the past. I still don't want to write a hack for each
> > openstack middlewares.
> > 
> > 
> > Then I have digged into other middlewares and applications to see how
> > they handle their conf.
> > 
> > oslo_middlewarre.sizelimit and oslo_middlewarre.ssl take options only
> > via the global oslo_config.cfg.CONF. So they are unusable for application
> > that doesn't use this global object.
> > 
> > oslo_middleware.healthcheck take options as dict like any other python
> > middleware. This is suitable for 'paste-deploy'. But doesn't allow
> > configuration via oslo.config, doesn't have a strong config options
> > type checking and co.
> > 
> > Zaqar seems got same kind of issue about keystonemiddleware, and just
> > write a hack to workaround the issue (monkeypatch the cfg.CONF of
> > keystonemiddleware with their local version of the object [3] and then
> > transform the loaded options into a dict to pass them via the legacy
> > middleware dict options [4]) .
> > 
> > Most applications, just still use the global object for the
> > configuration and don't, yet, see those issues.
> > 
> > 
> > All of that is really not consistent.
> > 
> > This is confusing for developer to have some middlewares that need
> > pre-setup,
> > enforce them to rely on global python object, and some others not.
> > This is confusing for deployer their can't do the configuration of
> > middlewares in the same way for each middlewares and each projects.
> > 
> > But keystonemiddleware, oslo.middleware.cors,... are supposed to be wsgi
> > middlewares, something that is independant of the app.
> > And this is not really the case.
> > 
> > From my point of view and what wsgi looks like generally in python, the
> > middleware object should be just MyMiddleware(app, options_as_dict),
> > if the middleware want to rely to another configuration system it should
> > do the setup/initialisation itself.
> > 
> > 
> > 
> > So, how to solve that ?
> > ------------------------
> > 
> > Do you agree:
> > 
> > * all openstack middlewares should load their options with oslo.config ?
> >   this permits type checking and all other features it provides, it's cool
> >   :)
> >   configuration in paste-deploy conf is thing of past
> > 
> > * we must support local AND global oslo.config object ?
> >   This is an application choice not something enforced by middleware.
> >   The deployer experience should be the same in both case.
> > 
> > * the middleware must be responsible of the section name in the oslo.config
> > ?
> >   Gnocchi/Zaqar hack have to hardcode the section name in their code,
> >   this doesn't looks good.
> > 
> > * we must support legacy python signature for WSGI object,
> >   MyMiddleware(app, options_as_dict) ? To be able to use paste for
> >   application/deployer that want it and not break already deployed things.
> > 
> > 
> > I really think all our middlewares should be consistent:
> > 
> > * to be usable by all applications without enforcing them to write crap
> > around them.
> > * and to made the deployer life easier.
> > 
> > 
> > Possible solution:
> > ------------------
> > 
> > I have already started to work on something that do all of that for all
> > middlewares [5], [6]
> > 
> > The idea is, the middleware should create a oslo_config.cfg.ConfigOpts()
> > (instead of rely on the global one) and load the configuration file of the
> > application in. oslo.config will discover the file location just with the
> > name of application as usual.
> > 
> > So the middleware can now be loaded like this:
> > 
> > code example:
> > 
> >    app = MyMiddleware(app, {"oslo_config_project": "aodh"})
> > 
> > paste-deploy example:
> > 
> >    [filter:foobar]
> >    paste.filter_factory = foobar:MyMiddleware.filter_factory
> >    oslo_config_project = aodh
> > 
> > oslo_config.cfg.ConfigOpts() will easly find the /etc/aodh/aodh.conf,
> > This cut the hidden links between middleware and the application
> > (through the global object).
> > 
> > And of course if oslo_config_project is not provided, the middleware
> > fallback the global oslo.config object.
> > The middleware options can still be passed via the legacy dict.
> > The backward compatibility is conserved.
> > 
> > I have already tested that in Gnocchi and Aodh, and that solves all of
> > my issues. Remove all hacks, the application doesn't need special pre
> > setup. All our middleware become normal middleware but still can use
> > oslo.config.
> > 
> > 
> > WDYT ?
> 
> So i agree on most of your points, the middlewares are certainly
> inconsistent, i would like to make things easier for deployers and I don't
> like global CONF objects (but i've now had that argument so many times i'm
> done).
> 
> Also to support some of the newer services that don't use paste i think we
> should absolutely make it so that the CONF object is passed to middleware
> rather than sourced globally. I think gnochhi and zaqar both fit into this
> case.
> 
> The problem i see with what you are saying is that it is mixing deployment
> methodologies in a way that is unintended. Paste is designed to allow
> deployers to add and remove middleware independent of configuring the
> service. This means that at paste time there is no CONF object unless it's
> globally registered and this is why most middlewares allow taking options
> from paste.ini because if you don't have global CONF then it's the only way
> to actually get options into the middleware.
> 
> The proposed solution is then to provide enough options (via paste) to load a
> CONF object local to the middleware that will be an exact copy of what the
> service will have already loaded. In your test so far this has only required
> project_name which might be enough because i don't know how often services
> specify their own default_config_files[1] and it looks like CONF will use
> sys.argv[1:] by default [2] for parsing CLI arguments. This conf loading
> code would then have to be replicated into all middleware that wants to use
> oslo.config for its options.
> 
> Fixing this problem is always where i loose enthusiasm for removing global
> CONF files.
> 
> From a developer perspective I feel the solution is for us to reconsider how
> we deploy and configure middleware. If you are using paste those pieces of
> middleware should each be able to be configured without any integration with
> the service underneath it. Otherwise if your service needs a piece of
> middleware like auth_token middleware to operate or relies on oslo.config
> options like cors then that is not something that should be controlled by
> paste.
> 
> From a deployer perspective there is no great answer i just want everything
> to be in oslo.config.
> 
> Equally from a deployer perspective this wasn't an issue until aodh (et al)
> decided to remove the global CONF object which developers from all the
> projects hate but live with. I don't mean to imply we shouldn't look at
> better ways to do things, but if you're main concern is deployers the
> easiest thing we can do for consistency is add back the global CONF object
> or remove paste from aodh. :)
> 
> Jamie

On re-reading this i realize the :) doesn't prevent this coming acrross a bit snarky or aggressive. That's not my intent and i apologize.

I would love to figure out ways we can make all this more consistent, but i'm not a fan of having to load and handle CONF objects in multiple places, especially once per middleware object.


> 
> 
> [1]
> https://git.openstack.org/cgit/openstack/oslo.config/tree/oslo_config/cfg.py#n1832
> [2]
> https://git.openstack.org/cgit/openstack/oslo.config/tree/oslo_config/cfg.py#n1880
>  
> > 
> > Cheers,
> > 
> > [1]
> > https://github.com/openstack/gnocchi/blob/master/gnocchi/rest/app.py#L140
> > [2]
> > https://github.com/openstack/gnocchi/blob/master/gnocchi/service.py#L64-L73
> > [3]
> > https://github.com/openstack/zaqar/blob/87fd1aa93dafb64097f731dbd416c2eeb697d403/zaqar/transport/auth.py#L63
> > [4]
> > https://github.com/openstack/zaqar/blob/87fd1aa93dafb64097f731dbd416c2eeb697d403/zaqar/transport/auth.py#L70
> > [5] https://review.openstack.org/#/c/208965/
> > [6] https://review.openstack.org/#/c/209817/
> > 
> > 
> > --
> > Mehdi Abaakouk
> > mail: sileht at sileht.net
> > irc: sileht
> > 
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>