Open Stack

Doug, I respect you very much as an engineer and as a community member, 
so I want to preface this with a very important dislaimer: don't take 
these opinions as anything more than what they are...a discussion of 
differing viewpoints.

Comments inline.

On 11/04/2013 06:58 PM, Doug Hellmann wrote:
> On Sun, Nov 3, 2013 at 10:54 PM, Jay Pipes <jaypipes at gmail.com
> <mailto:jaypipes at gmail.com>> wrote:
>
>     On 11/02/2013 11:26 PM, Russell Bryant wrote:
>
>         On 11/02/2013 11:54 AM, Adrian Otto wrote:
>
>             Noorul,
>
>             I agree that key decisions should be tracked in blueprints.
>             This is the
>             one for this decision which was made in our 2013-10-18
>             public meeting.
>             Jay's submission is consistent with the direction indicated
>             by the team.
>
>             https://blueprints.launchpad.__net/solum/+spec/rest-api-base
>             <https://blueprints.launchpad.net/solum/+spec/rest-api-base>
>
>             Transcript log:
>             http://irclogs.solum.io/2013/__solum.2013-10-08-16.01.log.__html
>             <http://irclogs.solum.io/2013/solum.2013-10-08-16.01.log.html>
>             <http://irclogs.solum.io/2013/__solum.2013-10-08-16.01.log.__html
>             <http://irclogs.solum.io/2013/solum.2013-10-08-16.01.log.html>>
>
>
>         Heh, not much discussion there.  :-)
>
>
>     Agreed. I actually didn't know anything about the discussion -- I
>     wasn't at the meeting. I just figured I would throw some example
>     code up to Gerrit that shows how Falcon can be used for the API
>     plumbing. Like I mentioned in a previous email, I believe it's much
>     easier to discuss things when there is sample code...
>
>
>         Here's my take ... Pecan+WSME has been pushed as the thing to
>         standardize on across most OpenStack APIs.  Ceilometer (and maybe
>         others?) are already using it.  Others, such as Nova, are
>         planning to
>         use it this cycle. [1][2]
>
>
>     I've used both actually, and I've come to prefer Falcon because of
>     its simplicity and specifically because of the following things:
>
>     * It's lack of integration with WSME, which I don't care for. I
>     don't care for WSME because I believe it tries to put the model at
>     the view layer, instead of where it belongs, at the model layer.
>
>
> The "models" defined in WSME are completely different from the database
> models, and should not be used outside of the API code. Think of them as
> declaring the models for the consumer of the API, rather than the
> implementer of the API.

I don't really see the point of the distinction. At the end of the day, 
the consumer of the API is using the API to manipulate and retrieve 
data. That data is really the model, with some syntactic sugar like Atom 
links, etc. Even in a control API -- as opposed to a data API like 
Glance, Ceilometer or Swift -- the benefits of a heavy API layer like 
Pecan and WSME are pretty small, IMO.

I would much rather see the Ceilometer models [1] actually be models 
that can validate the data that is used to construct the model object 
instead of having duplicated WSME "models" repeated in the WSGI 
controller code [2]. The reason is because if/when I decide to make a 
Ceilometer API that uses a different protocol, say AMQP, instead of 
HTTP, now I need to duplicate all of the validation code that WSME is 
providing on the data model layer... however if the validation was in 
the models themselves, I could easily create an API on a different 
protocol using just the models for validation.

> The benefits of declaring WSME classes include automatic serialization
> in JSON and XML, generating WADL files to be included in the API docs
> (work is already happening to make this available for everyone), and
> consistent input and output types for API endpoints (making it easier
> for consumers of the API to use it and for implementers to validate
> inputs and assume consistent defaults).

I can't stand XML. I believe it should be retired to the dustbin of 
coding history, like Basic.

That said, consumers of a RESTful API don't care how the API is 
implemented. They care that it's documented and consistent, and if WSME 
makes API documentation easier, then that's A Good Thing, agreed.

It's true that WSME includes some stuff to make validating inputs 
"easier", but it does so, IMHO, at the expense of readability because 
everything is decorated and hidden away somewhere other than the models 
themselves. See note above...

> And, to be clear, Pecan and WSME are integrated by Pecan can definitely
> be used without WSME. I included WSME in the proposal to replace the
> home-grown WSGI framework because I thought it added significant
> benefits, but it is not going to be appropriate for all situations
> (streaming large image files is one example).
>
>     * It doesn't need a configuration file, specifically a configuration
>     file that is a Python file as opposed to an .ini file.
>
> Pecan does not require a configuration file. It can use one, but we set
> up the WSGI app factory in ceilometer to not use one and I expect the
> other projects to work the same way.

OK, that's good to know.

Here's a third reason I don't care for Pecan/WSME: it uses Webob. Other 
than eventlet, I don't know of a single library that OpenStack projects 
have used over the years that we've had more issues with than Webob.

> Tuesday (today) at 2:00 there is a session in the Oslo track
> (http://icehousedesignsummit.sched.org/event/b2680d411aa7f5d432438a435ac21fee)
> to discuss tips and pain points with Pecan & WSME.

Unfortunately, I won't be at the summit. :(

 >I didn't intend to
> revisit the decision to start adopting either (we've spent an hour at
> each of the last summits going over that, as well as many email
> threads),

Yeah, you know the argument that a couple dozen folks have discussed 
something at one or two of the past summits and therefore it should be 
considered settled fact is a bit annoying to me.

Partly it is annoying because the summit has a vast number of sessions, 
with many tracks going simultaneously, making it impossible to get to 
all the sessions that people feel strongly about. I wish I could have 
been at several design summit discussions, especially around database 
issues, at the last couple summits, but I was in the QA track most of 
the summit, for good or bad.

Secondly, it is the sign of strength that a contributor community 
consider -- and continually consider -- alternative libraries or 
implementations of various things. Change is good, and projects that are 
just starting off are a good place to experiment with newer libraries 
and see if something is better than what existed previously. I seem to 
remember that was your position on Pecan when the community was 
considering whether to standardize on some WSGI pipeline. Why didn't you 
just use what was in other projects instead of Pecan and WSME? Likely 
the same reasons that I'm saying "let's give Falcon a chance".

I believe we discussed why Bazaar and Launchpad merge proposals were the 
code contribution platform at length for the Austin, Bexar and Cactus 
summits, with GitHub fans at every one of those discussions, and lo and 
behold, the community switched to Git and GitHub and Gerrit during the 
Diablo timeframe. So, long story short, stuff changes, opinions change, 
and that's totally fine...

 > but I do want to clear up any other misconceptions and discuss
> issues with either tool so that feedback can be incorporated upstream.
> Now that both Pecan and WSME are on stackforge, we have already had a
> few patches from OpenStack developers intended to improve and adjust
> them to meet our needs better.

I think that is a fine thing to continue, and I don't believe this 
particular debate by the Solum project should affect that at all.

Best,
-jay

[1] 
https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/models.py
[2] 
https://github.com/openstack/ceilometer/blob/master/ceilometer/api/controllers/v2.py

> Doug
>
>
>
>         I care less about the particular choice and more about
>         consistency.  It
>         brings a lot of value, such as making it a lot easier for
>         developers to
>         jump around between the OpenStack projects.  Can we first at
>         least agree
>         that there is value in standardizing on *something* for most
>         OpenStack APIs?
>
>
>     I completely understand the need for consistency. I pushed my patch
>     as an example of how to do things the Falcon way. While I would
>     prefer Falcon over Pecan (and certainly over Pecan+WSME), I will
>     respect the push towards consistency if that's what is most important.
>
>     That said, I also believe that the projects in Stackforge should be
>     the "laboratories of experiment", and these projects may serve as a
>     good playground for various implementations of things. I remind the
>     reader that over time, the development community has standardized on
>     various things, only to find a better implementation in an incubated
>     project. Pecan+WSME is actually an example of that experimentation
>     turned accepted standard.
>
>
>     Best,
>     -jay
>
>
>         I understand that there may be cases where the needs for an API
>         justify
>         being different.  Marconi being more of a data-plane API vs
>         control-plane means that performance concerns are much higher,
>         for example.
>
>         If we agree that consistency is good, does Solum have needs that
>         make it
>         different than the majority of OpenStack APIs?  IMO, it does not.
>
>         Can someone lay out a case for why all OpenStack projects should be
>         using Falcon, if that's what you think Solum should use?
>
>         Also, is anyone willing to put up the equivalent of Jay's review
>         [3],
>         but with Pecan+WSME, to help facilitate the discussion?
>
>         [1]
>         http://icehousedesignsummit.__sched.org/event/__b2680d411aa7f5d432438a435ac21f__ee
>         <http://icehousedesignsummit.sched.org/event/b2680d411aa7f5d432438a435ac21fee>
>         [2]
>         http://icehousedesignsummit.__sched.org/event/__4a7316a4f5c6f783e362cbba2644ba__e2
>         <http://icehousedesignsummit.sched.org/event/4a7316a4f5c6f783e362cbba2644bae2>
>         [3] https://review.openstack.org/#__/c/55040/
>         <https://review.openstack.org/#/c/55040/>
>
>
>
>     _________________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.__org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>