Open Stack

Solum Team,

First of all I wanted to say that I have been thinking a lot about this thread, and have been seeking input from a number of you who attended the Summit this week. I do *not* see this decision as a critical one, because if something really mattered a ton we could rip our one framework and put another in. 

What I love about this discussion is that we are having a healthy debate about different points of view, which was very thought provoking. I have heard input from skeptics who think that engineering decisions have to be made in a small group in order to be efficient. I know that thinking is wrong, and it's examples like these that convince me further that discussions like these help us make better choices, and make development more efficient over the long run.

We should give the most weight to the preferences of the engineers who will write and maintain the code. If there are team members who are volunteering to write and maintain the bulk of our API code over a period of time who really want Falcon over Pecan, then we should give that fair consideration. The API is exceedingly important to Solum, and all of us will be working on it, so we need a choice that all of us can live with.

I suggest that we settle on Pecan+WSME, for the following reasons:

1) It is a known quantity, and is likely to work well for Solum's needs, considering that Solum is primarily a control plane API system, and that performance is not a primary motivator.

2) Pecan+WSME allows us to offer data serializations in both JSON and XML to help satisfy the preferences of the API consumers. 

3) It allows us to have multiple models that are loosely coupled, which can aid in data validation.

4) At this point in time, other OpenStack core/integrated/ecosystem projects are using Pecan+WSME, and several Solum contributors will be switching between projects. There is an advantage to a higher level of consistency among various projects.

I accept that Webob may be problematic for us, that performance may be less than ideal, and that some Solum developers may dislike working with WSME, and that Falcon may actually be more pleasant to work with. We have team members with a deep understanding of Falcon, and could definitely make it work. We can proceed with Pecan+WSME accepting these (and  other) trade-offs.

Are there any other critically important considerations that we should consider before converging on this choice? I'd like to hear that input prior to our next IRC meeting.

Thanks,

Adrian


On Nov 8, 2013, at 5:59 AM, Steven Dake <sdake at redhat.com> wrote:

> On 11/07/2013 07:28 AM, Jay Pipes wrote:
>> Hey Doug, following up, but realize you are likely very busy at the summit :)
>> 
>> On 11/04/2013 10:20 PM, Doug Hellmann wrote:
>>> On Tue, Nov 5, 2013 at 10:37 AM, Jay Pipes <jaypipes at gmail.com
>>> 
>>>        The "models" defined in WSME are completely different from the
>>>        database
>>>        models, and should not be used outside of the API code. Think of
>>>        them as
>>>        declaring the models for the consumer of the API, rather than the
>>>        implementer of the API.
>>> 
>>> 
>>>    I don't really see the point of the distinction. At the end of the
>>>    day, the consumer of the API is using the API to manipulate and
>>>    retrieve data. That data is really the model, with some syntactic
>>>    sugar like Atom links, etc. Even in a control API -- as opposed to a
>>>    data API like Glance, Ceilometer or Swift -- the benefits of a heavy
>>>    API layer like Pecan and WSME are pretty small, IMO.
>>> 
>>> 
>>> That approach binds your API tightly to the database representation,
>>> which we were trying to avoid.
>> 
>> I wasn't referring to the database representation, actually. I was referring to the data model, which is different in that it abstracts away any underlying schema or storage engine...
>> 
>>>    I would much rather see the Ceilometer models [1] actually be models
>>>    that can validate the data that is used to construct the model
>>>    object instead of having duplicated WSME "models" repeated in the
>>>    WSGI controller code [2]. The reason is because if/when I decide to
>>>    make a Ceilometer API that uses a different protocol, say AMQP,
>>>    instead of HTTP, now I need to duplicate all of the validation code
>>>    that WSME is providing on the data model layer... however if the
>>>    validation was in the models themselves, I could easily create an
>>>    API on a different protocol using just the models for validation.
>>> 
>>> We do that in some cases. However, there is also a difference in some
>>> cases between the validation at the API layer (a value must be a number,
>>> or a UUID, etc.) and the validation in the database layer (a number must
>>> fall within a range or a UUID must refer to an existing object). So
>>> there is a place for both, and the validation done in the WSME classes
>>> is not meant to be the only validation performed.
>> 
>> OK, understood.
>> 
>>>        The benefits of declaring WSME classes include automatic
>>>        serialization
>>>        in JSON and XML, generating WADL files to be included in the API
>>>        docs
>>>        (work is already happening to make this available for everyone), and
>>>        consistent input and output types for API endpoints (making it
>>>        easier
>>>        for consumers of the API to use it and for implementers to validate
>>>        inputs and assume consistent defaults).
>>> 
>>> 
>>>    I can't stand XML. I believe it should be retired to the dustbin of
>>>    coding history, like Basic.
>>> 
>>> You've made that clear in the past. :-) I agree, for what it's worth.
>>> Some of our users do seem to want it, and with WSME *you don't have to
>>> care*.
>> 
>> Also understood. :)
>> 
>>>    That said, consumers of a RESTful API don't care how the API is
>>>    implemented. They care that it's documented and consistent, and if
>>>    WSME makes API documentation easier, then that's A Good Thing, agreed.
>>> 
>>>    It's true that WSME includes some stuff to make validating inputs
>>>    "easier", but it does so, IMHO, at the expense of readability
>>>    because everything is decorated and hidden away somewhere other than
>>>    the models themselves. See note above...
>>> 
>>> I'm not sure what that means. Hidden where? The validation is either
>>> described in the attribute specifier for the model, or in the model's
>>> class, or in the controller (depending on the scope of the rule being
>>> applied).
>> 
>> Sorry, "hidden" wasn't the best word to describe that. I meant more "it's less obvious to folks who expect to find validation of the data model at the data model layer" :)
>> 
>>>        And, to be clear, Pecan and WSME are integrated by Pecan can
>>>        definitely
>>>        be used without WSME. I included WSME in the proposal to replace the
>>>        home-grown WSGI framework because I thought it added significant
>>>        benefits, but it is not going to be appropriate for all situations
>>>        (streaming large image files is one example).
>> 
>> Yes, in one way it's a bit unfortanate that we're now comparing Falcon with Pecan + WSME as opposed to just comparing Falcon with Pecan and then evaluating whether WSME should be used (on top of either one...). It's unfortunate because we've spent all this email thread actually talking about WSME and not much at all about Pecan ;)
>> 
>>>    Here's a third reason I don't care for Pecan/WSME: it uses Webob.
>>>    Other than eventlet, I don't know of a single library that OpenStack
>>>    projects have used over the years that we've had more issues with
>>>    than Webob.
>>> 
>>> Yes, I felt the pain of updating us to the latest WebOb. The project has
>>> evolved since those days, and the current maintainers are committed to
>>> not breaking the API. That new attitude, combined with the long history
>>> of addressing edge cases from misbehaving web client libraries makes m e
>>> less reluctant to use WebOb than in the past.
>> 
>> OK, that's good to hear. However, I remain guarded and pessimistic...it's my nature I suppose :)
>> 
>>>    Secondly, it is the sign of strength that a contributor community
>>>    consider -- and continually consider -- alternative libraries or
>>>    implementations of various things. Change is good, and projects that
>>>    are just starting off are a good place to experiment with newer
>>>    libraries and see if something is better than what existed
>>>    previously. I seem to remember that was your position on Pecan when
>>>    the community was considering whether to standardize on some WSGI
>>>    pipeline. Why didn't you just use what was in other projects instead
>>>    of Pecan and WSME? Likely the same reasons that I'm saying "let's
>>>    give Falcon a chance".
>>> 
>>> My main arguments were "do not build something new from scratch when
>>> there are working tools" and "the existing library is too hard to use".
>>> We can argue about whether Falcon or Pecan solves the second issue
>>> better, but Pecan was used in production deployments far earlier than
>>> Falcon even existed.
>> 
>> Yes, that is a very fair point.
>> 
>> I suppose at the end of the day, a lot of this does come down to personal preference/opinion for me. I just prefer the style of Falcon over Pecan. That said, I'm a community member that does not wish to see Solum's early pace slowed down just because I personally prefer a different library! So, I'll back down if the direction that the Solum contributors want to take is to go with Pecan. I'll still likely bitch and moan about WSME though... :P
>> 
> Jay,
> 
> +1
> 
> This is a great point.  Best for new projects to focus on key problems in their mission, rather then dealing with dependency management which can always be addressed later as some OpenStack programs like nova are doing in their planned migration for Icehouse.
> 
> Regards
> -steve
> 
>> Best,
>> -jay
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev