Open Stack

Hey everyone,

I agree that we need to be preparing for the summit. Using Google docs
mixed with Openstack wiki works for me right now. I need to become more
familiar the gerrit process and I agree with Samuel that it is not
conducive to "large" design discussions. That being said I'd like to add
my thoughts on how I think we can most effectively get stuff done.

As everyone knows there are many new players from across the industry that
have an interest in Neutron LBaaS. Companies I currently see
involved/interested are Mirantis, Blue Box Group, HP, PNNL, Citrix,
eBay/Paypal and Rackspace. We also have individuals involved as well. I
echo Kyle's sentiment on the passion everyone is bringing to the project!
Coming into this project a few months ago I saw that a few things needed
to be done. Most notably, I realized that gathering everyone's
expectations on what they wanted Neutron LBaaS to be was going to be
crucial. Hence, I created the requirements document. Written requirements
are important within a single organization. They are even more important
when multiple organizations are working together because everyone is
spread out across the world and every organization has a different
development process. Again, my goal with the requirements document is to
make sure that everyone's voice in the community is taken into
consideration. The benefit I've seen from this document is that we ask
"Why?" to each other, iterate on the document and in the end have a clear
understanding of everyone's motives. We also learn from each other by
doing this which is one of the great benefits of open source.

Now that we have a set of requirements the next question to ask is, "How
doe we prioritize requirements so that we can start designing and
implementing them"? If this project were a completely new piece of
software I would argue that we iterate on individual features based on
anecdotal information. In essence I would argue an agile approach.
However, most of the companies involved have been operating LBaaS for a
while now. Rackspace, for example, has been operating LBaaS for the better
part of 4 years. We have a clear understanding of what features our
customers want and how to operate at scale. I believe other operators of
LBaaS have the same understanding of their customers and their operational
needs. I guess my main point is that, collectively, we have data to back
up which requirements we should be working on. That doesn't mean we
preclude requirements based on anecdotal information (i.e. "Our customers
are saying they want new shiny feature X"). At the end of the day I want
to prioritize the community's requirements based on factual data and
anecdotal information.

Assuming requirements are prioritized (which as of today we have a pretty
good idea of these priorities) the next step is to design before laying
down any actual code. I agree with Samuel that pushing the cart before the
horse is a bad idea in this case (and it usually is the case in software
development), especially since we have a pretty clear idea on what we need
to be designing for. I understand that the current code base has been
worked on by many individuals and the work done thus far is the reason why
so many new faces are getting involved. However, we now have a completely
updated set of requirements that the community has put together and trying
to fit the requirements to existing code may or may not work. In my
experience, I would argue that 99% of the time duct-taping existing code
to fit in new requirements results in buggy software. That being said, I
usually don't like to rebuild a project from scratch. If I can I try to
refactor as much as possible first. However, in this case we have a
particular set of requirements that changes the game. Particularly,
operator requirements have not been given the attention they deserve.

I think of Openstack as being cloud software that is meant to operate at
scale and have the necessary operator tools to do so. Otherwise, why do we
have so many companies interested in Openstack if you can't operate a
cloud that scales? In the case of LBaaS, user/feature requirements and
operator requirements are not necessarily mutually exclusive. How you
design the system in regards to one set of requirements affects the design
of the system in regards to the other set of requirements. SSL
termination, for example, affects the ability to scale since it is CPU
intensive. As an operator, I need to know how to provision load balancer
instances efficiently so that I'm not having to order new hardware more
than I have to. With this in mind, I am assuming that most of us are
vendor-agnostic and want to cooperate in developing an open source driver
while letting vendors create their own drivers. If this is not the case
then perhaps a lot of the debates we have been having are moot since we
can separate efforts depending on what driver we want to work on. The only
item of Neutron LBaaS that we need to have consensus on then is the API
(web app, database, messaging system, etc.). Keep in mind that the API
implies what feature/user requirements are satisfied, but no so much for
operator requirements. I think this is one reason why we have been working
on API proposals. Samuel, thank you for the time you spent on your
proposal as we know how much time and effort it takes.

Because several of us have been spending large amounts of time on API
proposals, and because we can safely assume that most operational
requirements are abstracted into the driver layer I say we continue the
conversation around the different proposals since this is the area we
definitely need consensus on. So far there are three proposals--Stephen's,
Rackspace's and Eugene's. To date, we honestly haven't had an actual
discussion on the pros and cons of each proposal. To give structure to
this we here at Rackspace have been going to great lengths to make sure we
have enough tangible documentation in order to clearly convey our
thoughts. We also went to great lengths to satisfy the user/feature
requirements in our API. Here is what we have done:

- An API specification located here ==>
https://docs.google.com/document/d/1mTfkkdnPAd4tWOMZAdwHEx7IuFZDULjG9bTmWyX
e-zo/edit
- Single API call workflows & multiple API call workflows of each of the
use cases (#1 through #9 for now) from
https://docs.google.com/document/d/1Ewl95yxAMq2fO0Z6Dz6fL-w2FScERQXQR1-mXuS
INis/edit#heading=h.48fieovwttzg. Our workflows are located here ==>
https://drive.google.com/#folders/0B2r4apUP7uPwRVc2MzQ2MHNpcE0
- A lightweight proof of concept that is in the works so that people that
need to actually send requests to an API to believe in it can do so. We
will send an update in a few days when this POC is complete.

In order to fairly compare proposals I think it would be nice if each
proposal give workflows on how their API will operate. This is isn't
necessary but I think it will definitely give structure in any discussions
we have when comparing. If others have thoughts on how to compare the
proposals on equal footing then by all means speak up.

Once we come to a consensus on the API then we can figure out how to make
iterative changes in order to get the API to the consensus state. That is
a separate conversation in my mind. First we need to agree on a spec
without the confines of looking at current implementation.


Cheers,
--Jorge


P.S. Sorry for the delay in responding to the ML. Just reading them takes
several hours.