Open Stack

Maru's concerns are that:
1. It is large
2. It is complex

And Armando's related concerns are:
3. Could dev/review cycles be better spent on refactoring
4. If refactored neutron was available, would a simpler option become more
viable

Let me address them in that order.

1. Re: It is large
Group policy has an ambitious goal  - provide devop teams with policy based
controls that are usable at scale and with automation (say a higher
governance layer like Congress). The fact that meeting a large challenge
requires more code is natural. We understand that challenge, and that is
why we did a prototype (as PoC that was demonstrated on the summit). And
based on that learning we are incrementally creating patches for building
the group based policy. Just because a task is large, we as neutron can not
shy away from building it. That will only drive people who need it out side
neutron (as we are seeing with the frustration that the LBaaS team had
because they have a requirement that is "large" as well).

2. Re: It is complex
Complexity depends on the context. Our goal was to make the end-user's life
simpler (and more automated). To achieve some of that simplicity, we
required a little more complexity in the implementation. We decide to make
that arbitrage - a little higher complexity in implementation to allow for
simpler usage. But we were careful and did not want to impose that
complexity on every use case - hence a lot of that is optional (and
exercised only if the use case needs it). Unfortunately the model, has to
model all of it so as it not add complexity later in upgrade and backward
compatibility issues. We choose to do architecture upfront, and then
implement it incrementally.

The team came up with the model currently in model based on that review and
evaluation all the proposals in the document that you refer. It is easy to
make general comments, but unless you participate in the process and sign
up to writing the code, those comments are not going to help with solving
the original problem. And this _is_ open-source. If you disagree, please
write code and the community can decide for itself as to what model is
actually simple to use for them. Curtailing efforts from other developers
just because their engineering trade-offs are different from what you
believe your use-case needs is not why we like open source. We enjoy the
mode where different developers try different things, we experiment, and
the software evolves to what the user demands. Or maybe, multiple models
live in harmony. Let the users decide that.

3. Re: Could dev/review cycles be better spent on refactoring
I think that most people agree that policy control is an important feature
that fundamentally improves neutron (by solving the automation and scale
issues). In a large project, multiple sub-projects can, and for a healthy
project should, work in parallel. I understand that the neutron core team
is stretched. But we still need to be able to balance the needs of today
(paying off the technical debt/existing-issues by doing refactoring) with
needs of tomorrow (new features like GP and LBaaS). GP effort was started
in Havana, and now we are trying to get this in Juno. I think that is
reasonable and a long enough cycle for a "high priority" project to be able
to get some core attention. Again I refer to LBaaS experience, as they
struggled with very similar issues.

4. Re: If refactored neutron was available, would a simpler option become
more viable
We would love to be able to answer that question. We have been trying to
understand the refactoring work to understand this (see another ML thread)
and we are open to understanding your position on that. We will call the
ad-hoc meeting that you suggested and we would like to understand the
refactoring work that might be reused for simpler policy implementation. At
the same time, we would like to build on what is available today, and when
the required refactored neutron becomes available (say Juno or K-release),
we are more than happy to adapt to it at that time. Serializing all
development around an effort that is still in inception phase is not a good
solution. We are looking forward to participating in the core refactoring
work, and based on the final spec that come up with, we would love to be
able to eventually make the policy implementation simpler.

Regards,
Mandeep




On Thu, May 22, 2014 at 11:44 AM, Armando M. <armamig at gmail.com> wrote:

> I would second Maru's concerns, and I would also like to add the following:
>
> We need to acknowledge the fact that there are certain architectural
> aspects of Neutron as a project that need to be addressed; at the
> summit we talked about the core refactoring, a task oriented API, etc.
> To me these items have been neglected far too much over the past and
> would need a higher priority and a lot more attention during the Juno
> cycle. Being stretched as we are I wonder if dev/review cycles
> wouldn't be better spent devoting more time to these efforts rather
> than GP.
>
> That said, I appreciate that GP is important and needs to move
> forward, but at the same time I am thinking that there must be a
> better way for addressing it and yet relieve some of the pressure that
> GP complexity imposes to the Neutron team. One aspect it was discussed
> at the summit was that the type of approach shown in [2] and [3]
> below, was chosen because of lack of proper integration hooks...so I
> am advocating: let's talk about those first before ruling them out in
> favor of a monolithic approach that seems to violate some engineering
> principles, like modularity and loose decoupling of system components.
>
> I think we didn't have enough time during the summit to iron out some
> of the concerns voiced here, and it seems like the IRC meeting for
> Group Policy would not be the right venue to try and establish a
> common ground among the people driving this effort and the rest of the
> core team.
>
> Shall we try and have an ad-hoc meeting and an ad-hoc agenda to find a
> consensus?
>
> Many thanks,
> Armando
>
> On 22 May 2014 11:38, Maru Newby <marun at redhat.com> wrote:
> >
> > On May 22, 2014, at 11:03 AM, Maru Newby <marun at redhat.com> wrote:
> >
> >> At the summit session last week for group-based policy, there were many
> concerns voiced about the approach being undertaken.  I think those
> concerns deserve a wider audience, and I'm going to highlight some of them
> here.
> >>
> >> The primary concern seemed to be related to the complexity of the
> approach implemented for the POC.  A number of session participants voiced
> concern that the simpler approach documented in the original proposal [1]
> (described in the section titled 'Policies applied between groups') had not
> been implemented in addition to or instead of what appeared in the POC
> (described in the section titled 'Policies applied as a group API').  The
> simpler approach was considered by those participants as having the
> advantage of clarity and immediate usefulness, whereas the complex approach
> was deemed hard to understand and without immediate utility.
> >>
> >> A secondary but no less important concern is related to the impact on
> Neutron of the approach implemented in the POC.  The POC was developed
> monolithically, without oversight through gerrit, and the resulting patches
> were excessive in size (~4700 [2] and ~1500 [3] lines).  Such large patches
> are effectively impossible to review.  Even broken down into reviewable
> chunks, though, it does not seem realistic to target juno-1 for merging
> this kind of complexity.  The impact on stability could be considerable,
> and it is questionable whether the necessary review effort should be
> devoted to fast-tracking group-based policy at all, let alone an approach
> that is considered by many to be unnecessarily complicated.
> >>
> >> The blueprint for group policy [4] is currently listed as a 'High'
> priority.  With the above concerns in mind, does it make sense to continue
> prioritizing an effort that at present would seem to require considerably
> more resources than the benefit it appears to promise?
> >>
> >>
> >> Maru
> >>
> >> 1: https://etherpad.openstack.org/p/group-based-policy
> >
> > Apologies, this link is to the summit session etherpad.  The link to the
> original proposal is:
> >
> >
> https://docs.google.com/document/d/1ZbOFxAoibZbJmDWx1oOrOsDcov6Cuom5aaBIrupCD9E/edit
> >
> >> 2: https://review.openstack.org/93853
> >> 3: https://review.openstack.org/93935
> >> 4:
> https://blueprints.launchpad.net/neutron/+spec/group-based-policy-abstraction
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140522/21cb6c6d/attachment.html>