Open Stack

On Sat, Oct 12, 2013 at 12:21 PM, Alessandro Pilotti <
apilotti at cloudbasesolutions.com> wrote:

>
>
> On 12.10.2013, at 20:22, "Dan Smith" <dms at danplanet.com> wrote:
>
> >> From the user perspective, splitting off the projects seems to be
> >> focussing on the ease of commit compared to the final user
> >> experience.
> >
> > I think what you describe is specifically the desire that originally
> > spawned the thread: making the merging of changes to the hyper-v driver
> > faster by having them not reviewed by the rest of the Nova team. It
> > seems to be what the hyper-v developers want, not necessarily what the
> > Nova team as a whole wants.
> >
> >> An 'extras' project without *strong* testing co-ordination with
> >> packagers such as SUSE and RedHat would end up with the consumers of
> >> the product facing the integration problems rather than resolving
> >> where they should be, within the OpenStack project itself.
> >
> > I don't think splitting out to -extras means that it loses strong
> > testing coordination (note that strong testing coordination does not
> > exist with the hyper-v driver at this point in time). Every patch to the
> > -extras tree could still be unit (and soon, integration) tested against
> > the current nova tree, using the proposed patch applied to the -extras
> > tree. It just means that a change against nova wouldn't trigger the
> > same, which is why the potential for "catch up" behavior would be
> required.
> >
> >> I am sympathetic to the 'extra' drivers problem such as Hyper-V and
> >> powervm, but I do not feel the right solution is to split.
> >>
> >> Assuming there is a summit session on how to address this, I can
> >> arrange a user representation in that session.
> >
> > Cool, I really think we're at the point where we know the advantages and
> > disadvantages of the various options and further face-to-face discussion
> > at the summit is what is going to move us to the next stage.
> >
>
> I agree. Looks like we are converging towards a common ground. I'm summing
> it up here, including a few additional details, for the benefit of who will
> not join us in HK (sorry, we'll party for you as well :-)):
>


This sounds like a very myopic solution to the issue you originally raised,
and I don't think it will solve the underlying issues.



Taking a step back, you originally raised a concern about how we prioritize
reviews with the havana-rc-potential tag.

"In the past weeks we diligently marked bugs that are related to Havana
features with the "havana-rc-potential" tag, which at least for what Nova
is concerned, had absolutely no effect.
Our code is sitting in the review queue as usual and, not being tagged for
a release or prioritised, there's no guarantee that anybody will take a
look at the patches in time for the release. Needless to say, this starts
to feel like a Kafka novel. :-)" [1]

If the issue is just better bug triage and prioritizing reviews, help us do
that!

[2] shows the current status of your hyper-v havana-rc-potential bugs.
Currently there are only 7 bugs that have both tags.  Of those 7, 3 have no
pending patches to trunk, and one doesn't sound like it warrants a back
port (https://bugs.launchpad.net/nova/+bug/1220256).

Looking at the remaining 4, one is marked as a WIP by you (
https://bugs.launchpad.net/nova/+bug/1231911
https://review.openstack.org/#/c/48645/) which leaves three patches for
nova team to review.  Three reviews open for a week doesn't sound like an
issue that warrants a whole new repository.

You went on to clarify your position.

"I'm not putting into discussion how much and well you guys are working (I
actually firmly believe that you DO work very well), I'm just discussing
about the way in which blueprints and bugs get prioritised.

<snip>

On the other side, to get our code reviewed and merged we are always
dependent on the good will and best effort of core reviewers that don't
necessarily know or care about specific driver, plugin or agent internals.
This brings to even longer review cycles even considering that reviewers
are clearly doing their best in understanding the patches and we couldn't
be more thankful.

"Best effort" has also a very specific meaning: in Nova all the Havana
Hyper-V blueprints were marked as "low priority" (which can be translated
in: "the only way to get them merged is to beg for reviews or maybe commit
them on day 1 of the release cycle and pray") while most of the Hyper-V
bugs had no priority at all (which can be translated in "make some noise on
the ML and IRC or nobody will care"). :-)

This reality unfortunately applies to most of the sub-projects (non only
Hyper-V) and can be IMHO solved only by delegating more authonomy to the
sub-project teams on their specific area of competence across OpenStack as
a whole. Hopefully we'll manage to find a solution during the design summit
as we are definitely not the only ones feeling this way, by judging on
various threads in this ML." [3]


Once again you raise the issue of bug triage and prioritization of reviews
(and blueprints), so help us fix that!  This isn't a virt driver only issue
though.

The issues you originally raise are only incidentally related to virt
drivers and hyper-v.  The same issues can be brought up as you point out by
any sub-project (scheduling, APIs, DB, etc).  So a fix for only virt
drivers hardly sounds like an appropriate solution.




I would like t propose an alternate  two part solution to the issues you
originally raised.

* It sounds like nova needs to do a better job of bug triage, prioritizing
reviews and blueprints. With so many bugs and reviews its very easy for us
to mess this up.  How to do a better job?  I think we need to brainstorm on
this a little.

* Do a better job of tracking how knowledgeable a reviewer is about a
subset of code, just because someone is in core doesn't mean they know the
most about a specific component. I think its safe to say that on average
the more someone commits patches and reviews a file the more he knows about
it.  With git and gerrit we have all this information we just need to use
it.  Here is a quick proof of concept of the idea, (as a proof of concept
it just lists previous authors of the modified files):
https://gist.github.com/jogo/6955514, along with some sample outputs
http://paste.openstack.org/show/48340/.


[1]
http://lists.openstack.org/pipermail/openstack-dev/2013-October/016382.html
<http://lists.openstack.org/pipermail/openstack-dev/2013-October/016382.html>

[2]
https://bugs.launchpad.net/nova/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=havana-rc-potential+hyper-v+&field.tags_combinator=ALL&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on
[3]
http://lists.openstack.org/pipermail/openstack-dev/2013-October/016400.html


>
> 1) All the drivers will still be part of Nova.
>
> 2) One official project (nova-drivers-incubator?) or more than one will be
> created for the purpose of supporting a leaner and faster development pace
> of the drivers.
>
> 3) Current driver sub-project teams will informally elect their
> maintainer(s) which will have +2a rights on the new project or specific
> subtrees.
>
> 4) Periodically, code from the new project(s) must be merged into Nova.
> Only Nova core reviewers will have obviously +2a rights here.
> I propose to do it on scheduled days before every milestone,
> differentiated per driver to distribute the review effort (what about also
> having Nova core reviewers assigned to each driver? Dan was suggesting
> something similar some time ago).
>
> 5) All drivers will be treated equally and new features and bug fixes for
> master (except security ones) should land in the new project before moving
> to Nova.
>
> 6) CI gates for all drivers, once available, will be added to the new
> project as well. Only drivers code with a CI gate will be merged in Nova
> (starting with the Icehouse release as we already discussed).
>
> 7) Active communication should be maintained between the Nova core team
> and the drivers maintainers. This means something more than: "I wrote it on
> the ML didn't you see it?" :-)
>
> A couple if questions: will we keep version branches on the new project or
> just master?
>
> Bug fixes for older releases will be proposed to the incubator for the
> current release in development and to Nova for past versions branches?
>
> Please correct me if I missed something!
>
> Thanks,
>
> Alessandro
>
> > --Dan
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131012/c9058911/attachment.html>