[openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers

John Griffith john.griffith at solidfire.com
Fri Sep 5 04:44:17 UTC 2014


On Thu, Sep 4, 2014 at 4:32 PM, Jay Pipes <jaypipes at gmail.com> wrote:

>
>
> On 09/04/2014 12:11 PM, Duncan Thomas wrote:
>
>> I think that having a shared review team across all of the drivers
>> has definite benefits in terms of coherency and consistency - it is
>> very easy for experts on one technology to become tunnel-visioned on
>> some points and miss the wider, cross project picture. A common
>> drivers team is likely to have a broad enough range of opinions to
>> keep things healthy, compared to one repo (and team) per driver, and
>> also they are able to speak collectively to teh core nova team, which
>> helps set priorities there when they need to be influenced on behalf
>> of the drivers team.
>>
>
> In theory, the above sounds good. In practice, it doesn't happen. The code
> in the virt drivers is horribly inconsistent, duplicative and yet slightly
> and pointlessly different, and uses paradigms that make sense for the one
> platform but don't necessarily make sense for another platform.
>
> The testing/CI benefits that Dan highlighted -- in terms of patches to
> non-related virt drivers not interfering with the stability and progress of
> a patch to another virt driver -- is the #1 critical benefit to Dan's
> proposal, and doing a single virt drivers core team and repo totally throws
> that benefit away.
>
> Best,
> -jay
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Just some thoughts and observations I've had regarding this topic in Cinder
the past couple of years.  I realize this is a Nova thread so hopefully
some of this can be applied in a more general context.

TLDR:
1. I think moving drivers into their own repo is just shoveling the pile to
make a new pile (not really solving anything)

2. Removal of drivers other than the reference implementation for each
project could be the healthiest option
    a. Requires transparent, public, automated 3'rd party CI
    b. Requires a TRUE plugin architecture and mentality
    c. Requires a stable and well defined API

3. While I'm still sort of a fan of the removal of drivers, I do think
Cinder is "making it work", there have been missteps and yes it's a pain
sometimes but it's working "ok" and we've got plans to try and improve

4. Adding restrictions like drivers only in first milestone and more
intense scrutinization of features will go a long way to help resolve the
issues we do have currently

Now the long winded version with a little more detail and context;

````````````````````````````````````````````````



I've spent a fair amount of time thinking about the explosive number of
drivers being added to Cinder over the past year or so.  I've been a pretty
vocal proponent of the idea of removing all drivers except the LVM
reference implementation from Cinder.  I'd rather see Vendors drivers
maintained in their own Github Repo and truly follow a "plugin" model.
 This of course means that Cinder has to be truly designed and maintained
with a real plugin architecture kept in mind in every aspect of development
(experience proves this harder to do than it sounds).  I think with things
stable and well defined interfaces as well as 3'rd party CI this is
actually a reasonable approach and could be effective.  I do not see how
creating a separate repo and in essence yet another set of OpenStack
Projects really helps with the problem.  The fact is that the biggest issue
most people see with driver contributions is those that are made by
organizations that work on their driver only and don't contribute back to
the core project (whether that be in the form of reviews of core
contributions).  I'm not sure I understand why that would be any different
by just putting the code in a separate bucket.  In other words, getting a
solid and consistent team working on that "project" seems like you've just
kicked the can down the road so you don't have to deal with it.

Any time I've mentioned the removal approach the response is typically that
there's no quality control, or that Vendors won't be as willing to invest
in OpenStack because they can focus on their own interests and get by with
that.  The quality control one was a tough one to counter, but now that
we're moving towards things like 3'rd party CI I'm not sure that's quite as
significant as it was a year ago.  I'd still like to see a public record of
testing in the form of CI, NOT just Vendor-A submitting something that
says.. "yeah, I'm awesome".  I suspect that OpenStack adopters would look
at things like public CI postings to determine what's worth pursuing and
what's not.

The other concern I had in the past was "we'd loose valuable contributors".
 There are vendors that are directly responsible for providing us with some
great contributors in the Core of the Cinder project.  They do a great job
of balancing the tactical and strategic interests, and the concern is that
if the drivers aren't in Cinder then maybe they won't see the need to make
the investment.  I don't think this is true any longer.  I think that
OpenStack in general has become "real" so to speak and any Vendor that has
sense is going to realize they need to be involved in the core projects and
providing input to the future direction of things.  I honestly think that
many of them would continue to operate as they do today, the difference
being their driver is maintained externally.

Over the past year I've become even less concerned about the issues of
"loosing contributors".  The fact is that more an more vendors are choosing
to strip down the driver implementation they submit to Cinder to a simple
Interface that calls their external python library.  Some of you may
remember a long time ago I voiced my concern that this was an AWFUL
direction and nothing good would come of it.  Fact is, I was VERY WRONG.
 It's actually not a bad model, vendors that have taken that approach are
focused more heavily on Cinder than they are on their own drivers.  Really,
IMO since a number of vendors are already half way there, maybe it's not
that big of a deal to just go all the way.

All in all Cinder as tried to do things like requiring features in the API
be implemented on every backend and such.  We've had mixed success with
this IMO.  Over the years there have been things that creep in that
probably shouldn't and what I call "sponsored features" are in most cases
only interesting for the Vendor sponsoring them.  That being said, I think
we're doing "ok", and I really believe that moving to a model of limiting
when features and drivers can be added will go a long way to helping some
of the short-comings we currently face.

All of that being said, from Cinder's perspective I'd rather take an
incremental step and try focusing on restricting new driver submissions to
the first milestone, and being more "careful" about features, as well as
also requiring those features to be available earlier in the release cycle
(we ran into a real mess with Juno IMO having major features added the last
week of J3 that now nobody can implement).

Thanks,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140904/da84b3ea/attachment-0001.html>


More information about the OpenStack-dev mailing list