Open Stack

I know I'm arriving late to this thread, but I want to chime in with a
resounding "yes!" to this approach.

We've held to a fairly strict policy around maintaining compatibility
within the driver API since early on, and we'll continue to do that as
we add new interfaces (see the new ManagementInterface) or enhance
existing ones (like the new @clean_step decorator). I believe that, as
a hardware abstraction layer, we need to enable authors of
vendor-specific plugins/drivers to innovate out of tree -- whether
that is through a driver + library, such as stackforge/proliantutils,
or in a completely out of tree driver (I'm not aware of any open
source examples of this yet, but I have heard of private ones).

As far as the two (dis)advantages you list:

1) vendor drivers reviewed by the community

On the one hand, I believe there is some benefit from the broader
OpenStack community reviewing vendor code. Some... but not much.
Architecture and general principles (is it scalable? are you doing
something that operators will hate?) is valuable, but that's where it
stops. As you pointed out, it is impractical to expect open source
developers (or developers at other hardware companies) to have the
knowledge and expertise necessary (or hardware available to them) to
determine whether or not vendor-specific code will work with specific
vendor hardware.

As an interesting data point, we now have support for 8 different
types of hardware drivers in tree (10 if you count the ssh and vbox
drivers that we use in testing).

2) distros (not) packaging dependencies

Because we can not actually test every driver's dependencies in the
upstream gate (there's no hardware to exercise against), we do not
install those packages in devstack-gate, and so we do not list
driver's python dependencies in requirements.txt. We mock those
libraries and test the in-tree driver code, and rely on the library
and driver authors to ensure their fitness for use against real
hardware. So, yes, third-party-CI is essential to this -- and I wish
more vendors would step up and run CI on their drivers. But I
digress...

After talking with Thomas Goirand about debian packaging of Ironic, we
realized that there wasn't a clear list of our driver's dependencies.
So to provide distros with a single location for this information, we
are now maintaining a "driver-requirements.txt" file. It's fairly new
-- I added it about a month ago. See
https://github.com/openstack/ironic/blob/master/driver-requirements.txt


Thanks for writing up your experiences with this, Ramakrish. I believe
it is a great example to other driver authors -- and it would be great
to have a wiki or doc page describing the how's and why's of this
approach.

Cheers,
Devananda


On Fri, Feb 27, 2015 at 10:28 PM, Ramakrishnan G
<rameshg87.openstack at gmail.com> wrote:
>
> Hello All,
>
> This is about adding vendor drivers in Ironic.
>
> In Kilo, we have many vendor drivers getting added in Ironic which is a very
> good thing.  But something I noticed is that, most of these reviews have
> lots of hardware-specific code in them.  This is something most of the other
> Ironic folks cannot understand unless they go and read the hardware manuals
> of the vendor hardware about what is being done.  Otherwise we just need to
> blindly mark the file as reviewed.
>
> Now let me pitch in with our story about this.  We added a vendor driver for
> HP Proliant hardware (the *ilo drivers in Ironic).  Initially we proposed
> this same thing in Ironic that we will add all the hardware specific code in
> Ironic itself under the directory drivers/modules/ilo.  But few of the
> Ironic folks didn't agree on this (Devananda especially who is from my
> company :)). So we created a new module proliantutils, hosted in our own
> github and recently moved it to stackforge.  We gave a limited set of APIs
> for Ironic to use - like get_host_power_status(), set_host_power(),
> get_one_time_boot(), set_one_time_boot(), etc. (Entire list is here
> https://github.com/stackforge/proliantutils/blob/master/proliantutils/ilo/operations.py).
>
> We have only seen benefits in doing it.  Let me bring in some examples:
>
> 1) We tried to add support for some lower version of servers.  We could do
> this without making any changes in Ironic (Review in proliantutils
> https://review.openstack.org/#/c/153945/)
> 2) We are adding support for newer models of servers (earlier we use to talk
> to servers in protocol called RIBCL, newer servers we will use a protocol
> called RIS) - We could do this with just 14 lines of actual code change in
> Ironic (this was needed mainly because we didn't think we will have to use a
> new protocol itself when we started) -
> https://review.openstack.org/#/c/154403/
>
> Now talking about the advantages of putting hardware-specific code in
> Ironic:
>
> 1) It's reviewed by Openstack community and tested:
> No. I doubt if I throw in 600 lines of new iLO specific code that is here
> (https://github.com/stackforge/proliantutils/blob/master/proliantutils/ilo/ris.py)
> for Ironic folks, they will hardly take a look at it.  And regarding
> testing, it's not tested in the gate unless we have a 3rd party CI for it.
> [We (iLO drivers) also don't have 3rd party CI right now, but we are working
> on it.]
>
> 2) Everything gets packaged into distributions automatically:
> Now the hardware-specific code that we add in Ironic under
> drivers/modules/<vendor>/ will get packaged into distributions, but this
> code in turn will have dependencies  which needs to be installed manually by
> the operator (I assume vendor specific dependencies are not considered by
> Linux distributions while packaging Openstack Ironic). Anyone installing
> Ironic and wanting to manage my company's servers will again need to install
> these dependencies manually.  Why not install the wrapper if there is one
> too.
>
> I assume we only get these advantages by moving all of hardware-specific
> code to a wrapper module in stackforge and just exposing some APIs for
> Ironic to use:
> * Ironic code would be much cleaner and easier to maintain
> * Any changes related to your hardware - support for newer hardware, bug
> fixes in particular models of hardware, would be very easy. You don't need
> to change Ironic code for that. You could just fix the bug in your module,
> release a new version and ask your users to install a newer version of the
> module.
> * python-fooclient could be used outside Ironic to easily manage foo
> servers.
> * Openstack CI for free if you are in stackforge - unit tests, flake tests,
> doc generation, merge, pypi release everything handled automatically.
>
> I don't see any disadvantages.
>
> Now regarding the time taken to do this, if you have all the code ready now
> in Ironic (which assume you will already have), perhaps it will take a day
> to do this - half a day for putting into a separate module in python/github
> and half a day for stackforge. The request to add stackforge should get
> approved in the same day (if everything is all-right).
>
> Let me know all of your thoughts on this.  If we agree, I feel we should
> have some documentation on it in our Ironic docs directory.
>
> Thanks for reading :)
>
> Regards,
> Ramesh
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>