Open Stack

On 01/25/2016 03:56 PM, Steven Hardy wrote:
> On Fri, Jan 22, 2016 at 11:24:20AM -0600, Ben Nemec wrote:
>> So I haven't weighed in on this yet, in part because I was on vacation
>> when it was first proposed and missed a lot of the initial discussion,
>> and also because I wanted to take some time to order my thoughts on it.
>>  Also because my initial reaction...was not conducive to calm and
>> rational discussion. ;-)
>>
>> The tldr is that I don't like it.  To explain why, I'm going to make a
>> list (everyone loves lists, right? Top $NUMBER reasons we should stop
>> expecting other people to write our API for us):
>>
>> 1) We've been down this road before.  Except last time it was with Heat.
>>  I'm being somewhat tongue-in-cheek here, but expecting a general
>> service to provide us a user-friendly API for our specific use case just
>> doesn't make sense to me.
> 
> Actually, we've been down this road before with Tuskar, and discovered that
> designing and maintaining a bespoke API for TripleO is really hard.

My takeaway from Tuskar was that designing an API that none of the
developers on the project use is doomed to fail.  Tuskar also suffered
from a lack of some features in Heat that the new API is explicitly
depending on in an attempt to avoid many of the problems Tuskar had.

Problem #1 is still developer apathy IMHO though.

> 
> I somewhat agree that heat as an API is insufficient, but that doesn't
> necessarily imply you have to have a TripleO specific abstraction, just
> that *an* abstraction is required.
> 
>> 2) The TripleO API is not a workflow API.  I also largely missed this
>> discussion, but the TripleO API is a _Deployment_ API.  In some cases
>> there also happens to be a workflow going on behind the scenes, but
>> honestly that's not something I want our users to have to care about.
> 
> How would you differentiate between "deployment" in a generic sense in
> contrast to a generic workflow?
> 
> Every deployment I can think of involves a series of steps, involving some
> choices and interactions with other services.  That *is* a workflow?

Well, I mean if we want to take this to extremes then pretty much every
API is a workflow API.  You make a REST call, a "workflow" happens in
the service, and you get back a result.

Let me turn this around: Would you implement Heat's API on Mistral?  All
that happens when I call Heat is that a series of OpenStack calls are
made from heat-engine, after all.  Or is that a gross oversimplification
of what's happening?  I could argue that the same is true of this
discussion. :-)

> 
>> 3) It ties us 100% to a given implementation.  If Mistral proves to be a
>> poor choice for some reason, or insufficient for a particular use case,
>> we have no alternative.  If we have an API and decide to change our
>> implementation, nobody has to know or care.  This is kind of the whole
>> point of having an API - it shields users from all the nasty
>> implementation details under the surface.
> 
> This is a valid argument, but building (and maintining, forever) a bespoke
> API is a high cost to pay for this additional degree of abstraction, and
> when you think of the target audience, I'm not certain it's entirely
> justified (or, honestly, if our community can bear that overhead);
> 
> For example, take other single-purpose "deployment" projects, such as
> Sahara, Magnum, perhaps Trove.  These are designed primarily as user-facing
> API's, where the services will ultimately be consumed by public and private
> cloud customers.
> 
> Contrast with TripleO, where our target audience is, for the most part,
> sysadmins who deploy and maintain an openstack deployment over a long
> period of time.  There are two main groups here:
> 
> 1. PoC "getting started" folks who need a very simple on-ramp (generalizing
> somewhat, the audience for the opinionated deployments driven via UI's)
> 
> 2. Seasoned sysadmins who want plugability, control and flexibility above
> all else (and, simplicity and lack of extraneous abstractions)
> 
> A bespoke API potentially has a fairly high value to (1), but a very low or
> even negative value to (2).  Which is why this is turning out to be a tough
> and polarized discussion, unfortunately.

Well, to be honest I'm not sure we can satisfy the second type of user
with what we have today anyway.  Our Heat-driven puppet is hardly
lightweight or simple, and there are extraneous abstractions all over
the place (see also every place that we have a Heat template param that
exists solely to get plugged into a puppet hiera file :-).

To me, this is mostly an artifact of the original intent of the Heat
templates being _the_ abstraction that would then be translated into
os-*-config, puppet, or [insert deployment tool of choice] by the
templates, and I have to admit I'm not sure how to fix it for these users.

So I guess the question is, how does having an API hurt those power
users?  They'll still be able/have to edit Heat templates to deploy
additional services.  They'll still have all the usual openstack clients
to customize their Ironic or Nova setups.  They're already using an API
today, it's just one written entirely in the client.

On the other hand, an API that can help guide a user through the deploy
process (You say you want network isolation enabled?  Well here are the
parameters you need to configure...) could make a huge difference for
the first type of user, as would _any_ API usable by the GUI (people
just like pretty GUIs, whether it's actually better than the CLI or not :-).

I guess it is somewhat subjective as you say, but to me the API doesn't
significantly affect the power user experience, but it would massively
improve the newbie experience.  That makes it a win in my book.

> 
>> 4) It raises the bar even further for both new deployers and developers.
>>  You already need to have a pretty firm grasp of Puppet and Heat
>> templates to understand how our stuff works, not to mention a decent
>> understanding of quite a number of OpenStack services.
> 
> I'm not really sure if a bespoke WSGI app vs an existing one (mistral)
> really makes much difference at all wrt raising the bar.  I see it
> primarily as in implementation detail tbh.

I guess that depends.  Most people in OpenStack already know at least
some Python, and if you've done any work in the other projects there's a
fair chance you are familiar with the Python clients.  How many people
know Mistral YAML?

Maybe I'm overestimating the Python knowledge in the community, and
underestimating the Mistral knowledge, but I would bet we're talking
order(s?) of magnitude in terms of the difference.  And I'm not saying
learning Mistral is a huge task on its own, but it's one more thing in a
project full of one more things.

> 
> Looking at the prototypes Dan has done, it *is* possible to define a much
> more strongly versioned API with Mistral than it is with Heat, which is the
> main requirement in terms of lowering the bar (stable, relatively
> opinionated by default and easy to use).
> 
> Steve
>