Open Stack

Angus Salkeld <asalkeld at redhat.com> wrote on 19.06.2013 03:09:34:

> From: Angus Salkeld <asalkeld at redhat.com>
> To: openstack-dev at lists.openstack.org,
> Date: 19.06.2013 03:19
> Subject: Re: [openstack-dev] [Heat] Does it make sense to have a
> resource-create API?
>
> On 18/06/13 23:32 +0000, Adrian Otto wrote:
> >Yes. I think having a POST method in the API makes perfect sense.
> Assuming we reach agreement on that, the next question that comes up is:
>
> Err, I am not convinced.
>
> Before that, I think it's worth highlighting the different
> proposals/requirements here:
>
> 1) heat needs a way to setup an autoscaling group
> 2) autoscaling needs a way of telling heat to scale up/down
> 3) People might want to integrate other orchestration engines with Heat
>
> So one by one:
> 1) autoscaling has a post api that the Heat autoscaling group resource
>     posts to, Heat will provide a webhook in the event of a scaling
action.
> 2) when autoscaling determines that there should be a scaling action
>     it calls the webhook. The reason I suggest a webhook is in-instance
>     applications might want to scale their applications (like
>     openshift-gears) - so don't assume a heat endpoint.
>     Also we can have this as an action not a resource-create.
>     PUT /$tenant/stacks/$stack/resources/<autoscale-group>
>     (with an action of scale-up/down)
>
> I think we should discourage (make it impossible) for users from modifing
> stacks outside of stack-update.
>
> To me one of the most powerful and apealing things of Heat is the
> ability to reproducibly re-create a stack from a template. This
> new public API is going to make this difficult.
>
> Also creating a resource is not a trivial issue, the user would have
> create the resources in the correct order (with correct inter-resource
> references etc..) mostly throwing away the point of orchestation in
> the first place. If you are doing this you may as well talk directly
> to nova/cinder/networking directly.

I agree on all of the above. I've read other responses to this thread and
do not yet get why out of band creation of resources should be allowed in
Heat. Updates might be ok, but I am not convinced on the create case for
the reasons Angus brought up. Heat is doing orchestration based on
templates, so bypassing this takes the orchestration away.
Sure, this could be handy in some cases, but once you open up an API like
this, a lot of nasty things could start to evolve.

If am following this correctly, it all started with an autoscaling
discussion that raised the question for a create resource API. So why isn't
autoscaling like any other resource (an instance, a swift bucket, etc.) and
if someone likes to create it out of band, go ahead and just talk to the
respective service (like you would do with nova for an instance). If you
want to do it as part of orchestration, include the autoscaling resource in
a Heat template and use Heat.

>
> >
> >How to do you modify resources that have been created with a POST?
> >
> >You mention HTTP PUT as an answer to that. Unfortunately PUT is
> only really useful for doing a full resource replacement, not just
> tweaking something that's already there. For that, you really want
> HTTP PATCH (http://tools.ietf.org/html/rfc5789). You can make this
> really elegant for JSON with JSON Patch (http://tools.ietf.org/html/
> draft-ietf-appsawg-json-patch-10).
> >
> >We should note that offering API methods to adjust a stack (aka:
> assembly) means that there will be a divergence between what's
> described in the original template, and the actual running state of
> the stack/assembly created by the template, well beyond the results
> of an autoscale policy. In fact, it would be possible to build a
> stack/assembly with no template at all, if the right API methods are
> present. There are good use cases for this, particularly for higher
> level compatibility layers where it would be awkward to generate
> permutations of templates to immediately feed into an API, rather
> than just use an API method for adjusting the stack/assembly in
> place. it would be much more elegant, for example, to implement a
> CAMP implementation on top of Heat if Heat had a REST API for
> creating and managing individual resources within a stack/assembly.
> This same argument applies to integrating any other orchestration or
> configuration management system with Heat.
>
> So this is "3)":
> As I was alluding to above, if you are integrating at this level what
> value is Heat providing to you? If you are just using it for resource
> create this is a thin layer over the python clients. I'd suggest just
> using the resources as python plugins. I think exposing this would
> bring more harm than good.
>
> -Angus
>
> >More comments in-line:
> >
> >On Jun 18, 2013, at 3:44 PM, Christopher Armstrong
> <chris.armstrong at rackspace.com>
> > wrote:
> >
> >> tl;dr POST /$tenant/stacks/$stack/resources/ ?
> >
> >Yes.
> >
> >> == background ==
> >>
> >> While thinking about the Autoscaling API, Thomas Hervé and I had the
> >> following consideration:
> >>
> >> - autoscaling is implemented as a set of Heat Resources
> >> - there are already general APIs for looking at resources generically:
> >>  - resource-show (GET /$tenant/stacks/$stack/resources/$id)
> >>  - resource-metadata
(GET /$tenantt/stacks/$stack/resources/$id/metadata)
> >>  - resource-list (GET /$tenant/stacks/$stack/resources/)
> >> - we want to be able to create and configure autoscaling resources
> >> through the API
> >> - maybe we should implement POST for resources?
> >>
> >> This is basically the gist of the question. I believe the answer
> >> should be the same as the answer about any other type of resource we
> >> might want to manipulate through the API -- it seems best that either
> >> all resource types are manipulated through a generic resource
> >> manipulation API, or they should all have their own specific ReST
> >> collection.
> >
> >Give them specific collections, so they can be easily specialized.
> >
> >> Actually, I could also imagine a situation where only generic
> >> operations on common resource metadata are allowed via
> >> /$tenant/stacks/$stack/resources/, and resource-specific manipulation
> >> is done via resource-specific collections -- I don't know how ReSTy
> >> that is, though.
> >>
> >> I'll get to specifics. There are two ways I can imagine the autoscale
> >> API looking. I'll avoid the word "resource" when referring to ReST
> >> resources and just talk about "collections" and "paths", since
> >> "resource" in this context also means Heat resources.
> >>
> >> == resource-specific paths ==
> >>
> >> One is basically just like Otter's: http://docs.autoscale.apiary.io/
> >>
> >> This provides paths like /$tenant/groups/$id (for an autoscaling
> >> group), /$tenant/groups/$id/policies (for a policy), etc. These
> >> variously support GET for reading as well as POST and PUT for
> >> manipulation.
> >>
> >> We can use "/v1.0/{tenantId}/groups/{groupId}/policies" as an example
> >> operation. We POST JSON describing a new scaling policy to create to a
> >> new scaling policy.
> >
> >The above approach is definitely my preference.
> >
> >> == generic paths ==
> >>
> >> The alternative is to say that autoscaling groups, policies, etc are
> >> all Just Heat Resources, and Heat resources already have a ReST
> >> collection at /$tenant/resources/.
> >>
> >> In this option, the alternative to POSTing to
> >> /$tenant/groups/$id/policies would be to post directly to
> >> /$tenant/resources/, with a body exactly like in the previous example,
> >> but with two more JSON attributes:
> >>
> >> - the type of the resource, in this case something like
> >> "AWS::AutoScaling::ScalingPolicy"
> >> - the group ID that the new policy should be associated with, since
> >> it's not specified in the URL.
> >
> >This is approach is less elegant than resource-specific paths.
> Bringing in types is a whole other layer of complexity that can be
> avoided when using resource-specific paths. You actually still have
> types, but they are implicit. You still have to document them, but
> you don't have to individually name them.
> >
> >> One concern I have is about how well we can specify a strict schema of
> >> inputs and outputs to the resources/ collection -- I'm particular
> >> interested in JSON hyperschema. I'm not sure how it handles
> >> heterogeneous collections like this.
> >
> >In the JSON world, that's what documentation is for. Although
> having a JSON Schema can help simplify client development, it is yet
> another body of code that Heat developers will need to maintain. The
> question here comes down to how many developers will work on
> building clients, and could simplifying their development experience
> help boost adoption of Heat. My suggestion is (even if we like the
> idea of a schema) to start lean without a schema, and add that in if
> we agree it's something we want later. With some discipline, it's
> possible to implement things in a way that lends itself to using a
> schema even if we don't explicitly define, publish, and maintain one.
> >
> >Maintaining the schema (and the documentation for that matter)
> should be something that all OpenStack development teams feel
> comfortable with, because basically anytime someone changes any of
> the downstream OpenStack API's, there's a good chance it will need
> to change here too in order to expose whatever feature they add. A
> tweak to a documentation page may be less burdensome if you think of
> it from that perspective.
> >
> >Adrian
> >_______________________________________________
> >OpenStack-dev mailing list
> >OpenStack-dev at lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>