Open Stack

Some comments on primitive management:

There could be 2 approaches to primitives:
1) Primitives belong to container (in our case, a loadbalancer) object
In that case, when you delete container, child objects are deleted (which
also implies un-deploying them from backends)
When a primitive is deleted, it doesn't affect container or other
primitives.

2) Primitives are independent
Independent primitives which can belong to more then 1 parent object
(container) imply that there is a special association object defining the
link between those.
More complex logic may be applied for such case:
When 'parent' object is deleted, child object is deleted as well iff it
isn't associated with other parent objects.
When child object is deleted, parent object persists.

Existing API uses mixed approach where some primitives are managed as in
(1) and others as in (2) (particular primitive can't be deleted if it is
used by other objects)

Thanks,
Eugene.




On Sat, Apr 19, 2014 at 9:50 AM, Stephen Balukoff <sbalukoff at bluebox.net>wrote:

> Hi y'all!
>
> Yes-- I agree that is a very bad idea to delete a primitive that's being
> shared by other load balancing configurations. The only case where this
> seems acceptable to me is the case that German mentioned where all assets
> on a given user's account are being wrapped up. But even in this case,
> assuming each load balancing service is being deleted from the root level,
> eventually none of the primitives will be shared anymore...
>
> In fact, what do y'all think of this policy?  If a primitive is shared at
> all, an attempt to delete it directly should return an error indicating
> it's still in use.
>
> Also, I've been making this assumption the whole time, but it's probably
> worth mentioning: I'm assuming primitives cannot be shared among tenants.
> (It seems to me that we'd be opening up a whole nasty can of worms
> security-wise if we allowed tenants to share primitives.)
>
> Here's an alternate idea for handling the case of abandoned primitives
> suggested by my team-mate Dustin Lundquist:
>
> Instead of doing any kind of cascading delete when a root object gets
> deleted, follow Brandon's idea and simply detach the primitives from the
> object being deleted. Later, a periodic garbage-collection process goes
> through and removes any orphaned primitives a reasonable interval after it
> was orphaned (this interval could be configurable per operator, per tenant,
> per user... whatever). If we do this:
>
> * Each primitive would need an additional 'orphaned_at' attribute
> * Primitives (other than the root object) created outside of a single-call
> interface would be created in an orphaned state.
> * Viewing the attributes of a primitive should also show whether it's an
> orphan, and if so, when the garbage collector will delete it.
> * Update and creation tasks that reference any connected primitives would
> clear the above orphaned_at attributes of said primitives
> * Deletion tasks would need to check connected primitives and update their
> orphaned_at attribute if said deletion orphans the primitive (ie. do an
> immediate reference check, set orphaned_at if references = 0)
> * It probably also makes sense to allow some primitives to have a flag set
> by the user to prevent garbage collection from ever cleaning them up (ex.
> SSL certificates are probably a good candidate for this). Maybe a "persist"
> boolean attribute or something?
> * Garbage collection never cleans up root objects.
>
> It seems to me the above:
> * Would prevent the CI user from having a bazillion orphaned primitives.
> * Does not immediately nuke anything the customer wasn't planning on nuking
> * Probably follows the principle of least surprise a little better than my
> previous "conditional cascading delete" suggestion
> * Still allows the simple act of deleting all the root objects to
> eventually delete all primitives on an account (ie. the "delete this
> account" task German was talking about.)
> * Has a secondary side-benefit of not immediately destroying all the
> supporting primitives of an object if the user accidentally nukes the wrong
> object.
>
> Also, it may be appropriate for some primitives to skip the garbage
> collection process and simply get deleted when their immediate parent
> primitive is deleted. This applies to primitives that aren't allowed to be
> shared, like:
> * Members (ie, deleting a pool deletes all its members, immediately)
> * "join" objects, like the 'rule' object that associates a listener with a
> non-default pool in the case of L7 switching
>
> I still like the idea of also having a cascading delete of some kind, in
> case the user wants to see all the primitives go away immediately (though
> they could always step through these themselves), or, again, as an
> immediate way to ensure all objects associated with a customer account are
> cleaned up quickly when the account is deleted. Though, I'm also thinking
> this should be an explicit flag the user has to set. (And again, even with
> this flag set, shared primitives are not deleted.)
>
> What do y'all think of the above idea?
>
> Thanks,
> Stephen
>
>
>
> On Fri, Apr 18, 2014 at 2:35 PM, Carlos Garza <carlos.garza at rackspace.com>wrote:
>
>>
>>  On Apr 17, 2014, at 8:39 PM, Stephen Balukoff <sbalukoff at bluebox.net>
>>  wrote:
>>
>>  Hello German and Brandon!
>>
>>  Responses in-line:
>>
>>
>> On Thu, Apr 17, 2014 at 3:46 PM, Brandon Logan <
>> brandon.logan at rackspace.com> wrote:
>>
>>> Stephen,
>>> I have responded to your questions below.
>>>
>>>
>>> On 04/17/2014 01:02 PM, Stephen Balukoff wrote:
>>>
>>>  Howdy folks!
>>>
>>>  Based on this morning's IRC meeting, it seems to me there's some
>>> contention and confusion over the need for "single call" functionality for
>>> load balanced services in the new API being discussed. This is what I
>>> understand:
>>>
>>>  * Those advocating "single call" are arguing that this simplifies the
>>> API for users, and that it more closely reflects the users' experience with
>>> other load balancing products. They don't want to see this functionality
>>> necessarily delegated to an orchestration layer (Heat), because
>>> coordinating how this works across two OpenStack projects is unlikely to
>>> see success (ie. it's hard enough making progress with just one project). I
>>> get the impression that people advocating for this feel that their current
>>> users would not likely make the leap to Neutron LBaaS unless some kind of
>>> functionality or workflow is preserved that is no more complicated than
>>> what they currently have to do.
>>>
>>>  Another reason, which I've mentioned many times before and keeps
>>> getting ignored, is because the more primitives you add the longer it will
>>> take to provision a load balancer.  Even if we relied on the orchestration
>>> layer to build out all the primitives, it still will take much more time to
>>> provision a load balancer than a single create call provided by the API.
>>> Each request and response has an inherent time to process.  Many primitives
>>> will also have an inherent build time.  Combine this in an environment that
>>> becomes more and more dense, build times will become very unfriendly to end
>>> users whether they are using the API directly, going through a UI, or going
>>> through an orchestration layer.  This industry is always trying to improve
>>> build/provisioning times and there are no reasons why we shouldn't try to
>>> achieve the same goal.
>>>
>>
>>  Noted.
>>
>>
>>>   * Those (mostly) against the idea are interested in seeing the API
>>> provide primitives and delegating "higher level" single-call stuff to Heat
>>> or some other orchestration layer. There was also the implication that if
>>> "single-call" is supported, it ought to support both simple and advanced
>>> set-ups in that single call. Further, I sense concern that if there are
>>> multiple ways to accomplish the same thing supported in the API, this
>>> redundancy breeds complication as more features are added, and in
>>> developing test coverage. And existing Neutron APIs tend to expose only
>>> primitives. I get the impression that people against the idea could be
>>> convinced if more compelling reasons were illustrated for supporting
>>> single-call, perhaps other than "we don't want to change the way it's done
>>> in our environment right now."
>>>
>>>  I completely disagree with "we dont want to change the way it's done
>>> in our environment right now".  Our proposal has changed the way our
>>> current API works right now.  We do not have the notion of primitives in
>>> our current API and our proposal included the ability to construct a load
>>> balancer with primitives individually.  We kept that in so that those
>>> operators and users who do like constructing a load balancer that way can
>>> continue doing so.  What we are asking for is to keep our users happy when
>>> we do deploy this in a production environment and maintain a single create
>>> load balancer API call.
>>>
>>>
>>  There's certainly something to be said for having a less-disruptive
>> user experience. And after all, what we've been discussing is so radical a
>> change that it's close to starting over from scratch in many ways.
>>
>>
>>
>>      Its not disruptive. There is nothing preventing them from
>> continuing to use  the multiple primitive operations philosophy, so they
>> can continue that approach.
>>
>>
>>>  I've mostly stayed out of this debate because our solution as used by
>>> our customers presently isn't "single-call" and I don't really understand
>>> the requirements around this.
>>>
>>>  So! I would love it if some of you could fill me in on this,
>>> especially since I'm working on a revision of the proposed API.
>>> Specifically, what I'm looking for is answers to the following questions:
>>>
>>>  1. Could you please explain what you understand single-call API
>>> functionality to be?
>>>
>>>  Single-call API functionality is a call that supports adding multiple
>>> features to an entity (load balancer in this case) in one API request.
>>> Whether this supports all features of a load balancer or a subset is up for
>>> debate.  I prefer all features to be supported.  Yes it adds complexity,
>>> but complexity is always introduced by improving the end user experience
>>> and I hope a good user experience is a goal.
>>>
>>
>>  Got it. I think we all want to improve the user experience.
>>
>>
>>>  2. Could you describe the simplest use case that uses single-call API
>>> in your environment right now? Please be very specific--  ideally, a couple
>>> examples of specific CLI commands a user might run, or API (along with
>>> specific configuration data) would be great.
>>>
>>>
>>> http://docs.rackspace.com/loadbalancers/api/v1.0/clb-devguide/content/Create_Load_Balancer-d1e1635.html
>>>
>>> This page has many different ways to configure a load balancer with one
>>> call.  It ranges from a simple load balancer to a load balancer with a much
>>> more complicated configuration.  Generally, if any of those features are
>>> allowed on a load balancer then it is supported through the single call.
>>>
>>>
>>>    I'm going to use example 4.10 as the "simplest" case I'm seeing
>> there. (Also because I severely dislike XML ;) )
>>
>>
>>
>>>    3. Could you describe the most complicated use case that your
>>> single-call API supports? Again, please be very specific here.
>>>
>>>  Same data can be derived from the link above.
>>>
>>>
>>  Ok, I'm actually not seeing and complicated examples, but I'm guessing
>> that any attributes at the top of the page could be expanded on according
>> the the syntax described.
>>
>>  Hmmm...  one of the draw-backs I see with a "one-call" approach is
>> you've got to have really good syntax checking for everything right from
>> the start, or (if you plan to handle primitives one at a time) a really
>> solid roll-back strategy if anything fails or has problems, cleaning up any
>> primitives that might already have been created before the whole call
>> completes.
>>
>>
>>      Agreed. We usually take the approach that newly created objects
>> will be removed if the call rolls back but already existing objects that
>> where referenced before would not be since they already existed before the
>> call.  I don't see much syntax checking just methods that detect if the
>> passed in child objects were new objects(new objects) or referenced( via an
>> ID), or in place new referenced(via a transient id). But just remember this
>> does not affect the multiple primitive operations strategy. The API must be
>> orthogonal to the multi primitive approach.
>>
>>
>>  The alternative is to not do this with primitives... but then I don't
>> see how that's possible either. (And certainly not easy to write tests for:
>>  The great thing about small primitives is their methods tend to be easier
>> to unit test.)
>>
>>
>>      I agree this will add complexity to test scenarios but we braved
>> that as we wanted our users to have a high level api. Don't get me wrong
>> When an interface is low level you should offer primitives not solutions
>> as, for interfaces like Instruction sets for a processor but for higher
>> level level APIs customers need high level solutions (Were providing a rest
>> interface which is pretty high level).  Just remember we plan on exposing
>> this API for our end users in addition to a control panel(UI) so we are
>> only asking that we be allowed to provide a single create call method that
>> will not interfere with the multiple primitive operations. We are committed
>> to the testing scenarios for the single call method as we are the ones that
>> would be most offected by it should it fail so I hope this would alleviate
>> the fears others have about the single call method.
>>
>>
>>>  4. What percentage of your customer base are used to using single-call
>>> functionality, and what percentage are used to manipulating primitives?
>>>
>>>  100% but just like others it is the only way to create a load balancer
>>> in our API.  So this data doesn't mean much.
>>>
>>>     Oh! One other question:
>>>
>>>      5. Should "single-call" stuff work for the lifecycle of a load
>>> balancing service? That is to say, should "delete" functionality also clean
>>> up all primitives associated with the service?
>>>
>>>  How we were thinking was that it would just "detach" the primitives
>>> from the load balancer but keep them available for association with another
>>> load balancer.  A user would only be able to actually delete a primitive if
>>> it went through the root primitive resource (i.e. /pools, /vips).  However,
>>> this is definitely up for debate and there are pros and cons to doing it
>>> both ways.  If the system completely deletes the primitives on the deletion
>>> of the load balancer, then the system has to handle when one of those
>>> primitives is being shared with another load balancer.
>>>
>>
>>      That makes sense--  but I think it could end in disaster for the
>> poor fool who isn't aware of that and makes "deploying load balancing
>> services" part of their continuous integration run. In very little time,
>> they'd have bazillions of abandoned primitives. At the same time, it
>> doesn't make sense to delete shared primitives, lest you horribly break
>> things for service B by nuking service A.
>>
>>
>>      Foremost we believe it would be horrible to delete a primitive that
>> is shared by another loadbalancer." And I'll state now that its an even
>> worse Idea to suggest "on delete cascade" to remove the then broken
>> loadbalancers. We think it is also impolite to delete non referenced
>> objects when deleting a loadbalancer. If a user of the current api deletes
>> a loadbalancer do you for "conveniently" search for unreferenced objects
>> then delete them our of courtesy? No? so why add this expectation just
>> because a single call exists for creation. If you really want a courtesy
>> clean up delete call we can debate this
>> but that would already but just know that that would require a change to
>> the primitive delete call since they are the same url endpoint. And I'm
>> almost certain changing delete behavior is against everyones interests in
>> this. We don't have this problem in our API for our customers as child
>> objects are not generally shared. Meaning every LB is deleting only its
>> copy of the primitives. But we would never ask for this due to the need to
>> support multiple primitives calls based on reference IDs. We believe that
>> if the user didn't explicitly request it we shouldn't do it.  We believe
>> its rude to delete unused child elements as they may be planning to use
>> them in future loadbalancers and will want to reference them by id. The
>> only real way for this to work is add a boolean field in all operations
>> that would allow them to specify delete unused children. But that would
>> require a change to the DELETE calls on all PRIMITIVES as well. several
>> options are apparent to me. Either specify in the spec that deletes do not
>> destroy unused primitives. I would assume this was default behavior as
>> create calls already have a way to specify new objects on the fly but
>> considering how delicate people here are about changing existing methods
>> are we would not suggest adding a boolean to them.
>>
>>      Suggesting any kind of change in general has resulted in somewhat
>> harsh criticism thus far so we are of the approach of trying to minimize
>> our impact on existing features and do not propose adding a boolean to the
>> request some how. If you feel we should we would be definitely be open to
>> discuss this, but we suspect the proper approach is to document that
>> DELETES don't remove unused objects. I would never expect when I work with
>> single create call APIs and most people I work with seem to agree its rude
>> to wipe unused primitives in an API that allows sharing based on id
>> references. But we accept what ever approach is used as we were interested
>> in a Single create call and not a single delete call.
>>
>>
>>   So, going with the principle of least surprise here, it seems to me
>> that most people attempting a delete in a single call are going to want all
>> the non-shared primitives deleted (in a cascading effect) unless they
>> specify that they want the primitives preserved. It would be great if there
>> were a good way to set this as an option somehow (though I know an HTTP
>> DELETE doesn't allow for this kind of flexibility-- maybe something
>> appended to the URI if you want to preserve non-shared primitives?)
>>
>>
>>       Yes something like "?deleteUnusedPrimitives" or better still
>> something more along the lines of search for an optional attribute in the
>> requesting dictionary like
>> {"delete_unshared_objects":true} that we could search for via
>> delete_unused_primitives =
>> req_object.get("delete_unshared_objects",false).  I'd prefer to default to
>> false. We are not against this suggestion if it becomes a requirement.
>>
>>
>> Deleting a primitive (ie. not using single-call) should clearly just
>> delete the primitive. Though, of course, it would be nice to specify (using
>> some flag) that the delete should be ignored if the primitive happens to be
>> shared.
>>
>>
>>      Are you suggesting a separate delete method with a different uri to
>> represent the primitive and another URI for a single delete call case?  If
>> we are committed to this then I would caution that we use the flag mention
>> like in my earlier comment but default to false if the flag is not present
>> in the DELETE request. Ie get the user to explicit request deleting of
>> shared primitives that seems like the only sane thing to do. Initially we
>> were more interested in a single create call but we see no problem
>> supporting a delete with a new flag dictating this.
>>
>>
>>
>> On Thu, Apr 17, 2014 at 2:23 PM, Eichberger, German <
>> german.eichberger at hp.com> wrote:
>>
>>>  Hi Stephen,
>>>
>>>
>>>
>>> 1. Could you please explain what you understand single-call API
>>> functionality to be?
>>>
>>> From my perspective most of our users will likely create load balancers
>>> via  a web interface. Thought not necessary, having a single API call makes
>>> it easier to develop the web interface.
>>>
>>>
>>>
>>> For the “expert” users I envision them to create a load balancer, tweak
>>> with the settings, and when they arrive at the load balancer they need to
>>> automate the creation of it. So if they have to create several objects with
>>> multiple calls in a particular order that is far too complicated and makes
>>> the learning curve very steep from the GUI to the API. Hence, I like being
>>> able to do one call and get a functioning load balancer. I like that aspect
>>> from Jorge’s proposal. On the other hand making a single API call contain
>>> all possible settings might make it too complex for the casual user who
>>> just wants some feature activated the GUI doesn’t provide….
>>>
>>
>>  That makes sense. Are you envisioning having a function in the GUI to
>> "show me the CLI or API command to do this" once a user has ticked all the
>> check-boxes they want and filled in the fields they want?
>>
>>  For our power users-- I could see some of them occasionally updating
>> primitives. Probably the most common API command we see has to do with
>> users who have written their own scaling algorithms which add and remove
>> members from a pool as they see load on their app servers change throughout
>> the day (and spin up / shut down app server clones in response).
>>
>>
>>>
>>>
>>> 2. Could you describe the simplest use case that uses single-call API in
>>> your environment right now?
>>>
>>> Please be very specific--  ideally, a couple examples of specific CLI
>>> commands a user might run, or API (along with specific configuration data)
>>> would be great.
>>>
>>>
>>>
>>>
>>> http://libra.readthedocs.org/en/latest/api/rest/load-balancer.html#create-a-new-load-balancer
>>>
>>>
>>>
>>  Got it. Looks straight-forward.
>>
>>
>>>
>>>
>>>
>>> 5. Should "single-call" stuff work for the lifecycle of a load balancing
>>> service? That is to say, should "delete" functionality also clean up all
>>> primitives associated with the service?
>>>
>>>
>>>
>>> Yes. If a customer doesn’t like a load balancer any longer one call will
>>> remove it. This makes a lot of things easier:
>>>
>>> -          GUI development – one call does it all
>>>
>>> -          Cleanup scripts: If a customer leaves us we just need to run
>>> delete on a list of load balancers – ideally if the API had a call to
>>> delete all load balancers of a specific user/project that would be even
>>> better J
>>>
>>> -          The customer can tear down test/dev/etc. load balancer very
>>> quickly
>>>
>>>
>>>   What do you think of my "conditional cascading delete" idea (ie. nuke
>> everything but shared primitives) above for the usual / least surprise case?
>>
>>
>> Thanks,
>> Stephen
>>
>>
>>
>>  --
>> Stephen Balukoff
>> Blue Box Group, Inc.
>> (800)613-4305 x807
>>  _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Stephen Balukoff
> Blue Box Group, LLC
> (800)613-4305 x807
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140419/b58c6dfb/attachment.html>