Open Stack

The Unified Limits session in Boston was interesting and fruitful.
Here's the current summary of where I think we stand.

* Current Status

** Conceptual Spec Landed


https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/unified-limits.html

** Keystone detailed interface spec - UP FOR REVIEW

   https://review.openstack.org/#/c/455709/
   Definitely needs lots of looking at, this is very fresh and raw

** Quota Models doc - UP FOR DISCUSSION

   https://review.openstack.org/#/c/441203/ - This is far less than
   review, but has been useful to drive discussion. Need some user
   contributed models to make further progress.

* Boston Forum
  https://etherpad.openstack.org/p/BOS-forum-quotas

  The Boston Forum sessions was a double length session. The first
  half was spent going through the basics of the conceptual spec, then
  a lot of talk about models.

  Some key points of understanding.

  - Limits amounts are only integers at this point. If you want to
    work in percentages, that's the role of an external tool to
    rebalance your limits in your cluster.
  - Strict validation means that keystone will only store limits for
    services keystone understands
  - None of this is rate limiting, it's absolute usage, not usage over
    time.
  - User limits should go away, everything really needs to be project
    level owned.

  The models conversation was interesting, and goes about how it
  always goes. It turns out these are complicated enough algorithms
  that you can't do them in your head, even though people think they
  can. Especially the moment you require the example to work in N>=3
  levels of hierarchy. The pre-made diagrams helped push the
  conversation a bit, and a couple of ad-hoc examples helped as well.

  It does mean we need to prepare for "why can't you just", by having
  a template and some standard pathological use cases that every
  example runs through. A quota models worksheet.

* Next Steps

  Most of these come out of the Boston Forum (notes:
  https://etherpad.openstack.org/p/BOS-forum-quotas )


** TODO CERN ideal ruleset

   Get someone from the CERN team to write down what their ideal
   ruleset looks like, so it can be turned into a worked example

** TODO Chet ruleset

   Get Chet to write down his ideal ruleset here, so it can be turned
   into a worked example.

** TODO Jay's usage computing algorithm

   Jay suggested there is algorithm that doesn't need to take into
   account the tree structure when computing usage. Get the algorithm,
   "unit test" with examples to validate whether that is viable or not.

* Unresolved Issues

** Who is going to work on Unified Limits?

   There is a ton of interest in getting this work done, recent
   organizational priority changes and cuts mean this entire effort is
   pretty high risk of failure largely due to lack of people having
   time to work on it.

** Can we actually put together an interface in Keystone without having
a complex model worked out?

   Good point from Colleen. As we go through these different examples
   (Flatland, and various hierarchies) it does seem like every one
   might need different details to compute usage correctly. Perhaps
   the best we can get is tagging the return datastructures with the
   model name so that we can understand a datastructure is only good
   enough to describe a single model.

** How much load will this add to Keystone?

   This moves a set of API calls from services back to Keystone, and
   is looking to enable more complicated functions, which will mean
   more work. How does this change Keystone scaling?

   NOTE: this means some early optimizations are going to need to be
   put into place to address load concerns.

** Templated Catalog vs. Strict validation of service types

   Nectar and others don't add to keystone catalog, they use a
   templated catalog. That effectively means that Keystone doesn't
   actually know anything about service types. :(

** When we talk about project root, how far up are we talking?

   In my mind a hierarchical structure isn't going to be single
   rooted. However, I think the conversation about the magic domain 0
   made it clear that we may need some mechanism where a project is
   annotated to be a root (and thus ignores usage further up). This
   remains one of those areas where I think the Inside Keystone
   Internals vs. Outside view is still hitting communication gaps.

-- 
Sean Dague
http://dague.net