[Openstack] Distributed rate-limiting
John Dickinson
me at not.mn
Sat Mar 17 16:57:26 UTC 2012
If you're looking for wsgi rate limiting, take a look at swift's rate limiting middleware. It's distributed, simple (uses memcached to keep track of what's going on), supports multiple tiers of rate limiting (with interpolation between them), and white/black lists.
While swift's rate limiting is tuned for swift, its technique could be pretty simply adapted to other applications. Swift's rate limiting is swift-specific in what it checks to rate limit on. Those checks should be able to be adapted to other use cases.
source: https://github.com/openstack/swift/blob/master/swift/common/middleware/ratelimit.py
docs: https://github.com/openstack/swift/blob/master/doc/source/ratelimit.rst
sample config: https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L118
--John
On Mar 16, 2012, at 4:45 PM, Kevin L. Mitchell wrote:
> Howdy, folks. I've been working on a replacement for nova's
> rate-limiting middleware that will handle the multiple-node case, and
> I've developed a fairly generic rate-limiting package, along with a
> second package that adapts it to nova. (This means you could also use
> this rate-limiting setup with, say, glance, or with any other project
> that uses Python middleware.) Here is some information:
>
> * Turnstile
> Turnstile is a piece of WSGI middleware that performs true distributed
> rate-limiting. System administrators can run an API on multiple
> nodes, then place this middleware in the pipeline prior to the
> application. Turnstile uses a Redis database to track the rate at
> which users are hitting the API, and can then apply configured rate
> limits, even if each request was made against a different API node.
>
> - https://github.com/klmitch/turnstile
> - http://pypi.python.org/pypi/turnstile
>
> * nova_limits
> This package provides the ``nova_limits`` Python module, which
> contains the ``nova_preprocess()`` preprocessor, the
> ``NovaClassLimit`` limit class, and the ``NovaTurnstileMiddleware``
> replacement middleware class, all for use with Turnstile. These
> pieces work together to provide class-based rate limiting integration
> with nova.
>
> - https://github.com/klmitch/nova_limits
> - http://pypi.python.org/pypi/nova_limits
>
> Both packages should be fairly well documented (start with README.rst),
> and please feel free to log issues or make pull requests.
> --
> Kevin L. Mitchell <kevin.mitchell at rackspace.com>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4329 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120317/236f938b/attachment.bin>
More information about the Openstack
mailing list