[openstack-dev] [ceilometer] [swift] Improving ceilometer.objectstore.swift_middleware
osanai.hisashi at jp.fujitsu.com
Fri Aug 8 11:30:24 UTC 2014
Is there any way to proceed ahead the following topic?
On Friday, August 01, 2014 7:32 PM, Hisashi Osanai wrote:
> I would like to follow this discussion so I picked up points.
> - There are two way to collect info from swift, one is pollster and
> the other is notification. And we discussed about how to solve the
> performance degradation of swift_middleware here.
> - storage.objects
> - storage.objects.size
> - storage.objects.containers
> - storage.containers.objects
> - storage.containers.objects.size
> - storage.objects.incoming.bytes
> - storage.objects.outgoing.bytes
> - storage.api.request
> - storage.objects.imcoming.bytes, storage.objects.outgoing.bytes and
> storage.api.request are handled with swift_middleware because
> need to have the info with per-user and per-tenant basis.
> - swift has statsd but there is no per-user and per-tenant related info
> because to realize this swift has to have keystone-isms into core swift
> - improves swift_middleware with stopping the 1:1 mapping b/w API calls
> - swift may consume 10s of thousands of event per second and this case
> is fairly
> unique so far.
> I would like to think this performance problem with the following point
> of view.
> - need to handle 10s of thousands of event per second
> - possibility to lost events (i.e. swift proxy goes down when events queued
> in a swift process)
> With the notification style there are restriction for above points.
> Therefore I change the style
> to get storage.objects.imcoming.bytes, storage.objects.outgoing.bytes
> storage.api.request from notification to pollster.
> Here I met a problem that pointed out by Mr. Merritt, swift has dependency
> with keystone.
> But I prefer to solve this problem rather than a problem for notification
> style. What do you think?
> My rough idea to solve the dependency problem is
> - enable statsd (or similar function) in swift
> - put a middleware in swift proxy
> - this middleware does not have any communication with ceilometer but
> put a mark to followed middleware or swift proxy
> - store metrics with a tenant and a user by statsd if there is the mark
> store metrics by statsd if there is no mark
> - Ceilometer (central agent) call APIs to get the metrics
> Is there any way to solve the dependency problem?
> Best Regards,
> Hisashi Osanai
More information about the OpenStack-dev