Open Stack

Hi,

Is there any way to proceed ahead the following topic?

Best Regards,
Hisashi Osanai

On Friday, August 01, 2014 7:32 PM, Hisashi Osanai wrote:
> I would like to follow this discussion so I picked up points.
> 
> - There are two way to collect info from swift, one is pollster and
>   the other is notification. And we discussed about how to solve the
>   performance degradation of swift_middleware here.
>   pollster:
>    - storage.objects
>    - storage.objects.size
>    - storage.objects.containers
>    - storage.containers.objects
>    - storage.containers.objects.size
>   notification:
>    - storage.objects.incoming.bytes
>    - storage.objects.outgoing.bytes
>    - storage.api.request
> 
> - storage.objects.imcoming.bytes, storage.objects.outgoing.bytes and
>   storage.api.request are handled with swift_middleware because
> ceilometer
>   need to have the info with per-user and per-tenant basis.
> - swift has statsd but there is no per-user and per-tenant related info
>   because to realize this swift has to have keystone-isms into core swift
> code.
> - improves swift_middleware with stopping the 1:1 mapping b/w API calls
> and
>   notifications
> - swift may consume 10s of thousands of event per second and this case
> is fairly
>   unique so far.
> 
> I would like to think this performance problem with the following point
> of view.
> - need to handle 10s of thousands of event per second
> - possibility to lost events (i.e. swift proxy goes down when events queued
> in a swift process)
> 
> With the notification style there are restriction for above points.
> Therefore I change the style
> to get storage.objects.imcoming.bytes, storage.objects.outgoing.bytes
> and
> storage.api.request from notification to pollster.
> Here I met a problem that pointed out by Mr. Merritt, swift has dependency
> with keystone.
> But I prefer to solve this problem rather than a problem for notification
> style. What do you think?
> 
> My rough idea to solve the dependency problem is
> - enable statsd (or similar function) in swift
> - put a middleware in swift proxy
> - this middleware does not have any communication with ceilometer but
>   put a mark to followed middleware or swift proxy
> - store metrics with a tenant and a user by statsd if there is the mark
>   store metrics by statsd if there is no mark
> - Ceilometer (central agent) call APIs to get the metrics
> 
> Is there any way to solve the dependency problem?
> 
> Best Regards,
> Hisashi Osanai