[Openstack-operators] Small openstack (part 2), distributed glance

George Shuklin george.shuklin at gmail.com
Wed Jan 21 10:36:37 UTC 2015

Ok, news so far:

It works like a magic. Nova have option

And I do not need to cheat with endpoint resolving (my initial plan was 
to resolve glance endpoint to with /etc/hosts magic). Normal 
glance-api reply to external clients requests 
(image-create/download/list/etc), and local glance-apis (per compute) 
are used to connect to swift.

Glance registry works in normal mode (only on 'official' api servers).

I don't see any reason why we should centralize all traffic to swift 
through special dedicated servers, investing in fast CPU and 10G links.

With that solution CPU load on glance-api is distributed evenly on all 
compute nodes, and overall snapshot traffic (on ports) was cut down 3 times!

Why I didn't thought about this earlier?

On 01/16/2015 12:20 AM, George Shuklin wrote:
> Hello everyone.
> One more thing in the light of small openstack.
> I really dislike tripple network load caused by current glance 
> snapshot operations. When compute do snapshot, it playing with files 
> locally, than it sends them to glance-api, and (if glance API is 
> linked to swift), glance sends them to swift. Basically, for each 
> 100Gb disk there is 300Gb on network operations. It is specially 
> painful for glance-api, which need to get more CPU and network 
> bandwidth than we want to spend on it.
> So idea: put glance-api on each compute node without cache.
> To help compute to go to the proper glance, endpoint points to fqdn, 
> and on each compute that fqdn is pointing to localhost (where 
> glance-api is live). Plus normal glance-api on API/controller node to 
> serve dashboard/api clients.
> I didn't test it yet.
> Any ideas on possible problems/bottlenecks? And how many 
> glance-registry I need for this?

More information about the OpenStack-operators mailing list