Open Stack

Hi,

I am using puppet-swift to deploy a swift multi node cluster (Icehouse), 
following the setup in supplied tests/site.pp. I am running into two 
issues that seem to be related to the subject above:

1/ Errors when the storage replication services try to start before the 
ring files exist. e.g:

Error: Could not start Service[swift-object-replicator]: Execution of 
'/sbin/start swift-object-replicator' returned 1: start: Job failed to start
Wrapped exception:
Execution of '/sbin/start swift-object-replicator' returned 1: start: 
Job failed to start
Error: 
/Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Service[swift-object-replicator]/ensure: 
change from stopped to running failed: Could not start 
Service[swift-object-replicator]: Execution of '/sbin/start 
swift-object-replicator' returned 1: start: Job failed to start

Now these will be fixed the *next* time I do a puppet run (provided I've 
performed a run on the appropriate proxy/ringmaster). However the 
failing services make scripted testing difficult as we have to put in 
logic to the effect "don't worry about errors the 1st time".

2/ Container and object stats not updated without full restart of services

This one is a bit more subtle - everything works but 'swift stat' always 
shows zero objects and bytes for every container. The only way to fix 
this is to stop and start all services on each storage node.

Again this complicates scripted builds as there is the need to go and 
stop + start all the swift storage services! Not to mention an extra 
little quirk for ops to remember at zero dark 30 oclock...

I've made a patch that prevents these services starting until the ring 
files exist (actually for now it just checks the object ring) - see 
attached.

Now while I'm not entirely sure that this is the best way to solve the 
issue (custom fact that changes the service start flag)...I *do* think 
that making the storage services await the ring existence *is* a needed 
change, so any thoughts on better ways to do this are appreciated.

Also note that this change *does* require one more puppet run on the 
storage nodes:
- one to create the storage servers config and drives
- one to get the ring from the proxy/ringmaster
- one to start the services

Regards

Mark
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Make-storage-service-startup-wait-for-the-ring.patch
Type: text/x-patch
Size: 6654 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150710/c6f68a41/attachment.bin>