Open Stack

Hey Pete,  we were thinking specifically about ports usage at the storage nodes.  For every incoming object there's an outgoing TCP connection for the container update.  If the workload is somewhat bursty, and if object creation activity has container temporal locality, then batching increases the likelihood that multiple requests would get queued up for a single container server and then could be sent at once (using some kind of hold time mechanism to batch them up).  

I suppose one consideration though is hardening the operations as they're being batched.  I realized right now If they're in the async file on the storage node or in the pending file at the container server, they're effectively power cycle safe.

Thanks,

Stephen

-----Original Message-----
From: Pete Zaitcev [mailto:zaitcev at redhat.com] 
Sent: Monday, July 14, 2014 4:38 PM
To: Luse, Paul E
Cc: John Dickinson; Shrinand Javadekar; Blinick, Stephen L; openstack at lists.openstack.org
Subject: Re: [Openstack] [Swift] Running out of ports or fds?

On Fri, 11 Jul 2014 20:48:54 +0000
"Luse, Paul E" <paul.e.luse at intel.com> wrote:

> So Stephen (presented the perf work at the last hackathon) had an idea 
> about maybe saving on some connections via batching up groups of 
> container updates before making the connection to the container server 
> in async_update().

I don't see how improving container updates would help proxies, unless you run PACO.

-- Pete