Open Stack

Hi, 

I am doing some tests using SWIFT and S3 Python client API. Test is very simple I have multiple file (around 1GB ) in the SWIFT store and  I am accessing 64k data by specifying byte range. The result is a bit surprising and I couldn't really understand the reason. I need some hits to get the proper understanding.  

I am getting significantly low performance with SWIFT python client compare to S3 python Client. I am running 8 clients and with S3 API the successful download requests per second are 240 where as with SWIFT API the successful download requests per second are 157. Similarly 

Client,  S3,  SWIFT
16, 490, 215
24, 709, 238

Each client sends a request to get 64k block. 

Following is how I am using SWIFT python code. 

To make things simple I am using port 80 and simple HTTP requests. 

--------------------------------------------------------------------------------------------
from common.client import ClientException, get_auth, get_container, get_object, get_object_with_range, put_container, put_object, delete_container, delete_objec
t, head_object

def getAuth(self):
        return  get_auth(self.url, self.user, self.passwd)

def get(self, container, objectName, range, dataSize):
        try:    
            res = get_object_with_range(self.credentials[0], self.credentials[1], container, objectName, range, None, dataSize)
        except ClientException:
            res = 'no'
        return res


def get_object_with_range(url, token, container, name, range, http_conn=None,
               resp_chunk_size=None):

    """
    Get an object with the byte range

    :param url: storage URL
    :param token: auth token
    :param container: container name that the object is in
    :param name: object name to get
    :param range: object byte range for example "bytes=0-300" 
    :param http_conn: HTTP connection object (If None, it will create the
                      conn object)
    :param resp_chunk_size: if defined, chunk size of data to read. NOTE: If
                            you specify a resp_chunk_size you must fully read
                            the object's contents before making another
                            request.
    :returns: a tuple of (response headers, the object's contents) The response
              headers will be a dict and all header names will be lowercase.
    :raises ClientException: HTTP GET request failed
    """
    if range == '':
        return 'valid range is required.'
    h = {}
    if http_conn:
        parsed, conn = http_conn
    else:
        parsed, conn = http_connection(url)
    path = '%s/%s/%s' % (parsed.path, quote(container), quote(name))
    h['X-Auth-Token'] = token
    h['Range'] = range
    #conn.request('GET', path, '', {'X-Auth-Token': token, 'Range':'bytes=0-400'})
    conn.request('GET', path, '', h)
    resp = conn.getresponse()
    if resp.status < 200 or resp.status >= 300:
        resp.read()
        raise ClientException('Object GET failed', http_scheme=parsed.scheme,
                http_host=conn.host, http_port=conn.port, http_path=path,
                http_status=resp.status, http_reason=resp.reason)
    if resp_chunk_size:

        def _object_body():
            buf = resp.read(resp_chunk_size)
            while buf:
                yield buf
                buf = resp.read(resp_chunk_size)
        object_body = _object_body()
    else:
        object_body = resp.read()
    resp_headers = {}
    for header, value in resp.getheaders():
        resp_headers[header.lower()] = value
    return resp_headers, object_body



--------------------------------------------------------------------------------------------


Where 
url= http://myserv.uu.se:80/auth/v1.0
dataSize = 64000
range = 'bytes=1-64000'
getAuth return the URL and the token in credentials[0] and credentials[1].

method  get_object_with_range is exactly same as get_object except it handles the range tag as well in the header (highlighted in the red color). 

And since its is simple http request so it shouldn't make much difference.

Before the test I was expecting that both of the clients should be similar or even Swift Client should have a bit better performance as in I am passing the url and the token to the get method. The result I am getting is strange to me. 

Can someone please point out what I am doing wrong ?   

Thanks in advance.

Regards..
Salman.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20120110/5bf6ae12/attachment-0002.html>