[openstack-dev] how to write python object in openstack swift

Jialin Liu jalnliu at lbl.gov
Thu Oct 5 18:24:43 UTC 2017


Thank you John,

I think I figured out the way. My case is a little bit rare as I'm dealing
with the HDF5 file format, I can not think of any way to retrieve the
entire in memory file as a single python object and then serialize it.
What I did is to use the hdf5.get_file_image function to get a memory image
of the file, and then use, io.ByteIO(image) to make it a file-like object.

So far, it works well, but I believe it is not the best way in terms of
performance.

Best,
Jialin

On Thu, Oct 5, 2017 at 8:25 AM, John Dickinson <me at not.mn> wrote:

> If you've got an arbitrary object in Python, you'll need to serialize it
> to a file-like object. You could keep it in memory and use a StringIO
> type, or you could serialize it to disk and open() it like any other file.
>
> Ultimately, Swift is storing arbitrary bytes and doesn't care what they
> are. You, as the Swift client (i.e. API user), need to dump those bytes on
> the network to send them to Swift. As long as you're transforming your
> Python object[s] in some regular way that makes sense in your application,
> it doesn't matter what bytes you send to Swift.
>
> --John
>
> On 5 Oct 2017, at 8:07, Jialin Liu wrote:
>
> Hi,
> It seems to me that openstack swift only supports file upload/download, is
> it possible to put a python object to swift store?
> The doc says we could use file-like object, e.g., StringIO, but this is
> very limited. I'd like to write a numpy array or other python object into
> the swift store, can anybody tell me the solution?
>
> Best,
> Jialin
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171005/3b7d244e/attachment.html>


More information about the OpenStack-dev mailing list