[Openstack] [Swift] XFS extended attribute performance

Shrinand Javadekar shrinand at maginatics.com
Thu Mar 26 20:32:14 UTC 2015


Thanks for the reply Sam. Some comments below.

> Maybe(TM). It depends on your workload. When you make inodes bigger, fewer
> of them fit in the kernel's buffer cache, possibly resulting in more work.
> On the other hand, when you make them smaller, then you always get xattrs
> spilled to extents.
>
> Also, it's affected by what your users do. Are your users storing lots of
> metadata (X-Object-Meta-*) on their objects? Are you using middleware that
> stores lots of sysmeta (X-Object-Sysmeta-*) on your objects? Are you running
> SELinux, which uses lots of xattrs for its own purposes?

Good point. Our users currently do not use any other metadata nor are
we using SELinux. In fact, the object name and container name are of a
fixed size. So I could find out precisely how long the extended
attributes will be and size the inodes accordingly.

>
> Further, I think the benchmarking script is too naive to really get at the
> key improvement here: avoiding seeks. If the xattrs live in the inode, then
> we get them essentially for free when we open() the file. If the xattrs live
> in extents, then at least one random IO is required to fetch them. The
> improvement shown here is on the order of 30 microseconds, while an extra
> disk seek is more like 10-20 *milliseconds* (on spinning platters). If you
> really want to measure the performance gain, you'll need to drop the buffer
> cache each time and fsync() when you're done.

Is there some sample code for how to drop the buffer cache in python.
Presumably this will be for each file and not the entire buffer cache.

The tests I ran were in a VM. I can run it on hardware with spinning
disks underneath to get more accurate numbers.

-Shri

>
>
> [1]
> ============
> #!/usr/bin/python
>
> import xattr
> prefix = "/srv/node/r1/"
>
> for i in range (100000):
>    fd = open(prefix + "file-" + str(i), 'w')
>    fd.write("Some text")
>    xattr.setxattr(fd, "user.swift.metadata",
> "U.Content-Lengthq.U.166U.nameq.U'/AUTH_admin/container-test/testxattr.pyq.U.X-Object-Meta-MtimeU.1426996406.379914q.U.ETagq.U
> bab3674eed1a8725793f2d2de21f50a9q.U.X-Timestampq.U.1426996448.35449U.Content-Typeq.U.text/x-pythonq.u.")
>    fd.flush()
>    fd.close()
>
> ============
>
>
> [2]
> http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure/tmp/en-US/html/Extended_Attributes.html
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




More information about the Openstack mailing list