Open Stack

>From my some testing I did a couple of months ago, we decided to move to
XFS to avoid the issue

I was poking around with after my file system inadvertently filled and
> found that in ex3/4 all of the inodes in the file system have to be zeroed
> prior to mkfs completing (unless the kernel is above 2.6.37 in which case
> the inode table is lazily initialized (in the background) after the first
> mount]). Initializing the inode table with the current default
> bytes-per-inode ratio of 16Ki and 256Bi (inode-size) per inode results in
> 16GiB of inodes per 1TiB of volume.
>


There appear to be two viable options to reducing the format time.
> 1. We can increase the bytes-per-inode value during mkfs, at the cost of
> total files the file system can store. Moving this so that 1GiB per TiB are
> initialized (setting bytes-per-inode to 256Ki) would result in 4Mi inodes
> per TiB of disk (down from 64Mi)
> 2. We can format all non-os partitions as XFS. Which does a lot less
> upfront allocation. By my observation, it appears to initialize around
> 700MiB per TiB


The performance increase you saw is likely to the deferred updates that
will still occur on the first mount of the device (but in the background).
However this wont occur for RHEL4,5,6 as the kernel won't do it so they
will still sit there and require the pre-allocation. You can short the
inodes by making them larger as I initially tested which could be OK since
most images are going many MByte and usualy many GByte

Just my feedback on what you were seeing. moving to ext4 preferred over ext3


On Thu, Dec 19, 2013 at 12:30 PM, Sean Dague <sean at dague.net> wrote:

> On 12/19/2013 03:21 PM, Robert Collins wrote:
> > The default ephemeral filesystem in Nova is ext3 (for Linux). However
> > ext3 is IMNSHO a pretty poor choice given ext4's existence. I can
> > totally accept that other fs's like xfs might be contentious - but is
> > there any reason not to make ext4 the default?
> >
> > I'm not aware of any distro that doesn't have ext4 support - even RHEL
> > defaults to ext4 in RHEL5.
> >
> > The reason I'm raising this is that making a 1TB ext3 ephemeral volume
> > does (way) over 5GB of writes due to zeroing all the inode tables, but
> > an ext4 one does less than 1% of the IO - 14m vs 7seconds in my brief
> > testing. (We were investigating why baremetal deploys were slow :)).
> >
> > -Rob
>
> Seems like a fine change to me. I assume that's all just historical
> artifact.
>
>         -Sean
>
> --
> Sean Dague
> Samsung Research America
> sean at dague.net / sean.dague at samsung.com
> http://dague.net
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
If google has done it, Google did it right!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131219/a960c18b/attachment.html>