[Openstack-operators] /var/lib/nova/instances fs filled up corrupting my Linux instances

Joe Topjian joe.topjian at cybera.ca
Wed Mar 13 21:42:18 UTC 2013


It would, yes, but I think your caveat trumps that idea. Having x nodes be
able to work with a shared _base directory is great for saving space and
centrally using images. As an example, one of my OpenStack's _base
directory is 650gb in size. It's currently shared via NFS. If it was not
shared or used a _base_$host scheme, that would be 650gb per compute node.
10 nodes and you're already at 6.5TB.

There was a recent
discussion<http://www.gossamer-threads.com/lists/openstack/dev/24749>
that
is kind of related (supplemental?). One idea I saw in that thread was to
combine Glance and _base. That would be extremely dangerous since removal /
corruption of a _base image would affect Glance as well.


On Wed, Mar 13, 2013 at 4:27 PM, Nathanael Burton <
nathanael.i.burton at gmail.com> wrote:

> If using NFS, if base_dir_name was set such that each host used a separate
> directory like base_dir_name=_base_$host, wouldn't the image cache manager
> perform the desired behavior? (albeit worst case using up to #hosts times
> as much storage in the cache).
>
> Thanks,
>
> Nate
>
>
> On Wed, Mar 13, 2013 at 4:58 PM, Lorin Hochstein <lorin at nimbisservices.com
> > wrote:
>
>> Michael:
>>
>> Is this related to the remove_unused_base_images flag?
>>
>> cf
>> https://review.openstack.org/#/c/22040/
>> https://bugs.launchpad.net/nova/+bug/1126375
>>
>> If so, I think this is a strong argument for changing the default.
>>
>>
>>
>> On Wed, Mar 13, 2013 at 1:47 PM, Michael Still <mikal at stillhq.com> wrote:
>>
>>> On Wed, Mar 13, 2013 at 10:38 AM, Conor Glynn <cglynn at sportingindex.com>
>>> wrote:
>>> > by the way, yes both compute nodes were using that shared NFS
>>> storage...
>>>
>>> Ok, I think that is likely to have caused base images to go missing.
>>> If that's the case, you should be able to rescue the images from
>>> snapshots if you have that enabled on your netapp device. If you don't
>>> have snapshots or backups, then I think we could tool up a refect from
>>> glance, although I suspect that might involve writing code (and
>>> therefore delay).
>>>
>>> Please do let me know how you go with recovering.
>>>
>>> Michael
>>>
>>> _______________________________________________
>>> OpenStack-operators mailing list
>>> OpenStack-operators at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>
>>
>>
>>
>> --
>> Lorin Hochstein
>> Lead Architect - Cloud Services
>> Nimbis Services, Inc.
>> www.nimbisservices.com
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>


-- 
Joe Topjian
Systems Administrator
Cybera Inc.

www.cybera.ca

Cybera is a not-for-profit organization that works to spur and support
innovation, for the economic benefit of Alberta, through the use
of cyberinfrastructure.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20130313/25e7ea3e/attachment.html>


More information about the OpenStack-operators mailing list