[openstack-dev] [Fuel] Diagnostic snapshot generation is broken due to lack of disk space

Maciej Kwiek mkwiek at mirantis.com
Thu Jan 14 08:31:04 UTC 2016


Thanks for your insight guys!

I agree with Oleg, I will see what I can do to make this work this way.

About hardlinks - wouldn't it be better to use symlinks? This way we don't
occupy more space than necessary, and we can link to files and directories
that are in other block device than /var. Please see [1] review for a
proposed change that introduces symlinks.

This doesn't really give us much right now, because most of the logs are
fetched from master node via ssh due to shotgun being run in mcollective
container, but it's something! When we remove containers, this will prove
more useful.

Regards,
Maciej Kwiek

[1] https://review.openstack.org/#/c/266964/

On Tue, Jan 12, 2016 at 1:51 PM, Oleg Gelbukh <ogelbukh at mirantis.com> wrote:

> I think we need to find a way to:
>
> 1) verify the size of snapshot without actually making it and compare to
> the available disk space beforehand.
> 2) refuse to create snapshot if space is insufficient and notify user
> (otherwise it breaks Admin node as we have seen)
> 3) provide a way to prioritize elements of the snapshot and exclude them
> based on the priorities or user choice.
>
> This will allow for better and safer UX with the snapshot.
>
> --
> Best regards,
> Oleg Gelbukh
>
> On Tue, Jan 12, 2016 at 1:47 PM, Maciej Kwiek <mkwiek at mirantis.com> wrote:
>
>> Hi!
>>
>> I need some advice on how to tackle this issue. There is a bug [1]
>> describing the problem with creating a diagnostic snapshot. The issue is
>> that /var/log has 100GB available, while /var (where diagnostic snapshot is
>> being generated - /var/www/nailgun/dump/fuel-snapshot according to [2]) has
>> 10GB available, so dumping the logs can be an issue when logs size exceed
>> free space in /var.
>>
>> There are several things we could do, but I am unsure on which course to
>> take. Should we
>> a) Allocate more disk space for /var/www (or for whole /var)?
>> b) Make the snapshot location share the diskspace of /var/log?
>> c) Something else? What?
>>
>> Please share your thoughts on this.
>>
>> Cheers,
>> Maciej Kwiek
>>
>> [1] https://bugs.launchpad.net/fuel/+bug/1529182
>> [2]
>> https://github.com/openstack/fuel-web/blob/2855a9ba925c146b4802ab3cd2185f1dce2d8a6a/nailgun/nailgun/settings.yaml#L717
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160114/1537242e/attachment.html>


More information about the OpenStack-dev mailing list