[openstack-dev] [Fuel] Diagnostic snapshot generation is broken due to lack of disk space

Bartlomiej Piotrowski bpiotrowski at mirantis.com
Thu Jan 14 10:22:33 UTC 2016


Igor,

I took a glance on Maciej's patch and it adds a switch to tar command to
make it follow symbolic links, so it looks good to me.

Bartłomiej

On Thu, Jan 14, 2016 at 10:39 AM, Igor Kalnitsky <ikalnitsky at mirantis.com>
wrote:

> Hey Maceij -
>
> > About hardlinks - wouldn't it be better to use symlinks?
> > This way we don't occupy more space than necessary
>
> AFAIK, hardlinks won't occupy much space. They are the links, after all. :)
>
> As for symlinks, I'm afraid shotgun (and fabric underneath) won't
> resolve them and links are get to snapshot As Is. That means if there
> will be no content in the snapshot they are pointing to, they are
> simply useless. Needs to be checked, though.
>
> - Igor
>
> On Thu, Jan 14, 2016 at 10:31 AM, Maciej Kwiek <mkwiek at mirantis.com>
> wrote:
> > Thanks for your insight guys!
> >
> > I agree with Oleg, I will see what I can do to make this work this way.
> >
> > About hardlinks - wouldn't it be better to use symlinks? This way we
> don't
> > occupy more space than necessary, and we can link to files and
> directories
> > that are in other block device than /var. Please see [1] review for a
> > proposed change that introduces symlinks.
> >
> > This doesn't really give us much right now, because most of the logs are
> > fetched from master node via ssh due to shotgun being run in mcollective
> > container, but it's something! When we remove containers, this will prove
> > more useful.
> >
> > Regards,
> > Maciej Kwiek
> >
> > [1] https://review.openstack.org/#/c/266964/
> >
> > On Tue, Jan 12, 2016 at 1:51 PM, Oleg Gelbukh <ogelbukh at mirantis.com>
> wrote:
> >>
> >> I think we need to find a way to:
> >>
> >> 1) verify the size of snapshot without actually making it and compare to
> >> the available disk space beforehand.
> >> 2) refuse to create snapshot if space is insufficient and notify user
> >> (otherwise it breaks Admin node as we have seen)
> >> 3) provide a way to prioritize elements of the snapshot and exclude them
> >> based on the priorities or user choice.
> >>
> >> This will allow for better and safer UX with the snapshot.
> >>
> >> --
> >> Best regards,
> >> Oleg Gelbukh
> >>
> >> On Tue, Jan 12, 2016 at 1:47 PM, Maciej Kwiek <mkwiek at mirantis.com>
> wrote:
> >>>
> >>> Hi!
> >>>
> >>> I need some advice on how to tackle this issue. There is a bug [1]
> >>> describing the problem with creating a diagnostic snapshot. The issue
> is
> >>> that /var/log has 100GB available, while /var (where diagnostic
> snapshot is
> >>> being generated - /var/www/nailgun/dump/fuel-snapshot according to
> [2]) has
> >>> 10GB available, so dumping the logs can be an issue when logs size
> exceed
> >>> free space in /var.
> >>>
> >>> There are several things we could do, but I am unsure on which course
> to
> >>> take. Should we
> >>> a) Allocate more disk space for /var/www (or for whole /var)?
> >>> b) Make the snapshot location share the diskspace of /var/log?
> >>> c) Something else? What?
> >>>
> >>> Please share your thoughts on this.
> >>>
> >>> Cheers,
> >>> Maciej Kwiek
> >>>
> >>> [1] https://bugs.launchpad.net/fuel/+bug/1529182
> >>> [2]
> >>>
> https://github.com/openstack/fuel-web/blob/2855a9ba925c146b4802ab3cd2185f1dce2d8a6a/nailgun/nailgun/settings.yaml#L717
> >>>
> >>>
> >>>
> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160114/f4e2977d/attachment.html>


More information about the OpenStack-dev mailing list