<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi,<br>
<br>
using symlinks is a bit dangerous, here is a quote from the man you
mentioned [0]:<br>
<br>
>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<span style="color: rgb(0, 0, 0); font-family: 'Times New Roman';
font-size: medium; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 1;
word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline
!important; float: none; background-color: rgb(255, 255, 255);">The<span
class="Apple-converted-space"> </span></span><samp style="color:
rgb(0, 0, 0); font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 1;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(255, 255, 255);">`--dereference'</samp><span
style="color: rgb(0, 0, 0); font-family: 'Times New Roman';
font-size: medium; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 1;
word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline
!important; float: none; background-color: rgb(255, 255, 255);"><span
class="Apple-converted-space"> </span>option is unsafe if an
untrusted user can modify directories while<span
class="Apple-converted-space"> </span></span><code style="color:
rgb(0, 0, 0); font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 1;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(255, 255, 255);">tar</code><span
style="color: rgb(0, 0, 0); font-family: 'Times New Roman';
font-size: medium; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 1;
word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline
!important; float: none; background-color: rgb(255, 255, 255);"><span
class="Apple-converted-space"> </span>is running.<br>
</span><br>
Hard links usage is much safer, because you can't use them for
directories. But at the same time implementation in shotgun would be
more complicated than with symlinks.<br>
<br>
Anyway, in order to determine what linking to use we need to decide
where (/var/log or another partition) diagnostic snapshot will be
stored. <br>
<br>
p.s.<br>
<pre wrap="">>This doesn't really give us much right now, because most of the logs are fetched from master node via ssh due to shotgun being run in mcollective container
</pre>
AFAIK '/var/log/docker-logs/' is available from mcollective
container and mounted to /var/log/:<br>
<br>
[root@fuel-lab-cz5557 ~]# dockerctl shell mcollective mount -l |
grep os-varlog<br>
/dev/mapper/os-varlog on /var/log type ext4
(rw,relatime,stripe=128,data=ordered)<br>
<br>
From my experience '/var/log/docker-logs/remote' folder is most '
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<span style="color: rgb(34, 34, 34); font-family: arial, sans-serif;
font-size: 16px; font-style: normal; font-variant: normal;
font-weight: normal; letter-spacing: normal; line-height: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: 1;
word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline
!important; float: none; background-color: rgb(255, 255, 255);">heavy</span>'
thing in snapshot.<br>
<br>
[0]
<a class="moz-txt-link-freetext" href="http://www.gnu.org/software/tar/manual/html_node/dereference.html">http://www.gnu.org/software/tar/manual/html_node/dereference.html</a><br>
<br>
Thanks!<br>
<br>
<div class="moz-cite-prefix">On 14.01.16 13:00, Igor Kalnitsky
wrote:<br>
</div>
<blockquote
cite="mid:CACo6NWBWEY0c=XvNL_UyeQ77iiqUL=iHv4FB4jAdM-ghbbXh2w@mail.gmail.com"
type="cite">
<blockquote type="cite">
<pre wrap="">I took a glance on Maciej's patch and it adds a switch to tar command
to make it follow symbolic links
</pre>
</blockquote>
<pre wrap="">
Yeah, that should work. Except one thing - we previously had fqdn ->
ipaddr links in snapshots. So now they will be resolved into full
copy?
</pre>
<blockquote type="cite">
<pre wrap="">I meant that symlinks also give us the benefit of not using additional
space (just as hardlinks do) while being able to link to files from
different filesystems.
</pre>
</blockquote>
<pre wrap="">
I'm sorry, I got you wrong. :)
- Igor
On Thu, Jan 14, 2016 at 12:34 PM, Maciej Kwiek <a class="moz-txt-link-rfc2396E" href="mailto:mkwiek@mirantis.com"><mkwiek@mirantis.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Igor,
I meant that symlinks also give us the benefit of not using additional space
(just as hardlinks do) while being able to link to files from different
filesystems.
Also, as Barłomiej pointed out the `h` switch for tar should do the trick
[1].
Cheers,
Maciej
[1] <a class="moz-txt-link-freetext" href="http://www.gnu.org/software/tar/manual/html_node/dereference.html">http://www.gnu.org/software/tar/manual/html_node/dereference.html</a>
On Thu, Jan 14, 2016 at 11:22 AM, Bartlomiej Piotrowski
<a class="moz-txt-link-rfc2396E" href="mailto:bpiotrowski@mirantis.com"><bpiotrowski@mirantis.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
Igor,
I took a glance on Maciej's patch and it adds a switch to tar command to
make it follow symbolic links, so it looks good to me.
Bartłomiej
On Thu, Jan 14, 2016 at 10:39 AM, Igor Kalnitsky <a class="moz-txt-link-rfc2396E" href="mailto:ikalnitsky@mirantis.com"><ikalnitsky@mirantis.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
Hey Maceij -
</pre>
<blockquote type="cite">
<pre wrap="">About hardlinks - wouldn't it be better to use symlinks?
This way we don't occupy more space than necessary
</pre>
</blockquote>
<pre wrap="">
AFAIK, hardlinks won't occupy much space. They are the links, after all.
:)
As for symlinks, I'm afraid shotgun (and fabric underneath) won't
resolve them and links are get to snapshot As Is. That means if there
will be no content in the snapshot they are pointing to, they are
simply useless. Needs to be checked, though.
- Igor
On Thu, Jan 14, 2016 at 10:31 AM, Maciej Kwiek <a class="moz-txt-link-rfc2396E" href="mailto:mkwiek@mirantis.com"><mkwiek@mirantis.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Thanks for your insight guys!
I agree with Oleg, I will see what I can do to make this work this way.
About hardlinks - wouldn't it be better to use symlinks? This way we
don't
occupy more space than necessary, and we can link to files and
directories
that are in other block device than /var. Please see [1] review for a
proposed change that introduces symlinks.
This doesn't really give us much right now, because most of the logs
are
fetched from master node via ssh due to shotgun being run in
mcollective
container, but it's something! When we remove containers, this will
prove
more useful.
Regards,
Maciej Kwiek
[1] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/266964/">https://review.openstack.org/#/c/266964/</a>
On Tue, Jan 12, 2016 at 1:51 PM, Oleg Gelbukh <a class="moz-txt-link-rfc2396E" href="mailto:ogelbukh@mirantis.com"><ogelbukh@mirantis.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
I think we need to find a way to:
1) verify the size of snapshot without actually making it and compare
to
the available disk space beforehand.
2) refuse to create snapshot if space is insufficient and notify user
(otherwise it breaks Admin node as we have seen)
3) provide a way to prioritize elements of the snapshot and exclude
them
based on the priorities or user choice.
This will allow for better and safer UX with the snapshot.
--
Best regards,
Oleg Gelbukh
On Tue, Jan 12, 2016 at 1:47 PM, Maciej Kwiek <a class="moz-txt-link-rfc2396E" href="mailto:mkwiek@mirantis.com"><mkwiek@mirantis.com></a>
wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
Hi!
I need some advice on how to tackle this issue. There is a bug [1]
describing the problem with creating a diagnostic snapshot. The issue
is
that /var/log has 100GB available, while /var (where diagnostic
snapshot is
being generated - /var/www/nailgun/dump/fuel-snapshot according to
[2]) has
10GB available, so dumping the logs can be an issue when logs size
exceed
free space in /var.
There are several things we could do, but I am unsure on which course
to
take. Should we
a) Allocate more disk space for /var/www (or for whole /var)?
b) Make the snapshot location share the diskspace of /var/log?
c) Something else? What?
Please share your thoughts on this.
Cheers,
Maciej Kwiek
[1] <a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/fuel/+bug/1529182">https://bugs.launchpad.net/fuel/+bug/1529182</a>
[2]
<a class="moz-txt-link-freetext" href="https://github.com/openstack/fuel-web/blob/2855a9ba925c146b4802ab3cd2185f1dce2d8a6a/nailgun/nailgun/settings.yaml#L717">https://github.com/openstack/fuel-web/blob/2855a9ba925c146b4802ab3cd2185f1dce2d8a6a/nailgun/nailgun/settings.yaml#L717</a>
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<pre wrap="">
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Artem Panchenko
QA Engineer</pre>
</body>
</html>