<div dir="ltr">Making diagnostic snapshot for a particular environment is a good idea. But the issue is still there. <div><br></div><div>We often have the situation when user actually doesn't care of old logs at all. He downloads ISO, installs it and tries various installation options (Ubuntu, Centos, HA, Ceph, etc.). Sooner or later his hard drive is full and he even can not make the diagnostic snapshot. Dealing with that stuff about taking care of available free space inside shotgun seems to be not a good idea. But we still need to address this. The easiest way to do that is to delete old log directories (logrotate or nailgun itself). In this case the issue at least will be rather seldom. But, of course, the right way is to have a kind of monitoring system on the master node and notify user when disk is full or launch a kind of cleaning task. <div>
<br></div><div>Ok, right place where we should deal with that stuff about removing old logs is logrotate. Currently it just moves files like this</div><div>/var/log/remote/<a href="http://old-node.example.com/some.log">old-node.example.com/some.log</a> -> /var/log/remote/<a href="http://old-node.example.com/some.log.1.gz">old-node.example.com/some.log.1.gz</a>. But what it actually should do is to remove the whole directories which are related to nonexistent nodes, right?</div>
<div><br></div><div> </div><div><br></div><div><br></div></div></div><div class="gmail_extra"><br clear="all"><div><div>Vladimir Kozhukalov</div></div>
<br><br><div class="gmail_quote">On Tue, Jun 24, 2014 at 9:19 PM, Andrey Danin <span dir="ltr"><<a href="mailto:adanin@mirantis.com" target="_blank">adanin@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">+1 to @Aleksandr<br></div><div class="gmail_extra"><div><div class="h5"><br><br><div class="gmail_quote">On Tue, Jun 24, 2014 at 8:32 PM, Aleksandr Didenko <span dir="ltr"><<a href="mailto:adidenko@mirantis.com" target="_blank">adidenko@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Yes, of course, snapshot for all nodes at once (like currently) should also be available.<br></div><div>
<div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jun 24, 2014 at 7:27 PM, Igor Kalnitsky <span dir="ltr"><<a href="mailto:ikalnitsky@mirantis.com" target="_blank">ikalnitsky@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div><div>@Aleks, it's a good idea to make snapshot per environment, but I think</div>
<div>we can keep functionality to make snapshot for all nodes at once too.</div></div><span><font color="#888888"><div><br></div>
<div>- Igor</div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jun 24, 2014 at 6:38 PM, Aleksandr Didenko <span dir="ltr"><<a href="mailto:adidenko@mirantis.com" target="_blank">adidenko@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Yeah, I thought about diagnostic snapshot too. Maybe it would be better to implement per-environment diagnostic snapshots? I.e. add diagnostic snapshot generate/download buttons/links in the environment actions tab. Such snapshot would contain info/logs about Fuel master node and nodes assigned to the environment only.<br>
</div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jun 24, 2014 at 6:27 PM, Igor Kalnitsky <span dir="ltr"><<a href="mailto:ikalnitsky@mirantis.com" target="_blank">ikalnitsky@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi guys,</div><div><br></div><div>What about our diagnostic snapshot?</div><div><br></div><div>I mean we're going to make snapshot of entire /var/log and obviously</div>
<div>this old logs will be included in snapshot. Should we skip theem or</div>
<div>such situation is ok?</div><span><font color="#888888"><div><br></div><div>- Igor</div><div><br></div><div><br></div></font></span></div><div><div><div class="gmail_extra"><br>
<br><div class="gmail_quote">On Tue, Jun 24, 2014 at 5:57 PM, Aleksandr Didenko <span dir="ltr"><<a href="mailto:adidenko@mirantis.com" target="_blank">adidenko@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi,<br><br></div>If user runs some experiments with creating/deleting clusters, then taking care of old logs is under user's responsibility, I suppose. Fuel configures log rotation with compression for remote logs, so old logs will be gzipped and will not take much space.<br>
<br>In case of additional boolean parameter, the default value should be "0-don't touch old logs".<br><br></div>--</div>Regards,<br>Alex<br></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div>
<div>On Tue, Jun 24, 2014 at 4:07 PM, Vladimir Kozhukalov <span dir="ltr"><<a href="mailto:vkozhukalov@mirantis.com" target="_blank">vkozhukalov@mirantis.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><div>Guys,</div><div><br></div><div>What do you think of removing node logs on master node right after removing node from cluster? </div>
<div><br></div><div>The issue is when user do experiments he creates and deletes clusters and old unused directories remain and take disk space. On the other hand, it is not so hard to imaging the situation when user would like to be able to take a look in old logs.</div>
<div><br></div><div>My suggestion here is to add a boolean parameter into settings which will manage this piece of logic (1-remove old logs, 0-don't touch old logs).</div><div><br></div><div>Thanks for your opinions.</div>
<span><font color="#888888">
<div><br></div><div>Vladimir Kozhukalov<br></div>
</font></span></div>
<br></div></div>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><br></div></div><div class="">-- <br>Andrey Danin<br><a href="mailto:adanin@mirantis.com" target="_blank">adanin@mirantis.com</a><br>skype: gcon.monolake<br>
</div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>