<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 21, 2014 at 2:48 PM, Dan Genin <span dir="ltr"><<a href="mailto:daniel.genin@jhuapl.edu" target="_blank">daniel.genin@jhuapl.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>So then it is probably best to leave
existing Cinder LVM code in lib/cinder_backends/lvm alone and
create a similar set of lvm scripts for Nova,<br>
perhaps in lib/nova_backends/lvm?<span class=""><br>
<br>
Dan<br>
<br>
On 10/21/2014 03:10 PM, Duncan Thomas wrote:<br>
</span></div><div><div class="h5">
<blockquote type="cite">
<p>Sharing the vg with cinder is likely to cause some pain testing
proposed features cinder reconciling backend with the cinder db.
Creating a second vg sharing the same backend pv is easy and
avoids all such problems.</p>
<p>Duncan Thomas</p>
<div class="gmail_quote">On Oct 21, 2014 4:07 PM, "Dan Genin" <<a href="mailto:daniel.genin@jhuapl.edu" target="_blank">daniel.genin@jhuapl.edu</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hello,<br>
<br>
I would like to add to DevStack the ability to stand up Nova
with LVM ephemeral<br>
storage. Below is a draft of the blueprint describing the
proposed feature.<br>
<br>
Suggestions on architecture, implementation and the blueprint
in general are very<br>
welcome.<br>
<br>
Best,<br>
Dan<br>
<br>
========================<br>
Enable LVM ephemeral storage for Nova<br>
========================<br>
<br>
Currently DevStack supports only file based ephemeral storage
for Nova, e.g.,<br>
raw and qcow2. This is an obstacle to Tempest testing of Nova
with LVM ephemeral<br>
storage, which in the past has been inadvertantly broken<br>
(see for example, <a href="https://bugs.launchpad.net/nova/+bug/1373962" target="_blank">https://bugs.launchpad.net/nova/+bug/1373962</a>),
and to Tempest<br>
testing of new features based on LVM ephemeral storage, such
as LVM ephemeral<br>
storage encryption.<br>
<br>
To enable Nova to come up with LVM ephemeral storage it must
be provided a<br>
volume group. Based on an initial discussion with Dean Troyer,
this is best<br>
achieved by creating a single volume group for all services
that potentially<br>
need LVM storage; at the moment these are Nova and Cinder.<br>
<br>
Implementation of this feature will:<br>
<br>
* move code in lib/cinder/cinder_backends/lvm to lib/lvm with
appropriate<br>
modifications<br>
<br>
* rename the Cinder volume group to something generic, e.g.,
devstack-vg<br>
<br>
* modify the Cinder initialization and cleanup code
appropriately to use<br>
the new volume group<br>
<br>
* initialize the volume group in stack.sh, shortly before
services are<br>
launched<br>
<br>
* cleanup the volume group in unstack.sh after the services
have been<br>
shutdown<br>
<br>
The question of how large to make the common Nova-Cinder
volume group in order<br>
to enable LVM ephemeral Tempest testing will have to be
explored. Although,<br>
given the tiny instance disks used in Nova Tempest tests, the
current<br>
Cinder volume group size may already be adequate.<br>
<br>
No new configuration options will be necessary, assuming the
volume group size<br>
will not be made configurable.<br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
OpenStack-dev mailing list
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div><div class="gmail_extra">One thing to keep in mind when using AWS as an example is that the equivalent isn't "Use LVM on Compute Node", the model is more use EBS (in our case Cinder) Volumes for Instances. We most certainly have this currently and there is some testing in the gate. It seems to be coming up in all sorts of tangential conversations lately. Personally I'd LOVE to see it more widely used, which in turn would likely lead to improvements.</div><div class="gmail_extra"><br></div><div class="gmail_extra">As far as the comment made about "poor performance" I'm not sure where that comes from. Sure, if you run iSCSI over a 1G network to a Cinder Volume on a loop back device I wouldn't expect great perf. If you run over a 10 Gig Network to a Cinder Volume backed by a descent/real disk I don't think the performance is as significant as some seem to think (full disclosure it's been a while since I've actually tested and looked at this closely). That being said, there's a good deal of work that should be done here to tweak it and improve the LVM driver in Cinder for example. LVM in general tends to be perceived as providing poor performance, but meh... some of that is based more in fud than in data (note I said "some of it").</div><div class="gmail_extra"><br></div><div class="gmail_extra">All of that is kinda off topic but it did come up so I thought I'd chime in. The real question of LVM on Compute Nodes.... I'd mostly be interested in more feedback from folks that use that and why? I don't have any background or experience with it so I'd love some insight on why one chooses LVM on the Compute Node versus File System based instances? At some point consolidating the LVM code in Cinder and Nova should really happen as well (i.e. a shared library or having LVM code in oslo). Yet another topic and fortunately there are a few folks working on that for Kilo.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Thanks,</div><div class="gmail_extra">John</div></div>