<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Hi,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I'm research methods to get around high storage latency for some applications where redundancy does not matter, so using local NVMe drives in compute nodes seems to be the practical choice.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>However, there does not appear to be a good solution from what I have read. For example, BlockDeviceDriver has been deprecated/removed, LVM is only supported via iSCSI (which is slow) and localization of LVM volumes onto the same compute node as VMs is impossible, and other methods (PCI pass-through, etc.) would require direct access to the local drives, where device cleansing would need to occur after a device was removed from a VM, and I don't believe there is a hook for this.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Ephemeral storage appears to be an option, but I believe it has the same issue as PCI pass-through, in that there is no abiilty to automatically cleanse a device after it has been used. In our default configuration, ephemeral storage is redirected to use Ceph, which solves the cleansing issue, but isn't suitable due to its high latency. Also, ephemeral storage appears as a second device, not the root disk, so that complicates a few configurations we have.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Is there any other way to write an operating system image onto a local drive and boot from it? Or preferably assign an LVM /dev/mapper path as a device in libvirt (no iSCSI) after configuring a logical volume? or am I missing something?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Thanks!<o:p></o:p></p><p class=MsoNormal><br>Eric<o:p></o:p></p></div></body></html>