<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-IN" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Devs,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">As another alternative we can use start/stop API’s instead of shelve/unshelve the instance.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<table class="MsoTableLightShadingAccent1" border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none">
<tbody>
<tr>
<td width="123" valign="top" style="width:92.15pt;border-top:solid #4F81BD 1.0pt;border-left:none;border-bottom:solid #4F81BD 1.0pt;border-right:none;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">API’s<o:p></o:p></span></b></p>
</td>
<td width="189" valign="top" style="width:5.0cm;border-top:solid #4F81BD 1.0pt;border-left:none;border-bottom:solid #4F81BD 1.0pt;border-right:none;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">cpu/memory released<o:p></o:p></span></b></p>
</td>
<td width="187" valign="top" style="width:140.15pt;border-top:solid #4F81BD 1.0pt;border-left:none;border-bottom:solid #4F81BD 1.0pt;border-right:none;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Disk released<o:p></o:p></span></b></p>
</td>
<td width="86" valign="top" style="width:64.8pt;border-top:solid #4F81BD 1.0pt;border-left:none;border-bottom:solid #4F81BD 1.0pt;border-right:none;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Fast respawning<o:p></o:p></span></b></p>
</td>
<td width="250" valign="top" style="width:187.4pt;border-top:solid #4F81BD 1.0pt;border-left:none;border-bottom:solid #4F81BD 1.0pt;border-right:none;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Notes<o:p></o:p></span></b></p>
</td>
</tr>
<tr>
<td width="123" valign="top" style="width:92.15pt;border:none;background:#D3DFEE;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">start/stop<o:p></o:p></span></b></p>
</td>
<td width="189" valign="top" style="width:5.0cm;border:none;background:#D3DFEE;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">No<o:p></o:p></span></p>
</td>
<td width="187" valign="top" style="width:140.15pt;border:none;background:#D3DFEE;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">No<o:p></o:p></span></p>
</td>
<td width="86" valign="top" style="width:64.8pt;border:none;background:#D3DFEE;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Yes<o:p></o:p></span></p>
</td>
<td width="250" valign="top" style="width:187.4pt;border:none;background:#D3DFEE;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</td>
</tr>
<tr>
<td width="123" valign="top" style="width:92.15pt;border:none;border-bottom:solid #4F81BD 1.0pt;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">shelve/unshelve<o:p></o:p></span></b></p>
</td>
<td width="189" valign="top" style="width:5.0cm;border:none;border-bottom:solid #4F81BD 1.0pt;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Yes<o:p></o:p></span></p>
</td>
<td width="187" valign="top" style="width:140.15pt;border:none;border-bottom:solid #4F81BD 1.0pt;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Yes (Not released if shelved_offload_time = -1)<o:p></o:p></span></p>
</td>
<td width="86" valign="top" style="width:64.8pt;border:none;border-bottom:solid #4F81BD 1.0pt;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">No<o:p></o:p></span></p>
</td>
<td width="250" valign="top" style="width:187.4pt;border:none;border-bottom:solid #4F81BD 1.0pt;padding:0cm 5.4pt 0cm 5.4pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Instance does not respawn faster in case of instance is booted from image<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">In order to make unshelve fast enough, we need to preserve instance root disk in compute node,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">which I have proposed in spec [1] of shelve-partial-offload.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">In case of start/stop API’s cpu/memory are not released/reassigned. We can modify these API’s to release<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">the cpu and memory while stopping the instance and reassign the same while starting the instance. In this case<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">also rescheduling logic need to be modified to reschedule the instance on different host, if required resources<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">are not available while starting the instance. This is similar to what I have implemented in [2] Improving the<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">performance of unshelve API.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[1]
<a href="https://review.openstack.org/#/c/135387/">https://review.openstack.org/#/c/135387/</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">[2]
<a href="https://review.openstack.org/#/c/150344/">https://review.openstack.org/#/c/150344/</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Please let me know your opinion, whether we can modify start/stop API’s as an alternative to shelve/unshelve API’s.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank You,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Abhishek Kekane<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Kekane, Abhishek [mailto:Abhishek.Kekane@nttdata.com]
<br>
<b>Sent:</b> 24 February 2015 12:47<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] Unshelve Instance Performance Optimization Questions<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Duncan,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank you for the inputs.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">@Community-Members<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I want to know if there are any other alternatives to improve the performance of unshelve api ((booted from image only).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Please give me your opinion on the same.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank You,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Abhishek Kekane<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Duncan Thomas [mailto:duncan.thomas@gmail.com]
<br>
<b>Sent:</b> 16 February 2015 16:46<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [nova] Unshelve Instance Performance Optimization Questions<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">There has been some talk in cinder meetings about making cinder<->glance interactions more efficient. They are already optimised in some deployments, e.g. ceph glance and ceph cinder, and some backends cache
glance images so that many volumes created from the same image becomes very efficient. (search the meeting logs or channel logs for 'public snapshot' to get some entry points into the discussions)
<br>
<br>
I'd like to see more work done on this, and perhaps re-examine a cinder backend to glance. This would give some of what you're suggesting (particularly fast, low traffic un-shelve), and there is more that can be done to improve that performance, particularly
if we can find a better performing generic CoW technology than QCOW2.<o:p></o:p></p>
</div>
<p class="MsoNormal">As suggested in the review, in the short term you might be better experimenting with moving to boot-from-volume instances if you have a suitable cinder deployed, since that gives you some of the performance improvements already.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 16 February 2015 at 12:10, Kekane, Abhishek <<a href="mailto:Abhishek.Kekane@nttdata.com" target="_blank">Abhishek.Kekane@nttdata.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Devs,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Problem Statement: Performance and storage efficiency of shelving/unshelving instance booted from image is far worse than instance booted from volume.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">When you unshelve hundreds of instances at the same time, instance spawning time varies and it mainly depends on the size of the instance snapshot and<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">the network speed between glance and nova servers.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">If you have configured file store (shared storage) as a backend in Glance for storing images/snapshots, then it's possible to improve the performance of<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">unshelve instance dramatically by configuring nova.image.download.FileTransfer in nova. In this case, it simply copies the instance snapshot as if it is<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">stored on the local filesystem of the compute node. But then again in this case, it is observed the network traffic between shared storage servers and<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">nova increases enormously resulting in slow spawning of the instances.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I would like to gather some thoughts about how we can improve the performance of unshelve api (booted from image only) in terms of downloading large<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">size instance snapshots from glance.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I have proposed a nova-specs [1] to address this performance issue. Please take a look at it.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">During the last nova mid-cycle summit,
<a href="https://review.openstack.org/#/q/owner:mikal%2540stillhq.com+status:open,n,z" target="_blank" title="Michael Still <mikal@stillhq.com>">
<span style="color:windowtext;text-decoration:none">Michael Still</span></a> has suggested alternative solutions to tackle this issue.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Storage solutions like ceph (Software based) and NetApp (Hardare based) support exposing images from glance to nova-compute and cinder-volume with<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">copy in write feature. This way there will be no need to download the instance snapshot and unshelve api will be pretty faster than getting it<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">from glance.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Do you think the above performance issue should be handled in the OpenStack software as described in nova-specs [1] or storage solutions like<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">ceph/NetApp should be used in production environment? Apart from ceph/NetApp solutions, are there any other options available in the market.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">[1]
<a href="https://review.openstack.org/#/c/135387/" target="_blank">https://review.openstack.org/#/c/135387/</a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thank You,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Abhishek Kekane<o:p></o:p></p>
</div>
<p class="MsoNormal"><br clear="all">
______________________________________________________________________<br>
Disclaimer: This email and any attachments are sent in strictest confidence<br>
for the sole use of the addressee and may contain legally privileged,<br>
confidential, and proprietary data. If you are not the intended recipient,<br>
please advise the sender by replying promptly to this email and then delete<br>
and destroy this email and any attachments without any further use, copying<br>
or forwarding.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<br>
-- <o:p></o:p></p>
<div>
<p class="MsoNormal">Duncan Thomas<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><br>
______________________________________________________________________<br>
Disclaimer: This email and any attachments are sent in strictest confidence<br>
for the sole use of the addressee and may contain legally privileged,<br>
confidential, and proprietary data. If you are not the intended recipient,<br>
please advise the sender by replying promptly to this email and then delete<br>
and destroy this email and any attachments without any further use, copying<br>
or forwarding.<o:p></o:p></p>
</div>
<br clear="both">
______________________________________________________________________<BR>
Disclaimer: This email and any attachments are sent in strictest confidence<BR>
for the sole use of the addressee and may contain legally privileged,<BR>
confidential, and proprietary data. If you are not the intended recipient,<BR>
please advise the sender by replying promptly to this email and then delete<BR>
and destroy this email and any attachments without any further use, copying<BR>
or forwarding.<BR>
</body>
</html>