<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Pavlo Shchelokovskyy [mailto:pshchelokovskyy@mirantis.com]
<br>
<b>Sent:</b> Thursday, December 03, 2015 1:20 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> [openstack-dev] [heat][infra] heat-templates dsvm gate<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi all,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I would like to discuss how to fix and improve $subject, which targets to validate the templates present in the repo. Current state is the following:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">the gate does merely validate the "parseability" of YAML/JSON templates and very basic check on template structure. Any other real checks as would be actually performed in real deployment are not executed because:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">- only Heat and Keystone are installed on this gate;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">- for resources for any other service service-based resource exposure kicks in early, producing "Service {name} does not have required endpoint in service catalog for the resource type {resource}" errors,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">[Manickam, Kanagaraj] </span></i></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">‘Service not available’ error could
be handled once the ignore_erros options for validate template blueprint [3] is completed. Below command line option would help for it<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">heat template-validate –ignore-errors 990002 –f <template file><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">This helps to ignore the given errors and continue the validation of templates.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></i></b></p>
<p class="MsoNormal">- which are ignored by the script running the validation (to not block the gate, bug 1492942)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><b><i><span style="color:#1F497D"><o:p> </o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></i></b></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">[Manickam, Kanagaraj] </span></i></b><span style="color:#1F497D">once blueprint [3] is implemented, this tool could be updated as mentioned above, which
gives more validation control over existing logic.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal">Thus for example the check that properties of the resources are confirming to the schema of a particular resource is not performed, and we might be having faulty templates (e.g. with typos) in the repo. I hoped to fix this with mapping
all resources to OS::Heat::None, but it turned out this would not be really helpful, as the None-resource has any property and attribute.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">[Manickam, Kanagaraj]
</span></i></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">For unsupported resource types (not part of global-requirments.txt), we are currently masking with None resource. for other resource types which does not have keystone
endpoint in the gate, I think we ignore it in the tool. [4]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal">So I propose that heat-templates repo can configure its environment itself by using the "post_test_hook" facilities provided and supported by gate setup. We need that to better control the environment the tests are being run in. In particular,
I'd like to add some fake service endpoints to Keystone, so service-based exposure is out of the validation way.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thereby I ask Heat and Infra team to take a look at these two patches:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">- [0] in heat-templates and <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">- [1] in project-config (depends on [0]).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Though these are simply moving couple of bash lines from project-config to heat-templates, we want to be sure they work so that we do not break the gate. Unfortunately I can not prove my further patches [2] are working as it seems Depend-On
does not work for referencing project-config changes.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">[0] <a href="https://review.openstack.org/#/c/252515/">https://review.openstack.org/#/c/252515/</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[1] <a href="https://review.openstack.org/#/c/252523/">https://review.openstack.org/#/c/252523/</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[2] <a href="https://review.openstack.org/#/q/status:open+project:openstack/heat-templates+branch:master+topic:bug/1492942,n,z">https://review.openstack.org/#/q/status:open+project:openstack/heat-templates+branch:master+topic:bug/1492942,n,z</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">[3] </span><span class="apple-converted-space"><span style="font-family:"Arial",sans-serif;color:black;background:white"> </span></span><a href="https://blueprints.launchpad.net/heat/+spec/heat-template-validate-improvements" target="_blank"><span style="font-family:"Arial",sans-serif;color:#0654AC;background:white">https://blueprints.launchpad.net/heat/+spec/heat-template-validate-improvements</span></a><o:p></o:p></p>
<p class="MsoNormal"><b><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">[4]
<a href="https://github.com/openstack/heat-templates/blob/master/tools/validate-templates#L41">
https://github.com/openstack/heat-templates/blob/master/tools/validate-templates#L41</a><o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal">Best regards,<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">-- <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Dr. Pavlo Shchelokovskyy<o:p></o:p></p>
<div>
<p class="MsoNormal">Senior Software Engineer<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Mirantis Inc<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.mirantis.com">www.mirantis.com</a><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>