<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Interesting, what version of Heat and Nova are you using? That looks like a nova client error which I guess Heat is using to make calls to Nova.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 24 Jul 2020, at 12:52, Alexander Dibbo - UKRI STFC <<a href="mailto:alexander.dibbo@stfc.ac.uk" class="">alexander.dibbo@stfc.ac.uk</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Ok, This definitely seems to be a heat issue. A heat stack created with a server group gives the same error<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Regards<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Alexander Dibbo – Cloud Architect / Cloud Operations Group Leader<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">For STFC Cloud Documentation visit<span class="Apple-converted-space"> </span><a href="https://stfc-cloud-docs.readthedocs.io/" style="color: purple; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">https://stfc-cloud-docs.readthedocs.io</span></a><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">To raise a support ticket with the cloud team please email<span class="Apple-converted-space"> </span><a href="mailto:cloud-support@gridpp.rl.ac.uk" style="color: purple; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">cloud-support@gridpp.rl.ac.uk</span></a><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">To receive notifications about the service please subscribe to our mailing list at:<a href="https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD" style="color: purple; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD</span></a><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">To receive fast notifications or to discuss usage of the cloud please join our Slack:<a href="https://stfc-cloud.slack.com/" style="color: purple; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">https://stfc-cloud.slack.com/</span></a><o:p class=""></o:p></span></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><b class=""><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">From:</span></b><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class="Apple-converted-space"> </span>Alexander Dibbo - UKRI STFC <<a href="mailto:alexander.dibbo@stfc.ac.uk" class="">alexander.dibbo@stfc.ac.uk</a>><span class="Apple-converted-space"> </span><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>24 July 2020 09:58<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Bharat Kunwar <<a href="mailto:bharat@stackhpc.com" class="">bharat@stackhpc.com</a>><br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:openstack-discuss@lists.openstack.org" class="">openstack-discuss@lists.openstack.org</a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>RE: Magnum: invalid format of client version<o:p class=""></o:p></span></div></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">From doing some more digging I see this in the debug logs on heat<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif; background-color: rgb(12, 12, 12);" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">2020-07-24 08:33:10.618 2167043 INFO heat.engine.stack [req-249dfd80-48f4-44f5-9f0e-7c9ac743f047 - admin - default default] Exception in stack validation<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack Traceback (most recent call last):<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 909, in validate<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack result = res.validate()<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/nova/server_group.py", line 69, in validate<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack MICROVERSION_SOFT_POLICIES)<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack File "/usr/lib/python2.7/site-packages/heat/engine/clients/os/nova.py", line 107, in is_version_supported<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack self.get_max_microversion())<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 236, in get_api_version<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack api_version = APIVersion(version_string)<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 77, in __init__<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack raise exceptions.UnsupportedVersion(msg)<br class="">2020-07-24 08:33:10.618 2167043 ERROR heat.engine.stack UnsupportedVersion: Invalid format of client version ''. Expected format 'X.Y', where X is a major part and Y is a minor part of ve<br class="">rsion.<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">So it looks like heat is trying to not specify a microversion which is causing an issue with creating server groups. I have tried and I am able to create a server group and assign members with the cli.<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div class=""><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Regards<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Alexander Dibbo – Cloud Architect / Cloud Operations Group Leader<o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">For STFC Cloud Documentation visit<span class="Apple-converted-space"> </span></span><a href="https://stfc-cloud-docs.readthedocs.io/" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(5, 99, 193);" class="">https://stfc-cloud-docs.readthedocs.io</span></a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">To raise a support ticket with the cloud team please email<span class="Apple-converted-space"> </span></span><a href="mailto:cloud-support@gridpp.rl.ac.uk" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(5, 99, 193);" class="">cloud-support@gridpp.rl.ac.uk</span></a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">To receive notifications about the service please subscribe to our mailing list at:</span><a href="https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(5, 99, 193);" class="">https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD</span></a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">To receive fast notifications or to discuss usage of the cloud please join our Slack:</span><a href="https://stfc-cloud.slack.com/" style="color: purple; text-decoration: underline;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(5, 99, 193);" class="">https://stfc-cloud.slack.com/</span></a><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div></div><p style="margin-right: 0cm; margin-left: 0cm; font-size: 12pt; font-family: "Times New Roman", serif;" class=""><span style="font-size: 6pt;" class="">This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. Opinions, conclusions or other information in this message and attachments that are not related directly to UKRI business are solely those of the author and do not represent the views of UKRI.</span><o:p class=""></o:p></p></div><p style="margin-right: 0cm; margin-left: 0cm; font-size: 12pt; font-family: "Times New Roman", serif; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="font-size: 6pt;" class="">This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. Opinions, conclusions or other information in this message and attachments that are not related directly to UKRI business are solely those of the author and do not represent the views of UKRI.</span></p></div></blockquote></div><br class=""></body></html>