<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi Alexander,</p>
<p>Firstly, could you please help understand your Heat version and
Magnum version? Secondly, I don't really think it's related to
Magnum. As long as Heat API get the request from Magnum, Magnum
conductor just query Heat API to sync the status. <br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 14/08/20 10:49 pm, Alexander Dibbo -
UKRI STFC wrote:<br>
</div>
<blockquote type="cite"
cite="mid:08439410328b4d1ab7ca684d5af2c7c7@stfc.ac.uk">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Cascadia Mono";
panose-1:2 11 6 9 2 0 0 2 0 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am having an issue with magnum creating
clusters when I have multiple active heat-engine daemons
running.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I get the following error in the heat
engine logs:<o:p></o:p></p>
<p class="MsoNormal">2020-08-14 10:36:30.237 598383 INFO
heat.engine.resource [req-a2c862eb-370c-4e91-a2c6-dca32c7872ce
- - - - -] signal SoftwareDeployment
"master_config_deployment" [67ba9ce2-aba5-4c15-a7ea<br>
-6b774659a0e2] Stack
"kubernetes-test-26-3uzjqqob47fh-kube_masters-mhctjio2b4gh-0-pbhumflm5mn5"
[dc66e4d9-0c9b-4b18-a2c6-dd9724fa51a9] : Authentication cannot
be scoped to multiple target<br>
s. Pick one of: project, domain, trust or unscoped<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
Traceback (most recent call last):<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/heat/engine/resource.py",
line 2462, in _handle_signal<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
signal_result = self.handle_signal(details)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/software_deployment.py",
line 514, in handle_signal<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
timeutils.utcnow().isoformat())<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File "/usr/lib/python2.7/site-packages/heat/rpc/client.py",
line 788, in signal_software_deployment<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
version='1.6')<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File "/usr/lib/python2.7/site-packages/heat/rpc/client.py",
line 89, in call<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
return client.call(ctxt, method, **kwargs)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py",
line 165, in call<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
msg_ctxt = self.serializer.serialize_context(ctxt)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/heat/common/messaging.py",
line 46, in serialize_context<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
_context = ctxt.to_dict()<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/heat/common/context.py",
line 185, in to_dict<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
'roles': self.roles,<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/heat/common/context.py",
line 315, in roles<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
self._load_keystone_data()<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File "/usr/lib/python2.7/site-packages/tenacity/__init__.py",
line 292, in wrapped_f<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
return self.call(f, *args, **kw)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File "/usr/lib/python2.7/site-packages/tenacity/__init__.py",
line 358, in call<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
do = self.iter(retry_state=retry_state)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File "/usr/lib/python2.7/site-packages/tenacity/__init__.py",
line 319, in iter<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
return fut.result()<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/concurrent/futures/_base.py",
line 422, in result<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
return self.__get_result()<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File "/usr/lib/python2.7/site-packages/tenacity/__init__.py",
line 361, in call<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
result = fn(*args, **kwargs)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/heat/common/context.py",
line 306, in _load_keystone_data<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
auth_ref = self.auth_plugin.get_access(self.keystone_session)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py",
line 134, in get_access<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
self.auth_ref = self.get_auth_ref(session)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py",
line 208, in get_auth_ref<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
return self._plugin.get_auth_ref(session, **kwargs)<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
File
"/usr/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py",
line 144, in get_auth_ref<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
message='Authentication cannot be scoped to multiple'<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource
AuthorizationFailure: Authentication cannot be scoped to
multiple targets. Pick one of: project, domain, trust or
unscoped<br>
2020-08-14 10:36:30.237 598383 ERROR heat.engine.resource<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
[req-a2c862eb-370c-4e91-a2c6-dca32c7872ce - - - - -] Unhandled
error in asynchronous task: ResourceFailure:
AuthorizationFailure:
<br>
resources.master_config_deployment: Authentication cannot be
scoped to multiple targets. Pick one of: project, domain,
trust or unscoped<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
Traceback (most recent call last):<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File
"/usr/lib/python2.7/site-packages/heat/engine/service.py",
line 132, in log_exceptions<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
gt.wait()<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File
"/usr/lib/python2.7/site-packages/eventlet/greenthread.py",
line 181, in wait<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
return self._exit_event.wait()<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File "/usr/lib/python2.7/site-packages/eventlet/event.py",
line 132, in wait<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
current.throw(*self._exc)<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File
"/usr/lib/python2.7/site-packages/eventlet/greenthread.py",
line 221, in main<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
result = function(*args, **kwargs)<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File
"/usr/lib/python2.7/site-packages/heat/engine/service.py",
line 123, in _start_with_trace<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
return func(*args, **kwargs)<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File
"/usr/lib/python2.7/site-packages/heat/engine/service.py",
line 1871, in _resource_signal<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
needs_metadata_updates = rsrc.signal(details, need_check)<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File
"/usr/lib/python2.7/site-packages/heat/engine/resource.py",
line 2500, in signal<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
self._handle_signal(details)<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
File
"/usr/lib/python2.7/site-packages/heat/engine/resource.py",
line 2480, in _handle_signal<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
raise failure<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service
ResourceFailure: AuthorizationFailure:
resources.master_config_deployment: Authentication cannot be
scoped to multiple targets. Pi<br>
ck one of: project, domain, trust or unscoped<br>
2020-08-14 10:36:30.890 598383 ERROR heat.engine.service<span
style="font-size:12.0pt;font-family:"Cascadia
Mono";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Each of the individual heat-engine daemons
create magnum clusters correctly when they are the only ones
online.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Attached are the heat and magnum config
files.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any ideas where to look would be
appreciated?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Regards<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Alexander
Dibbo – Cloud Architect / Cloud Operations Group Leader<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">For
STFC Cloud Documentation visit
<a href="https://stfc-cloud-docs.readthedocs.io/"
moz-do-not-send="true"><span style="color:#0563C1">https://stfc-cloud-docs.readthedocs.io</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">To
raise a support ticket with the cloud team please email
<a href="mailto:cloud-support@gridpp.rl.ac.uk"
moz-do-not-send="true"><span style="color:#0563C1">cloud-support@gridpp.rl.ac.uk</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">To
receive notifications about the service please subscribe to
our mailing list at:
<a
href="https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD"
moz-do-not-send="true"><span style="color:#0563C1">https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">To
receive fast notifications or to discuss usage of the cloud
please join our Slack:
<a href="https://stfc-cloud.slack.com/"
moz-do-not-send="true"><span style="color:#0563C1">https://stfc-cloud.slack.com/</span></a>
<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p><span style="font-size: 6pt;">This email and any attachments
are intended solely for the use of the named recipients. If
you are not the intended recipient you must not use, disclose,
copy or distribute this email or any of its attachments and
should notify the sender immediately and delete this email
from your system. UK Research and Innovation (UKRI) has taken
every reasonable precaution to minimise risk of this email or
any attachments containing viruses or malware but the
recipient should carry out its own virus and malware checks
before opening the attachments. UKRI does not accept any
liability for any losses or damages which the recipient may
sustain due to presence of any viruses. Opinions, conclusions
or other information in this message and attachments that are
not related directly to UKRI business are solely those of the
author and do not represent the views of UKRI.</span></p>
</blockquote>
<pre class="moz-signature" cols="72">--
Cheers & Best regards,
Feilong Wang (王飞龙)
------------------------------------------------------
Senior Cloud Software Engineer
Tel: +64-48032246
Email: <a class="moz-txt-link-abbreviated" href="mailto:flwang@catalyst.net.nz">flwang@catalyst.net.nz</a>
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
------------------------------------------------------ </pre>
</body>
</html>