On 27/02/2019 20:49, Zane Bitter wrote:
> On 20/02/19 1:40 PM, Jonathan Rosser wrote:
>> In openstack-ansible we are trying to help a number of our end users
>> with their heat deployments, some of them in conjunction with magnum.
>>
>> There is some uncertainty with how the following heat.conf sections
>> should be configured:
>>
>> [clients_keystone]
>> auth_uri = ...
>>
>> [keystone_authtoken]
>> www_authenticate_uri = ...
>>
>> It does not appear to be possible to define a set of internal or
>> external keystone endpoints in heat.conf which allow the following:
>>
>>   * The orchestration panels being functional in horizon
>>   * Deployers isolating internal openstack from external networks
>>   * Deployers using self signed/company cert on the external endpoint
>>   * Magnum deployments completing
>>   * Heat delivering an external endpoint at [1]
>>   * Heat delivering an external endpoint at [2]
>>
>> There are a number of related bugs:
>>
>> https://bugs.launchpad.net/openstack-ansible/+bug/1814909
>> https://bugs.launchpad.net/openstack-ansible/+bug/1811086
>> https://storyboard.openstack.org/#!/story/2004808
>> https://storyboard.openstack.org/#!/story/2004524
>
> Based on this and your comment on IRC[1] - and correct me if I'm
> misunderstanding here - the crux of the issue is that the Keystone
> auth_url must be accessed via different addresses depending on which
> network the request is coming from?
>
The most concrete example I can give is that of a Magnum k8s deployment,
where heat is used to create several VM and deploy software. Callback
URLs are embedded into those VM and SoftwareDeployments, and the
Callback URL must be accessible from the VM, this would always need to
be something that could reasonably be called a "Public" endpoint.

Conversely, heat itself needs to be able to talk to many other openstack
components, defined in the [clients_*] config sections. It is reasonable
to describe these interactions as being "Internal" - I may misunderstand
some of this though.

So here lies the issue - appropriate entries in heat.conf to make
internal interactions between heat and horizon (one example) work in
real-world deployments results in the keystone internal URL being placed
in callbacks, and then SoftwareDemployments never complete as the
internal keystone URL is not usually accessible to a VM. I suspect that
there is not much coverage for this kind of network separation in gate
tests.

> I don't think this was ever contemplated as a use case in developing
> Heat. For my part, I certainly always assumed that while the Keystone
> catalog could contain different Public/Internal/Admin endpoints for
> each service, there was only a single place to access the catalog
> (i.e. each cloud had a single unique auth_url).
>
I think that as far as heat itself interacting with other openstack
components is concerned there does not need to be more than one
auth_url. However it is very important to make a distinction between the
context in which the heat code runs and the context of a VM created by
heat - any callback URL created must be valid for the context of the VM,
not the heat code.

> It's entirely possible this wasn't a valid assumption about the how
> clouds would/should be deployed in practice. If that's the case then
> we likely need some richer configuration options. The design of the
> Keystone catalog predates both the existence of Heat and the idea that
> cloud workloads might have reason to access the OpenStack APIs, and
> nobody is really an expert on both although we've gotten better at
> communicating.
>
> [1]
> http://eavesdrop.openstack.org/irclogs/%23heat/%23heat.2019-02-26.log.html#t2019-02-26T17:14:14
>
Colleen makes some observations about the use of keystone config in heat
- and interestingly suggests a seperate config entry for cases where a
keystone URL should be handed on to another service. Mohammed and I have
already discussed additional config options being a potential solution
whilst trying to debug this.

There are already examples of similar config options in heat.conf, such
as "heat_waitcondition_server_url" - would additonal config items such
as server_base_auth_url and signal_responder_auth_url be appropriate so
that we can be totally explicit about the endpoints handed on to created VM?