[openstack-dev] [heat] [devstack] [infra] heat api services with uwsgi

Rabi Mishra ramishra at redhat.com
Thu May 25 04:42:19 UTC 2017


On Wed, May 24, 2017 at 9:24 AM, Rabi Mishra <ramishra at redhat.com> wrote:

> On Tue, May 23, 2017 at 11:57 PM, Zane Bitter <zbitter at redhat.com> wrote:
>
>> On 23/05/17 01:23, Rabi Mishra wrote:
>>
>>> Hi All,
>>>
>>> As per the updated community goal[1]  for api deployment with wsgi,
>>> we've to transition to use uwsgi rather than mod_wsgi at the gate. It
>>> also seems mod_wsgi support would be removed from devstack in Queens.
>>>
>>> I've been working on a patch[2] for the transition and encountered a few
>>> issues as below.
>>>
>>> 1. We encode stack_indentifer(<stack_name/stack_id> along with the path
>>> separator in heatclient. So, requests with encoded path separators are
>>> dropped by apache (with 404), if we don't have 'AllowEncodedSlashes On'
>>> directive in the site/vhost config[3].
>>>
>>
>> We'd probably want 'AllowEncodedSlashes NoDecode'.
>>
>
> Yeah, that would be ideal  for supporting slashes in stack and resource
> names where we take care of the encoding and decoding.
>
>
>> Setting this for mod_proxy_uwsgi[4] seems to work on fedora but not
>>> ubuntu.  From my testing It seems, it has to be set in 000-default.conf
>>> for ubuntu.
>>>
>>> Rather than messing with the devstack plugin code, I went ahead proposed
>>> a change to not encode the path separators in heatclient[5] ( Anyway
>>> they would be decoded by apache with the directive 'AllowEncodedSlashes
>>> On' before it's consumed by the service) which seem to have fixed those
>>> 404s.
>>>
>>
>> Pasting my comment from the patch:
>>
>> One potential problem with this is that you can probably craft a stack
>> name in such a way that heatclient ends up calling a real but unexpected
>> URL. (I don't think this is a new problem, but it's likely the problem that
>> the default value of AllowEncodedSlashes is designed to fix, and we're
>> circumventing it here.)
>>
>
>> It seems to me the ideal would be to force '/'s to be encoded when they
>> occur in the stack and resource names. Clearly they should never have been
>> encoded when they're actual path separators (e.g. between the stack name
>> and stack ID).
>>
> It'd be even better if Apache were set to "AllowEncodedSlashes NoDecode"
>> and we could then decode stack/resource names that include slashes after
>> splitting at the path separators, so that those would actually work. I
>> don't think the routing framework can handle that though.
>>
>>
> I don't think we even support slashes (encoded or not) in stack name. The
> validation below would not allow it.
>
> https://git.openstack.org/cgit/openstack/heat/tree/heat/
> engine/stack.py#n143
>
> As far as resource names are concerned, we don't encode or decode them
> appropriately for it to work as expected. Creating a stack with resource
> name containing '/' fails with validation error as it's not encoded for
> being inside the template snippet and the validation below would fail.
>
> https://git.openstack.org/cgit/openstack/heat/tree/heat/
> engine/resource.py#n214
>
> For that reason I believe we disallow slashes in stack/resource names. So
>> with "AllowEncodedSlashes Off" we'd get the right behaviour (which is to
>> always 404 when the stack/resource name contains a slash).
>>
>
>>
> Is there a generic way to set the above directive (when using
>>> apache+mod_proxy_uwsgi) in the devstack plugin?
>>>
>>> 2.  With the above, most of the tests seem to work fine other than the
>>> ones using waitcondition, where we signal back from the vm to the api
>>>
>>
>> Not related to the problem below, but I believe that when signalling
>> through the heat-cfn-api we use an arn to identify the stack, and I suspect
>> that slashes in the arn are escaped at or near the source. So we may have
>> no choice but to find a way to turn on AllowEncodedSlashes. Or is it in the
>> query string part anyway?
>>
>> Yeah, it's not related to the problem below as the request not reaching
> apache at all. I've  taken care of the above issue in the patch itself[1]
> and the signal url looks ok to me[2].
>
> [1] https://review.openstack.org/#/c/462216/11/heat/common/identifier.py
>
> [2] http://logs.openstack.org/16/462216/11/check/gate-heat-
> dsvm-functional-convg-mysql-lbaasv2-non-apache-ubuntu-
> xenial/e7d9e90/console.html#_2017-05-20_07_04_30_500696
>
> services. I could see " curl: (7) Failed to connect to 10.0.1.78 port
>>> 80: No route to host" in the vm console logs[6].
>>>
>>> It could connect to heat api services using ports 8004/8000 without this
>>> patch, but not sure why not port 80? I tried testing this locally and
>>> didn't see the issue though.
>>>
>>> Is this due to some infra settings or something else?
>>>
>>>
I finally found out the reason for the above issue. We're explicitly
allowing nova vms to access heat api services with some iptables rules.

I've submitted a project-config patch[1] to add port 80.

[1] https://review.openstack.org/#/c/467703


>
>>> [1] https://governance.openstack.org/tc/goals/pike/deploy-api-in
>>> -wsgi.html
>>>
>>> [2] https://review.openstack.org/#/c/462216/
>>>
>>> [3]
>>> https://github.com/openstack/heat/blob/master/devstack/files
>>> /apache-heat-api.template#L9
>>>
>>> [4]
>>> http://logs.openstack.org/16/462216/6/check/gate-heat-dsvm-f
>>> unctional-convg-mysql-lbaasv2-non-apache-ubuntu-xenial/fbd06
>>> d6/logs/apache_config/heat-wsgi-api.conf.txt.gz
>>>
>>> [5] https://review.openstack.org/#/c/463510/
>>>
>>> [6]
>>> http://logs.openstack.org/16/462216/11/check/gate-heat-dsvm-
>>> functional-convg-mysql-lbaasv2-non-apache-ubuntu-xenial/e7d9
>>> e90/console.html#_2017-05-20_07_04_30_718021
>>>
>>>
>>> --
>>> Regards,
>>> Rabi Mishra
>>>
>>>
>>>
>>> ____________________________________________________________
>>> ______________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.op
>>> enstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>


-- 
Regards,
Rabi Misra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170525/4eb93419/attachment.html>


More information about the OpenStack-dev mailing list