Open Stack

Further things that someone may want to read/try (if the below is true),

https://en.wikipedia.org/wiki/ReDoS

Joshua Harlow wrote:
> Just the paranoid person in me, but is it safe to say that the filter
> that you are showing here does not come from user text?
>
> Ie these two lines don't come from a user input directly (without going
> through some filter) do they?
>
> https://github.com/openstack/nova/blob/16.0.0/nova/compute/api.py#L2458-L2459
>
>
>  From reading it seems like perhaps they do come at least partially from
> a user, so I am hoping that its not possible for a user to present a
> 'ip' that is really a complicated regex that takes a long time to
> compile (and therefore can DOS the nova-api component); but I don't know
> the surrounding code so I might be wrong...
>
> Just wondering :-/
>
> -Josh
>
> Matt Riedemann wrote:
>> Nova has had this long-standing known performance issue if you're
>> filtering a large number of instances by IP. The instance IPs are stored
>> in a JSON blob in the database so we don't do filtering in SQL. We pull
>> the instances out of the database, deserialize the JSON and then apply a
>> regex filter match in the nova-api python code.
>>
>> At the Queens PTG we talked about possible ways to fix this and came up
>> with this nova spec:
>>
>> https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/improve-filter-instances-by-ip-performance.html
>>
>>
>>
>> The idea is to have nova get ports from neutron and apply the IP filter
>> in neutron to whittle down the ports, then from that list of ports get
>> the instances to pull out of the nova database.
>>
>> One issue that has come up with this is neutron does not currently
>> support regex filters when listing ports. There is an RFE for adding
>> that:
>>
>> https://bugs.launchpad.net/neutron/+bug/1718605
>>
>> The proposed neutron implementation is to just do SQL LIKE substring
>> matching in the database.
>>
>> However, one issue that has come up is that the compute API accepts a
>> python regex filter and uses re.match():
>>
>> https://github.com/openstack/nova/blob/16.0.0/nova/compute/api.py#L2469
>>
>> At least one good thing about that is match() only matches from the
>> beginning of the string unlike search().
>>
>> So for example I can filter on "192.16.*[1-5]$" if I wanted to, but
>> that's not going to work with just a LIKE substring filter in SQL.
>>
>> The question is, does anyone actually do more than basic substring
>> matching with the IP filter today? Because if we started using neutron,
>> that behavior would be broken. We've never actually documented the match
>> restrictions on the IP filter, but that's not a good reason to break it.
>>
>> One option is to make this configurable such that deployments which rely
>> on the complicated pattern matching can just use the existing nova code
>> despite performance issues. However, that's not interoperable, I hate
>> config-driven API behavior, and it would mean maintaining two code paths
>> in nova, which is also terrible.
>>
>> I was trying to think of a way to determine if the IP filter passed to
>> nova is basic or a complicated pattern match and let us decide that way,
>> but I'm not sure if there are good ways to detect that - maybe by simply
>> looking for special characters like (, ), - and $? But then there is []
>> and we have an IPv6 filter, so that gets messy too...
>>
>> For now I'd just like to know if people rely on the regex match or not.
>> Other ideas on how to handle this are appreciated.
>>
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators