Open Stack

On 16/06/14 06:26, Clint Byrum wrote:
> Excerpts from Steven Hardy's message of 2014-06-15 02:40:14 -0700:
>> Hi all,
>>
>> So, I stumbled accross an issue while fixing up some tests, which is that
>> AFAICS since Icehouse we continually revalidate every property every time
>> they are accessed:
>>
>> https://github.com/openstack/heat/blob/stable/havana/heat/engine/properties.py#L716
>>
>> This means that, for example, we revalidate every property every time an
>> event is created:
>>
>> https://github.com/openstack/heat/blob/stable/havana/heat/engine/event.py#L44
>>
>> And obviously also every time the property is accessed in the code
>> implementing whatever action we're handling, and potentially also before
>> the action (e.g the explicit validate before create/update).
>>
>> This repeated revalidation seems like it could get very expensive - for
>> example there are several resources (Instance/Server resources in
>> particular) which validate against glance via a custom constraint, so we're
>> probably doing at least 6 calls to glance validating the image every
>> create.  My suspicion is this is one of the reasons for the performance
>> regression observed in bug #1324102.
>>
>> I've been experimenting with some code which implements local caching of
>> the validated properties, but according to the tests this introduces some
>> problems where the cached value doesn't always match what is expected,
>> still investigating why but I guess it's updates where we need to
>> re-resolve what is cached during the update.
>>
>> Does anyone (and in particular Zane and Thomas who I know have deep
>> experience in this area) have any ideas on what strategy we might employ to
>> reduce this revalidation overhead?
> tl;dr: I think we should only validate structure in validate, and leave
> runtime validation to preview.
>
> I've been wondering about what we want to achieve with validation
> recently. It seems to me that the goal is to assist template authors
> in finding obvious issues in structure and content before they cause a
> runtime failure. But the error messages are so unhelpful we basically
> get this:
>
> http://cdn.memegenerator.net/instances/500x/50964597.jpg
>
> What holds us back from improving that is the complexity of doing
> runtime validation.
>
> To me, runtime is more of a 'preview' problem than a validate problem. A
> template that validates once should continue to validate on any version
> that supports the template format. But a preview will actually want to
> measure runtime things and use parameters, and thus is where runtime
> concerns belong.
>
> I wonder if we could move validation out of any runtime context, and
> remove any attempts to validate runtime things like image names/ids and
> such. That would allow us to remove any but pre-action validation calls.
>
Agreed, and parameter validation needs to be included in this too -
especially with custom constraints which can make API calls.