Open Stack

On 03/11/2014 10:15 AM, Steven Dake wrote:
> On 03/11/2014 04:04 AM, Sean Dague wrote:
>> On 03/04/2014 12:39 PM, Steven Hardy wrote:
>>> Hi all,
>>>
>>> As some of you know, I've been working on the instance-users blueprint[1].
>>>
>>> This blueprint implementation requires three new items to be added to the
>>> heat.conf, or some resources (those which create keystone users) will not
>>> work:
>>>
>>> https://review.openstack.org/#/c/73978/
>>> https://review.openstack.org/#/c/76035/
>>>
>>> So on upgrade, the deployer must create a keystone domain and domain-admin
>>> user, add the details to heat.conf, as already been done in devstack[2].
>>>
>>> The changes requried for this to work have already landed in devstack, but
>>> it was discussed to day and Clint suggested this may be unacceptable
>>> upgrade behavior - I'm not sure so looking for guidance/comments.
>>>
>>> My plan was/is:
>>> - Make devstack work
>>> - Talk to tripleo folks to assist in any transition (what prompted this
>>>   discussion)
>>> - Document the upgrade requirements in the Icehouse release notes so the
>>>   wider community can upgrade from Havana.
>>> - Try to give a heads-up to those maintaining downstream heat deployment
>>>   tools (e.g stackforge/puppet-heat) that some tweaks will be required for
>>>   Icehouse.
>>>
>>> However some have suggested there may be an openstack-wide policy which
>>> requires peoples old config files to continue working indefinitely on
>>> upgrade between versions - is this right?  If so where is it documented?
>> This is basically enforced in code in grenade, the language for this
>> actually got lost in the project requirements discussion in the TC, I'll
>> bring that back in the post graduation requirements discussion we're
>> having again.
>>
>> The issue is - Heat still doesn't materially participate in grenade.
>> Heat is substantially far behind the other integrated projects in it's
>> integration with the upstream testing. Only monday did we finally start
>> gating on a real unit of work for Heat (the heat-slow jobs). If I was
>> letter grading projects right now on upstream testing I'd give Nova an
>> A, Neutron a C (still no full run, no working grenade), and Heat a D.
> Sean,
> 
> I agree the Heat community hasn't done a bang-up job of getting
> integrated with Tempest.  We only have 50 functional tests implemented. 
> The community clearly needs to do more and provide better functional
> coverage with Heat.
> 
> It is inappropriate to say "Only monday did we finally start gating"
> because that was a huge move in the right direction.  It took alot of
> effort and should not be so easily dismissed.  Clearly the community,
> and especially the core developers, are making an effort.  Keep in mind
> we have to balance upstream development work, answering user questions,
> staying on top of a 5 page review queue, keeping relationships and track
> of the various integrated projects which are consuming Heat as a
> building block, plus all of the demands of our day jobs.

I agree it was a huge step in the right direction. It's not clear to me
why expressing that this was very recent was inappropriate.

Recent conversations have made me realize that a lot of the Heat core
team doesn't realize that Heat's participation in upstream gating is
below average, so I decided to be blunt about it. Because it was only
after being blunt about that with the Neutron team in Hong Kong did we
get any real motion on it (Neutron has seen huge gains this cycle).

All the integrated projects have the same challenges.

Upstream QA is really important. It not only protects heat from itself,
it protects it from changes in other projects.

> We just don't have enough bandwidth on the core team to tackle writing
> all of the tempest test cases ourselves.  We have made an effort to
> distribute this work to the overall heat community via wishlist bugs in
> Heat which several new folks have picked up.  I hope to see our coverage
> improve over time, especially with more advanced scenario tests through
> this effort.

Bandwidth is a problem for everyone. It's a matter of priorities. The
fact that realistic upstream gating is considered wishlist priority in
from a Heat perspective is something I find troubling.

Putting the investment into realistic scenarios in Tempest / gate is
going to be a huge timesaving for the Heat team. It will ensure Heat is
functioning at every commit (not just releases), it will protect Heat
from chasing breaking issues in Keystone or Nova, and it will mean that
we'll expose more subtle issues that only come with being able to do
data analysis on 10k runs.

I get it's never fun to hear that a project is below average on a metric
that's important to the OpenStack community. But if we aren't honest and
open about these things they never change.

	-Sean

--
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140311/af274d87/attachment.pgp>