Open Stack

On 09/04/2014 11:23 AM, John Garbutt wrote:
> Sorry for another top post, but I like how Nikola has pulled this
> problem apart, and wanted to respond directly to his response.
> 

Thanks John - I'm glad this caught your eye.

> On 3 September 2014 10:50, Nikola Đipanov <ndipanov at redhat.com> wrote:
>> The reason many features including my own may not make the FF is not
>> because there was not enough buy in from the core team (let's be
>> completely honest - I have 3+ other core members working for the same
>> company that are by nature of things easier to convince), but because of
>> any of the following:
>>
>> * Crippling technical debt in some of the key parts of the code
> 
> +1
> 
> We have problems that need solving.
> 
> One of the ideas behind the slots proposal is to "encourage" work on
> the urgent technical debt, before related features are even approved.
> 

As I stated before, my issue with slots was more about the fact that
they seem to me like needlessly elaborate process to communicate a
simple list of important things we focus on, not what it was trying to
accomplish, which I fully support.

Not sure where I stand on it still, but if it will get us closer to
fixing stuff we really need to fix, then I won't argue with it.

>> * that we have not been acknowledging as such for a long time
> 
> -1
> 
> We keep saying "thats cool, but we have to fix/finish XXX first".
> 
> But... we have been very bad at:
> * remembering that, and recording that
> * actually fixing those problems
> 

This seems to me ties in with prioritizing important work, like jog0 is
proposing on [1]. I am not sure if just prioritizing it will work
though, we've had blueprints before that we all agreed were high
priority that got delayed and punted mostly because of lack of hands to
do the work. I am not sure how to solve this problem.

Even with what danpb is proposing on a parallel thread with the drivers
- this is still work that needs to be done on the core.

Objects is a good example of how this can work, but we must not forget
that it had a strong backing and several highly skilled people working
on it. This is the part prioritizing won't solve.

[1] https://review.openstack.org/#/c/112733/

>> * which leads to proposed code being arbitrarily delayed once it makes
>> the glaring flaws in the underlying infra apparent
> 
> Sometimes we only spot this stuff in code reviews, where you throw up
> reading all the code around the change, and see all the extra
> complexity being added to a fragile bit of the code, and well, then
> you really don't want to be the person who clicks approve on that.
> 
> We need to track this stuff better. Every time it happens, we should
> try make a not to go back there and do more tidy ups.
> 

+1 - absolutely - we definitely lack the "grumpy developer who goes in
and fixes stuff" mentality!

>> * and that specs process has been completely and utterly useless in
>> helping uncover (not that process itself is useless, it is very useful
>> for other things)
> 
> Yeah, it hasn't helped for this.
> 
> I don't think we should do this, but I keep thinking about making
> specs two step:
> * write generally direction doc
> * go write the code, maybe upload as WIP
> * write the "documentation" part of the spec
> * get docs merged before any code
> 

I would say that we need to keep the spec approval as lightweight as
possible so that we can make sure we get to the details (where devil
resides), sooner rather than later... so maybe a 2-phase process along
the lines of:

* This feature makes sense for Nova and it is proposed in a reasonable
manner (merge quickly in /tentative dir)
* This now looks like a coherent whole with the POC code, move spec to a
/approved dir, work on the details of the code OR we need to fix some
issues, let's see how we can do that and still not make the life of
feature proposer a living hell just for hitting a snag.

>> I am almost positive we can turn this rather dire situation around
>> easily in a matter of months, but we need to start doing it! It will not
>> happen through pinning arbitrary numbers to arbitrary processes.
> 
> +1
> 
> This is ongoing, but there are some major things, I feel we should
> stop and fix in kilo.
> 
> ...and that will make getting features in much worse for a little
> while, but it will be much better on the other side.
> 

I really do hope so because all things considered - I don't think Nova
is in horrible state, but we need to work on it _now_. Many of these
issues have been known for a long time and are just piling up.

>> I will follow up with a more detailed email about what I believe we are
>> missing, once the FF settles and I have applied some soothing creme to
>> my burnout wounds
> 
> Awesome, please catch up with jogo who was also trying to build this
> list. I would love to continue to contribute to that too.
> 

Yes - as already said on mu response to jog0 - I missed his proposal and
will comment on there.

> Might be working moving into here:
> https://etherpad.openstack.org/p/kilo-nova-summit-topics
> 
> The idea was/is to use that list to decide what fills up the majority
> of code slots in Juno.
> 
>> but currently my sentiment is:
>> Contributing features to Nova nowadays SUCKS!!1 (even as a core
>> reviewer) We _have_ to change that!
> 
> Agreed.
> 
> 
> In addition, our bug list would suggest our users are seeing the
> impact of this technical debt.
> 
> 
> My personal feeling is we also need to tidy up our testing debt too:
> * document major bits that are NOT tested, so users are clear
> * document what combinations and features we actually see tested up stream
> * support different levels of testing: on-demand+daily vs every commit
> * making it easier to interested parties to own and maintain some testing
> * plan for removing the untested code paths in L
> * allow for untested code to enter the tree, as experimental, with the
> expectation it gets removed in the following release if not tested,
> and architected so that is possible (note this means supporting
> "experimental" APIs that can be ripped out at a later date.)
> 
> We have started doing some of the above work. But I think we need to
> hold ALL code to the same standard. It seems it will take time to
> agree on that standard, but the above is an attempt to compromise
> between speed of innovation and stability.
> 

Some of this seems to me touches on our release process as well - and
what a release means - I don't have good answers to this other than it
seems to me we may want to evolve that too.

N.