Open Stack

On 08/27/2014 02:54 PM, Sean Dague wrote:
> Note: thread intentionally broken, this is really a different topic.
>
> On 08/27/2014 02:30 PM, Doug Hellmann wrote:>
>> On Aug 27, 2014, at 1:30 PM, Chris Dent <chdent at redhat.com> wrote:
>>
>>> On Wed, 27 Aug 2014, Doug Hellmann wrote:
>>>
>>>> I have found it immensely helpful, for example, to have a written set
>>>> of the steps involved in creating a new library, from importing the
>>>> git repo all the way through to making it available to other projects.
>>>> Without those instructions, it would have been much harder to split up
>>>> the work. The team would have had to train each other by word of
>>>> mouth, and we would have had constant issues with inconsistent
>>>> approaches triggering different failures. The time we spent building
>>>> and verifying the instructions has paid off to the extent that we even
>>>> had one developer not on the core team handle a graduation for us.
>>> +many more for the relatively simple act of just writing stuff down
>> "Write it down.” is my theme for Kilo.
> I definitely get the sentiment. "Write it down" is also hard when you
> are talking about things that do change around quite a bit. OpenStack as
> a whole sees 250 - 500 changes a week, so the interaction pattern moves
> around enough that it's really easy to have *very* stale information
> written down. Stale information is even more dangerous than no
> information some times, as it takes people down very wrong paths.
>
> I think we break down on communication when we get into a conversation
> of "I want to learn gate debugging" because I don't quite know what that
> means, or where the starting point of understanding is. So those
> intentions are well meaning, but tend to stall. The reality was there
> was no road map for those of us that dive in, it's just understanding
> how OpenStack holds together as a whole and where some of the high risk
> parts are. And a lot of that comes with days staring at code and logs
> until patterns emerge.
>
> Maybe if we can get smaller more targeted questions, we can help folks
> better? I'm personally a big fan of answering the targeted questions
> because then I also know that the time spent exposing that information
> was directly useful.
>
> I'm more than happy to mentor folks. But I just end up finding the "I
> want to learn" at the generic level something that's hard to grasp onto
> or figure out how we turn it into action. I'd love to hear more ideas
> from folks about ways we might do that better.
>
> 	-Sean
>
Race conditions are what makes debugging very hard. I think we are in 
the process of experimenting with such an idea: asymetric gating by 
moving functional tests to projects, making them deeper and more 
extensive, and gating against their own projects. The result should be 
that when a code change is made, we will spend much more time running 
tests of code that is most likely to be growing a race bug from the 
change. Of course there is a risk that we will impair integration 
testing and we will have to be vigilant about that. One mitigating 
factor is that if cross-project interaction uses apis (official or not) 
that are well tested by the functional tests, there is less risk that a 
bug will only show up only when those apis are used by another project.

  -David