Open Stack

On 08/27/2014 03:43 PM, Sean Dague wrote:
> On 08/27/2014 03:33 PM, David Kranz wrote:
>> On 08/27/2014 02:54 PM, Sean Dague wrote:
>>> Note: thread intentionally broken, this is really a different topic.
>>>
>>> On 08/27/2014 02:30 PM, Doug Hellmann wrote:>
>>>> On Aug 27, 2014, at 1:30 PM, Chris Dent <chdent at redhat.com> wrote:
>>>>
>>>>> On Wed, 27 Aug 2014, Doug Hellmann wrote:
>>>>>
>>>>>> I have found it immensely helpful, for example, to have a written set
>>>>>> of the steps involved in creating a new library, from importing the
>>>>>> git repo all the way through to making it available to other projects.
>>>>>> Without those instructions, it would have been much harder to split up
>>>>>> the work. The team would have had to train each other by word of
>>>>>> mouth, and we would have had constant issues with inconsistent
>>>>>> approaches triggering different failures. The time we spent building
>>>>>> and verifying the instructions has paid off to the extent that we even
>>>>>> had one developer not on the core team handle a graduation for us.
>>>>> +many more for the relatively simple act of just writing stuff down
>>>> "Write it down.” is my theme for Kilo.
>>> I definitely get the sentiment. "Write it down" is also hard when you
>>> are talking about things that do change around quite a bit. OpenStack as
>>> a whole sees 250 - 500 changes a week, so the interaction pattern moves
>>> around enough that it's really easy to have *very* stale information
>>> written down. Stale information is even more dangerous than no
>>> information some times, as it takes people down very wrong paths.
>>>
>>> I think we break down on communication when we get into a conversation
>>> of "I want to learn gate debugging" because I don't quite know what that
>>> means, or where the starting point of understanding is. So those
>>> intentions are well meaning, but tend to stall. The reality was there
>>> was no road map for those of us that dive in, it's just understanding
>>> how OpenStack holds together as a whole and where some of the high risk
>>> parts are. And a lot of that comes with days staring at code and logs
>>> until patterns emerge.
>>>
>>> Maybe if we can get smaller more targeted questions, we can help folks
>>> better? I'm personally a big fan of answering the targeted questions
>>> because then I also know that the time spent exposing that information
>>> was directly useful.
>>>
>>> I'm more than happy to mentor folks. But I just end up finding the "I
>>> want to learn" at the generic level something that's hard to grasp onto
>>> or figure out how we turn it into action. I'd love to hear more ideas
>>> from folks about ways we might do that better.
>>>
>>>      -Sean
>>>
>> Race conditions are what makes debugging very hard. I think we are in
>> the process of experimenting with such an idea: asymetric gating by
>> moving functional tests to projects, making them deeper and more
>> extensive, and gating against their own projects. The result should be
>> that when a code change is made, we will spend much more time running
>> tests of code that is most likely to be growing a race bug from the
>> change. Of course there is a risk that we will impair integration
>> testing and we will have to be vigilant about that. One mitigating
>> factor is that if cross-project interaction uses apis (official or not)
>> that are well tested by the functional tests, there is less risk that a
>> bug will only show up only when those apis are used by another project.
> So, sorry, this is really not about systemic changes (we're running
> those in parallel), but more about skills transfer in people getting
> engaged. Because we need both. I guess that's the danger of breaking the
> thread is apparently I lost part of the context.
>
> 	-Sean
>
I agree we need both. I made the comment because if we can make gate 
debugging less daunting
then less skill will be needed and I think that is key. Honestly, I am 
not sure the full skill you have can be transferred. It was gained 
partly through
learning in simpler times.

  -David