Open Stack

No disagreement with anything you say below, Matt. More testing of all
kinds, including unit tests, should be a priority.

Cheers,
jay

On Wed, Feb 16, 2011 at 11:37 PM, Matt Dietz <matt.dietz at rackspace.com> wrote:
> These are all great points Jay.
>
> I'd like to re-echo the comment about unit tests. Obviously they aren't
> the panacea, but they can protect against some of the dumber errors that
> have made their way into trunk. One particular bug stopped one developer
> on my team dead in his tracks, and it ended up being a semi-colon in place
> of a colon. There's a lot of utility in simply "exercising" code...
>
> I think we may want to consider everyone's favorite topic of code coverage
> as well in all of this. Specifically, we may want to take note of code
> coverage on any given merge, and if subsequent merges reduce that number,
> we throw a fit/reject the patch. I know that won't be a popular solution,
> but it would definitely put a stop to the lack of unit tests.
>
> On 2/16/11 4:27 PM, "Jay Pipes" <jaypipes at gmail.com> wrote:
>
>>Hey all,
>>
>>It's come to my attention that a number of folks are not happy that
>>Nova's trunk branch (lp:nova) is, shall we say, "less than stable". :)
>>
>>First, before going into some suggestions on keeping trunk more
>>stable, I'd like to point out that trunk is, by nature, an actively
>>developed source tree. Nobody should have an expectation that they can
>>simply bzr branch lp:nova and everything will magically work with a)
>>their existing installations of software packages, b) whatever code
>>commits they have made locally, or c) whatever specific
>>hypervisor/volume/network environment that they test their local code
>>with. The trunk branch is, after all, in active development.
>>
>>That said, there's *no* reason we can't *improve* the relative
>>stability of the trunk branch to make life less stressful for
>>contributors.  Here are a few suggestions on how to keep trunk a bit
>>more stable for those developers who actively develop from trunk.
>>
>>1) Participate fully in code reviews. If you suspect a proposed branch
>>merge will "mess everything up for you", then you should notify
>>reviewers and developers about your concerns. Be proactive.
>>
>>2) If you pull trunk and something breaks, don't just complain about
>>it. Log a bug immediately and talk to the reviewers/approvers of the
>>patch that broke your environment. Be constructive in your criticism,
>>and be clear about why the patch should have been more thoroughly or
>>carefully reviewed. If you don't, we're bound to repeat mistakes.
>>
>>3) Help us to write functional and integration tests. It's become
>>increasingly clear from the frequency of breakages in trunk (and other
>>branches) that our unit tests are nowhere near sufficient to catch a
>>large portion of bugs. This is to be expected. Our unit tests use
>>mocks and stubs for virtually everything, and they only really test
>>code interfaces, and they don't even test that very well. We're
>>working on adding functional tests to Hudson that will run, as the
>>unit test do, before any merge into trunk, with any failure resulting
>>in a failed merge. However, we need your help to create functional
>>tests and integration tests (tests that various *real* components work
>>together properly).  We also need help writing test cases that ensure
>>software library dependencies and other packaging issues are handled
>>properly and don't break with minor patches.
>>
>>4) If you have a specific environment/setup that you use (Rackers and
>>Anso guys, here...), then we need your assistance to set up test
>>clusters that will pull trunk into a wiped test environment and ensure
>>that a series of realistic calls to the Nova API are properly handled.
>>I know some of you are working on getting hardware ready. We need help
>>from the software teams to ensure that these environments are
>>initialized with the exact setups you use.
>>
>>The more testing we fire off against each potential merge into trunk,
>>and the more those tests are hitting real-life deployment
>>environments, the more stable trunk will become and the easier your
>>life as a contributor will be.
>>
>>Thanks in advance for your assistance, and please don't hesitate to
>>expand on any more suggestions you might have to stabilize trunk.
>>
>>-jay
>>
>>_______________________________________________
>>Mailing list: https://launchpad.net/~openstack
>>Post to     : openstack at lists.launchpad.net
>>Unsubscribe : https://launchpad.net/~openstack
>>More help   : https://help.launchpad.net/ListHelp
>
>
>
> Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use of the
> individual or entity to which this message is addressed, and unless otherwise
> expressly indicated, is confidential and privileged information of Rackspace.
> Any dissemination, distribution or copying of the enclosed material is prohibited.
> If you receive this transmission in error, please notify us immediately by e-mail
> at abuse at rackspace.com, and delete the original message.
> Your cooperation is appreciated.
>
>