Open Stack

On 05/13/2015 05:23 PM, Ben Swartzlander wrote:
> We (the Manila community) have known for a while that we would like to
> have 3rd-party vendor CI similar to the Cinder project, but up until now
> the details of what that would look like have been vague. I would like
> to make a proposal for how we should proceed so we can set deadlines and
> socialize them at the design summit next week.
> 
> I'm assuming based on previous discussions that everyone understands the
> value of CI systems and there's no disagreement that Liberty is the
> right time frame to add a requirement. If not, then we need to have a
> different discussion. But if so, then I'd like to get consensus from the
> community about what the deadlines should be and how we should deal with
> drivers that can't or won't set up CI systems to test them.
> 
> 
> My proposal consists of 4 deadlines, aligned to the Liberty milestones.
> 
> liberty-1 (Jun 25)
> * Every driver must have an associated maintainer with contact
> information. We will create a wiki/etherpad to collect this information.

I suggest starting off in the expected place for contact information:
https://wiki.openstack.org/wiki/ThirdPartySystems

> * Maintainers should understand the 3rd party CI requirement

http://ci.openstack.org/third_party.html

> and have a
> plan for how to meet the deadlines. A plan includes how and when any
> needed materials will be obtained (hardware, other compute resources,
> accounts, firewall exceptions etc).

You can ask for a plan if you want one but infra won't be tracking,
assessing or approving plans. We can however answer questions about our
supported workflow, using puppet modules to deploy various tools.

> * Maintainers should already be in contact with #openstack-infra and
> other resources to get help with how to build a CI system if help is
> needed.

Maintainers are best to start off with attending a third party meeting:
https://wiki.openstack.org/wiki/Meetings#Third_Party_Meeting

Tossing new folks into the infra channel is usually a frustrating
endeavour for all involved, it is best if they read the infra
requirements (ensuring Manila folks have also done so)
http://ci.openstack.org/third_party.html and attend a third party
meeting to ask the getting started questions. Once they have a
beginning, asking for help in infra is fine but bootstrapping every
single new operator gets old, hence the meeting as a good first stop.

> 
> liberty-2 (July 30)
> * Maintainers should have created any needed accounts and obtained any
> needed resources and should be making progress towards getting the
> system to run tests on their backends.
> * Maintainers are be expected to be able to show logs of Tempest running
> on Manila with their drivers if they aren't able to report results
> directly to gerrit.
> * Any tests that aren't passing should have bugs filed to get them fixed.

Let's add a step here where they acknowledge the ci-sandbox repo exists
and have demonstrated that they can test their system on it, I don't
care the time line but best to have that prior to commenting on manila
repos: http://git.openstack.org/cgit/openstack-dev/ci-sandbox/

> 
> liberty-3 (Sept 3)
> * CI systems are expected to be running by this point, and posting
> results to gerrit (whether successful or not).
> * Maintainers shouldn't be working on the CI system itself at this point
> but only fixing driver bugs or other issues related to stability.
> 
> liberty-rc1 (Sept 24)
> * Drivers which don't have CI systems posting successful results
> reliably will get removed from the master branch before the RC1 cut.
> 
> 
> My theory is that by creating multiple checkpoints, we can identify
> anyone who's is having issues early on and assist them, to avoid nasty
> surprises late in the release.

It is a good theory, the extent to which it will be successful is
entirely dependent on your team paying attention and following up. I
suggest you designate one individual to be responsible, not that they
have to do all the work but that they act as point person.

> Also, this forces maintains to at least
> start thinking about the requirement early on so they don't
> underestimate the difficultly/effort required and try to do it all a the
> last moment.

Again a good theory, time will tell how it plays out.

> 
> The Cinder community took roughly 12 months to go from a decision that
> CI was a good idea to having the requirement fully implemented. I
> believe the Manila community can do it in 6 months, by taking lessons
> learned from Cinder's experience, and also taking advantage of the
> excellent resources that the infra team has built up over the last year.

I think you have to start somewhere, so it appears you are off to a good
beginning. Don't forget Neutron has also been through this too, and has
many lessons learned. Avail yourself of the knowledge and wisdom from
that team at summit as well.

> 
> The specifics of the requirement are open to debate, as far as what
> deadlines we should enforce, and how we deal with drivers that don't
> meet the requirement, so please add feedback to this thread if you have
> any suggestions or disagreements. We will discuss this topic at the
> Manila weekly meeting tomorrow as well (May 14, 1500 UTC).
> 
> -Ben Swartzlander

I have no opinion on the Manila requirements and deadlines other than
you have some, which is a good idea. Make sure Manila team knows the
infra requirements and expectations as outlined
http://ci.openstack.org/third_party.html so your interested parties get
accurate information from the beginning.

Thanks Ben,
Anita.

> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev