Open Stack

On Fri, Aug 26, 2016 at 09:48:26AM -0700, Clark Boylan wrote:
> On Fri, Aug 26, 2016, at 09:03 AM, Joshua Harlow wrote:
> > Hi folks (dev and more!),
> > 
> > I was having a conversation with some folks at godaddy around our future 
> > plans for a developer lab (where we can have various setups of 
> > networking, compute, storage...) for 'exploring' purposes (testing out a 
> > new LBAAS for example or ...) and as well as for 'developer' purposes 
> > (aka, the 'I need to reproduce a bug or work on a feature that requires 
> > having a setup mimicking closer to what we have in staging or
> > production').
> > 
> > And it got me thinking about how other developers (and other companies) 
> > are doing this. Do various companies have shared labs that their 
> > developers get partitions of for (periods of) usage (for example for a 
> > volume vendor I would expect this) or if you are a networking company do 
> > you hand out miniature networks (with associated gear) as needed (or do 
> > you build out such labs via SDN and software only)?
> > 
> > Then of course there are the people developing libraries (somewhat of my 
> > territory), part of that development can just be done locally and 
> > running of tox and such via that, but often times even that is not 
> > sufficient (for example pick oslo.messaging or oslo.db, altering this in 
> > ways that could pass unittests could still end up breaking its 
> > integration with other projects); so the gate helps here (but the gate 
> > really is a 'last barrier') so have folks that have been working on say 
> > zeromq or the newer amqp versions, what is the daily life of testing and 
> > exploring features and development for you?
> > 
> > Are any of the environments that people may be getting build-out on 
> > demand (ie in a cloud-like manner)? For example I could see how it could 
> > be pretty nifty to request a environment be built out with say something 
> > like the following as a descriptor language:
> > 
> > build_out:
> >     nova:
> >        git_url: git://git.openstack.org/openstack/nova
> >        git_ref: ....
> >     neutron:
> >        git_url: <my personal git repo>
> >        git_ref: my sha
> >     ....
> >     topology:
> >        use_standard_config: true
> >        build_with_switch_type: XYZ...
> > 
> > I hope this info is not just useful to myself (and maybe it's been 
> > talked about before, but nothing of recent that I can recall) and I'd be 
> > very much interested in hearing what other companies (big and small) are 
> > doing here (and also from folks that are not associated with any 
> > company, which I guess brings in the question of the OSIC lab).
> 
> As someone that semi frequently has to reproduce gate results my current
> setup involves semi frequently building the infra test images locally
> using openstack-infra/project-config/tools/build-image.sh then booting
> this image on my workstation using kvm. With that I can easily run a
> devstack-gate reproduce.sh or tox -e whatever and have a high degree of
> confidence that my setup mirrors the gate's.
> 
> When I worked for HP I did similar on top of HPCloud. This actually
> worked very well as I could much more easily share the results. If I had
> my choice of setup I would just ask for a set of openstack cloud
> credentials with a reasonable amount of quota. Dogfooding has been a
> great way to understand how openstack works and I think it has produced
> valuable feedback helping openstack improve.
> 
> Granted this is much harder to hand out when you need specific hardware
> resources (network switches, server hardware, whatever), but with Ironic
> most of that should be doable with the "give devs cloud credentials"
> model.
> 
> As for requesting an environment with nova version foo and neutron
> version bar and topology baz devstack does this a few thousand times per
> day. It may not be everyone's preferred tool, but my suggestion would be
> to not make another tool that does this and never gets tested. Instead
> use what is being tested.
> 
> TL;DR dogfood and use openstack, it solves this problem well.
> 
With zuulv2.5 we've started to make this easier too, we now publish our
ansible-playbooks along side our job logs[1]. I have visions where we also
publish minimal images create by nodepool (minus openstack-infra SSH keys and
git / package cache). Then write a new ansible-playbook to bootstrap a node to
your local environment (ansible-role-cloud-launcher) and or public cloud
account, then fire off the same ansible-playbook we did in zuul-launcher
(obviously using the reproduce.sh script).

It sounds like a mouth full, but I don't actually think it will be too much
work.  I've started pushing up patches to zuul-launcher to do this, but haven't
asked for the code to be merged. Mostly because I think we want to wait until
zuulv3 for this.

[1] http://logs.openstack.org/49/361449/2/check/gate-puppet-iptables-puppet-lint/797d4da/_zuul_ansible/
[2] https://review.openstack.org/#/c/352627/