Open Stack

On Mon, Jan 12, 2015 at 1:20 PM, Chris Dent <chdent at redhat.com> wrote:

>
> After some discussion with Sean Dague and a few others it became
> clear that it would be a good idea to introduce a new tool I've been
> working on to the list to get a sense of its usefulness generally,
> work towards getting it into global requirements, and get the
> documentation fleshed out so that people can actually figure out how
> to use it well.
>
> tl;dr: Help me make this interesting tool useful to you and your
> HTTP testing by reading this message and following some of the links
> and asking any questions that come up.
>
> The tool is called gabbi
>
>     https://github.com/cdent/gabbi
>     http://gabbi.readthedocs.org/
>     https://pypi.python.org/pypi/gabbi
>
> It describes itself as a tool for running HTTP tests where requests
> and responses are represented in a declarative form. Its main
> purpose is to allow testing of APIs where the focus of test writing
> (and reading!) is on the HTTP requests and responses, not on a bunch of
> Python (that obscures the HTTP).
>
>
Hi Chris,

I'm interested, sure. What did you use to write the HTTP tests, as in, what
was the source of truth for what the requests and responses should be?

Thanks,
Anne


> The tests are written in YAML and the simplest test file has this form:
>
> ```
> tests:
> - name: a test
>   url: /
> ```
>
> This test will pass if the response status code is '200'.
>
> The test file is loaded by a small amount of python code which transforms
> the file into an ordered sequence of TestCases in a TestSuite[1].
>
> ```
> def load_tests(loader, tests, pattern):
>     """Provide a TestSuite to the discovery process."""
>         test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
>         return driver.build_tests(test_dir, loader, host=None,
>                                   intercept=SimpleWsgi,
>                                   fixture_module=sys.modules[__name__])
> ```
>
> The loader provides either:
>
> * a host to which real over-the-network requests are made
> * a WSGI app which is wsgi-intercept-ed[2]
>
> If an individual TestCase is asked to be run by the testrunner, those tests
> that are prior to it in the same file are run first, as prerequisites.
>
> Each test file can declare a sequence of nested fixtures to be loaded
> from a configured (in the loader) module. Fixtures are context managers
> (they establish the fixture upon __enter__ and destroy it upon
> __exit__).
>
> With a proper group_regex setting in .testr.conf each YAML file can
> run in its own process in a concurrent test runner.
>
> The docs contain information on the format of the test files:
>
>     http://gabbi.readthedocs.org/en/latest/format.html
>
> Each test can state request headers and bodies and evaluate both response
> headers and response bodies. Request bodies can be strings in the
> YAML, files read from disk, or JSON created from YAML structures.
> Response verifcation can use JSONPath[3] to inspect the details of
> response bodies. Response header validation may use regular
> expressions.
>
> There is limited support for refering to the previous request
> to construct URIs, potentially allowing traversal of a full HATEOAS
> compliant API.
>
> At the moment the most complete examples of how things work are:
>
> * Ceilometer's pending use of gabbi:
>   https://review.openstack.org/#/c/146187/
> * Gabbi's testing of gabbi:
>   https://github.com/cdent/gabbi/tree/master/gabbi/gabbits_intercept
>   (the loader and faked WSGI app for those yaml files is in:
>   https://github.com/cdent/gabbi/blob/master/gabbi/test_intercept.py)
>
> One obvious thing that will need to happen is a suite of concrete
> examples on how to use the various features. I'm hoping that
> feedback will help drive that.
>
> In my own experimentation with gabbi I've found it very useful. It's
> helped me explore and learn the ceilometer API in a way that existing
> test code has completely failed to do. It's also helped reveal
> several warts that will be very useful to fix. And it is fast. To
> run and to write. I hope that with some work it can be useful to you
> too.
>
> Thanks.
>
> [1] Getting gabbi to play well with PyUnit style tests and
>     with infrastructure like subunit and testrepository was one of
>     the most challenging parts of the build, but the result has been
>     a lot of flexbility.
>
> [2] https://pypi.python.org/pypi/wsgi_intercept
> [3] https://pypi.python.org/pypi/jsonpath-rw
>
> --
> Chris Dent tw:@anticdent freenode:cdent
> https://tank.peermore.com/tanks/cdent
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150112/24904766/attachment-0001.html>