Open Stack

On Oct 24, 2014, at 6:58 AM, Chris Dent <chdent at redhat.com> wrote:

> On Thu, 23 Oct 2014, Doug Hellmann wrote:
> 
> Thanks for the feedback Doug, it's useful.
> 
>> WebTest isn’t quite what you’re talking about, but does provide a
>> way to talk to a WSGI app from within a test suite rather simply. Can
>> you expand a little on why “declarative” tests are better suited
>> for this than the more usual sorts of tests we write?
> 
> I'll add a bit more on why I think declarative tests are useful, but
> it basically comes down to explicit transparency of the on-the-wire
> HTTP requests and responses. At least within Ceilometer and at least
> for me, unittest form tests are very difficult to read as a
> significant portion of the action is somewhere in test setup or a
> super class. This may not have much impact on the effectiveness of the
> tests for computers, but it's a total killer of the effectiveness of
> the tests as tools for discovery by a developer who needs to make
> changes or, heaven forbid, is merely curious. I think we can agree
> that a more informed developer, via a more learnable codebase, is a
> good thing?

OK, at first I thought you were talking about writing out literal HTTP request/response sets, but looking at the example YAML file in [1] I see you’re doing something more abstract. I was worried about minor changes in a serialization library somewhere breaking a bunch of tests by changing their formatting in insignificant ways, but that shouldn’t be a problem if you’re testing the semantic contents rather than the literal contents of the response.

[1] https://github.com/tiddlyweb/tiddlyweb/blob/master/test/httptest.yaml

> 
> I did look at WebTest and while it looks pretty good I don't particular
> care for its grammar. This might be seen as a frivolous point (because
> under the hood the same thing is happening, and its not this aspect that
> would be exposed the declarations) but calling a method on the `app`
> provided by WebTest feels different than using an http library to make
> a request of what appears to be a web server hosting the app.
> 
> If people prefer WebTest over wsgi-intercept, that's fine, it's not
> something worth fighting about. The positions I'm taking in the spec,
> as you'll have seen from my responses to Eoghan, are trying to emphasize
> the focus on HTTP rather than the app itself. I think this could lead,
> eventually, to better HTTP APIs.

I’m not familiar with wsgi-intercept, so I can’t really comment on the difference there. I find WebTest tests to be reasonably easy to read, but since (as I understand it) I would write YAML files instead of Python tests, I’m not sure I care which library is used to build the tool as long as we don’t have to actually spin up a web server listening on a network port in order to run tests.

> 
>> I definitely don’t think the ceilometer team should build
>> something completely new for this without a lot more detail in the
>> spec about which projects on PyPI were evaluated and rejected as not
>> meeting the requirements. If we do need/want something like this I
>> would expect it to be built within the QA program. I don’t know if
>> it’s appropriate to put it in tempestlib or if we need a
>> completely new tool.
> 
> I don't think anyone wants to build a new thing unless that turns out
> to be necessary, thus this thread. I'm hoping to get input from people
> who have thought about or explored this before. I'm hoping we can
> build on the shoulders of giants and all that. I'm also hoping to
> short circuit extensive personal research by collaborating with others
> who may have already done it before.
> 
> I have an implementation of the concept that I've used in a previous
> project (linked from the spec) which could work as a starting point
> from which we could iterate but if there is something better out there
> I'd prefer to start there.
> 
> So the questions from the original post still stand, all input
> welcome, please and thank you.
> 
>> * Is this a good idea?
>> * Do other projects have similar ideas in progress?
>> * Is this concept something for which a generic tool should be
>>  created _prior_ to implementation in an individual project?
>> * Is there prior art? What's a good format?
> 
> -- 
> Chris Dent tw:@anticdent freenode:cdent
> https://tank.peermore.com/tanks/cdent_______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev