<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<br>
<div class="moz-cite-prefix">On 4/6/15 2:52 PM, Morgan Fainberg
wrote:<br>
</div>
<blockquote
cite="mid:A2FF07C6-47BA-42E6-AA6F-A4C9EF13C36D@gmail.com"
type="cite">
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
<div><br>
</div>
<div><br>
On Apr 6, 2015, at 11:41, Mike Bayer <<a
moz-do-not-send="true" href="mailto:mbayer@redhat.com"><a class="moz-txt-link-abbreviated" href="mailto:mbayer@redhat.com">mbayer@redhat.com</a></a>>
wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
<div class="moz-cite-prefix">On 4/6/15 12:49 PM, David Stanek
wrote:<br>
</div>
<blockquote
cite="mid:CAO69NdmFwgJLh-=VwBV3Z-KKC+Prwb+m46HsAn95LyAZRbzK_w@mail.gmail.com"
type="cite">
<p dir="ltr">Exactly. This is the direction I have been
going. Functional tests are written using the public APIs
using the client.</p>
<p dir="ltr">I would also add that I don't like that the
Keystone unit tests are so database heavy. I would not
want MySQL or ant RDBMS to be a requirement for running
the tests.</p>
</blockquote>
is that because you'd prefer that the unit tests were more
isolated, or just that an external service is being used?<br>
<br>
I've done some work with extensive mocking of SQL databases;
specifically mocking at the ORM level. It is nice when you
get it to run, but it's also a much bigger job to write
fine-grained mocks like that, the mocks can be brittle in
relation to the code they're targeting, and you also need to
come up with some solution to actually functional test your
database access code.<br>
<br>
I tend to think that having a mysqld service running is the
lesser of two evils and you get a lot more code coverage by
going all the way out to the DB.<br>
<br>
</div>
</blockquote>
<div><br>
</div>
<div>What David is specifically referencing is that we want our
functional tests to only require direct API access. There is
almost no reason to need access to the DB backend. We have many
ways to perform isolation where needed (tempest does a lot of
this today). </div>
<div><br>
</div>
<div>The goal is to allow the functional test suite to run against
any keystone deployment (be agnostic to db, non-db, etc driver
used). This makes environment setup a separate concern the tests
don't need to be involved in/aware of. It makes our functional
tests more useful for validating a driver or configuration
passes muster. <br>
</div>
</blockquote>
<br>
ah. for sure. I would think functional tests could run against a
backend that was entirely mock / file-based. <br>
<br>
<blockquote
cite="mid:A2FF07C6-47BA-42E6-AA6F-A4C9EF13C36D@gmail.com"
type="cite">
<div><br>
</div>
<div>If there are legitimately cases we need to test a specific db
function in isolation we will make specific efforts to support
it. Those are apt to be the exception to the rule. </div>
<div><br>
</div>
<div>--Morgan</div>
<div>Sent via mobile </div>
<br>
<blockquote type="cite">
<div> <br>
<br>
<blockquote
cite="mid:CAO69NdmFwgJLh-=VwBV3Z-KKC+Prwb+m46HsAn95LyAZRbzK_w@mail.gmail.com"
type="cite"> <br>
<div class="gmail_quote">On Mon, Apr 6, 2015, 12:42 Morgan
Fainberg <<a moz-do-not-send="true"
href="mailto:morgan.fainberg@gmail.com">morgan.fainberg@gmail.com</a>>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
<br>
> On Apr 6, 2015, at 09:20, Mike Bayer <<a
moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:mbayer@redhat.com"><a class="moz-txt-link-abbreviated" href="mailto:mbayer@redhat.com">mbayer@redhat.com</a></a>>
wrote:<br>
><br>
><br>
><br>
>> On 4/6/15 12:06 PM, Clint Byrum wrote:<br>
>> Excerpts from Boris Bobrov's message of
2015-04-03 18:29:08 -0700:<br>
>>>> On Saturday 04 April 2015 03:55:59
Morgan Fainberg wrote:<br>
>>>> I am looking forward to the Liberty
cycle and seeing the special casing we<br>
>>>> do for SQLite in our migrations (and
elsewhere). My inclination is that we<br>
>>>> should (similar to the deprecation of
eventlet) deprecate support for<br>
>>>> SQLite in Keystone. In Liberty we will
have a full functional test suite<br>
>>>> that can (and will) be used to validate
everything against much more real<br>
>>>> environments instead of in-process
“eventlet-like” test-keystone-services;<br>
>>>> the “Restful test cases” will no longer
be part of the standard unit tests<br>
>>>> (as they are functional testing). With
this change I’m inclined to say<br>
>>>> SQLite (being the non-production usable
DB) what it is we should look at<br>
>>>> dropping migration support for SQLite
and the custom work-arounds.<br>
>>>><br>
>>>> Most deployers and developers (as far
as I know) use devstack and MySQL or<br>
>>>> Postgres to really suss out DB
interactions.<br>
>>>><br>
>>>> I am looking for feedback from the
community on the general stance for<br>
>>>> SQLite, and more specifically the
benefit (if any) of supporting it in<br>
>>>> Keystone.<br>
>>> +1. Drop it and clean up tons of code used
for support of sqlite only.<br>
>>><br>
>>> Doing tests with mysql is as easy, as with
sqlite ("mysqladmin drop -f;<br>
>>> mysqladmin create" for "reset"), and using
it by default will finally make<br>
>>> people test their code on real rdbmses.<br>
>> Please please please be careful with that and
make sure the database<br>
>> name is _always_ random in tests... or even
better, write a fixture to<br>
>> spin up a mysqld inside a private tempdir. That
would be a really cool<br>
>> thing for oslo.db to provide actually.<br>
>><br>
>> I'm just thinking some poor sap runs the test
suite with the wrong<br>
>> .my.cnf in the wrong place and <poof>
there went keystone's db.<br>
> The standard approach here is that tests based on
the oslo.db approach by default connect using a username
openstack_citest on localhost, which is then used to
create databases of random names. If you base your
database tests on oslo.db, you get that right now.
This username/host/etc is also configurable through
environment variables. I don't see how my.cnf is
involved in this nor how it would impact someone's
keystone database, unless we're talking about tests that
happen to load down and/or crash the whole database
server.<br>
><br>
> spinning up a whole mysqld seems really
heavy-handed and unnecessary. Not to mention the tests
run on other backends as well such as Postgresql.<br>
><br>
<br>
The reasons outlined by both Clint and Mike are why we
won't be attempting this until we are happy with our
functional test suite. But once we are happy dropping
SQLite is on the table. The way I see it the functional
tests should be performed against a real keystone
server, even if it is one spun up for testing
specifically.<br>
<br>
Per test db creation / other such stuff will be part of
that discussion.<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>