Open Stack

On 06/09/2013 04:40 AM, Attila Fazekas wrote:
<snip>
> Long time ago I removed the compute_admin credentials from the config file,
> and you was not happy about this.
> Do you need it nowadays?

compute_admin is merely a role. Nothing more. By removing it that 
original time, you were equating a keystone admin with a compute admin, 
and that was wrong. And it's still wrong.

> Extending into the roles can have two goals:
> - Covering deployment scenarios when the administrative roles spitted to multiple teams.
>   For example a network admin does not have admin permission on the Image system.

I'm in complete agreement with you on this, Attila. If you look at my 
proof of concept code, you will see I am using role-based test user 
management:

https://github.com/jaypipes/tempest/blob/api/tempest/tests/api/base.py#L68

> - Covering policy decisions
>   The tempest in this is very weak, if any of the component forgets to
>   validate the  user-role-teant-policy combination in a particular step, we would not see it.
>   In this area an user without any capability would be the simplest and most beneficial "tool".
>   But the reality is little bit more complex, if we want it to test fully.

I support this 100%. No idea why you think I do not support this.

>>>    * The identity admin credentials does not necessarily be known (not my
>>>    use case), AFAIK several user
>>>     wants to use tempest just with the predefined users.
>>>     (If this use case does not really exists, it can simplify lot of things)
>>
>> Are you asking whether Tempest should not create users for use in its
>> tests? As I've said previously, this was done for two reasons:
>>
>> 1) To get past many issues with quotas for a tenant
>
> Those very bad leakages are fixed since long, the other will be fixed soon,
> and leakage  will be prevented.
> https://blueprints.launchpad.net/tempest/+spec/stop-leaking

It wasn't at all about leakages. It was about quotas being reached for a 
tenant and Tempest needing to either extend a tenant's quota or wait 
while resources were spun down. Nothing about this has changed.

>> 2) To provide the basis for parallel execution -- if you are running X
>> compute tests in parallel with the exact same user, you are needlessly
>> going to run into issues where calls to things like GET
>> /<TENANT>/servers is going to return non-deterministic results between
>> processes
>>
>> What are you thinking will be simplified by removing Tempest's test
>> tenant/user isolation?
>
> Yes.
> Our list tests are written in way which tolerate this situation.
> If you find it works in different way, it is a HIGH priority issue.

We go through hoops in some tests to do this, and we wouldn't need to if 
we could rely on an isolated tenant for each test case. That was the 
point of tenant isolation in the first place: to provide guarantees that 
the tenant's set of resources would not be modified outside of the test 
case itself.

> If we really want to test something in clean tenant, we can create tenants.

We already do this.

> These test will not be part of the service provider OS API validation test case set.
> Bit if we need it for something what worth to test, we will test it, and create new tenant.

I have no idea why you think we do not *already* do this. We already 
operation in tenant isolation in the gate. For each test case class, a 
new tenant and user is created in Keystone.

Best.
-jay

>>> 2. keystone stress
>>>    * if we have all test cases to request multiple tokens
>>>      and create users and tenants and set the quota set,  it will be a lot
>>>      of overhead.
>>
>> The volume is the same on Keystone tokens regardless of number of
>> Keystone users you use to run tests. In the default Keystone, every
>> request to any endpoint is going to involve a token, and that token will
>> need to be validated on each REST call. Makes no difference whether it's
>> 1 user or >1 user running the tests.
>>
>>>    * token request if the token request math /test case is
>>>      number_of_used_users * number_of_used_clients *
>>>      number_of_impersonate_switch,
>>>      is not too efficient.
>>
>> That is incorrect. Number of token requests is number of requests made
>> to any endpoint. Has nothing to do with number of users.
>
> I am speaking about the token-get requests: http://paste.openstack.org/show/38294/
>
> The token-get response contains all endpoints and a token (id) which usable with all services.
>
> The tempest current model does not takes it into consideration.
>
> You need to do a new token get only once per user in 24h.
> You juts need a new token when the old one is expired.
>
>> With PKI, the auth_token middlewares can cache a user's token and reduce
>> the number of requests to the Keystone API server, true, and the number
>> of users would affect the number of cached tokens the auth_token
>> middleware would cache.  But, I believe that focusing on these
>> micro-performance issues/optimizations is missing the point of using
>> multiple users: to better isolate the test cases from the actions of
>> other users.
>
> Yes, I am speaking about the token-get, not about the token validation.
>
>>>      The RSA signing is not cheap operation, the request/response latency is
>>>      measurable side
>>>      effect anyway.
>>>      Does it scales well with multi-thread ?.
>>>      Keystone uses cPython and we just have 1 keystone process on the gate.
>>
>> This is a deployment issue, frankly. We can change the gate to put
>> Keystone in an Apache wsgi servlet container and multiplex Keystone API
>> that way.
>>
>>> 3. multiple client tool chain usage
>>>    * how will this model will be usable for multiple client usage /per test
>>>    case ?
>>
>> Not sure what you mean by this. Could you elaborate a bit more, perhaps
>> with an example piece of code?
>
> See above.
>>> 4. resource sharing
>>>    * several resource creation does not scales well for example the server
>>>    creation.
>>>      The owner tenant members has more capabilities on given server
>>>      resource,
>>>      if you want to combine the servers with another resources, the owner
>>>      tenant is a more important fact.
>>
>> You lost me on this one, Attila :) I don't understand what you mean by this.
>
> If you create a server in tenant A and you try do anything with an user who
>   just have roles on tenant B.
> - You cannot see the server
> - You cannot attache volume/.. created in tenant B..
>
> --> If you create a new dynamic user in dynamic tenant,
>   you cannot touch the existing resources.
>
> (These negative test are mostly missing)
>
>> Best,
>> -jay
>>
>>> ----- Original Message -----
>>>> From: "Jay Pipes" <jaypipes at gmail.com>
>>>> To: openstack-qa at lists.openstack.org
>>>> Sent: Thursday, June 6, 2013 4:04:35 PM
>>>> Subject: Re: [openstack-qa] Tempest Architecture/Design brainstorming
>>>>
>>>> On 06/06/2013 01:57 AM, Attila Fazekas wrote:
>>>>> Hi All,
>>>>>
>>>>> I am looking for ideas for unifying the tempest "Object Model" and to
>>>>> enhance it's capabilities.
>>>>>
>>>>> Several interesting topic:
>>>>> - Plug-ability
>>>>> - Identity management ("impersonate the right person efficiently")
>>>>
>>>> I put together a proof of concept of this exact functionality using
>>>> fixtures.Fixture and a context manager called acting_as() [1] that was
>>>> extended by subclasses' do_as() method [2] to make it easy to write
>>>> clean impersonation code [3]:
>>>>
>>>> class TestFlavorsNonAdmin(base.ComputeApiTest):
>>>>
>>>>        def test_list_flavors(self):
>>>>            with self.do_as('non_admin') as client:
>>>>                flavors = client.flavors.list()
>>>>                self.assertTrue(len(flavors) > 0)
>>>>
>>>>        def test_unauthorized(self):
>>>>            with self.do_as('non_admin') as client:
>>>>                self.assertRaises(Exception, client.flavors.create,
>>>> 'flavorname')
>>>>
>>>>
>>>> class TestFlavorsAdmin(base.ComputeApiTest):
>>>>
>>>>        def test_list_flavors(self):
>>>>            with self.do_as('admin') as client:
>>>>                flavors = client.flavors.list()
>>>>                self.assertTrue(len(flavors) > 0)
>>>>
>>>>        def test_crud(self):
>>>>            with self.do_as('admin') as client:
>>>>                flavor_name = self.getUniqueString("flavor")
>>>>                flavor = client.flavors.create(flavor_name, 64, 1, 10)
>>>>                self.assertEqual(flavor_name, flavor.name)
>>>>                self.assertEqual(64, flavor.ram)
>>>>                self.assertEqual(1, flavor.vcpus)
>>>>                self.assertEqual(10, flavor.disk)
>>>>                flavors = client.flavors.list()
>>>>                flavor_ids = [x.id for x in flavors]
>>>>                self.assertIn(flavor.id, flavor_ids)
>>>>                flavor = client.flavors.find(name=flavor_name)
>>>>                self.assertEqual(flavor_name, flavor.name)
>>>>                self.assertEqual(64, flavor.ram)
>>>>                self.assertEqual(1, flavor.vcpus)
>>>>                self.assertEqual(10, flavor.disk)
>>>>                client.flavors.delete(flavor.id)
>>>>
>>>> Best,
>>>> -jay
>>>>
>>>> [1]
>>>> https://github.com/jaypipes/tempest/blob/api/tempest/tests/api/base.py#L89
>>>> [2]
>>>> https://github.com/jaypipes/tempest/blob/api/tempest/tests/api/compute/base.py#L27
>>>> [3]
>>>> https://github.com/jaypipes/tempest/blob/api/tempest/tests/api/compute/test_flavors.py
>>>>
>>>>
>>>>> - test case selection (based on supported feature, installed services..)
>>>>>
>>>>> An ideal proposal should answer the
>>>>> - What you would like to achieve.
>>>>> - Why do you want/need this.
>>>>> - How do you want to do it.
>>>>>
>>>>> Feel free to add incomplete idea as well.
>>>>>
>>>>> Best Regards,
>>>>> Attila
>>>>>
>>>>> PS.: python is not juts OOP language,
>>>>> we should use the imperative and functional programming styles as well.
>>>>>
>>>>> _______________________________________________
>>>>> openstack-qa mailing list
>>>>> openstack-qa at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> openstack-qa mailing list
>>>> openstack-qa at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>>
>>>
>>> _______________________________________________
>>> openstack-qa mailing list
>>> openstack-qa at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>
>>
>>
>> _______________________________________________
>> openstack-qa mailing list
>> openstack-qa at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>