Open Stack

----- Original Message -----
> Hello Stackers,
> 
> Disclaimer: I'm on a quest to develop a new agent, already have coding a few
> lines. The next step is Unit Testing and enabling me to create a patch to
> test on a devstack. So, this is what brings me here, the need to understand
> and learn the best-pratices or rules to develop Unit Tests on the Ceilometer
> project.
> 
> I probably should mention that my Unit Tests knowledge should be classified
> as Basic (on a Java environment). I could synthesise it as: "I need to
> create a block the code and assert the output against with I expected".
> In this project seems a little bit more complex.
> I have read it about Mock and Fixtures, the first one nevertheless the
> complexity it's much more easy to read and understand.
> 
> I will try to put this problem simply as possible.
> I have analyze several files:
> 
> 
>     * ceilometer/central/test_manager.py
>     * ceilometer/compute/test_manager.py
>     * ceilometer/compute/pollsters/test_cpu.py
>     * ceilometer/agentbase.py
>     * ceilometer/base.py
> And I need to confess I'm a little lost on this part. I have realised that I
> should create a test for the manager and for every pollster and to the
> plugin if I think it is necessary.

Yes, as I pointed out yesterday when you raised this question on IRC,
there are two main threads of logic within an agent that need unit test
coverage:

* the manager's interaction with the pipeline logic: generally tested by
  extending the ceilometer.tests.agentbase.BaseAgentManagerTestCase base
  class and adding agent-specific setup/mocking & assertions

* each individual pollster's polling logic: generally tested by mocking
  up the interactions with the target interface (e.g. the hypervisor
  inspector in the case of the compute agent, or public RESTful APIs
  in the case of the central agent's pollsters)

> So that being said, I infered the following topics:
> 
> 
>     * on my test_manager.py I should declare two classes: TestManager and
>     TestRunTasks
> 
> 
>         * the first one should be responsible by test the manager.py of my
>         agent, with 'test_load_plugins' verify if there is any pollster
>         associated to this agent (seems to be straightforward)

Yes.

>         * the second one should be responsible by test the pollsters/*.py of
>         my agent and in this case the things complicate a little bit...

Note that this is not so much testing the pollsters/*.py, rather the
intent is to test the agent's interaction with the pipeline logic.

For a typical agent, most of that logic is inherited from the superclass,
ceilometer.agent.AgentManager.

Similarly, most of the test coverage is inherited from the test superclass,
ceilometer.tests.agentbase.BaseAgentManagerTestCase.

> Have I infered correctly? Anyone disagrees?

See above.

> Moving forward...
> Following the code on ceilometer/{central, compute}/test_manager.py, the
> method setUp(...) raises some questions, why to use setUp if we have already
> have a create_manager(...)? 

The purpose of the create_manager() abstract method is simply to push the
instantiation of a *single* object, the relevant agent manager, down to
the agent-specific sub-class (from a thread of logic that is *common* to
the test set up for multiple different agents).

The setUp method OTOH does a whole lot more and is *common* to each of the
agent-specific tests. It delegates to the subclass-specific create_manager()
to create the relevant agent manager:

  https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/agentbase.py#L225
 
> Should be for creating samples/alarms?

No (alarms are irrelevant to these tests).

> I have already figure out that .useFixture() comes from fixtures package:
> "The method useFixture will use another fixture, call setUp on it, call
> self.addCleanup(thefixture.cleanUp), attach any details from it and return
> the fixture. This allows simple composition of different fixtures."
> Again, lost! Where are declarations of agents/ceilometer fixtures? How should
> I use them?

Fixtures simply extend fixtures.fixture.Fixture *if* they need to do
specific setUp, cleanup, reset etc. logic. Otherwise they can simply
be mocked. In the case of the compute agent manager testcases, we need
to ensure that the compute agent's call out to nova-api is mocked up.
 
> After that every test_* in TestRunTasks should execute a different test on
> which data? In every of them I must create the sample?

Not really - the agent-specific TestRunTasks extends from the superclass:

  ceilometer.tests.agentbase.BaseAgentManagerTestCase

Please read the code for the fake pollsters used by the test:

  https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/agentbase.py#L72

and:

  https://github.com/openstack/ceilometer/blob/master/ceilometer/tests/agentbase.py#L84

Note that a "real" sample isn't needed in this case. We're asserting
that the agent's interaction with any pollster works as expected, not
that the specific *concrete* pollsters do their thing.

In this case, we simply create a "sample" that does double-duty in terms
of recording the manager and resources references that were passed by the
agent to the fake pollster's get_samples() method. It's just a tuple with
those those references, not a real sample.

> Where do I test the get_samples(), publish(), pipepline(sources, sinks)
> matters? In this files too?

The operation of get_samples() method on the individual pollsters is
asserted in the individual pollster unit test, as explained before.

The operation of the agent-manager-level logic (i.e. interaction with the
pipeline setup and publishing) is asserted in the agent manager unit test
(mainly inherited from the super-class, as stated before).

Please come to the #openstack-ceilometer IRC channel if you have further
questions on unit testing in python.

Cheers,
Eoghan

> I appreciate a lot your atention and patience with me. Thank you in advance.
> Best Regards,
> Paulo J. Nascimento Oliveira
> http://about.me/pnascimento
> Advanced Telecommunications and Networks Group - http://atnog.av.it.pt
> Follow us - @ATNoG_ITAv
> 
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>