Open Stack

Akihiro Motoki <amotoki at gmail.com> writes:

> I added 'oslo' and 'cliff' subject tag as well.
>
> 2015-12-31 1:37 GMT+09:00 Sofer Athlan-Guyot <sathlang at redhat.com>:
>> Hi,
>>
>> I have added neutron people as they may help to find a solution.
>>
>> After banging my head against the wall for a whole afternoon I
>> discovered a serious bug in puppet neutron module.
>>
>> I not going to repeat here the detail of the bug report[1].  Basically:
>>  - neutron_port
>>  - neutron_subnet
>>  - neutron_router
>>  - neutron_network
>>
>> may break idempotency randomly and won't work at all when clifftablib is
>> removed from the package dependency of python-openstackclient
>> (Mitaka[2])
>>
>> So the problem is that neutron cli json output on liberty (at least, and
>> maybe before) is not consistent and may come from cliff or clifftablib.
>> I didn't test it but the same may apply to openstack cli. As we don't
>> use the openstack cli json output it's not a issue (for puppet modules)
>
> Apparently the problem comes from the fact the JSON output from
> cliff 1.15.0 and cliff-tablib are different.

Yes indeed.  One is a clean JSON (cliff), the other is a JSON "table"
with "Field" and "Value" as key for every parameter.

>
> Another tricky point is that we have two entry points with a single name.
> If we install cliff 1.15.0 and cliff-tablib, we will see two json and two yaml
> formatters in the help message....

Yes.  The problem being that whichever comes first win.  It's usually
clifftab, but no guaranty.

>
>> The available solution I can see are:
>>  1. go back to parsing csv, shell output (revert [3])
>>  2. find a way to leverage openstacklib parsing for neutron as well
>>  3. keep json and parse the right output (cliff) and find a way to make
>>     sure that it is always used by stevedore
>>  4. ?
>
> A variation of option 3 is to release a new version of cliff-tablbi
> without  json and yaml formatter.
> cliff 1.15.0 and later supports JSON and YAML formatters,
> so there is no need that cliff-tablib supports them.
>

Actually the clifftab dependency comes from openstackcli.  This will be
completly remove in Mitaka.  Packager will adjust then I guess.

> When cliff 1.14.0 is used in some distribution, we can use the current version
> of cliff-tablib. If someone uses the latest versions, he can use cliff
> 1.15.0 and
> a new cliff-tablib without json and yaml formatters.
> I believe it helps this confusing situation in liberty.
>
>> From my point of view 3) is not a option, but other may disagree.
>
> My vote is option 3. cliff (or neutronclient as workaround) can ensure
> that either of cliff json formatter is chosen if two json formatters
> are available.

Hum,  not sure I get this right.  neutronclient will get what stevedore
gives him as formatter and it cannot be "set" at run time.  You have to
edit file and disable the json output you don't like.  Not something
neutronclient can do.

> However, I don't think it is a good idea that neutronclient cannot
> provide cliff-tablib version
> of JSON formatter if cliff 1.15.0 is installed.

This double negation with conditional lost me :) You mean that it if
cliff 1.15.0 is installed then neutronclient should use cliff-tablib ?

> Can puppet-neutron can handle the difference?

Well, the problem is that we have to check the output of the cli for
warning or the like that get mixed with the actual output.  So we rely
on a starting tag.  This tag is different if cliff - a plain "{" - or
clifftab - a "[{" - is used (on a limited set of test I did, there could
be more subtleties here).  Then we have to maintain two parsers one for
clifftab and one for cliff.  The more I think about it the less I can
see a proper way to maintain this.

Then there is the fact that the clifftab JSON output is not the expected
one.


For the record I end up reverting the patch[1] (solution 1) and adding
the code to filter the output[2] (which was the original motivation for
switching to json).  It was fast, didn't have anything to handle on the
json side and seems to work ok.

[1] https://review.openstack.org/262809
[2] https://review.openstack.org/263874

>
> Akihiro
>
>>
>> The problem is tricky and the fact that the CI cannot detect this is
>> trickier[4].
>>
>> So before Mitaka, the json parsing should go.  I would love to see an
>> interface that all puppet modules would use (solution 2).  The current
>> openstacklib parses openstack client well enough.  The neutron command
>> is not that different and I think there is space for code reuse.  This
>> would be a long term solution.  It would bring the advantage of having
>> only one interface to change if it was decided to use the API directly
>> for instance[5]
>>
>> In the meantime, a quick solution to this bug must be found.
>>
>> Looking forward to your comments.
>>
>> Regards,
>>
>> [1] https://bugs.launchpad.net/puppet-neutron/+bug/1530163
>> [2] https://bugs.launchpad.net/python-neutronclient/+bug/1529914
>> [3] https://review.openstack.org/#/c/238156/
>> [4] https://review.openstack.org/#/c/262223/
>> [5] http://lists.openstack.org/pipermail/openstack-dev/2015-October/076439.html
>> --
>> Sofer Athlan-Guyot
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
Sofer Athlan-Guyot