Open Stack

It's good to have trackers for "cleaning up" the implementation of these
features. I hope we might be able soon to provide more details concerning
what cleaning up activities should happen here.

Core vs extension is often incorrectly misunderstood for "essential" vs
"optional". I believe this classification is wrong, in my opinion. Indeed I
believe that there will be a lot of people who consider the network entity
optional (why not just create subnets?), whereas there are a lot of people
who consider the floating IP concept essential (ever tried to run any sort
of app on OpenStack clouds without them?).
The API extensions serve two purposes: flexibility and evolution of the
API. While they've been successful at the former (even too much), they've
been a bit of failure at the latter. This is why, as Akihiro suggested, we
would like to move away from extensions as a mechanisms for evolving the
API, through a process which starts with condensing a lot of them into a
"super-core" or "unified API".
On the other hand, it is also important to note that this plan has not been
approved anywhere, so reviewers, in theory should stick to the "everything
is an extension" model.

Now everything I wrote in the last paragraphs is of little to no importance
in the present situation.
I think the answers we want are:
1) Does it make sense to present these attributes to *all* users?
2) If the answer to #1 is yes, what should be the correct behaviour for
plugins which are unable to understand these new attributes?


I think this is probably a good point to separate the discussion between
the MTU extension and the vlan_transparent one.

The MTU issue has been a long-standing problem for neutron users. What this
extension is doing is simply, in my opinion, enabling API control over an
aspect users were dealing with previously through custom made scripts.
It solves a fundamental issue, and it seems like it does that in a
reasonable way. The implementation is also "complete", in the sense that
there is a component shipped with neutron which supports it - the DHCP
agent. It's not merely a management layer change that then somehow
something will implement.

My personal opinion is that I welcome an appropriate MTU management. Is not
exposing details of the backend technology as it's the tenant network MTU
we are talking about.
If a plugin does not support specifically setting the MTU parameter, I
would raise a 500 NotImplemented error. This will probably create a
precedent, but as I also stated in the past, I tend to believe this might
actually be better than the hide & seek game we do with extension.

The vlan_transparent feature serves a specific purpose of a class of
applications - NFV apps.
The requirement is well understood, and the goal of these feature is to
support exactly this; It has been speculated during the review process
whether this was actually a "provider network" attribute. In theory it is
something that characterises how the network should be implemented in the
backend. However it was not possible to make this ad "admin" attribute
because also non-admins might require a vlan_transparent network. Proper
RBAC might allow us to expose this attribute only to a specific class of
users, but Neutron does not yet have RBAC [1]

Because of its nature vlan_transparent is an attribute that probably
several plugins will not be able to understand. Regardless of what the
community decides regardless extensions vs non-extension, the code as it is
implies that this flag is present in every request - defaulting to False.
This can lead to somewhat confusing situation, because users can set it to
True, and a get a 200 response. As a user, I would think that Neutron has
prepared for me a nice network which is vlan transparent... but if Neutron
is running any plugin which does not support this extension I would be in a
for a huge disappointment when I discover my network is not vlan
transparent at all!
Users need feedback, and need consistency between the desired state they
express in the API and the actual state which is implemented in the control
plane. I think this is something which should be fixed in the "cleanup
activities".
I reckon that perhaps, as a short term measure, the configuration flag
Armando mentioned might be used to obscure completely the API attribute if
a deployer chooses not to support vlan transparent networks.
Anyway, since I did not follow the review of the implementation I'll leave
to the authors and reviewers any comment pertaining implementation.

One more thing I do not see is how a transparent VLAN is implemented. Going
back to the spec history it is unclear whether this was at all in the scope
of this spec.
Indeed it seems this feature simply gives a ML2 driver a solution to
specify whether it support transparent networks or not, and then inform the
user in a fairly backend-agnostic way.
Indeed, quoting comment in the spec review [2]:

"there's no change to the implementation of the networking because we're
not changing it - so the change doesn't get down into the agents [..] But I
don't want to get into fixing the OVS driver to do VLAN trunks because it's
a can of worms and there is already a totally functional open source
implementation in core with LB"

Even if we're not gating on it, it would be great at least to clarify how
the LB agent will configure such trunk networks; because otherwise we will
have to document this attribute stating something like the following
"the vlan_transparent attribute can be set to True to request a network
which can be used for doing vlan trunks in guests. Whether these kind of
networks are attainable depends on the backend configuration. If the
neutron deployment is running a plugin which is aware of vlan transparent
networks then an error code will be returned if the vlan transparent
network cannot be implemented. If the neutron deployment is running a
plugin which is oblivious to vlan transparent networks the plugin will
simply ignore the setting even if the attribute will be successfully set on
the network resource. Unfortunately the API at the moment does not offer a
way to understand whether a particular neutron deployment is aware of
transparent vlans or not".
As a user, I would probably find this bit of documentation slightly
irritating. Snides apart, from what I gather there might be a few tweaks
needed to ensure sanity of the API in the face of attributes like this.

Salvatore

PS: One final note - I have seen several references in this thread to
approved specification as if they were immutable laws cast in stone.
Personally, I don't think that an approved specification is any sort of
binding contract between the submitter and the rest of the neutron
community - I am always glad to receive feedback on my specs after their
approval; similarly I want to be free to do object as I wish on approved
specifications, even after the relevant code is merged.

[1] https://review.openstack.org/#/c/132661/
[2] https://review.openstack.org/#/c/136554/

On 20 March 2015 at 19:54, Armando M. <armamig at gmail.com> wrote:

> In order to track this, and for Kyle's sanity, I have created these two
> RC1 bugs:
>
> - https://bugs.launchpad.net/neutron/+bug/1434667
> - https://bugs.launchpad.net/neutron/+bug/1434671
>
> Please, let's make sure that whatever approach we decide on, the resulting
> code fix targets those two bugs.
>
> Thanks,
> Armando
>
> On 20 March 2015 at 09:51, Armando M. <armamig at gmail.com> wrote:
>
>>
>>
>> On 19 March 2015 at 23:59, Akihiro Motoki <amotoki at gmail.com> wrote:
>>
>>> Forwarding my reply to the other thread too...
>>>
>>> Multiple threads on the same topic is confusing.
>>> Can we use this thread if we continue the discussion?
>>> (The title of this thread looks approapriate)
>>>
>>> ----
>>> API extension is the only way that users know which features are
>>> available unitl we support API microversioning (v2.1 or something).
>>> I believe VLAN transparency support should be implemented as an
>>> extension, not by changing the core resources attribute directly.
>>> Otherwise users (including Horizon) cannot know we field is available or
>>> not.
>>>
>>> Even though VLAN transparency and MTU suppotrs are basic features, it
>>> is better to be implemented as an extension.
>>> Configuration does not help from API perspective as it is not visible
>>> through the API.
>>>
>>
>> I was only suggesting the configuration-based approach because it was
>> simpler and it didn't lead to the evil mixin business. Granted it does not
>> help from the API perspective, but we can hardly claim good discoverability
>> of the API capabilities anyway :)
>>
>> That said, I'd be ok with moving one or both of these attributes to the
>> extension framework. I thought that consensus on having them as core
>> resources had been reached at the time the spec proposal.
>>
>>
>>>
>>> We are discussing moving away from extension attributes as Armando
>>> commented,
>>> but I think it is discussed about resources/attributes which are
>>> already used well and required.
>>> It looks natural to me that new resources/attributes are implemented
>>> via an extension.
>>> The situation may be changed once we have support of API microversioning.
>>> (It is being discussed in the context of Nova API microvesioning in
>>> the dev list started by Jay Pipes.)
>>>
>>> In my understanding, the case of IPv6 two mode is an exception.
>>> From the initial design we would like to have fully support of IPv6 in
>>> subnet resource,
>>> but through the discussion of IPv6 support it turns out some more
>>> modes are required,
>>> and we decided to change the subnet "core" resource. It is the exception.
>>>
>>> Thanks,
>>> Akihiro
>>>
>>> 2015-03-20 8:23 GMT+09:00 Armando M. <armamig at gmail.com>:
>>> > Forwarding my reply to the other thread here:
>>> >
>>> > ~~~~
>>> >
>>> > If my memory does not fail me, changes to the API (new resources, new
>>> > resource attributes or new operations allowed to resources) have
>>> always been
>>> > done according to these criteria:
>>> >
>>> > an opt-in approach: this means we know the expected behavior of the
>>> plugin
>>> > as someone has coded the plugin in such a way that the API change is
>>> > supported;
>>> > an opt-out approach: if the API change does not require explicit
>>> backend
>>> > support, and hence can be deemed supported by all plugins.
>>> > a 'core' extension (ones available in neutron/extensions) should be
>>> > implemented at least by the reference implementation;
>>> >
>>> > Now, there might have been examples in the past where criteria were
>>> not met,
>>> > but these should be seen as exceptions rather than the rule, and as
>>> such,
>>> > fixed as defects so that an attribute/resource/operation that is
>>> > accidentally exposed to a plugin will either be honored as expected or
>>> an
>>> > appropriate failure is propagated to the user. Bottom line, the server
>>> must
>>> > avoid to fail silently, because failing silently is bad for the user.
>>> >
>>> > Now both features [1] and [2] violated the opt-in criterion above: they
>>> > introduced resources attributes in the core models, forcing an
>>> undetermined
>>> > behavior on plugins.
>>> >
>>> > I think that keeping [3,4] as is can lead to a poor user experience;
>>> IMO
>>> > it's unacceptable to let a user specify the attribute, and see that
>>> > ultimately the plugin does not support it. I'd be fine if this was an
>>> > accident, but doing this by design is a bit evil. So, I'd suggest the
>>> > following, in order to keep the features in Kilo:
>>> >
>>> > Patches [3, 4] did introduce config flags to control the plugin
>>> behavior,
>>> > but it looks like they were not applied correctly; for instance, the
>>> > vlan_transparent case was only applied to ML2. Similarly the MTU
>>> config flag
>>> > was not processed server side to ensure that plugins that do not
>>> support
>>> > advertisement do not fail silently. This needs to be rectified.
>>> > As for VLAN transparency, we'd need to implement work item 5 (of 6) of
>>> spec
>>> > [2], as this extension without at least a backend able to let tagged
>>> traffic
>>> > pass doesn't seem right.
>>> > Ensure we sort out the API tests so that we know how the features
>>> behave.
>>> >
>>> > Now granted that controlling the API via config flags is not the best
>>> > solution, as this was always handled through the extension mechanism,
>>> but
>>> > since we've been talking about moving away from extension attributes
>>> with
>>> > [5], it does sound like a reasonable stop-gap solution.
>>> >
>>> > Thoughts?
>>> > Armando
>>> >
>>> > [1]
>>> >
>>> http://specs.openstack.org/openstack/neutron-specs/specs/kilo/mtu-selection-and-advertisement.html
>>> > [2]
>>> >
>>> http://specs.openstack.org/openstack/neutron-specs/specs/kilo/nfv-vlan-trunks.html
>>> > [3]
>>> >
>>> https://review.openstack.org/#/q/project:openstack/neutron+branch:master+topic:bp/mtu-selection-and-advertisement,n,z
>>> > [4]
>>> >
>>> https://review.openstack.org/#/q/project:openstack/neutron+branch:master+topic:bp/nfv-vlan-trunks,n,z
>>> > [5] https://review.openstack.org/#/c/136760/
>>> >
>>> > On 19 March 2015 at 14:56, Ian Wells <ijw.ubuntu at cack.org.uk> wrote:
>>> >>
>>> >> On 19 March 2015 at 11:44, Gary Kotton <gkotton at vmware.com> wrote:
>>> >>>
>>> >>> Hi,
>>> >>> Just the fact that we did this does not make it right. But I guess
>>> that
>>> >>> we
>>> >>> are starting to bend the rules. I think that we really need to be far
>>> >>> more
>>> >>> diligent about this kind of stuff. Having said that we decided the
>>> >>> following on IRC:
>>> >>> 1. Mtu will be left in the core (all plugins should be aware of this
>>> and
>>> >>> treat it if necessary)
>>> >>> 2. Vlan-transparency will be moved to an extension. Pritesh is
>>> working on
>>> >>> this.
>>> >>
>>> >>
>>> >> The spec started out as an extension, and in its public review people
>>> >> requested that it not be an extension and that it should instead be
>>> core.  I
>>> >> accept that we can change our minds, but I believe there should be a
>>> good
>>> >> reason for doing so.  You haven't given that reason here and you
>>> haven't
>>> >> even said who the 'we' is that decided this.  Also, as the spec
>>> author, I
>>> >> had a conversation with you all but there was no decision at the end
>>> of it
>>> >> (I presume that came afterward) and I feel that I have a reasonable
>>> right to
>>> >> be involved.  Could you at least summarise your reasoning here?
>>> >>
>>> >> I admit that I prefer this to be in core, but I'm not terribly choosy
>>> and
>>> >> that's not why I'm asking.  I'm more concerned that this is changing
>>> our
>>> >> mind at literally the last moment, and in turn wasting a developer's
>>> time,
>>> >> when there was a perfectly good process to debate this before coding
>>> was
>>> >> begun, and again when the code was up for review, both of which
>>> apparently
>>> >> failed.  I'd like to understand how we avoid getting here again in the
>>> >> future.  I'd also like to be certain we are not simply reversing a
>>> choice on
>>> >> a whim.
>>> >> --
>>> >> Ian.
>>> >>
>>> >>
>>> __________________________________________________________________________
>>> >> OpenStack Development Mailing List (not for usage questions)
>>> >> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >>
>>> >
>>> >
>>> >
>>> __________________________________________________________________________
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>>
>>> --
>>> Akihiro Motoki <amotoki at gmail.com>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150320/cc92c773/attachment.html>