[neutron][OVN] Multiple mechanism drivers
We are looking at using OVN and are having some issues with it in our ML2 environment.
We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network)
Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc. It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks.
Am I missing something here? (We’re using Stein version)
Thanks, Sam
Hi,
I think that this may be true that networking-ovn will not work properly with other drivers. I don't think it was tested at any time. Also the problem may be that when You are using networking-ovn than whole neutron topology is different. There are different agents for example.
Please open a bug for that for networking-ovn. I think that networking-ovn team will take a look into that.
On Mon, Nov 25, 2019 at 04:32:50PM +1100, Sam Morrison wrote:
We are looking at using OVN and are having some issues with it in our ML2 environment.
We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network)
Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc. It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks.
Am I missing something here? (We’re using Stein version)
Thanks, Sam
hi,
On Mon, Nov 25, 2019 at 5:00 PM Slawek Kaplonski skaplons@redhat.com wrote:
Hi,
I think that this may be true that networking-ovn will not work properly with other drivers. I don't think it was tested at any time. Also the problem may be that when You are using networking-ovn than whole neutron topology is different. There are different agents for example.
Please open a bug for that for networking-ovn. I think that networking-ovn team will take a look into that.
networking-midonet ignores networks without "midonet" type segments to avoid interfering other mechanism drivers. maybe networking-ovn can have something similar.
wrt agents, last time i checked there was no problem with running midonet agent and ovs agent on the same host, sharing the kernel datapath. so i guess there's no problem with ovn either.
wrt l3, unfortunately neither midonet or ovn have implemented "l3 flavor" thing yet. so you have to choose a single l3 plugin. iirc, Sam's deployment doesn't use l3 for linuxbridge, right?
On Mon, Nov 25, 2019 at 04:32:50PM +1100, Sam Morrison wrote:
We are looking at using OVN and are having some issues with it in our ML2 environment.
We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network)
Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc. It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks.
Am I missing something here? (We’re using Stein version)
Thanks, Sam
-- Slawek Kaplonski Senior software engineer Red Hat
On Thu, 2019-11-28 at 11:12 +0900, Takashi Yamamoto wrote:
hi,
On Mon, Nov 25, 2019 at 5:00 PM Slawek Kaplonski skaplons@redhat.com wrote:
Hi,
I think that this may be true that networking-ovn will not work properly with other drivers. I don't think it was tested at any time.
it should work with other direver if you use vlan or flat networks. it will not form mesh tunnel networks with other drivers event if you use geneve for the other ml2 driver.
Also the problem may be that when You are using networking-ovn than whole neutron topology is different. There are different agents for example.
Please open a bug for that for networking-ovn. I think that networking-ovn team will take a look into that.
networking-midonet ignores networks without "midonet" type segments to avoid interfering other mechanism drivers. maybe networking-ovn can have something similar.
that is actully the opiste of how that shoudl work. you are ment to be able to have multiple ml2 drivers share the same segmentation type and you are not ment to have a segmentation type that is specific to a mech driver. give we dont scheduler based on the segmenation type supprot today either (we shoudl by the way) it would be very fagile to use a dedicated ovn segmentation type and i woudl not advise doing it for midio net.
ideally we would create placement aggregate or triats to track which segmentation types are supported by which hosts. traits are proably better for the segmentation types but modelling network segments them selves would be better as aggreates.
if we really wanted to model the capsity of the segmenation types we woudl addtionally create shareing resouce providers with inventories of network segmenation types resouce classes per physnet with a singel gloabl rp for the tunneled types. then every time you allocated a network in neuton you would create an allocation for that network and tag ports with the approreate aggreate requrest.
on the nova side wew could combine the segment and segmenation type aggreate request form the port with any other aggreates form nova and pass all of them as member_of requriements to placment to ensure we land on a host that can provide the required network connectivty. today we litrallly just assme all nodes are connected to all networks with all segmenation types and hope for the best.
thats a bit of a tangent but just pointing out we should schduler on network connectivity and segmenation types but we shoudl not have backend specific segmenation types.
wrt agents, last time i checked there was no problem with running midonet agent and ovs agent on the same host, sharing the kernel datapath. so i guess there's no problem with ovn either.
you can run ml2/ovn and ml2/ovs on the same cloud. just put the ml2/ovs first. it will fail to bind if a host dose not have the ovs neutron agent running and will then bind with ml2/ovn instead.
it might work the other way too but i have nto tested that.
wrt l3, unfortunately neither midonet or ovn have implemented "l3 flavor" thing yet. so you have to choose a single l3 plugin. iirc, Sam's deployment doesn't use l3 for linuxbridge, right?
if you have dedicated network nodes that is not really a proable. just make sure that they are all ovn or all ovs or whatever makes sense. its the same way that if you deploy with ml2/ovs and want to use ovs-dpdk that you only instlal ovs-dpdk on the comptue nodes and use kernel ovs on the networking nodes to avoid the terible network performace when using network namespace for nat and routing.
if you have tunneled networks it would be an issue butin that case you just need to ensure that at least 1 router is created on each plugin so you would use ha routers by default and set the ha factor so that it created routers on node with both mechinmum dirvers. again however since the different ml2/driver do not form a mesh you should really only use different ml2 drivers if you are using vlan or flat networks.
On Mon, Nov 25, 2019 at 04:32:50PM +1100, Sam Morrison wrote:
We are looking at using OVN and are having some issues with it in our ML2 environment.
We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network)
Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc.
i would try adding ovn last so it is only used if the other two cannot bind the port. the mech driver list is orderd for this reason so you can express preference.
It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks.
that would be a bug if so.
Am I missing something here? (We’re using Stein version)
Thanks, Sam
-- Slawek Kaplonski Senior software engineer Red Hat
On Thu, Nov 28, 2019 at 9:27 PM Sean Mooney smooney@redhat.com wrote:
On Thu, 2019-11-28 at 11:12 +0900, Takashi Yamamoto wrote:
hi,
On Mon, Nov 25, 2019 at 5:00 PM Slawek Kaplonski skaplons@redhat.com wrote:
Hi,
I think that this may be true that networking-ovn will not work properly with other drivers. I don't think it was tested at any time.
it should work with other direver if you use vlan or flat networks. it will not form mesh tunnel networks with other drivers event if you use geneve for the other ml2 driver.
Also the problem may be that when You are using networking-ovn than whole neutron topology is different. There are different agents for example.
Please open a bug for that for networking-ovn. I think that networking-ovn team will take a look into that.
networking-midonet ignores networks without "midonet" type segments to avoid interfering other mechanism drivers. maybe networking-ovn can have something similar.
that is actully the opiste of how that shoudl work. you are ment to be able to have multiple ml2 drivers share the same segmentation type and you are not ment to have a segmentation type that is specific to a mech driver. give we dont scheduler based on the segmenation type supprot today either (we shoudl by the way) it would be very fagile to use a dedicated ovn segmentation type and i woudl not advise doing it for midio net.
IMO it doesn't make sense to use the same segmentation types unless the on-wire protocol among nodes is actually compatible with the reference implementations. (in case of midonet, it isn't.)
ideally we would create placement aggregate or triats to track which segmentation types are supported by which hosts. traits are proably better for the segmentation types but modelling network segments them selves would be better as aggreates.
if we really wanted to model the capsity of the segmenation types we woudl addtionally create shareing resouce providers with inventories of network segmenation types resouce classes per physnet with a singel gloabl rp for the tunneled types. then every time you allocated a network in neuton you would create an allocation for that network and tag ports with the approreate aggreate requrest.
on the nova side wew could combine the segment and segmenation type aggreate request form the port with any other aggreates form nova and pass all of them as member_of requriements to placment to ensure we land on a host that can provide the required network connectivty. today we litrallly just assme all nodes are connected to all networks with all segmenation types and hope for the best.
thats a bit of a tangent but just pointing out we should schduler on network connectivity and segmenation types but we shoudl not have backend specific segmenation types.
wrt agents, last time i checked there was no problem with running midonet agent and ovs agent on the same host, sharing the kernel datapath. so i guess there's no problem with ovn either.
you can run ml2/ovn and ml2/ovs on the same cloud. just put the ml2/ovs first. it will fail to bind if a host dose not have the ovs neutron agent running and will then bind with ml2/ovn instead.
it might work the other way too but i have nto tested that.
wrt l3, unfortunately neither midonet or ovn have implemented "l3 flavor" thing yet. so you have to choose a single l3 plugin. iirc, Sam's deployment doesn't use l3 for linuxbridge, right?
if you have dedicated network nodes that is not really a proable. just make sure that they are all ovn or all ovs or whatever makes sense. its the same way that if you deploy with ml2/ovs and want to use ovs-dpdk that you only instlal ovs-dpdk on the comptue nodes and use kernel ovs on the networking nodes to avoid the terible network performace when using network namespace for nat and routing.
if you have tunneled networks it would be an issue butin that case you just need to ensure that at least 1 router is created on each plugin so you would use ha routers by default and set the ha factor so that it created routers on node with both mechinmum dirvers. again however since the different ml2/driver do not form a mesh you should really only use different ml2 drivers if you are using vlan or flat networks.
On Mon, Nov 25, 2019 at 04:32:50PM +1100, Sam Morrison wrote:
We are looking at using OVN and are having some issues with it in our ML2 environment.
We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network)
Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc.
i would try adding ovn last so it is only used if the other two cannot bind the port. the mech driver list is orderd for this reason so you can express preference.
It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks.
that would be a bug if so.
Am I missing something here? (We’re using Stein version)
Thanks, Sam
-- Slawek Kaplonski Senior software engineer Red Hat
Sorry just picking this up again after a bit of a hiatus.
We have compute nodes that support instances on midonet and on linuxbridge and everything works great, we don’t want or expect these networks to interact with each other. This works because the use different network_types. Our linux bridge networks are flat provider networks and midonet is used for tenant networks (tunnelled).
Replacing midonet with OVN almost works too except it shares flat network type with linuxbridge.
Ideally in our environment we’d want to say this is a linuxbridge network, use linuxbridge to bind, this is an OVN network and use ovn driver to bind etc. It kind of seems weird that it just iterates over all mechanism drivers and stops once one works, I’m sure there is a good reason for it but I can’t see it. Is anyone using neutron ml2 like this? What mech drivers can interact and so it doesn’t matter which driver binds the port?
Sam
On 28 Nov 2019, at 11:27 pm, Sean Mooney smooney@redhat.com wrote:
On Thu, 2019-11-28 at 11:12 +0900, Takashi Yamamoto wrote:
hi,
On Mon, Nov 25, 2019 at 5:00 PM Slawek Kaplonski skaplons@redhat.com wrote:
Hi,
I think that this may be true that networking-ovn will not work properly with other drivers. I don't think it was tested at any time.
it should work with other direver if you use vlan or flat networks. it will not form mesh tunnel networks with other drivers event if you use geneve for the other ml2 driver.
Also the problem may be that when You are using networking-ovn than whole neutron topology is different. There are different agents for example.
Please open a bug for that for networking-ovn. I think that networking-ovn team will take a look into that.
networking-midonet ignores networks without "midonet" type segments to avoid interfering other mechanism drivers. maybe networking-ovn can have something similar.
that is actully the opiste of how that shoudl work. you are ment to be able to have multiple ml2 drivers share the same segmentation type and you are not ment to have a segmentation type that is specific to a mech driver. give we dont scheduler based on the segmenation type supprot today either (we shoudl by the way) it would be very fagile to use a dedicated ovn segmentation type and i woudl not advise doing it for midio net.
ideally we would create placement aggregate or triats to track which segmentation types are supported by which hosts. traits are proably better for the segmentation types but modelling network segments them selves would be better as aggreates.
if we really wanted to model the capsity of the segmenation types we woudl addtionally create shareing resouce providers with inventories of network segmenation types resouce classes per physnet with a singel gloabl rp for the tunneled types. then every time you allocated a network in neuton you would create an allocation for that network and tag ports with the approreate aggreate requrest.
on the nova side wew could combine the segment and segmenation type aggreate request form the port with any other aggreates form nova and pass all of them as member_of requriements to placment to ensure we land on a host that can provide the required network connectivty. today we litrallly just assme all nodes are connected to all networks with all segmenation types and hope for the best.
thats a bit of a tangent but just pointing out we should schduler on network connectivity and segmenation types but we shoudl not have backend specific segmenation types.
wrt agents, last time i checked there was no problem with running midonet agent and ovs agent on the same host, sharing the kernel datapath. so i guess there's no problem with ovn either.
you can run ml2/ovn and ml2/ovs on the same cloud. just put the ml2/ovs first. it will fail to bind if a host dose not have the ovs neutron agent running and will then bind with ml2/ovn instead.
it might work the other way too but i have nto tested that.
wrt l3, unfortunately neither midonet or ovn have implemented "l3 flavor" thing yet. so you have to choose a single l3 plugin. iirc, Sam's deployment doesn't use l3 for linuxbridge, right?
if you have dedicated network nodes that is not really a proable. just make sure that they are all ovn or all ovs or whatever makes sense. its the same way that if you deploy with ml2/ovs and want to use ovs-dpdk that you only instlal ovs-dpdk on the comptue nodes and use kernel ovs on the networking nodes to avoid the terible network performace when using network namespace for nat and routing.
if you have tunneled networks it would be an issue butin that case you just need to ensure that at least 1 router is created on each plugin so you would use ha routers by default and set the ha factor so that it created routers on node with both mechinmum dirvers. again however since the different ml2/driver do not form a mesh you should really only use different ml2 drivers if you are using vlan or flat networks.
On Mon, Nov 25, 2019 at 04:32:50PM +1100, Sam Morrison wrote:
We are looking at using OVN and are having some issues with it in our ML2 environment.
We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network)
Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc.
i would try adding ovn last so it is only used if the other two cannot bind the port. the mech driver list is orderd for this reason so you can express preference.
It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks.
that would be a bug if so.
Am I missing something here? (We’re using Stein version)
Thanks, Sam
-- Slawek Kaplonski Senior software engineer Red Hat
On Tue, 7 Apr 2020 at 03:11, Sam Morrison sorrison@gmail.com wrote:
Sorry just picking this up again after a bit of a hiatus.
We have compute nodes that support instances on midonet and on linuxbridge and everything works great, we don’t want or expect these networks to interact with each other. This works because the use different network_types. Our linux bridge networks are flat provider networks and midonet is used for tenant networks (tunnelled).
Replacing midonet with OVN almost works too except it shares flat network type with linuxbridge.
Ideally in our environment we’d want to say this is a linuxbridge network, use linuxbridge to bind, this is an OVN network and use ovn driver to bind etc. It kind of seems weird that it just iterates over all mechanism drivers and stops once one works, I’m sure there is a good reason for it but I can’t see it. Is anyone using neutron ml2 like this? What mech drivers can interact and so it doesn’t matter which driver binds the port?
When mixing mechanism drivers in the past I've separated them by physical network. Often drivers accept a list of physnets they manage. Not sure if this works for your case though.
Sam
On 28 Nov 2019, at 11:27 pm, Sean Mooney smooney@redhat.com wrote:
On Thu, 2019-11-28 at 11:12 +0900, Takashi Yamamoto wrote:
hi,
On Mon, Nov 25, 2019 at 5:00 PM Slawek Kaplonski skaplons@redhat.com wrote:
Hi,
I think that this may be true that networking-ovn will not work properly with other drivers. I don't think it was tested at any time.
it should work with other direver if you use vlan or flat networks. it will not form mesh tunnel networks with other drivers event if you use geneve for the other ml2 driver.
Also the problem may be that when You are using networking-ovn than whole neutron topology is different. There are different agents for example.
Please open a bug for that for networking-ovn. I think that networking-ovn team will take a look into that.
networking-midonet ignores networks without "midonet" type segments to avoid interfering other mechanism drivers. maybe networking-ovn can have something similar.
that is actully the opiste of how that shoudl work. you are ment to be able to have multiple ml2 drivers share the same segmentation type and you are not ment to have a segmentation type that is specific to a mech driver. give we dont scheduler based on the segmenation type supprot today either (we shoudl by the way) it would be very fagile to use a dedicated ovn segmentation type and i woudl not advise doing it for midio net.
ideally we would create placement aggregate or triats to track which segmentation types are supported by which hosts. traits are proably better for the segmentation types but modelling network segments them selves would be better as aggreates.
if we really wanted to model the capsity of the segmenation types we woudl addtionally create shareing resouce providers with inventories of network segmenation types resouce classes per physnet with a singel gloabl rp for the tunneled types. then every time you allocated a network in neuton you would create an allocation for that network and tag ports with the approreate aggreate requrest.
on the nova side wew could combine the segment and segmenation type aggreate request form the port with any other aggreates form nova and pass all of them as member_of requriements to placment to ensure we land on a host that can provide the required network connectivty. today we litrallly just assme all nodes are connected to all networks with all segmenation types and hope for the best.
thats a bit of a tangent but just pointing out we should schduler on network connectivity and segmenation types but we shoudl not have backend specific segmenation types.
wrt agents, last time i checked there was no problem with running midonet agent and ovs agent on the same host, sharing the kernel datapath. so i guess there's no problem with ovn either.
you can run ml2/ovn and ml2/ovs on the same cloud. just put the ml2/ovs first. it will fail to bind if a host dose not have the ovs neutron agent running and will then bind with ml2/ovn instead.
it might work the other way too but i have nto tested that.
wrt l3, unfortunately neither midonet or ovn have implemented "l3 flavor" thing yet. so you have to choose a single l3 plugin. iirc, Sam's deployment doesn't use l3 for linuxbridge, right?
if you have dedicated network nodes that is not really a proable. just make sure that they are all ovn or all ovs or whatever makes sense. its the same way that if you deploy with ml2/ovs and want to use ovs-dpdk that you only instlal ovs-dpdk on the comptue nodes and use kernel ovs on the networking nodes to avoid the terible network performace when using network namespace for nat and routing.
if you have tunneled networks it would be an issue butin that case you just need to ensure that at least 1 router is created on each plugin so you would use ha routers by default and set the ha factor so that it created routers on node with both mechinmum dirvers. again however since the different ml2/driver do not form a mesh you should really only use different ml2 drivers if you are using vlan or flat networks.
On Mon, Nov 25, 2019 at 04:32:50PM +1100, Sam Morrison wrote:
We are looking at using OVN and are having some issues with it in our ML2 environment.
We currently have 2 mechanism drivers in use: linuxbridge and midonet and these work well (midonet is the default tenant network driver for when users create a network)
Adding OVN as a third mechanism driver causes the linuxbridge and midonet networks to stop working in terms of CRUD operations etc.
i would try adding ovn last so it is only used if the other two cannot bind the port. the mech driver list is orderd for this reason so you can express preference.
It looks as if the OVN driver thinks it’s the only player and is trying to do things on ports that are in linuxbridge or midonet networks.
that would be a bug if so.
Am I missing something here? (We’re using Stein version)
Thanks, Sam
-- Slawek Kaplonski Senior software engineer Red Hat
participants (5)
-
Mark Goddard
-
Sam Morrison
-
Sean Mooney
-
Slawek Kaplonski
-
Takashi Yamamoto