Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/13/2015 01:42 PM, Miguel Ángel Ajo wrote:
> Hi, Ihar & Jiri, thank you for pointing this out.
> 
> I’m working on the following items:
> 
> 1) Doing Openflow traffic filtering (stateful firewall) based on
> OVS+CT[1] patch, which may
>     eventually merge. Here I want to build a good amount of benchmarks
> to be able to compare
>     the current network iptables+LB solution to just openflow.
> 
>      Openflow filtering should be fast, as it’s quite smart at using
> hashes to match OF rules
>      in the kernel megaflows (thanks Jiri & T.Graf for explaining me this)
>    
>      The only bad part is that we would have to dynamically change more
> rules based on security
> group changes (now we do that via ip sets without reloading all the rules).
> 
>       To do this properly, we may want to make the OVS plugin a real OF
> controller to be able to
> push OF rules to the bridge without the need of calling ovs-ofctl on the
> command line all the time.

I am actually a bit worried about this project being bound so tightly to
openvswitch. There are common linux alternatives in the wild that do not
require it at all (tc, eBPF). Of course, openvswitch could be utilized
in some exotic cases, like controlling HyperV instances in Windows
environment, but for usual Linux platform, utilizing something like
openvswitch seems to be an overkill, while we don't use any of
controller separation features and do propagate updates via AMQP to each
controlled node.

> 
> 2) Using OVS+OF to do QoS
> 
> other interesting stuff to look at:
> 
> 3) Doing routing in OF too, thanks to the NAT capabilities of having OVS+CT 
> 
> 4) The namespace problem, what kinds of statistics get broken by moving
> ports into namespaces now?,

I think some of those are already mentioned in the ovs tracker bug:
- - https://bugzilla.redhat.com/show_bug.cgi?id=1160340

Specifically, mac_in_use that is used to track mac addresses attached to
ports; or ifindex that is used for SNMP.

>     the short-term fix could be using vets, but “namespaceable” OVS
> ports would be perfect, yet I understand
>     the change is a big feature.
> 
>     If we had 1 & 3, may be 4 wouldn’t be a problem anymore.

I don't believe so. We still need namespaces for e.g. DHCP namespaces,
or in any other cases where we need to run a service that is not capable
of binding to a specific interface.

> 
> [1] https://github.com/justinpettit/ovs/tree/conntrack
> 
> Miguel Ángel Ajo
> 
> On Friday, 13 de February de 2015 at 13:14, Ihar Hrachyshka wrote:
> 
> Hi neutroners,
> 
> we** had several conversations recently with our Red Hat fellows who
> work on openvswitch (Jiri Benc and Jiri Pirko) regarding the way
> neutron utilizes their software. Those were beneficial to both sides
> to understand what we do right and wrong. I was asked to share some of
> the points from those discussions with broader community.
> 
> ===
> 
> One of the issues that came up during discussions is the way neutron
> connects ovs ports to namespaces. The short story is that openvswitch
> is not designed with namespaces in mind, and the fact that moving its
> ports into a different namespace works for neutron is mere
> coincidence, and is actually considered as a bug by openvswitch guys.
> 
> It's not just broken in theory from software design standpoint, but
> also in practice. Specifically,
> 
> 1. ovsdb dump does not work properly for namespaces:
> - https://bugzilla.redhat.com/show_bug.cgi?id=1160340
> 
> This results in wrong statistics and other data collected for these ports;
> 
> 2. We suspect that the following kernel crash is triggered because of
> our usage of the feature that is actually a bug:
> - https://bugs.launchpad.net/neutron/+bug/1418097
> 
> Quoting Jiri Benc,
> 
> "The problem is openvswitch does not support its ports to be moved to
> a different name space. The fact that it's possible to do that is a
> bug - such operation should have been denied. Unfortunately, due to a
> missing check, it's not been denied. Such setup does not work
> reliably, though, and leads to various issues from incorrect resource
> accounting to kernel crashes.
> 
> We're aware of the bug but we don't have any easy way to fix it. The
> most obvious option, disabling moving of ovs ports to different name
> spaces, would be easy to do but it would break Neutron. The other
> option, making ovs to work across net name spaces, is hard and will
> require addition of different kernel APIs and large changes in ovs
> user space daemons. This constitutes tremendous amount of work."
> 
> The tracker bug on openvswitch side is:
> - https://bugzilla.redhat.com/show_bug.cgi?id=1160340
> 
> So in the best case, we may expect openvswitch to properly support the
> feature in long term, but short term it won't work, especially while
> neutron expects other features implemented in openvswitch for it (like
> NAT, or connection tracking, or ipv6 tunnel endpoints, to name a few).
> 
> We could try to handle the issue neutron side. We can fix it by
> utilizing veth pairs to get into namespaces, but it may mean worse
> performance, and will definitely require proper benchmarking to see
> whether we can live with performance drop.
> 
> ===
> 
> There were other suggestions on how we can enhance our way of usage of
> openvswitch. Among those, getting rid of linux bridge used for
> security groups, with special attention on getting rid of ebtables
> (sic!) for they are a lot slower than iptables; getting rid of veth
> pair for instance ports.
> 
> ===
> 
> I really encourage everyone to check the following video from
> devconf.cz <http://devconf.cz> 2015 on all that and more at:
> 
> - https://www.youtube.com/watch?v=BlLD-qh9EBQ
> 
> Among other things, you will find presentation of plotnetcfg tool to
> create nice graphs of openstack state.
> 
> If you're lazy enough and want to switch directly to the analysis of
> neutron problems, skip to ~18:00.
> 
> I also encourage to check our the video around 30:00 on the way out of
> openvswitch for neutron (tc/eBPF). As crazy as encouraging.
> 
> ===
> 
> While at it, I encourage everyone interested in controlling switches
> the open source way to check out presentation of the new kernel
> subsystem for that (I guess vaporware atm):
> 
> - https://www.youtube.com/watch?v=awekaJ7xWuw
> 
> ===
> 
> So, that's what I have for you now. I really want to hear from
> everyone what is our plan to solve the problem neutron side.
> 
> Comments?
> 
> /Ihar
> 
> **Jakub Libosvar and me
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJU3fR/AAoJEC5aWaUY1u57CzgIALpCxiERaZZsN15BnSCXwTKE
xtADj80kycs7AMghEMblBVKdwFjKkzHEC8o9ie89U9CF1Fdb6Jl6MD2+2XcYFSki
hCWpWw5Z5GdpTCoIl5TZJBY4X6gEgBw4NMj+2iDKjTvx/m/jXrQk5nC//Y1AJ6BM
0pZ3ekPviwHdPmA3m0md7tj0FXc4uYkRQEDZkem8utXM1wmdgKfFrbuiO+3xP32J
mi9ehjMDXsZlWWqcc9OaulHx5yJyZR7XHgC0QkK86nkhdrIxoUA5eUFMSO0Fg1YC
d82OB/xLpm60nWJStixYmfgBxj4M5jci2P6dG5R9nr1sab0JixZ4XsaS5PSgGXc=
=Kgv0
-----END PGP SIGNATURE-----