Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02/13/2015 01:47 PM, Miguel Ángel Ajo wrote:
> Sorry, I forgot about
> 
> 5)  If we put all our OVS/OF bridge logic in just one bridge
> (instead of N: br-tun, br-int, br-ex, br-xxx), the performance
> should be yet higher, since, as far as I understood, flow rule
> lookup could be more optimized into the kernel megaflows without
> forwarding and re-starting evaluation due to patch ports. (Please
> correct me here where I’m wrong, I just have very high level view
> of this).

Indeed, that was also mentioned by Jiri in our private talks. That
said, I'm as unaware of details here as you probably are (or more).

> 
> Best, Miguel Ángel Ajo
> 
> On Friday, 13 de February de 2015 at 13:42, Miguel Ángel Ajo
> wrote:
> 
>> Hi, Ihar & Jiri, thank you for pointing this out.
>> 
>> I’m working on the following items:
>> 
>> 1) Doing Openflow traffic filtering (stateful firewall) based on 
>> OVS+CT[1] patch, which may eventually merge. Here I want to build
>> a good amount of benchmarks to be able to compare the current
>> network iptables+LB solution to just openflow.
>> 
>> Openflow filtering should be fast, as it’s quite smart at using 
>> hashes to match OF rules in the kernel megaflows (thanks Jiri &
>> T.Graf for explaining me this)
>> 
>> The only bad part is that we would have to dynamically change 
>> more rules based on security group changes (now we do that via ip
>> sets without reloading all the rules).
>> 
>> To do this properly, we may want to make the OVS plugin a real OF
>> controller to be able to push OF rules to the bridge without the
>> need of calling ovs-ofctl on the command line all the time.
>> 
>> 2) Using OVS+OF to do QoS
>> 
>> other interesting stuff to look at:
>> 
>> 3) Doing routing in OF too, thanks to the NAT capabilities of
>> having OVS+CT
>> 
>> 4) The namespace problem, what kinds of statistics get broken by 
>> moving ports into namespaces now?, the short-term fix could be
>> using vets, but “namespaceable” OVS ports would be perfect, yet I
>> understand the change is a big feature.
>> 
>> If we had 1 & 3, may be 4 wouldn’t be a problem anymore.
>> 
>> [1] https://github.com/justinpettit/ovs/tree/conntrack
>> 
>> Miguel Ángel Ajo
>> 
>> On Friday, 13 de February de 2015 at 13:14, Ihar Hrachyshka
>> wrote:
>> 
> Hi neutroners,
> 
> we** had several conversations recently with our Red Hat fellows
> who work on openvswitch (Jiri Benc and Jiri Pirko) regarding the
> way neutron utilizes their software. Those were beneficial to both
> sides to understand what we do right and wrong. I was asked to
> share some of the points from those discussions with broader
> community.
> 
> ===
> 
> One of the issues that came up during discussions is the way
> neutron connects ovs ports to namespaces. The short story is that
> openvswitch is not designed with namespaces in mind, and the fact
> that moving its ports into a different namespace works for neutron
> is mere coincidence, and is actually considered as a bug by
> openvswitch guys.
> 
> It's not just broken in theory from software design standpoint,
> but also in practice. Specifically,
> 
> 1. ovsdb dump does not work properly for namespaces: -
> https://bugzilla.redhat.com/show_bug.cgi?id=1160340
> 
> This results in wrong statistics and other data collected for
> these ports;
> 
> 2. We suspect that the following kernel crash is triggered because
> of our usage of the feature that is actually a bug: -
> https://bugs.launchpad.net/neutron/+bug/1418097
> 
> Quoting Jiri Benc,
> 
> "The problem is openvswitch does not support its ports to be moved
> to a different name space. The fact that it's possible to do that
> is a bug - such operation should have been denied. Unfortunately,
> due to a missing check, it's not been denied. Such setup does not
> work reliably, though, and leads to various issues from incorrect
> resource accounting to kernel crashes.
> 
> We're aware of the bug but we don't have any easy way to fix it.
> The most obvious option, disabling moving of ovs ports to different
> name spaces, would be easy to do but it would break Neutron. The
> other option, making ovs to work across net name spaces, is hard
> and will require addition of different kernel APIs and large
> changes in ovs user space daemons. This constitutes tremendous
> amount of work."
> 
> The tracker bug on openvswitch side is: -
> https://bugzilla.redhat.com/show_bug.cgi?id=1160340
> 
> So in the best case, we may expect openvswitch to properly support
> the feature in long term, but short term it won't work, especially
> while neutron expects other features implemented in openvswitch for
> it (like NAT, or connection tracking, or ipv6 tunnel endpoints, to
> name a few).
> 
> We could try to handle the issue neutron side. We can fix it by 
> utilizing veth pairs to get into namespaces, but it may mean worse 
> performance, and will definitely require proper benchmarking to
> see whether we can live with performance drop.
> 
> ===
> 
> There were other suggestions on how we can enhance our way of usage
> of openvswitch. Among those, getting rid of linux bridge used for 
> security groups, with special attention on getting rid of ebtables 
> (sic!) for they are a lot slower than iptables; getting rid of
> veth pair for instance ports.
> 
> ===
> 
> I really encourage everyone to check the following video from 
> devconf.cz <http://devconf.cz> 2015 on all that and more at:
> 
> - https://www.youtube.com/watch?v=BlLD-qh9EBQ
> 
> Among other things, you will find presentation of plotnetcfg tool
> to create nice graphs of openstack state.
> 
> If you're lazy enough and want to switch directly to the analysis
> of neutron problems, skip to ~18:00.
> 
> I also encourage to check our the video around 30:00 on the way out
> of openvswitch for neutron (tc/eBPF). As crazy as encouraging.
> 
> ===
> 
> While at it, I encourage everyone interested in controlling
> switches the open source way to check out presentation of the new
> kernel subsystem for that (I guess vaporware atm):
> 
> - https://www.youtube.com/watch?v=awekaJ7xWuw
> 
> ===
> 
> So, that's what I have for you now. I really want to hear from 
> everyone what is our plan to solve the problem neutron side.
> 
> Comments?
> 
> /Ihar
> 
> **Jakub Libosvar and me
>>> 
>>> __________________________________________________________________________
>>>
>>> 
OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: 
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe 
>>> <mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>
>>> 
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> 
> 
> __________________________________________________________________________
>
> 
OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJU3fTKAAoJEC5aWaUY1u57ZUIIAMLtl8r+baX1tutnWZ2aH8v1
WZZ60I8zlPFWUCax9NQbtWpXhQ6VqK+6BEKsDYZitFLno/suLyRJFQY7wldkdn73
QOzmZ24Xn2mxwTw4LHKNGkshCc42suZ1HD+NHqNup8JdYLts5gx2DJrZNlr7uj7B
OaTUqXzcinEgzwSdv2yLeIdYXjjHzsozP221AcqyFEWeiBJUm5/7hVwXOPmHJun+
7eTKJQp/mJlBrKQhD/FiOXuYpPGg9PisbKs0tODAV/VsUywqdXvQD10GmhMkcNhb
/j3w1DdBagXMExHYM4jqd5jgtX9iwujPiulbuxZoRjiyIW/2c6vWlmFh+LY+wp4=
=a17+
-----END PGP SIGNATURE-----