[kolla] neutron-l3-agent namespace NAT table not working?

Slawek Kaplonski skaplons at redhat.com
Mon Jan 6 19:56:36 UTC 2020


Hi,

> On 6 Jan 2020, at 16:15, Brian Haley <haleyb.dev at gmail.com> wrote:
> 
> On 1/6/20 7:33 AM, Radosław Piliszek wrote:
>> Folks, this seems to be about C7, not C8, and
>> "neutron_legacy_iptables" does not apply here.
>> @Jon - what is the kernel bug you mentioned but never referenced?
> 
> There was a previous kernel bug in a Centos kernel that broke DNAT, https://bugs.launchpad.net/neutron/+bug/1776778 but don't know if this is the same issue.  I would have hoped no one was using that kernel by now, and/or it was blacklisted.

This one also came to my mind when I read about kernel bug here. But this old bug was affecting only DNAT on dvr routers IIRC so IMO it doesn’t seems like same issue.

> 
> -Brian
> 
>> pon., 6 sty 2020 o 13:13 Jon Masters <jcm at jonmasters.org> napisał(a):
>>> 
>>> I did specifically check for such a conflict tho before proceeding down the path I went :)
>>> 
>>> --
>>> Computer Architect
>>> 
>>> 
>>>> On Jan 6, 2020, at 03:40, Sean Mooney <smooney at redhat.com> wrote:
>>>> 
>>>> On Mon, 2020-01-06 at 10:11 +0100, Radosław Piliszek wrote:
>>>>> If it's RHEL kernel's bug, then Red Hat would likely want to know
>>>>> about it (if not knowing already).
>>>>> I have my kolla deployment on c7.7 and I don't encounter this issue,
>>>>> though there is a pending kernel update so now I'm worried about
>>>>> applying it...
>>>> it sound more like a confilct between legacy iptables and the new nftables based replacement.
>>>> if you mix the two then it will appear as if the rules are installed but only some of the rules will run.
>>>> so the container images and the host need to be both configured to use the same versions.
>>>> 
>>>> that said fi you are using centos images on a centos host they should be providing your usnign centos 7 or centos 8 on
>>>> both. if you try to use centos 7 image on a centos 8 host or centos 8 images on a centos 7 host it would likely have
>>>> issues due to the fact centos 8 uses a differt iptables implemeantion
>>>> 
>>>>> 
>>>>> -yoctozepto
>>>>> 
>>>>> pon., 6 sty 2020 o 03:34 Jon Masters <jcm at jonmasters.org> napisał(a):
>>>>>> 
>>>>>> There’s no bug ID that I’m aware of. But I’ll go look for one or file one.
>>>>>> 
>>>>>> --
>>>>>> Computer Architect
>>>>>> 
>>>>>> 
>>>>>>> On Jan 5, 2020, at 18:51, Laurent Dumont <laurentfdumont at gmail.com> wrote:
>>>>>> 
>>>>>> 
>>>>>> Do you happen to have the bug ID for Centos?
>>>>>> 
>>>>>> On Sun, Jan 5, 2020 at 2:11 PM Jon Masters <jcm at jonmasters.org> wrote:
>>>>>>> 
>>>>>>> This turns out to a not well documented bug in the CentOS7.7 kernel that causes exactly nat rules not to run as I
>>>>>>> was seeing. Oh dear god was this nasty as whatever to find and workaround.
>>>>>>> 
>>>>>>> --
>>>>>>> Computer Architect
>>>>>>> 
>>>>>>> 
>>>>>>>> On Jan 4, 2020, at 10:39, Jon Masters <jcm at jonmasters.org> wrote:
>>>>>>>> 
>>>>>>>> Excuse top posting on my phone. Also, yes, the namespaces are as described. It’s just that the (correct) nat
>>>>>>>> rules for the qrouter netns are never running, in spite of the two interfaces existing in that ns and correctly
>>>>>>>> attached to the vswitch.
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Computer Architect
>>>>>>>> 
>>>>>>>> 
>>>>>>>>>> On Jan 4, 2020, at 07:56, Sean Mooney <smooney at redhat.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> On Sat, 2020-01-04 at 10:46 +0100, Slawek Kaplonski wrote:
>>>>>>>>>> Hi,
>>>>>>>>>> 
>>>>>>>>>> Is this qrouter namespace created with all those rules in container or in the host directly?
>>>>>>>>>> Do You have qr-xxx and qg-xxx ports from br-int in this qrouter namespace?
>>>>>>>>> 
>>>>>>>>> in kolla the l3 agent should be running with net=host so the container should be useing the hosts
>>>>>>>>> root namespace  and it will create network namespaces as needed for the different routers.
>>>>>>>>> 
>>>>>>>>> the ip table rules should be in the router sub namespaces.
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>>> On 4 Jan 2020, at 05:44, Jon Masters <jcm at jonmasters.org> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi there,
>>>>>>>>>>> 
>>>>>>>>>>> I've got a weird problem with the neutron-l3-agent container on my deployment. It comes up, sets up the
>>>>>>>>>>> iptables
>>>>>>>>>>> rules in the qrouter namespace (and I can see these using "ip netns...") but traffic isn't having DNAT or
>>>>>>>>>>> SNAT
>>>>>>>>>>> applied. What's most strange is that manually adding a LOG jump target to the iptables nat PRE/POSTROUTING
>>>>>>>>>>> chains
>>>>>>>>>>> (after enabling nf logging sent to the host kernel, confirmed that works) doesn't result in any log
>>>>>>>>>>> entries. It's as
>>>>>>>>>>> if the nat table isn't being applied at all for any packets traversing the qrouter namespace. This is
>>>>>>>>>>> driving me
>>>>>>>>>>> crazy :)
>>>>>>>>>>> 
>>>>>>>>>>> Anyone got some quick suggestions? (assume I tried the obvious stuff).
>>>>>>>>>>> 
>>>>>>>>>>> Jon.
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Computer Architect
>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Slawek Kaplonski
>>>>>>>>>> Senior software engineer
>>>>>>>>>> Red Hat
>>>>>>>>>> 
>>>>>>>>>> 
>>>>> 
>>>>> 
>>>> 
> 

— 
Slawek Kaplonski
Senior software engineer
Red Hat




More information about the openstack-discuss mailing list