<div dir="ltr"><a href="https://bugs.launchpad.net/kolla/+bug/1858505">https://bugs.launchpad.net/kolla/+bug/1858505</a><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 6, 2020 at 11:56 AM Slawek Kaplonski <<a href="mailto:skaplons@redhat.com">skaplons@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
> On 6 Jan 2020, at 16:15, Brian Haley <<a href="mailto:haleyb.dev@gmail.com" target="_blank">haleyb.dev@gmail.com</a>> wrote:<br>
> <br>
> On 1/6/20 7:33 AM, Radosław Piliszek wrote:<br>
>> Folks, this seems to be about C7, not C8, and<br>
>> "neutron_legacy_iptables" does not apply here.<br>
>> @Jon - what is the kernel bug you mentioned but never referenced?<br>
> <br>
> There was a previous kernel bug in a Centos kernel that broke DNAT, <a href="https://bugs.launchpad.net/neutron/+bug/1776778" rel="noreferrer" target="_blank">https://bugs.launchpad.net/neutron/+bug/1776778</a> but don't know if this is the same issue. I would have hoped no one was using that kernel by now, and/or it was blacklisted.<br>
<br>
This one also came to my mind when I read about kernel bug here. But this old bug was affecting only DNAT on dvr routers IIRC so IMO it doesn’t seems like same issue.<br>
<br>
> <br>
> -Brian<br>
> <br>
>> pon., 6 sty 2020 o 13:13 Jon Masters <<a href="mailto:jcm@jonmasters.org" target="_blank">jcm@jonmasters.org</a>> napisał(a):<br>
>>> <br>
>>> I did specifically check for such a conflict tho before proceeding down the path I went :)<br>
>>> <br>
>>> --<br>
>>> Computer Architect<br>
>>> <br>
>>> <br>
>>>> On Jan 6, 2020, at 03:40, Sean Mooney <<a href="mailto:smooney@redhat.com" target="_blank">smooney@redhat.com</a>> wrote:<br>
>>>> <br>
>>>> On Mon, 2020-01-06 at 10:11 +0100, Radosław Piliszek wrote:<br>
>>>>> If it's RHEL kernel's bug, then Red Hat would likely want to know<br>
>>>>> about it (if not knowing already).<br>
>>>>> I have my kolla deployment on c7.7 and I don't encounter this issue,<br>
>>>>> though there is a pending kernel update so now I'm worried about<br>
>>>>> applying it...<br>
>>>> it sound more like a confilct between legacy iptables and the new nftables based replacement.<br>
>>>> if you mix the two then it will appear as if the rules are installed but only some of the rules will run.<br>
>>>> so the container images and the host need to be both configured to use the same versions.<br>
>>>> <br>
>>>> that said fi you are using centos images on a centos host they should be providing your usnign centos 7 or centos 8 on<br>
>>>> both. if you try to use centos 7 image on a centos 8 host or centos 8 images on a centos 7 host it would likely have<br>
>>>> issues due to the fact centos 8 uses a differt iptables implemeantion<br>
>>>> <br>
>>>>> <br>
>>>>> -yoctozepto<br>
>>>>> <br>
>>>>> pon., 6 sty 2020 o 03:34 Jon Masters <<a href="mailto:jcm@jonmasters.org" target="_blank">jcm@jonmasters.org</a>> napisał(a):<br>
>>>>>> <br>
>>>>>> There’s no bug ID that I’m aware of. But I’ll go look for one or file one.<br>
>>>>>> <br>
>>>>>> --<br>
>>>>>> Computer Architect<br>
>>>>>> <br>
>>>>>> <br>
>>>>>>> On Jan 5, 2020, at 18:51, Laurent Dumont <<a href="mailto:laurentfdumont@gmail.com" target="_blank">laurentfdumont@gmail.com</a>> wrote:<br>
>>>>>> <br>
>>>>>> <br>
>>>>>> Do you happen to have the bug ID for Centos?<br>
>>>>>> <br>
>>>>>> On Sun, Jan 5, 2020 at 2:11 PM Jon Masters <<a href="mailto:jcm@jonmasters.org" target="_blank">jcm@jonmasters.org</a>> wrote:<br>
>>>>>>> <br>
>>>>>>> This turns out to a not well documented bug in the CentOS7.7 kernel that causes exactly nat rules not to run as I<br>
>>>>>>> was seeing. Oh dear god was this nasty as whatever to find and workaround.<br>
>>>>>>> <br>
>>>>>>> --<br>
>>>>>>> Computer Architect<br>
>>>>>>> <br>
>>>>>>> <br>
>>>>>>>> On Jan 4, 2020, at 10:39, Jon Masters <<a href="mailto:jcm@jonmasters.org" target="_blank">jcm@jonmasters.org</a>> wrote:<br>
>>>>>>>> <br>
>>>>>>>> Excuse top posting on my phone. Also, yes, the namespaces are as described. It’s just that the (correct) nat<br>
>>>>>>>> rules for the qrouter netns are never running, in spite of the two interfaces existing in that ns and correctly<br>
>>>>>>>> attached to the vswitch.<br>
>>>>>>>> <br>
>>>>>>>> --<br>
>>>>>>>> Computer Architect<br>
>>>>>>>> <br>
>>>>>>>> <br>
>>>>>>>>>> On Jan 4, 2020, at 07:56, Sean Mooney <<a href="mailto:smooney@redhat.com" target="_blank">smooney@redhat.com</a>> wrote:<br>
>>>>>>>>>> <br>
>>>>>>>>>> On Sat, 2020-01-04 at 10:46 +0100, Slawek Kaplonski wrote:<br>
>>>>>>>>>> Hi,<br>
>>>>>>>>>> <br>
>>>>>>>>>> Is this qrouter namespace created with all those rules in container or in the host directly?<br>
>>>>>>>>>> Do You have qr-xxx and qg-xxx ports from br-int in this qrouter namespace?<br>
>>>>>>>>> <br>
>>>>>>>>> in kolla the l3 agent should be running with net=host so the container should be useing the hosts<br>
>>>>>>>>> root namespace and it will create network namespaces as needed for the different routers.<br>
>>>>>>>>> <br>
>>>>>>>>> the ip table rules should be in the router sub namespaces.<br>
>>>>>>>>> <br>
>>>>>>>>>> <br>
>>>>>>>>>>>> On 4 Jan 2020, at 05:44, Jon Masters <<a href="mailto:jcm@jonmasters.org" target="_blank">jcm@jonmasters.org</a>> wrote:<br>
>>>>>>>>>>> <br>
>>>>>>>>>>> Hi there,<br>
>>>>>>>>>>> <br>
>>>>>>>>>>> I've got a weird problem with the neutron-l3-agent container on my deployment. It comes up, sets up the<br>
>>>>>>>>>>> iptables<br>
>>>>>>>>>>> rules in the qrouter namespace (and I can see these using "ip netns...") but traffic isn't having DNAT or<br>
>>>>>>>>>>> SNAT<br>
>>>>>>>>>>> applied. What's most strange is that manually adding a LOG jump target to the iptables nat PRE/POSTROUTING<br>
>>>>>>>>>>> chains<br>
>>>>>>>>>>> (after enabling nf logging sent to the host kernel, confirmed that works) doesn't result in any log<br>
>>>>>>>>>>> entries. It's as<br>
>>>>>>>>>>> if the nat table isn't being applied at all for any packets traversing the qrouter namespace. This is<br>
>>>>>>>>>>> driving me<br>
>>>>>>>>>>> crazy :)<br>
>>>>>>>>>>> <br>
>>>>>>>>>>> Anyone got some quick suggestions? (assume I tried the obvious stuff).<br>
>>>>>>>>>>> <br>
>>>>>>>>>>> Jon.<br>
>>>>>>>>>>> <br>
>>>>>>>>>>> --<br>
>>>>>>>>>>> Computer Architect<br>
>>>>>>>>>> <br>
>>>>>>>>>> —<br>
>>>>>>>>>> Slawek Kaplonski<br>
>>>>>>>>>> Senior software engineer<br>
>>>>>>>>>> Red Hat<br>
>>>>>>>>>> <br>
>>>>>>>>>> <br>
>>>>> <br>
>>>>> <br>
>>>> <br>
> <br>
<br>
— <br>
Slawek Kaplonski<br>
Senior software engineer<br>
Red Hat<br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Computer Architect</div></div>