[kolla] neutron-l3-agent namespace NAT table not working?

Jon Masters jcm at jonmasters.org
Mon Jan 6 22:27:43 UTC 2020


https://bugs.launchpad.net/kolla/+bug/1858505

On Mon, Jan 6, 2020 at 11:56 AM Slawek Kaplonski <skaplons at redhat.com>
wrote:

> Hi,
>
> > On 6 Jan 2020, at 16:15, Brian Haley <haleyb.dev at gmail.com> wrote:
> >
> > On 1/6/20 7:33 AM, Radosław Piliszek wrote:
> >> Folks, this seems to be about C7, not C8, and
> >> "neutron_legacy_iptables" does not apply here.
> >> @Jon - what is the kernel bug you mentioned but never referenced?
> >
> > There was a previous kernel bug in a Centos kernel that broke DNAT,
> https://bugs.launchpad.net/neutron/+bug/1776778 but don't know if this is
> the same issue.  I would have hoped no one was using that kernel by now,
> and/or it was blacklisted.
>
> This one also came to my mind when I read about kernel bug here. But this
> old bug was affecting only DNAT on dvr routers IIRC so IMO it doesn’t seems
> like same issue.
>
> >
> > -Brian
> >
> >> pon., 6 sty 2020 o 13:13 Jon Masters <jcm at jonmasters.org> napisał(a):
> >>>
> >>> I did specifically check for such a conflict tho before proceeding
> down the path I went :)
> >>>
> >>> --
> >>> Computer Architect
> >>>
> >>>
> >>>> On Jan 6, 2020, at 03:40, Sean Mooney <smooney at redhat.com> wrote:
> >>>>
> >>>> On Mon, 2020-01-06 at 10:11 +0100, Radosław Piliszek wrote:
> >>>>> If it's RHEL kernel's bug, then Red Hat would likely want to know
> >>>>> about it (if not knowing already).
> >>>>> I have my kolla deployment on c7.7 and I don't encounter this issue,
> >>>>> though there is a pending kernel update so now I'm worried about
> >>>>> applying it...
> >>>> it sound more like a confilct between legacy iptables and the new
> nftables based replacement.
> >>>> if you mix the two then it will appear as if the rules are installed
> but only some of the rules will run.
> >>>> so the container images and the host need to be both configured to
> use the same versions.
> >>>>
> >>>> that said fi you are using centos images on a centos host they should
> be providing your usnign centos 7 or centos 8 on
> >>>> both. if you try to use centos 7 image on a centos 8 host or centos 8
> images on a centos 7 host it would likely have
> >>>> issues due to the fact centos 8 uses a differt iptables implemeantion
> >>>>
> >>>>>
> >>>>> -yoctozepto
> >>>>>
> >>>>> pon., 6 sty 2020 o 03:34 Jon Masters <jcm at jonmasters.org>
> napisał(a):
> >>>>>>
> >>>>>> There’s no bug ID that I’m aware of. But I’ll go look for one or
> file one.
> >>>>>>
> >>>>>> --
> >>>>>> Computer Architect
> >>>>>>
> >>>>>>
> >>>>>>> On Jan 5, 2020, at 18:51, Laurent Dumont <laurentfdumont at gmail.com>
> wrote:
> >>>>>>
> >>>>>> 
> >>>>>> Do you happen to have the bug ID for Centos?
> >>>>>>
> >>>>>> On Sun, Jan 5, 2020 at 2:11 PM Jon Masters <jcm at jonmasters.org>
> wrote:
> >>>>>>>
> >>>>>>> This turns out to a not well documented bug in the CentOS7.7
> kernel that causes exactly nat rules not to run as I
> >>>>>>> was seeing. Oh dear god was this nasty as whatever to find and
> workaround.
> >>>>>>>
> >>>>>>> --
> >>>>>>> Computer Architect
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Jan 4, 2020, at 10:39, Jon Masters <jcm at jonmasters.org> wrote:
> >>>>>>>>
> >>>>>>>> Excuse top posting on my phone. Also, yes, the namespaces are as
> described. It’s just that the (correct) nat
> >>>>>>>> rules for the qrouter netns are never running, in spite of the
> two interfaces existing in that ns and correctly
> >>>>>>>> attached to the vswitch.
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Computer Architect
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>> On Jan 4, 2020, at 07:56, Sean Mooney <smooney at redhat.com>
> wrote:
> >>>>>>>>>>
> >>>>>>>>>> On Sat, 2020-01-04 at 10:46 +0100, Slawek Kaplonski wrote:
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> Is this qrouter namespace created with all those rules in
> container or in the host directly?
> >>>>>>>>>> Do You have qr-xxx and qg-xxx ports from br-int in this qrouter
> namespace?
> >>>>>>>>>
> >>>>>>>>> in kolla the l3 agent should be running with net=host so the
> container should be useing the hosts
> >>>>>>>>> root namespace  and it will create network namespaces as needed
> for the different routers.
> >>>>>>>>>
> >>>>>>>>> the ip table rules should be in the router sub namespaces.
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> On 4 Jan 2020, at 05:44, Jon Masters <jcm at jonmasters.org>
> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi there,
> >>>>>>>>>>>
> >>>>>>>>>>> I've got a weird problem with the neutron-l3-agent container
> on my deployment. It comes up, sets up the
> >>>>>>>>>>> iptables
> >>>>>>>>>>> rules in the qrouter namespace (and I can see these using "ip
> netns...") but traffic isn't having DNAT or
> >>>>>>>>>>> SNAT
> >>>>>>>>>>> applied. What's most strange is that manually adding a LOG
> jump target to the iptables nat PRE/POSTROUTING
> >>>>>>>>>>> chains
> >>>>>>>>>>> (after enabling nf logging sent to the host kernel, confirmed
> that works) doesn't result in any log
> >>>>>>>>>>> entries. It's as
> >>>>>>>>>>> if the nat table isn't being applied at all for any packets
> traversing the qrouter namespace. This is
> >>>>>>>>>>> driving me
> >>>>>>>>>>> crazy :)
> >>>>>>>>>>>
> >>>>>>>>>>> Anyone got some quick suggestions? (assume I tried the obvious
> stuff).
> >>>>>>>>>>>
> >>>>>>>>>>> Jon.
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Computer Architect
> >>>>>>>>>>
> >>>>>>>>>> —
> >>>>>>>>>> Slawek Kaplonski
> >>>>>>>>>> Senior software engineer
> >>>>>>>>>> Red Hat
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>
> >>>>>
> >>>>
> >
>
>> Slawek Kaplonski
> Senior software engineer
> Red Hat
>
>

-- 
Computer Architect
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200106/be93d2f0/attachment-0001.html>


More information about the openstack-discuss mailing list