Open Stack

Hi,

Thanks for the messages.

> You should check where exactly Your packets are dropped.
I am communicating with our Linux systems engineers to learn
how I can find out which rule is dropping the packets.
In the meantime, which method would you recommend for this purpose?

> Also, You didn't tell us what is the type of the Neutron network to which
> Your VM is plugged and
The VM is plugged to a provider network.

> how bridges are done on Your compute node.
We are using Linux Bridges.

Are there any more/other information that I should provide?

Thanks,
Hyunwoo
Application developer at Fermilab.

On Mon, Dec 7, 2020 at 3:33 PM Slawek Kaplonski <skaplons at redhat.com> wrote:

> Hi,
>
> On Mon, Dec 07, 2020 at 10:11:19AM -0600, Hyunwoo KIM wrote:
> > Summary of the problem
> >
> > This problem is in a compute node, not in a VM.
> >
> > Once a VM is running in a compute node,
> >
> > all outbound connections in a compute node (not VM) are blocked.
> >
> > For example:
> >
> > # telnet www.google.com 80
> >
> > Trying 172.217.5.4...
> >
> >
> >
> > Technical Details:
> >
> > We only use provider network.
> >
> > These 4 services are running  in each compute node:
> >
> > - neutron-linuxbridge-agent.service
> >
> > - neutron-dhcp-agent.service
> >
> > - neutron-metadata-agent.service
> >
> > - openstack-nova-compute.service
> >
> >
> >
> > Detailed description of the problem:
> >
> >
> > In a compute node, the following is the result of iptables -L when no VM
> is
> > running:
> >
> >
> > <begin>
> >
> > Chain INPUT (policy ACCEPT)
> >
> > target     prot opt source               destination
> >
> > neutron-linuxbri-INPUT  all  --  anywhere anywhere
> >
> > And our usual rules
> >
> >
> > Chain FORWARD (policy ACCEPT)
> >
> > target     prot opt source               destination
> >
> > neutron-filter-top        all -- anywhere anywhere
> >
> > neutron-linuxbri-FORWARD  all -- anywhere anywhere
> >
> >
> > Chain OUTPUT (policy ACCEPT)
> >
> > target     prot opt source               destination
> >
> > neutron-filter-top       all -- anywhere anywhere
> >
> > neutron-linuxbri-OUTPUT  all -- anywhere anywhere
> >
> >
> > Chain neutron-filter-top (2 references)
> >
> > target     prot opt source               destination
> >
> > neutron-linuxbri-local  all -- anywhere anywhere
> >
> >
> > Chain neutron-linuxbri-FORWARD (1 references)
> >
> > target prot opt source               destination
> >
> > ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-out tapb
> > --physdev-is-bridged
> >
> > ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-in  tapb
> > --physdev-is-bridged
> >
> > ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-out tap9
> > --physdev-is-bridged
> >
> > ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-in  tap9
> > --physdev-is-bridged
> >
> >
> > Chain neutron-linuxbri-INPUT (1 references)
> >
> > target     prot opt source               destination
> >
> >
> > Chain neutron-linuxbri-OUTPUT (1 references)
> >
> > target     prot opt source               destination
> >
> >
> > Chain neutron-linuxbri-local (1 references)
> >
> > target     prot opt source               destination
> >
> >
> > Chain neutron-linuxbri-sg-chain (0 references)
> >
> > target     prot opt source               destination
> >
> > ACCEPT     all  --  anywhere             anywhere
> >
> >
> > Chain neutron-linuxbri-sg-fallback (0 references)
> >
> > target     prot opt source               destination
> >
> > DROP       all  --  anywhere             anywhere
> >
> > </end>
> >
> >
> >
> > In the same compute node, when a VM is running,
> >
> > the following is the result of iptables -L:
> >
> >
> >
> > <begin>
> >
> > Chain INPUT (policy ACCEPT)
> >
> > target     prot opt source               destination
> >
> > neutron-linuxbri-INPUT  all  --  anywhere anywhere
> >
> > And our usual rules
> >
> >
> > Chain FORWARD (policy ACCEPT)
> >
> > target     prot opt source               destination
> >
> > neutron-filter-top        all -- anywhere anywhere
> >
> > neutron-linuxbri-FORWARD  all -- anywhere anywhere
> >
> >
> > Chain OUTPUT (policy ACCEPT)
> >
> > target     prot opt source               destination
> >
> > neutron-filter-top       all -- anywhere anywhere
> >
> > neutron-linuxbri-OUTPUT  all -- anywhere anywhere
> >
> >
> > Chain neutron-filter-top (2 references)
> >
> > target     prot opt source               destination
> >
> > neutron-linuxbri-local  all  --  anywhere             anywhere
> >
> >
> > Chain neutron-linuxbri-FORWARD (1 references)
> >
> > target     prot opt source               destination
> >
> > neutron-linuxbri-sg-chain all -- anywhere anywhere PHYSDEV match
> > --physdev-out tap8 --physdev-is-bridged
> >
> > neutron-linuxbri-sg-chain all -- anywhere anywhere PHYSDEV match
> > --physdev-in  tap8 --physdev-is-bridged
> >
> > ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out tapb
> > --physdev-is-bridged
> >
> > ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-in  tapb
> > --physdev-is-bridged
> >
> > ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out tap9
> > --physdev-is-bridged
> >
> > ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-in  tap9
> > --physdev-is-bridged
> >
> >
> > Chain neutron-linuxbri-INPUT (1 references)
> >
> > target     prot opt source               destination
> >
> > neutron-linuxbri-o8 all -- anywhere anywhere PHYSDEV match --physdev-in
> > tap8 --physdev-is-bridged
> >
> >
> > Chain neutron-linuxbri-OUTPUT (1 references)
> >
> > target     prot opt source               destination
> >
> >
> > Chain neutron-linuxbri-i8 (1 references)
> >
> > target     prot opt source               destination
> >
> > RETURN     all  --  anywhere anywhere  state RELATED,ESTABLISHED
> >
> > RETURN     udp  --  anywhere fermicloud248.fnal.gov udp spt:bootps
> > dpt:bootpc
> >
> > RETURN     udp  --  anywhere 255.255.255.255   udp spt:bootps dpt:bootpc
> >
> > RETURN     icmp --  anywhere anywhere
> >
> > RETURN     tcp  --  fermilab-net.fnal.gov/16 anywhere  tcp dpt:ssh
> >
> > RETURN     all  --  anywhere anywhere  match-set
> > NIPv41d69ba3c-68e3-414f-8f1b- src
> >
> > DROP       all  --  anywhere anywhere  state INVALID
> >
> > neutron-linuxbri-sg-fallback  all -- anywhere anywhere
> >
> >
> > Chain neutron-linuxbri-local (1 references)
> >
> > target     prot opt source               destination
> >
> >
> > Chain neutron-linuxbri-o8 (2 references)
> >
> > target     prot opt source               destination
> >
> > RETURN     udp  --  default              255.255.255.255 udp spt:bootpc
> > dpt:bootps
> >
> > neutron-linuxbri-s8 all -- anywhere anywhere
> >
> > RETURN     udp  --  anywhere anywhere udp spt:bootpc dpt:bootps
> >
> > DROP       udp  --  anywhere anywhere udp spt:bootps dpt:bootpc
> >
> > RETURN     all  --  anywhere anywhere state RELATED,ESTABLISHED
> >
> > RETURN     tcp  --  anywhere anywhere tcp dpt:https
> >
> > RETURN     all  --  anywhere anywhere
> >
> > RETURN     tcp  --  anywhere anywhere tcp dpt:http
> >
> > DROP       all  --  anywhere anywhere state INVALID
> >
> > neutron-linuxbri-sg-fallback  all -- anywhere anywhere
> >
> >
> > Chain neutron-linuxbri-s8 (1 references)
> >
> > target     prot opt source               destination
> >
> > RETURN     all  --  fermicloud248.fnal.gov  anywhere MAC FA:16:
> >
> > DROP       all  --  anywhere anywhere
> >
> >
> >
> > Chain neutron-linuxbri-sg-chain (2 references)
> >
> > target     prot opt source               destination
> >
> > neutron-linuxbri-i8 all  --  anywhere anywhere PHYSDEV match
> --physdev-out
> > tap8 --physdev-is-bridged
> >
> > neutron-linuxbri-o8 all  --  anywhere anywhere PHYSDEV match
> --physdev-in  tap8
> > --physdev-is-bridged
> >
> > ACCEPT     all  --  anywhere             anywhere
> >
> >
> > Chain neutron-linuxbri-sg-fallback (2 references)
> >
> > target     prot opt source               destination
> >
> > DROP       all  --  anywhere             anywhere
> >
> > </end>
> >
> >
> >
> > Let me summarize the differences from when no VM running:
> >
> >
> > Chain INPUT  : no change
> >
> > Chain FORWARD: no change
> >
> > Chain OUTPUT : no change
> >
> > Chain neutron-filter-top: no change
> >
> >
> > Chain neutron-linuxbri-FORWARD: Two new rules are added
> >
> >  neutron-linuxbri-sg-chain
> >
> >  neutron-linuxbri-sg-chain
> >
> >
> > Chain neutron-linuxbri-INPUT: One new rule is added
> >
> >  neutron-linuxbri-o8ae816b0-f
> >
> >
> > Chain neutron-linuxbri-sg-chain: Two new rules are added
> >
> >  neutron-linuxbri-i8
> >
> >  neutron-linuxbri-o8
>
> Those are chains which represents rules from Your Security Group used by a
> VM
>
> >
> >
> > Chain neutron-linuxbri-OUTPUT: no change
> >
> > Chain neutron-linuxbri-local: no change
> >
> > Chain neutron-linuxbri-sg-fallback: no change
> >
> >
> > Chain neutron-linuxbri-i8: A new chain with multiple rules
> >
> > Chain neutron-linuxbri-o8: A new chain with multiple rules
>
> In those 2 chains there are ingress and egress SG rules implemented
>
> >
> > Chain neutron-linuxbri-s8: A new chain with multiple rules
>
> And in this one there are antispoofing rules for Your port added.
>
> >
> >
> >
> > But now a problem arises here:
> >
> > All outbound connections are blocked (remember this is in a compute node,
> > not VM):
> >
> > For example:
> >
> > # telnet www.google.com 80
> >
> > Trying 172.217.5.4...
> >
> >
> > When there isn't any VM running, We don't see this problem.
> >
> >
> > I was wondering if I needed to create a new security group rule for the
> > port 80 (for example)
> >
> > but that didn't solve the issue.
> >
> >
> > Any technical advice will be appreciated,
>
> You should check where exactly Your packets are dropped.
> Also, You didn't tell us what is the type of the Neutron network to which
> Your VM is plugged and how bridges are done on Your compute node.
>
> >
> > Thanks,
> >
> > Hyunwoo
>
> --
> Slawek Kaplonski
> Principal Software Engineer
> Red Hat
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20201207/ca1fd9de/attachment-0001.html>