Open Stack

Hi,

On Mon, Dec 07, 2020 at 10:11:19AM -0600, Hyunwoo KIM wrote:
> Summary of the problem
> 
> This problem is in a compute node, not in a VM.
> 
> Once a VM is running in a compute node,
> 
> all outbound connections in a compute node (not VM) are blocked.
> 
> For example:
> 
> # telnet www.google.com 80
> 
> Trying 172.217.5.4...
> 
> 
> 
> Technical Details:
> 
> We only use provider network.
> 
> These 4 services are running  in each compute node:
> 
> - neutron-linuxbridge-agent.service
> 
> - neutron-dhcp-agent.service
> 
> - neutron-metadata-agent.service
> 
> - openstack-nova-compute.service
> 
> 
> 
> Detailed description of the problem:
> 
> 
> In a compute node, the following is the result of iptables -L when no VM is
> running:
> 
> 
> <begin>
> 
> Chain INPUT (policy ACCEPT)
> 
> target     prot opt source               destination
> 
> neutron-linuxbri-INPUT  all  --  anywhere anywhere
> 
> And our usual rules
> 
> 
> Chain FORWARD (policy ACCEPT)
> 
> target     prot opt source               destination
> 
> neutron-filter-top        all -- anywhere anywhere
> 
> neutron-linuxbri-FORWARD  all -- anywhere anywhere
> 
> 
> Chain OUTPUT (policy ACCEPT)
> 
> target     prot opt source               destination
> 
> neutron-filter-top       all -- anywhere anywhere
> 
> neutron-linuxbri-OUTPUT  all -- anywhere anywhere
> 
> 
> Chain neutron-filter-top (2 references)
> 
> target     prot opt source               destination
> 
> neutron-linuxbri-local  all -- anywhere anywhere
> 
> 
> Chain neutron-linuxbri-FORWARD (1 references)
> 
> target prot opt source               destination
> 
> ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-out tapb
> --physdev-is-bridged
> 
> ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-in  tapb
> --physdev-is-bridged
> 
> ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-out tap9
> --physdev-is-bridged
> 
> ACCEPT all  --  anywhere anywhere PHYSDEV match --physdev-in  tap9
> --physdev-is-bridged
> 
> 
> Chain neutron-linuxbri-INPUT (1 references)
> 
> target     prot opt source               destination
> 
> 
> Chain neutron-linuxbri-OUTPUT (1 references)
> 
> target     prot opt source               destination
> 
> 
> Chain neutron-linuxbri-local (1 references)
> 
> target     prot opt source               destination
> 
> 
> Chain neutron-linuxbri-sg-chain (0 references)
> 
> target     prot opt source               destination
> 
> ACCEPT     all  --  anywhere             anywhere
> 
> 
> Chain neutron-linuxbri-sg-fallback (0 references)
> 
> target     prot opt source               destination
> 
> DROP       all  --  anywhere             anywhere
> 
> </end>
> 
> 
> 
> In the same compute node, when a VM is running,
> 
> the following is the result of iptables -L:
> 
> 
> 
> <begin>
> 
> Chain INPUT (policy ACCEPT)
> 
> target     prot opt source               destination
> 
> neutron-linuxbri-INPUT  all  --  anywhere anywhere
> 
> And our usual rules
> 
> 
> Chain FORWARD (policy ACCEPT)
> 
> target     prot opt source               destination
> 
> neutron-filter-top        all -- anywhere anywhere
> 
> neutron-linuxbri-FORWARD  all -- anywhere anywhere
> 
> 
> Chain OUTPUT (policy ACCEPT)
> 
> target     prot opt source               destination
> 
> neutron-filter-top       all -- anywhere anywhere
> 
> neutron-linuxbri-OUTPUT  all -- anywhere anywhere
> 
> 
> Chain neutron-filter-top (2 references)
> 
> target     prot opt source               destination
> 
> neutron-linuxbri-local  all  --  anywhere             anywhere
> 
> 
> Chain neutron-linuxbri-FORWARD (1 references)
> 
> target     prot opt source               destination
> 
> neutron-linuxbri-sg-chain all -- anywhere anywhere PHYSDEV match
> --physdev-out tap8 --physdev-is-bridged
> 
> neutron-linuxbri-sg-chain all -- anywhere anywhere PHYSDEV match
> --physdev-in  tap8 --physdev-is-bridged
> 
> ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out tapb
> --physdev-is-bridged
> 
> ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-in  tapb
> --physdev-is-bridged
> 
> ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-out tap9
> --physdev-is-bridged
> 
> ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-in  tap9
> --physdev-is-bridged
> 
> 
> Chain neutron-linuxbri-INPUT (1 references)
> 
> target     prot opt source               destination
> 
> neutron-linuxbri-o8 all -- anywhere anywhere PHYSDEV match --physdev-in
> tap8 --physdev-is-bridged
> 
> 
> Chain neutron-linuxbri-OUTPUT (1 references)
> 
> target     prot opt source               destination
> 
> 
> Chain neutron-linuxbri-i8 (1 references)
> 
> target     prot opt source               destination
> 
> RETURN     all  --  anywhere anywhere  state RELATED,ESTABLISHED
> 
> RETURN     udp  --  anywhere fermicloud248.fnal.gov udp spt:bootps
> dpt:bootpc
> 
> RETURN     udp  --  anywhere 255.255.255.255   udp spt:bootps dpt:bootpc
> 
> RETURN     icmp --  anywhere anywhere
> 
> RETURN     tcp  --  fermilab-net.fnal.gov/16 anywhere  tcp dpt:ssh
> 
> RETURN     all  --  anywhere anywhere  match-set
> NIPv41d69ba3c-68e3-414f-8f1b- src
> 
> DROP       all  --  anywhere anywhere  state INVALID
> 
> neutron-linuxbri-sg-fallback  all -- anywhere anywhere
> 
> 
> Chain neutron-linuxbri-local (1 references)
> 
> target     prot opt source               destination
> 
> 
> Chain neutron-linuxbri-o8 (2 references)
> 
> target     prot opt source               destination
> 
> RETURN     udp  --  default              255.255.255.255 udp spt:bootpc
> dpt:bootps
> 
> neutron-linuxbri-s8 all -- anywhere anywhere
> 
> RETURN     udp  --  anywhere anywhere udp spt:bootpc dpt:bootps
> 
> DROP       udp  --  anywhere anywhere udp spt:bootps dpt:bootpc
> 
> RETURN     all  --  anywhere anywhere state RELATED,ESTABLISHED
> 
> RETURN     tcp  --  anywhere anywhere tcp dpt:https
> 
> RETURN     all  --  anywhere anywhere
> 
> RETURN     tcp  --  anywhere anywhere tcp dpt:http
> 
> DROP       all  --  anywhere anywhere state INVALID
> 
> neutron-linuxbri-sg-fallback  all -- anywhere anywhere
> 
> 
> Chain neutron-linuxbri-s8 (1 references)
> 
> target     prot opt source               destination
> 
> RETURN     all  --  fermicloud248.fnal.gov  anywhere MAC FA:16:
> 
> DROP       all  --  anywhere anywhere
> 
> 
> 
> Chain neutron-linuxbri-sg-chain (2 references)
> 
> target     prot opt source               destination
> 
> neutron-linuxbri-i8 all  --  anywhere anywhere PHYSDEV match --physdev-out
> tap8 --physdev-is-bridged
> 
> neutron-linuxbri-o8 all  --  anywhere anywhere PHYSDEV match --physdev-in  tap8
> --physdev-is-bridged
> 
> ACCEPT     all  --  anywhere             anywhere
> 
> 
> Chain neutron-linuxbri-sg-fallback (2 references)
> 
> target     prot opt source               destination
> 
> DROP       all  --  anywhere             anywhere
> 
> </end>
> 
> 
> 
> Let me summarize the differences from when no VM running:
> 
> 
> Chain INPUT  : no change
> 
> Chain FORWARD: no change
> 
> Chain OUTPUT : no change
> 
> Chain neutron-filter-top: no change
> 
> 
> Chain neutron-linuxbri-FORWARD: Two new rules are added
> 
>  neutron-linuxbri-sg-chain
> 
>  neutron-linuxbri-sg-chain
> 
> 
> Chain neutron-linuxbri-INPUT: One new rule is added
> 
>  neutron-linuxbri-o8ae816b0-f
> 
> 
> Chain neutron-linuxbri-sg-chain: Two new rules are added
> 
>  neutron-linuxbri-i8
> 
>  neutron-linuxbri-o8

Those are chains which represents rules from Your Security Group used by a VM

> 
> 
> Chain neutron-linuxbri-OUTPUT: no change
> 
> Chain neutron-linuxbri-local: no change
> 
> Chain neutron-linuxbri-sg-fallback: no change
> 
> 
> Chain neutron-linuxbri-i8: A new chain with multiple rules
> 
> Chain neutron-linuxbri-o8: A new chain with multiple rules

In those 2 chains there are ingress and egress SG rules implemented

> 
> Chain neutron-linuxbri-s8: A new chain with multiple rules

And in this one there are antispoofing rules for Your port added.

> 
> 
> 
> But now a problem arises here:
> 
> All outbound connections are blocked (remember this is in a compute node,
> not VM):
> 
> For example:
> 
> # telnet www.google.com 80
> 
> Trying 172.217.5.4...
> 
> 
> When there isn't any VM running, We don't see this problem.
> 
> 
> I was wondering if I needed to create a new security group rule for the
> port 80 (for example)
> 
> but that didn't solve the issue.
> 
> 
> Any technical advice will be appreciated,

You should check where exactly Your packets are dropped.
Also, You didn't tell us what is the type of the Neutron network to which
Your VM is plugged and how bridges are done on Your compute node.

> 
> Thanks,
> 
> Hyunwoo

-- 
Slawek Kaplonski
Principal Software Engineer
Red Hat