Hi all,<div><br></div><div>I am running multiple compute nodes and a single nova-network node, that is to act</div><div>as a central gateway for the tenant's VMs.</div><div><br></div><div>However, since this nova-network node (of course) knows all routes, every VM of</div>
<div>any tenant can talk to each other, including to the physical nodes, which</div><div>I highly disagree with and would like to restrict that. :-)</div><div><br></div><div><div><div><font face="courier new, monospace">root@gw1:~# ip route show</font></div>
<div><font face="courier new, monospace">default via $UPLINK_IP dev eth1 metric 100 </font></div><div><font face="courier new, monospace"><a href="http://10.10.0.0/19">10.10.0.0/19</a> dev eth0 proto kernel scope link src 10.10.30.5 </font></div>
<div><font face="courier new, monospace"><a href="http://10.10.40.0/21">10.10.40.0/21</a> dev br100 proto kernel scope link src 10.10.40.1 </font></div><div><font face="courier new, monospace"><a href="http://10.10.48.0/24">10.10.48.0/24</a> dev br101 proto kernel scope link src 10.10.48.1 </font></div>
<div><font face="courier new, monospace"><a href="http://10.10.49.0/24">10.10.49.0/24</a> dev br102 proto kernel scope link src 10.10.49.1 </font></div><div><font face="courier new, monospace">$PUBLIC_NET/28 dev eth1 proto kernel scope link src $PUBLIC_IP</font></div>
<div><font face="courier new, monospace"><a href="http://192.168.0.0/16">192.168.0.0/16</a> dev eth0 proto kernel scope link src 192.168.2.1</font></div></div></div><div><br></div><div>- <a href="http://10.10.0.0/19">10.10.0.0/19</a> is the network for bare metal nodes, switches, PDUs, etc.</div>
<div>- <a href="http://10.10.40.0/21(br100)">10.10.40.0/21(br100)</a> is the "production" tenant</div><div>- <a href="http://10.10.48.0/24">10.10.48.0/24</a> (br101) is the "staging" tenant</div><div>- <a href="http://10.10.49.0/24">10.10.49.0/24</a> (br102) is the "playground" tenant.</div>
<div>- <a href="http://192.168.0.0/16">192.168.0.0/16</a> is the legacy network (management and VM nodes)</div><div><br></div><div>No tenant's VM shall be able to talk to a VM of another tenant.</div><div>And ideally no tenant's VM should be able to talk to the management</div>
<div>network either.</div><div><br></div><div>Unfortunately, since we're migrating a live system, and we also have</div><div>production services on the bare-metal nodes, I had to add special routes</div><div>to allow the legacy installations to communicate to the new "production"</div>
<div>VMs for the transition phase. I hope I can remove that ASAP.</div><div><br></div><div>Now, checking iptables on the nova-network node:</div><div><br></div><div><div><font face="courier new, monospace">root@gw1:~# iptables -t filter -vn -L FORWARD</font></div>
<div><font face="courier new, monospace">Chain FORWARD (policy ACCEPT 64715 packets, 13M bytes)</font></div><div><font face="courier new, monospace"> pkts bytes target prot opt in out source destination </font></div>
<div><font face="courier new, monospace"> 36M 29G nova-filter-top all -- * * <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div><div><font face="courier new, monospace"> 36M 29G nova-network-FORWARD all -- * * <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">root@gw1:~# iptables -t filter -vn -L nova-filter-top</font></div><div><font face="courier new, monospace">Chain nova-filter-top (2 references)</font></div>
<div><font face="courier new, monospace"> pkts bytes target prot opt in out source destination </font></div><div><font face="courier new, monospace"> 36M 29G nova-network-local all -- * * <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">root@gw1:~# iptables -t filter -vn -L nova-network-local</font></div><div><font face="courier new, monospace">Chain nova-network-local (1 references)</font></div>
<div><font face="courier new, monospace"> pkts bytes target prot opt in out source destination </font></div><div><font face="courier new, monospace"> </font></div><div><font face="courier new, monospace">root@gw1:~# iptables -t filter -vn -L nova-network-FORWARD</font></div>
<div><font face="courier new, monospace">Chain nova-network-FORWARD (1 references)</font></div><div><font face="courier new, monospace"> pkts bytes target prot opt in out source destination </font></div>
<div><font face="courier new, monospace"> 0 0 ACCEPT all -- br102 * <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div><div><font face="courier new, monospace"> 0 0 ACCEPT all -- * br102 <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div>
<div><font face="courier new, monospace"> 0 0 ACCEPT udp -- * * <a href="http://0.0.0.0/0">0.0.0.0/0</a> 10.10.49.2 udp dpt:1194</font></div><div><font face="courier new, monospace"> 18M 11G ACCEPT all -- br100 * <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div>
<div><font face="courier new, monospace"> 18M 18G ACCEPT all -- * br100 <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div><div><font face="courier new, monospace"> 0 0 ACCEPT udp -- * * <a href="http://0.0.0.0/0">0.0.0.0/0</a> 10.10.40.2 udp dpt:1194</font></div>
<div><font face="courier new, monospace"> 106K 14M ACCEPT all -- br101 * <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div><div><font face="courier new, monospace">79895 23M ACCEPT all -- * br101 <a href="http://0.0.0.0/0">0.0.0.0/0</a> <a href="http://0.0.0.0/0">0.0.0.0/0</a> </font></div>
<div><font face="courier new, monospace"> 0 0 ACCEPT udp -- * * <a href="http://0.0.0.0/0">0.0.0.0/0</a> 10.10.48.2 udp dpt:1194</font></div></div><div><br></div><div>Now I see, that all traffic from tenant "staging" (br101) for example allows any traffic from/to any destination (-j ACCEPT).</div>
<div>I'd propose to reduce this limitation to the public gateway interface (eth1 in my case) and that this value</div><div>shall be configurable in the nova.conf file.</div><div><br></div><div>Is there any other thing, I might have overseen, to disallow inter-tenant communication and to disallow</div>
<div>tenant-VM-to-bare-metal communication?</div><div><br></div><div>Many thanks in advance,</div><div>Christian Parpart.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><div class="file" id="file_gistfile1.txt" style="margin:0px 0px 10px;padding:0px;border:1px solid rgb(217,217,217);background-color:rgb(245,245,245);text-align:left">
<div class="data type-text" style="text-align:-webkit-auto;margin:0px;padding:0px;overflow:auto;background-color:rgb(248,248,255)"><br></div></div></div>