<div dir="ltr"><div><div><div><div><div>Hi Germy,<br><br></div>Yes i understand now.<br></div>What you request is an enahncment to the API to be able to assing these port forwarding rules in bulks per subnet.<br></div>I will make sure to mention this in the spec that i am writing for this.<br><br></div>Thanks!<br></div>Gal.<br><div><div><div><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 8, 2015 at 10:33 AM, Germy Lure <span dir="ltr"><<a href="mailto:germy.lure@gmail.com" target="_blank">germy.lure@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Gal,<div><br></div><div>Thank you for your explanation.</div><div>As you mentioned, PF is a way of reusing floating IP to access several Neutron ports. I agree with your point of view completely.</div><div>Let me extend your example to explain where I was going.</div><div>T1 has 20 subnets behind a router, and one of them is <a href="http://10.0.0.0/24" target="_blank">10.0.0.0/24</a> named s1. There are 100 VMs named VM1~VM100 in the subnet s1 and T1 wants to update the same file(or something else) in those VMs. Let's have a look how will T1 do it.</div><div><br></div><div>T1 invokes Neutron API to create a port-mapping for VM1(Maybe that will be did by operator)</div><div>For example : <a href="http://172.20.20.10:4001" target="_blank">172.20.20.10:4001</a><span style="font-size:12.8000001907349px"> => </span><a href="http://10.0.0.1:80" target="_blank">10.0.0.1:80</a><br></div><div>And then T1 does the update task via <a href="http://172.20.20.10:4001" target="_blank">172.20.20.10:4001</a>.</div><div><br></div><div>Now for the VM2,VM3,...VM100, T1 must repeat the steps above with different ports. And T1 must clean those records(100 records in DB) after accessing. That's badly, I think.</div><div>Note that T1 still has 19 subnets to be dealt with. That's a nightmare to T1.</div><div>To PaaS, SaaS, that also is a big trouble.<br></div><div><br></div><div>So, can we do it like this?</div><div>T1 invokes Neutron API one time for s1(not VM1), and Neutron setups a group of port-mapping relation. For example:</div><span class=""><div><a href="http://172.20.20.10:4001" target="_blank">172.20.20.10:4001</a><span style="font-size:12.8000001907349px"> => </span><a href="http://10.0.0.1:80" target="_blank">10.0.0.1:80</a><br></div><div><a href="http://172.20.20.10:4002" target="_blank">172.20.20.10:4002</a><span style="font-size:12.8000001907349px"> => </span><a href="http://10.0.0.2:80" target="_blank">10.0.0.2:80</a><br></div></span><div><a href="http://172.20.20.10:4003" target="_blank">172.20.20.10:4003</a><span style="font-size:12.8000001907349px"> => </span><a href="http://10.0.0.3:80" target="_blank">10.0.0.3:80</a><br></div><div>...... ......</div><div><a href="http://172.20.20.10:4100" target="_blank">172.20.20.10:4100</a><span style="font-size:12.8000001907349px"> => </span><a href="http://10.0.0.100:80" target="_blank">10.0.0.100:80</a><br></div><div>Now T1 just needs focus on his/her business work not PF.</div><div><br></div><div>We just store one record in Neutron DB for such one time API invoking. For the single VM scene, we can specific private IP range instead of subnet. For example, 10.0.0.1 to 10.0.0.3. The mapped ports(like 4001,4002...) can be returned in the response body, for example, 4001 to 4003, also can just return a base number(4000) and upper layer rework it. For example, 4000+1, where 1 is the last number in the private IP address of VM1.</div><div><br></div><div>Forgive my poor E.</div><div>Hope that's clear enough and i am happy to discuss it further if necessary.</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Germy</div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 8, 2015 at 1:58 PM, Gal Sagie <span dir="ltr"><<a href="mailto:gal.sagie@gmail.com" target="_blank">gal.sagie@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div>Hi Germy,<br><br></div>Port forwarding the way i see it, is a way of reusing the same floating ip to access several different Neutron ports (VM's , Containers)<br></div>So for example if we have floating IP 172.20.20.10 , we can assign <a href="http://172.20.20.10:4001" target="_blank">172.20.20.10:4001</a> to VM1 and <a href="http://172.20.20.10:4002" target="_blank">172.20.20.10:4002</a> to VM2 (which are behind that same router<br></div>which has an external gw).<br></div><div>The user use the same IP but according to the tcp/udp port Neutron performs mapping in the virtual router namespace to the private IP and possibly to a different port <br></div><div>that is running on that instance for example port 80<br><br></div><div>So for example if we have two VM's with private IP's 10.0.0.1 and 10.0.0.2 and we have a floating ip assigned to the router of 172.20.20.10 <br></div><div>with port forwarding we can build the following mapping:<br><br><a href="http://172.20.20.10:4001" target="_blank">172.20.20.10:4001</a> => <a href="http://10.0.0.1:80" target="_blank">10.0.0.1:80</a><br><a href="http://172.20.20.10:4002" target="_blank">172.20.20.10:4002</a> => <a href="http://10.0.0.2:80" target="_blank">10.0.0.2:80</a><br><br></div><div>And this is only from the Neutron API, this feature is usefull when you offer PaaS, SaaS and have an automated framework that calls the API<br></div><div>to allocate these "client ports"<br><br></div>I am not sure why you think the operator will need to ssh the instances, the operator just needs to build the mapping of <floating_ip, port> to the instance private IP.<br></div>Of course keep in mind that we didnt yet discuss full API details but its going to be something like that (at least the way i see it)<br><br></div>Hope thats explains it.<span><font color="#888888"><br><br></font></span></div><span><font color="#888888">Gal.<br></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 7, 2015 at 5:21 AM, Germy Lure <span dir="ltr"><<a href="mailto:germy.lure@gmail.com" target="_blank">germy.lure@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Gal,<div><br></div><div>I'm sorry for my poor English. Let me try again.</div><div><br></div><div>What operator wants to access is several related instances, instead of only one or one by one. The use case is periodical check and maintain. RELATED means instance maybe in one subnet, or one network, or one host. The host's scene is similar to access the docker on the host as you mentioned before.</div><div><br></div><div>Via what you mentioned of API, user must ssh an instance and then invoke API to update the IP address and port, or even create a new PF to access another one. It will be a nightmare to a VPC operator who owns so many instances.</div><div><br></div><div>In a word, I think the "inside_addr" should be "subnet" or "host".</div><div><br></div><div>Hope this is clear enough.</div><span><font color="#888888"><div><br></div><div>Germy</div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Sep 6, 2015 at 1:05 PM, Gal Sagie <span dir="ltr"><<a href="mailto:gal.sagie@gmail.com" target="_blank">gal.sagie@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Hi Germy,<br><br></div>I am not sure i understand what you mean, can you please explain it further? <br><br></div><div>Thanks<br></div><div>Gal.<br></div></div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Sun, Sep 6, 2015 at 5:39 AM, Germy Lure <span dir="ltr"><<a href="mailto:germy.lure@gmail.com" target="_blank">germy.lure@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi, Gal<div><br></div><div>Thank you for bringing this up. But I have some suggestions for the API.</div><div><br></div><div>An operator or some other component wants to reach several VMs related NOT only one or one by one. Here, RELATED means that the VMs are in one subnet or network or a host(similar to reaching dockers on a host).</div><div><br></div><div>Via the API you mentioned, user must ssh one VM and update even delete and add PF to ssh another. To a VPC(with 20 subnets?) admin, it's totally a nightmare.</div><div><br></div><div>Germy</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Wed, Sep 2, 2015 at 1:59 PM, Gal Sagie <span dir="ltr"><<a href="mailto:gal.sagie@gmail.com" target="_blank">gal.sagie@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><div><div><div>Hello All,<br><br></div>I have searched and found many past efforts to implement port forwarding in Neutron.<br></div>I have found two incomplete blueprints [1], [2] and an abandoned patch [3].<br><br></div><div>There is even a project in Stackforge [4], [5] that claims<br></div><div>to implement this, but the L3 parts in it seems older then current master.<br><br></div><div>I have recently came across this requirement for various use cases, one of them is<br></div><div>providing feature compliance with Docker port-mapping feature (for Kuryr), and saving floating<br></div><div>IP's space.<br></div><div>There has been many discussions in the past that require this feature, so i assume<br></div><div>there is a demand to make this formal, just a small examples [6], [7], [8], [9]<br></div><div><br>The idea in a nutshell is to support port forwarding (TCP/UDP ports) on the external router<br>leg from the public network to internal ports, so user can use one Floating IP (the external<br>gateway router interface IP) and reach different internal ports depending on the port numbers.<br></div><div>This should happen on the network node (and can also be leveraged for security reasons).<br><br></div><div>I think that the POC implementation in the Stackforge project shows that this needs to be<br></div><div>implemented inside the L3 parts of the current reference implementation, it will be hard<br></div><div>to maintain something like that in an external repository.<br></div><div>(I also think that the API/DB extensions should be close to the current L3 reference <br></div><div>implementation)<br><br></div><div>I would like to renew the efforts on this feature and propose a RFE and a spec for this to the <br></div><div>next release, any comments/ideas/thoughts are welcome.<br></div><div>And of course if any of the people interested or any of the people that worked on this before<br></div><div>want to join the effort, you are more then welcome to join and comment.<br><br></div><div>Thanks<br></div><div>Gal.<br><br></div><div>[1] <a href="https://blueprints.launchpad.net/neutron/+spec/router-port-forwarding" target="_blank">https://blueprints.launchpad.net/neutron/+spec/router-port-forwarding</a><br>[2] <a href="https://blueprints.launchpad.net/neutron/+spec/fip-portforwarding" target="_blank">https://blueprints.launchpad.net/neutron/+spec/fip-portforwarding</a><br>[3] <a href="https://review.openstack.org/#/c/60512/" target="_blank">https://review.openstack.org/#/c/60512/</a><br>[4] <a href="https://github.com/stackforge/networking-portforwarding" target="_blank">https://github.com/stackforge/networking-portforwarding</a><br>[5] <a href="https://review.openstack.org/#/q/port+forwarding,n,z" target="_blank">https://review.openstack.org/#/q/port+forwarding,n,z</a><br><br>[6] <a href="https://ask.openstack.org/en/question/75190/neutron-port-forwarding-qrouter-vms/" target="_blank">https://ask.openstack.org/en/question/75190/neutron-port-forwarding-qrouter-vms/</a><br>[7] <a href="http://www.gossamer-threads.com/lists/openstack/dev/34307" target="_blank">http://www.gossamer-threads.com/lists/openstack/dev/34307</a><br>[8] <a href="http://openstack.10931.n7.nabble.com/Neutron-port-forwarding-for-router-td46639.html" target="_blank">http://openstack.10931.n7.nabble.com/Neutron-port-forwarding-for-router-td46639.html</a><br>[9] <a href="http://openstack.10931.n7.nabble.com/Neutron-port-forwarding-from-gateway-to-internal-hosts-td32410.html" target="_blank">http://openstack.10931.n7.nabble.com/Neutron-port-forwarding-from-gateway-to-internal-hosts-td32410.html</a><br><br><br>
</div></div>
<br></div></div>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><br></div></div><span><font color="#888888">-- <br><div>Best Regards ,<br><br>The G. </div>
</font></span></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div>Best Regards ,<br><br>The G. </div>
</div>
</div></div><br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">Best Regards ,<br><br>The G. </div>
</div>