Open Stack

On Tue, Oct 9, 2012 at 7:51 AM, Jānis Ģeņģeris <janis.gengeris at gmail.com> wrote:
> Hi,
>
> I have managed to create the Quantum router so that all the VMs created on
> provider network are going through it. I had to simply use
> router-interface-add.
>
> But I have now the problem with accessing metadata service. As L3 agent adds
> DNAT rule in router, the packets are rewritten and arrives at metadata host,
> but when coming back it doesn't go through the router but directly to the
> source host resulting in TCP handshake failure. Could it be because all the
> ports in br-int which are somehow related to this provider network are
> withing the same VLAN(have the same tag in ovs-vsctl output) thus being in
> the same broadcast domain?

Here's an example of the suggested physical topology for Quantum:
http://docs.openstack.org/trunk/openstack-network/admin/content/connectivity.html

Note the comment below that if metadata is in use, the IP address
space from tenant networks must be routed on both the External and API
networks, so that traffic from the metadata service can make it back
to the quantum router.  It sounds like your topology somehow differs
from this, and that tenant VMs are on the same L2 network and subnet
as your API server?  That won't work for the reason you describe
above.


>
> So the above could be solved with SNAT rule to router address, but then
> metadata service sees routers address instead of VMs and returns HTTP 404
> error.

Yes, the existing nova metadata service requires metadata traffic to
NOT be SNATed, since it looks up the VM based on source IP.


>
> Thanks,
> --janis
>
> On Mon, Oct 8, 2012 at 11:04 PM, Jānis Ģeņģeris <janis.gengeris at gmail.com>
> wrote:
>>
>> Here is the output, with few details.
>>
>> # quantum router-show router_vlan1501
>> +-----------------------+--------------------------------------+
>> | Field                 | Value                                |
>> +-----------------------+--------------------------------------+
>> | admin_state_up        | True                                 |
>> | external_gateway_info |                                      |
>> | id                    | 3c7383c7-7759-4db6-ba5b-19e754280cb8 |
>> | name                  | router_vlan1501                      |
>> | status                | ACTIVE                               |
>> | tenant_id             | 7246a7e9d61f42b8a644bc1551a2a396     |
>> +-----------------------+--------------------------------------+
>>
>> # quantum net-show vlan1501
>> +---------------------------+--------------------------------------+
>> | Field                     | Value                                |
>> +---------------------------+--------------------------------------+
>> | admin_state_up            | True                                 |
>> | id                        | c2161824-a439-40e5-8809-5599f80df2fe |
>> | name                      | vlan1501                             |
>> | provider:network_type     | vlan                                 |
>> | provider:physical_network | default                              |
>> | provider:segmentation_id  | 1501                                 |
>> | router:external           | False                                |
>> | shared                    | False                                |
>> | status                    | ACTIVE                               |
>> | subnets                   | af6eac1e-dfec-49a0-bfc2-3fbf9a7063b3 |
>> | tenant_id                 | 7246a7e9d61f42b8a644bc1551a2a396     |
>> +---------------------------+--------------------------------------+
>>
>> # quantum router-gateway-set 3c7383c7-7759-4db6-ba5b-19e754280cb8
>> c2161824-a439-40e5-8809-5599f80df2fe
>> Bad router request: Network c2161824-a439-40e5-8809-5599f80df2fe is not a
>> valid external network
>>
>> I assume this happens because 'router:external' if False, when switching
>> it to True the above command succeeds. But then if I want to switch back
>> later (with no ports attached, no routers, even no subnets), I get this:
>>
>> quantum net-update vlan1501 --router:external False
>> External network e011d68b-6abd-43a4-b6c3-9dbadf5344ee cannot be updated to
>> be made non-external, since it has existing gateway ports
>>
>> Anyway, I have set 'router:external' to True for 'vlan1501' network, set
>> the gateway for my newly created quantum router(router_vlan1501) with
>> 'router-gateway-set' as a result quantum have created router ns with single
>> interface from my configured provider network 'vlan1501' with default
>> gateway set to the one of the 'vlan1501' nets attached subnet.
>>
>> But now if I boot the VM with '--nic=<vlan1501_id>' option, I get VM with
>> default gateway set to networks('vlan1501') gateway(not the freshly created
>> router gateway, am I missing something in configs?), which is the same as
>> the created routers gateway. Another thing if I switch to newly created
>> 'router_vlan1501' namespace I can't actually ping the external gateway, that
>> is there as a default gw for the 'vlan1501' net.
>>
>> So my thinking is I need to change the default gw in VM to the virtual
>> router?
>>
>> On Mon, Oct 8, 2012 at 9:32 PM, Dan Wendlandt <dan at nicira.com> wrote:
>>>
>>> On Mon, Oct 8, 2012 at 12:27 PM, Jānis Ģeņģeris
>>> <janis.gengeris at gmail.com> wrote:
>>> > On Mon, Oct 8, 2012 at 6:24 PM, Dan Wendlandt <dan at nicira.com> wrote:
>>> >>
>>> >> On Mon, Oct 8, 2012 at 7:52 AM, Jānis Ģeņģeris
>>> >> <janis.gengeris at gmail.com>
>>> >> wrote:
>>> >> > Hello,
>>> >> >
>>> >> > When using provider networks in Quantum, where should the metadata
>>> >> > service
>>> >> > rule mapping (e.g. 169.254.169.254:80 ->
>>> >> > metadata_server:metadata_port)
>>> >> > must
>>> >> > be set?
>>> >> >
>>> >> > For example, for floating IPs l3-agent handles this, but for
>>> >> > provider
>>> >> > networks router is not used. I tried to set custom iptables rule for
>>> >> > this,
>>> >> > but have a hard time understanding where to set it, as there is
>>> >> > openvswitch
>>> >> > and namespaces.
>>> >> >
>>> >> > I'm using provider network configuration with VLANs.
>>> >>
>>> >> You actually could use the Quantum L3 router as your gateway even if
>>> >> VMs are on a provider network, but I suspect your question is actually
>>> >> more along the lines of: if I want my gateway to be a physical router
>>> >> not managed by Quantum, how do I does the DNAT rule for metadata get
>>> >> applied?  In this case, you need to apply the DNAT rule manually to
>>> >> the physical router, which I believe is the same as if you were using
>>> >> flat networking with Nova with a physical router.
>>> >
>>> > Adding the rule in physical router is not a good idea, because then the
>>> > configuration of the OpenStack crosses the actual software/server
>>> > border
>>> > into network equipment, than can add to complexity later.
>>>
>>> Yes, its hard to have it both ways...  if you want everything done
>>> automatically via software, I'd suggest using the quantum router as
>>> the gateway, not an external physical router.
>>>
>>> >
>>> > I tried to add provider network to quantum router, and the quantum CLI
>>> > was
>>> > rejecting it.
>>> > AFAIK router-interface-add is for internal networks, and
>>> > router-gateway-set
>>> > is also failing.
>>>
>>> Can you post what you've run and what the resulting error was?  In
>>> terms of the L3 API, quantum shouldn't care whether a network is
>>> provider or not for router-interface-add.  Perhaps this is a
>>> permissions issue?
>>>
>>> Dan
>>>
>>>
>>> >
>>> > Which CLI command to use for adding provider network to existing
>>> > quantum
>>> > router?
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> There may also be a more complex solution achievable via quantum in
>>> >> which the provider creates a quantum router with an interface on the
>>> >> provider network, VMs are each given a host route to route traffic
>>> >> destined for 169.254.169.254/32 to this quantum router IP, rather than
>>> >> the physical default gateway, and this quantum router performs the
>>> >> DNAT.  However, its probably much easier to just apply this rule to
>>> >> your physical router.
>>> >
>>> > No, this is no good.
>>> >>
>>> >>
>>> >> Dan
>>> >>
>>> >>
>>> >> >
>>> >> > Regards,
>>> >> > --janis
>>> >> >
>>> >> > _______________________________________________
>>> >> > Mailing list: https://launchpad.net/~openstack
>>> >> > Post to     : openstack at lists.launchpad.net
>>> >> > Unsubscribe : https://launchpad.net/~openstack
>>> >> > More help   : https://help.launchpad.net/ListHelp
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> >> Dan Wendlandt
>>> >> Nicira, Inc: www.nicira.com
>>> >> twitter: danwendlandt
>>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> >
>>> >
>>> > Regards,
>>> > --janis
>>>
>>>
>>>
>>> --
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> Dan Wendlandt
>>> Nicira, Inc: www.nicira.com
>>> twitter: danwendlandt
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>>
>>
>> --
>> --janis



-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~