Open Stack

Hi Bruno,

Here are the general troubleshooting steps that we follow when hunting networking issues, assuming that the Quantum configuration is correct:

1) If using VLANs, make sure that the Windows network drivers you are using are not blocking VLAN traffic on the Hyper-V node.

Here's a basic Powershell script that we are developed for that:
https://bitbucket.org/cloudbase/vlanfilter-check

2) On the Hyper-V host, check that the instance NICs are bound to the proper virtual switch (Powershell):

Get-VMNetworkAdapter <instancename>

3) Check that the external switch is bound to the proper host interface (Powershell):

Get-VMSwitch <switchname>

4) If everything is fine on the Hyper-V side, the general OpenVSwitch troubleshooting methodology applies, e.g.:

http://techbackground.blogspot.ro/2013/05/debugging-quantum-dhcp-and-open-vswitch.html

Please let me know if this helps.


Best,

Alessandro



On Jun 7, 2013, at 23:51 , Bruno Oliveira ~lychinus <brunnop.oliveira at gmail.com<mailto:brunnop.oliveira at gmail.com>> wrote:

"(...)Do you have your vSwitch properly configured on your hyper-v host?(...)"

I can't say for sure, Peter, but I think so...

In troubleshooting we did (and are still doing) I can tell that
regardless of the network model that we're using (FLAT or VLAN
Network),
the instance that is provisioned on Hyper-V (for some reason) can't
reach the quantum-l3-agent "by default"
(I said "default" because, we just managed to do it after a hard, long
and boring troubleshoting,
yet, we're not sure if that's how it should be done, indeed)

Since it's not something quick to explain, I'll present the scenario:
(I'm not sure if it might be a candidate for a fix in quantum-l3-agent,
so quantum-devs might be interested too)


Here's how our network interfaces turns out, in our network controller:

==============================
External bridge network
==============================

Bridge "br-eth1"
       Port "br-eth1"
           Interface "br-eth1"
               type: internal
       Port "eth1.11"
           Interface "eth1.11"
       Port "phy-br-eth1"
           Interface "phy-br-eth1"

==============================
Internal network
==============================

  Bridge br-int
       Port "int-br-eth1"
           Interface "int-br-eth1"
       Port br-int
           Interface br-int
               type: internal
       Port "tapb610a695-46"
           tag: 1
           Interface "tapb610a695-46"
               type: internal
       Port "qr-ef10bef4-fa"
           tag: 1
           Interface "qr-ef10bef4-fa"
               type: internal

==============================

There's another iface named "br-ex" that we're using for floating_ips,
but it has nothing to do with what we're doing right now, so I'm skipping it...


************ So, for the hands-on ****************

I know it may be a little bit hard to understand, but I'll do my best
trying to explain:

1) the running instance in Hyper-V, which is linked to Hyper-V vSwitch
is actually
communicating to bridge: "br-eth1" (that is in the network controller).

NOTE: That's where the DHCP REQUEST (from the instance) lands


2) The interface MAC Address, of that running instance on Hyper-V, is:
fa:16:3e:95:95:e4. (we're gonna use it on later steps)
Since DHCP is not fully working yet, we had to manually set an IP for
that instance: "10.5.5.3"


3) From that instance interface, the dhcp_broadcast should be forward ->
  FROM interface "eth1.12" TO  "phy-br-eth1"
  And FROM interface "phy-br-eth1" TO the bridge "br-int"   *** THIS
IS WHERE THE PACKETS ARE DROPPED  ***.

Check it out for the "actions:drop"
---------------------------------------------------------------------------------------------
root at osnetwork:~# ovs-dpctl dump-flows br-int  |grep 10.5.5.3

in_port(4),eth(src=fa:16:3e:f0:ac:8e,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.5.5.3,tip=10.5.5.1,op=1,sha=fa:16:3e:f0:ac:8e,tha=00:00:00:00:00:00),
packets:20, bytes:1120, used:0.412s, actions:drop
---------------------------------------------------------------------------------------------

4) Finally, when the packet reaches the bridge "br-int", the
DHCP_REQUEST should be forward to the
  dhcp_interface, that is: tapb610a695-46    *** WHICH IS NOT
HAPPENING EITHER ***


5) How to fix :: bridge br-eth1

-------------------------------------------
5.1. Getting to know the ifaces of 'br-eth1'
-------------------------------------------
root at osnetwork:~# ovs-ofctl show br-eth1

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000e0db554e164b
n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff

1(eth1.11): addr:e0:db:55:4e:16:4b
    config:     0
    state:      0
    current:    10GB-FD AUTO_NEG
    advertised: 1GB-FD 10GB-FD FIBER AUTO_NEG
    supported:  1GB-FD 10GB-FD FIBER AUTO_NEG

3(phy-br-eth1): addr:26:9b:97:93:b9:70
    config:     0
    state:      0
    current:    10GB-FD COPPER

LOCAL(br-eth1): addr:e0:db:55:4e:16:4b
    config:     0
    state:      0

OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0


-------------------------------------------
5.2. Adding flow rules to enable passing (instead of dropping)
-------------------------------------------

# the source mac_address (dl_src) is the from the interface of the
# running instance on Hyper-V. This fix the DROP (only)

root at osnetwork:~# ovs-ofctl add-flow br-eth1
priority=10,in_port=3,dl_src=fa:16:3e:95:95:e4,actions=normal



6) How to fix :: bridge br-int

-------------------------------------------
6.1. Getting to know the ifaces of 'br-int'
-------------------------------------------

root at osnetwork:~# ovs-ofctl show br-int

OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000092976d64274d

n_tables:255, n_buffers:256  features: capabilities:0xc7, actions:0xfff

1(tapb610a695-46): addr:19:01:00:00:00:00
    config:     PORT_DOWN
    state:      LINK_DOWN

4(int-br-eth1): addr:5a:56:e1:53:e9:90
    config:     0
    state:      0
    current:    10GB-FD COPPER

5(qr-ef10bef4-fa): addr:19:01:00:00:00:00
    config:     PORT_DOWN
    state:      LINK_DOWN

LOCAL(br-int): addr:92:97:6d:64:27:4d
    config:     0
    state:      0

-------------------------------------------
6.2. Adding flow rules to enable FORWARD
    FROM: interface int-br-eth1  (4)
    TO:   interface tapb610a695-46 (1) -> dhcp_interface

    and the REVERSE_FORWARD: from (1) to (4)
-------------------------------------------
root at osnetwork:~# ovs-ofctl add-flow br-int
priority=12,in_port=4,dl_src=fa:16:3e:f0:ac:8e,action=1,normal
root at osnetwork:~# ovs-ofctl add-flow br-int
priority=12,in_port=1,dl_dst=fa:16:3e:f0:ac:8e,action=4,normal


==================
Conclusion
==================

That way, and only *that way*, the Hyper-V instance is able to
exchange ARP with the dhcp (network controller).

Even though, it is functional, we're not sure if that's how it HAS to
be done. May I have your thoughts on it?

Should we really have to create those rules/actions in openVSwitch to
make the instance (of hyper-v) to reach out
DHCP ? It seems either bug or something is wierd in my configurations...

May I have your opinions on it?


We'd greatly appreciate your feedback. Thank you very much.

_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack at lists.launchpad.net<mailto:openstack at lists.launchpad.net>
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20130611/7eb098c9/attachment.html>