[openstack-dev] [Quantum] security groups now enforced w/devstack

Yoshihiro Kaneko ykaneko0929 at gmail.com
Tue Sep 18 06:12:28 UTC 2012


Hi Dan,

Thanks for the reply.

This is the result of "brctl show".
----------
$ brctl show
bridge name	bridge id		STP enabled	interfaces
br-ex		0000.6e47647aee44	no		qg-8c3a652b-f1
br-int		0000.4af0ded34a40	no		qr-f9a49be5-c1
							tap5a418ee0-53
qbr99eea189-fc		8000.5e15f4c7d4f2	no		qvb99eea189-fc
							vnet0
virbr0		8000.000000000000	yes		
virbr1		8000.525400d8a792	yes		virbr1-nic
----------

Following is the result that was running tcpdump on each terminal concurrently.
I don't know why each DHCP packet was received twice on vnet0. And the tap
device used by dnsmasq did not receive udp packet in qdhcp- namespace.
----------
# tcpdump -ni vnet0 udp
tcpdump: WARNING: vnet0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), capture size 65535 bytes
12:00:56.619340 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:00:56.619423 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:00:59.624796 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:00:59.624858 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:01:02.630474 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:01:02.630564 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280

# tcpdump -ni qvb99eea189-fc udp
tcpdump: WARNING: qvb99eea189-fc: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qvb99eea189-fc, link-type EN10MB (Ethernet), capture size
65535 bytes
12:00:56.619436 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:00:59.624872 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:01:02.630586 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280

# tcpdump -ni qvo99eea189-fc udp
tcpdump: WARNING: qvo99eea189-fc: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qvo99eea189-fc, link-type EN10MB (Ethernet), capture size
65535 bytes
12:00:56.619467 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:00:59.624904 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280
12:01:02.630633 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from fa:16:3e:48:aa:85, length 280

# tcpdump -ni tap5a418ee0-53 udp
tcpdump: tap5a418ee0-53: No such device exists
(SIOCGIFHWADDR: No such device)
# ip netns exec qdhcp-c95edcb8-3df6-4e92-a367-4fce5bce6e63 tcpdump -ni
tap5a418ee0-53 udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap5a418ee0-53, link-type EN10MB (Ethernet), capture size
65535 bytes
----------

When using "libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver",
VM can obtain an IP address by DHCP.
----------
$ grep libvirt_vif_driver /etc/nova/nova.conf
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver
$ nova console-log vm1
<snip>
Starting network...
udhcpc (v1.18.5) started
Sending discover...
Sending select for 10.0.0.3...
Lease of 10.0.0.3 obtained, lease time 120
deleting routers
route: SIOCDELRT: No such process
adding dns 10.0.0.2
<snip>
----------

Thanks,

Kaneko

2012/9/17 Dan Wendlandt <dan at nicira.com>:
> Hi Kaneko,
>
> Thanks for the detailed report.  At a glance, things look correct.
>
> I'd suggest running tcpdump on various interfaces to figure how far
> the DHCP requests/responses are reaching.
>
> tcpdump -ni vnetX udp (replace vnetX with correct device... can't tell
> from your output.  try running brctl show).
>
> tcpdump -ni qvb99eea189-fc udp   (veth device on VIF-specific bridge)
>
> tcpdump -ni qvo99eea189-fc udp   (veth device on main OVS bridge)
>
> tcpdump -ni tap5a418ee0-53 udp (tap device used by dnsmasq)
>
> Another interesting data point would be whether you only see these
> issues when using
> "libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver",
> or whether you also see it with
> "libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver
>
> Thanks,
>
> Dan
>
>
> On Fri, Sep 14, 2012 at 3:13 AM, Yoshihiro Kaneko <ykaneko0929 at gmail.com> wrote:
>> Hi Dan,
>>
>> I tried it on Ubuntu 12.04. However VM could not obtain an IP address from
>> a DHCP server.
>> Any advice?
>> If more information is needed, please let me know.
>>
>> $ git clone git://github.com/openstack-dev/devstack
>> Cloning into 'devstack'...
>> remote: Counting objects: 6415, done.
>> remote: Compressing objects: 100% (2186/2186), done.
>> remote: Total 6415 (delta 4352), reused 6145 (delta 4151)
>> Receiving objects: 100% (6415/6415), 1.10 MiB | 470 KiB/s, done.
>> Resolving deltas: 100% (4352/4352), done.
>> $ cd devstack/
>> $ git fetch https://review.openstack.org/openstack-dev/devstack
>> refs/changes/50/11650/8 && git checkout FETCH_HEAD
>> remote: Counting objects: 5, done
>> remote: Finding sources: 100% (3/3)
>> remote: Total 3 (delta 2), reused 3 (delta 2)
>> Unpacking objects: 100% (3/3), done.
>> From https://review.openstack.org/openstack-dev/devstack
>>  * branch            refs/changes/50/11650/8 -> FETCH_HEAD
>> Note: checking out 'FETCH_HEAD'.
>>
>> You are in 'detached HEAD' state. You can look around, make experimental
>> changes and commit them, and you can discard any commits you make in this
>> state without impacting any branches by performing another checkout.
>>
>> If you want to create a new branch to retain commits you create, you may
>> do so (now or later) by using -b with the checkout command again. Example:
>>
>>   git checkout -b new_branch_name
>>
>> HEAD is now at cea6c51... Quantum enhancements
>> $ vi localrc
>> $ cat localrc
>> disable_service n-net
>> enable_service q-svc q-agt q-dhcp q-l3 quantum
>>
>> $ cp samples/local.sh .
>> $ ./stack.sh
>> <snip>
>> $ . ./openrc demo demo
>> $ nova image-list
>> <snip>
>> $ quantum net-list
>> <snip>
>> $ nova boot --flavor 6 --image 7777d0ed-294c-4634-9304-769f64a52c81
>> --nic net-id=c95edcb8-3df6-4e92-a367-4fce5bce6e63 vm1
>> <snip>
>> $ nova list
>> +--------------------------------------+------+--------+---------------+
>> | ID                                   | Name | Status | Networks      |
>> +--------------------------------------+------+--------+---------------+
>> | 723d76c4-4dea-4fe4-a4d8-6ca8d08bb936 | vm1  | ACTIVE | net1=10.0.0.3 |
>> +--------------------------------------+------+--------+---------------+
>> $ nova console-log vm1
>> <snip>
>> Starting network...
>> udhcpc (v1.18.5) started
>> Sending discover...
>> Sending discover...
>> Sending discover...
>> No lease, failing
>> WARN: /etc/rc3.d/S40-network failed
>> <snip>
>>
>> $ sudo ovs-vsctl show
>> 6d7929f6-8b09-41bc-b775-1ef50915eb7b
>>     Bridge br-ex
>>         Port "qg-8c3a652b-f1"
>>             Interface "qg-8c3a652b-f1"
>>                 type: internal
>>         Port br-ex
>>             Interface br-ex
>>                 type: internal
>>     Bridge br-int
>>         Port "tap5a418ee0-53"
>>             tag: 1
>>             Interface "tap5a418ee0-53"
>>                 type: internal
>>         Port "qr-f9a49be5-c1"
>>             tag: 1
>>             Interface "qr-f9a49be5-c1"
>>                 type: internal
>>         Port "qvo99eea189-fc"
>>             tag: 1
>>             Interface "qvo99eea189-fc"
>>         Port br-int
>>             Interface br-int
>>                 type: internal
>>     ovs_version: "1.4.0+build0"
>>
>> $ ip netns
>> qrouter-aed60aac-288b-41ee-bc4a-9ed5a356a16d
>> qdhcp-c95edcb8-3df6-4e92-a367-4fce5bce6e63
>>
>> $ ps ax|grep dnsmasq
>>  6107 ?        S      0:00 dnsmasq --no-hosts --no-resolv
>> --strict-order --bind-interfaces --interface=tap5a418ee0-53
>> --except-interface=lo --domain=openstacklocal
>> --pid-file=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/pid
>> --dhcp-hostsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/host
>> --dhcp-optsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/opts
>> --dhcp-script=/opt/stack/quantum/bin/quantum-dhcp-agent-dnsmasq-lease-update
>> --leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s
>>  6108 ?        S      0:00 dnsmasq --no-hosts --no-resolv
>> --strict-order --bind-interfaces --interface=tap5a418ee0-53
>> --except-interface=lo --domain=openstacklocal
>> --pid-file=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/pid
>> --dhcp-hostsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/host
>> --dhcp-optsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/opts
>> --dhcp-script=/opt/stack/quantum/bin/quantum-dhcp-agent-dnsmasq-lease-update
>> --leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s
>> 28913 pts/1    S+     0:00 grep --color=auto dnsmasq
>>
>>
>> Thanks,
>> Kaneko
>>
>> 2012/9/14 Dan Wendlandt <dan at nicira.com>:
>>> Hi quantum hackers,
>>>
>>> We're pushing a change to devstack to use a new vif-driver for Quantum
>>> with Open vSwitch (https://review.openstack.org/#/c/11650/).  The
>>> benefit of this driver is that it is compatible with Nova's security
>>> group filtering.  This is "a good thing", since it more closely maps
>>> to how real users will deploy Quantum.
>>>
>>> However, this may catch developers by surprise who are suddenly unable
>>> to ping or SSH to instances because the security groups drop traffic
>>> by default.
>>>
>>> Preferred method of dealing with this is to add the following lines to
>>> local.sh in your devstack directory, which open up your VMs for ping
>>> and SSH for the 'demo' user:
>>>
>>> nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
>>> nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
>>>
>>> Another work around is to disable nova security groups by adding
>>> 'LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver' to
>>> your localrc
>>>
>>> Dan
>>>
>>>
>>>
>>>
>>>
>>> --
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> Dan Wendlandt
>>> Nicira, Inc: www.nicira.com
>>> twitter: danwendlandt
>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Dan Wendlandt
> Nicira, Inc: www.nicira.com
> twitter: danwendlandt
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list