[openstack-dev] [Quantum] security groups now enforced w/devstack
Dan Wendlandt
dan at nicira.com
Tue Sep 18 06:53:13 UTC 2012
On Mon, Sep 17, 2012 at 11:12 PM, Yoshihiro Kaneko
<ykaneko0929 at gmail.com> wrote:
> Hi Dan,
>
> Thanks for the reply.
>
> This is the result of "brctl show".
> ----------
> $ brctl show
> bridge name bridge id STP enabled interfaces
> br-ex 0000.6e47647aee44 no qg-8c3a652b-f1
> br-int 0000.4af0ded34a40 no qr-f9a49be5-c1
> tap5a418ee0-53
> qbr99eea189-fc 8000.5e15f4c7d4f2 no qvb99eea189-fc
> vnet0
> virbr0 8000.000000000000 yes
> virbr1 8000.525400d8a792 yes virbr1-nic
can you send the output of
ovs-vsctl list-ports br-int (or ovs-vsctl show)?
Its odd that the above output does not include qvo99eea189-fc, which
should be attached to br-int.
dan
> ----------
>
> Following is the result that was running tcpdump on each terminal concurrently.
> I don't know why each DHCP packet was received twice on vnet0. And the tap
> device used by dnsmasq did not receive udp packet in qdhcp- namespace.
> ----------
> # tcpdump -ni vnet0 udp
> tcpdump: WARNING: vnet0: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on vnet0, link-type EN10MB (Ethernet), capture size 65535 bytes
> 12:00:56.619340 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:00:56.619423 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:00:59.624796 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:00:59.624858 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:01:02.630474 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:01:02.630564 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
>
> # tcpdump -ni qvb99eea189-fc udp
> tcpdump: WARNING: qvb99eea189-fc: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on qvb99eea189-fc, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 12:00:56.619436 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:00:59.624872 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:01:02.630586 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
>
> # tcpdump -ni qvo99eea189-fc udp
> tcpdump: WARNING: qvo99eea189-fc: no IPv4 address assigned
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on qvo99eea189-fc, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 12:00:56.619467 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:00:59.624904 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
> 12:01:02.630633 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
> Request from fa:16:3e:48:aa:85, length 280
>
> # tcpdump -ni tap5a418ee0-53 udp
> tcpdump: tap5a418ee0-53: No such device exists
> (SIOCGIFHWADDR: No such device)
> # ip netns exec qdhcp-c95edcb8-3df6-4e92-a367-4fce5bce6e63 tcpdump -ni
> tap5a418ee0-53 udp
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on tap5a418ee0-53, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> ----------
>
> When using "libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver",
> VM can obtain an IP address by DHCP.
> ----------
> $ grep libvirt_vif_driver /etc/nova/nova.conf
> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver
> $ nova console-log vm1
> <snip>
> Starting network...
> udhcpc (v1.18.5) started
> Sending discover...
> Sending select for 10.0.0.3...
> Lease of 10.0.0.3 obtained, lease time 120
> deleting routers
> route: SIOCDELRT: No such process
> adding dns 10.0.0.2
> <snip>
> ----------
>
> Thanks,
>
> Kaneko
>
> 2012/9/17 Dan Wendlandt <dan at nicira.com>:
>> Hi Kaneko,
>>
>> Thanks for the detailed report. At a glance, things look correct.
>>
>> I'd suggest running tcpdump on various interfaces to figure how far
>> the DHCP requests/responses are reaching.
>>
>> tcpdump -ni vnetX udp (replace vnetX with correct device... can't tell
>> from your output. try running brctl show).
>>
>> tcpdump -ni qvb99eea189-fc udp (veth device on VIF-specific bridge)
>>
>> tcpdump -ni qvo99eea189-fc udp (veth device on main OVS bridge)
>>
>> tcpdump -ni tap5a418ee0-53 udp (tap device used by dnsmasq)
>>
>> Another interesting data point would be whether you only see these
>> issues when using
>> "libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver",
>> or whether you also see it with
>> "libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver
>>
>> Thanks,
>>
>> Dan
>>
>>
>> On Fri, Sep 14, 2012 at 3:13 AM, Yoshihiro Kaneko <ykaneko0929 at gmail.com> wrote:
>>> Hi Dan,
>>>
>>> I tried it on Ubuntu 12.04. However VM could not obtain an IP address from
>>> a DHCP server.
>>> Any advice?
>>> If more information is needed, please let me know.
>>>
>>> $ git clone git://github.com/openstack-dev/devstack
>>> Cloning into 'devstack'...
>>> remote: Counting objects: 6415, done.
>>> remote: Compressing objects: 100% (2186/2186), done.
>>> remote: Total 6415 (delta 4352), reused 6145 (delta 4151)
>>> Receiving objects: 100% (6415/6415), 1.10 MiB | 470 KiB/s, done.
>>> Resolving deltas: 100% (4352/4352), done.
>>> $ cd devstack/
>>> $ git fetch https://review.openstack.org/openstack-dev/devstack
>>> refs/changes/50/11650/8 && git checkout FETCH_HEAD
>>> remote: Counting objects: 5, done
>>> remote: Finding sources: 100% (3/3)
>>> remote: Total 3 (delta 2), reused 3 (delta 2)
>>> Unpacking objects: 100% (3/3), done.
>>> From https://review.openstack.org/openstack-dev/devstack
>>> * branch refs/changes/50/11650/8 -> FETCH_HEAD
>>> Note: checking out 'FETCH_HEAD'.
>>>
>>> You are in 'detached HEAD' state. You can look around, make experimental
>>> changes and commit them, and you can discard any commits you make in this
>>> state without impacting any branches by performing another checkout.
>>>
>>> If you want to create a new branch to retain commits you create, you may
>>> do so (now or later) by using -b with the checkout command again. Example:
>>>
>>> git checkout -b new_branch_name
>>>
>>> HEAD is now at cea6c51... Quantum enhancements
>>> $ vi localrc
>>> $ cat localrc
>>> disable_service n-net
>>> enable_service q-svc q-agt q-dhcp q-l3 quantum
>>>
>>> $ cp samples/local.sh .
>>> $ ./stack.sh
>>> <snip>
>>> $ . ./openrc demo demo
>>> $ nova image-list
>>> <snip>
>>> $ quantum net-list
>>> <snip>
>>> $ nova boot --flavor 6 --image 7777d0ed-294c-4634-9304-769f64a52c81
>>> --nic net-id=c95edcb8-3df6-4e92-a367-4fce5bce6e63 vm1
>>> <snip>
>>> $ nova list
>>> +--------------------------------------+------+--------+---------------+
>>> | ID | Name | Status | Networks |
>>> +--------------------------------------+------+--------+---------------+
>>> | 723d76c4-4dea-4fe4-a4d8-6ca8d08bb936 | vm1 | ACTIVE | net1=10.0.0.3 |
>>> +--------------------------------------+------+--------+---------------+
>>> $ nova console-log vm1
>>> <snip>
>>> Starting network...
>>> udhcpc (v1.18.5) started
>>> Sending discover...
>>> Sending discover...
>>> Sending discover...
>>> No lease, failing
>>> WARN: /etc/rc3.d/S40-network failed
>>> <snip>
>>>
>>> $ sudo ovs-vsctl show
>>> 6d7929f6-8b09-41bc-b775-1ef50915eb7b
>>> Bridge br-ex
>>> Port "qg-8c3a652b-f1"
>>> Interface "qg-8c3a652b-f1"
>>> type: internal
>>> Port br-ex
>>> Interface br-ex
>>> type: internal
>>> Bridge br-int
>>> Port "tap5a418ee0-53"
>>> tag: 1
>>> Interface "tap5a418ee0-53"
>>> type: internal
>>> Port "qr-f9a49be5-c1"
>>> tag: 1
>>> Interface "qr-f9a49be5-c1"
>>> type: internal
>>> Port "qvo99eea189-fc"
>>> tag: 1
>>> Interface "qvo99eea189-fc"
>>> Port br-int
>>> Interface br-int
>>> type: internal
>>> ovs_version: "1.4.0+build0"
>>>
>>> $ ip netns
>>> qrouter-aed60aac-288b-41ee-bc4a-9ed5a356a16d
>>> qdhcp-c95edcb8-3df6-4e92-a367-4fce5bce6e63
>>>
>>> $ ps ax|grep dnsmasq
>>> 6107 ? S 0:00 dnsmasq --no-hosts --no-resolv
>>> --strict-order --bind-interfaces --interface=tap5a418ee0-53
>>> --except-interface=lo --domain=openstacklocal
>>> --pid-file=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/pid
>>> --dhcp-hostsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/host
>>> --dhcp-optsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/opts
>>> --dhcp-script=/opt/stack/quantum/bin/quantum-dhcp-agent-dnsmasq-lease-update
>>> --leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s
>>> 6108 ? S 0:00 dnsmasq --no-hosts --no-resolv
>>> --strict-order --bind-interfaces --interface=tap5a418ee0-53
>>> --except-interface=lo --domain=openstacklocal
>>> --pid-file=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/pid
>>> --dhcp-hostsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/host
>>> --dhcp-optsfile=/opt/stack/data/dhcp/c95edcb8-3df6-4e92-a367-4fce5bce6e63/opts
>>> --dhcp-script=/opt/stack/quantum/bin/quantum-dhcp-agent-dnsmasq-lease-update
>>> --leasefile-ro --dhcp-range=set:tag0,10.0.0.0,static,120s
>>> 28913 pts/1 S+ 0:00 grep --color=auto dnsmasq
>>>
>>>
>>> Thanks,
>>> Kaneko
>>>
>>> 2012/9/14 Dan Wendlandt <dan at nicira.com>:
>>>> Hi quantum hackers,
>>>>
>>>> We're pushing a change to devstack to use a new vif-driver for Quantum
>>>> with Open vSwitch (https://review.openstack.org/#/c/11650/). The
>>>> benefit of this driver is that it is compatible with Nova's security
>>>> group filtering. This is "a good thing", since it more closely maps
>>>> to how real users will deploy Quantum.
>>>>
>>>> However, this may catch developers by surprise who are suddenly unable
>>>> to ping or SSH to instances because the security groups drop traffic
>>>> by default.
>>>>
>>>> Preferred method of dealing with this is to add the following lines to
>>>> local.sh in your devstack directory, which open up your VMs for ping
>>>> and SSH for the 'demo' user:
>>>>
>>>> nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
>>>> nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
>>>>
>>>> Another work around is to disable nova security groups by adding
>>>> 'LIBVIRT_FIREWALL_DRIVER=nova.virt.firewall.NoopFirewallDriver' to
>>>> your localrc
>>>>
>>>> Dan
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>> Dan Wendlandt
>>>> Nicira, Inc: www.nicira.com
>>>> twitter: danwendlandt
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> --
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Dan Wendlandt
>> Nicira, Inc: www.nicira.com
>> twitter: danwendlandt
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~
More information about the OpenStack-dev
mailing list