<div dir="ltr"><div><div><div><div>Thanks for all the hints. Finally I have a working network.<br><br></div>The biggest problem is neutron.agent.metadata.agent has bad auth param set in metadata_agent.ini config file.<br></div>
The metadata agent log file did not catch my attention earlier.<br><br></div>Another problem is net, router, vm must be all in the same tenant. If not, create vm works, but delete vm and other action may fail. The related bug report is here: <a href="https://bugs.launchpad.net/nova/+bug/1246258">https://bugs.launchpad.net/nova/+bug/1246258</a><br>
<br>--<br></div>Yuanle<br><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Nov 18, 2013 at 12:51 PM, sylecn <span dir="ltr"><<a href="mailto:sylecn@gmail.com" target="_blank">sylecn@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>Hi Remo,<br><br></div><div>That is a good point.<br>I am using the CirrOS 0.3.1 image to boot a vm. I notice in "nova console-log" output, ifconfig eth0 did not get the fixed ip.<br>
<br></div>
<div>Does that mean dhcp-agent is not working correctly? I see no relavant error log in dhcp-agent.<br><br></div><div>I have google this error log in dhcp-agent is harmless:<br>2013-11-18 10:13:35.379 7848 ERROR neutron.common.legacy [-] Skipping unknown group key: firewall_driver<br>
<br></div><div>How can I make sure dhcp agent is working?<br></div><div><br></div><div>Here is part of the console log:<br>=====================<br>info: initramfs: up at 1.04<br>GROWROOT: CHANGED: partition=1 start=16065 old: size=64260 end=80325 new: size=2072385,end=2088450<br>
info: initramfs loading root from /dev/vda1<br>info: /etc/init.d/rc.sysinit: up at 1.39<br>Starting logging: OK<br>Initializing random number generator... done.<br>Starting acpid: OK<br>cirros-ds 'local' up at 1.69<br>
no results found for mode=local. up 1.81. searched: nocloud configdrive ec2<br>Starting network...<br>udhcpc (v1.20.1) started<br>Sending discover...<br>Sending discover...<br>Sending discover...<br>No lease, failing<br>
WARN: /etc/rc3.d/S40-network failed<br>
cirros-ds 'net' up at 182.16<br>checking <a href="http://169.254.169.254/2009-04-04/instance-id" target="_blank">http://169.254.169.254/2009-04-04/instance-id</a><br>failed 1/20: up 182.21. request failed<br>failed 2/20: up 184.36. request failed<br>
failed 3/20: up 186.38. request failed<br>failed 4/20: up 188.40. request failed<br>failed 5/20: up 190.42. request failed<br>failed 6/20: up 192.45. request failed<br>failed 7/20: up 194.47. request failed<br>failed 8/20: up 196.49. request failed<br>
failed 9/20: up 198.51. request failed<br>failed 10/20: up 200.53. request failed<br>failed 11/20: up 202.55. request failed<br>failed 12/20: up 204.57. request failed<br>failed 13/20: up 206.59. request failed<br>failed 14/20: up 208.61. request failed<br>
failed 15/20: up 210.63. request failed<br>failed 16/20: up 212.65. request failed<br>failed 17/20: up 214.67. request failed<br>failed 18/20: up 216.69. request failed<br>failed 19/20: up 218.71. request failed<br>failed 20/20: up 220.73. request failed<br>
failed to read iid from metadata. tried 20<br>no results found for mode=net. up 222.75. searched: nocloud configdrive ec2<br>failed to get instance-id of datasource<br>Starting dropbear sshd: generating rsa key... generating dsa key... OK<br>
=== network info ===<br>if-info: lo,up,127.0.0.1,8,::1<br>if-info: eth0,up,,8,fe80::f816:3eff:fe65:1633<br>=== datasource: None None ===<br>=== cirros: current=0.3.1 uptime=223.30 ===<br>route: fscanf<br>=== pinging gateway failed, debugging connection ===<br>
############ debug start ##############<br>### /etc/init.d/sshd start<br>Starting dropbear sshd: OK<br>route: fscanf<br>### ifconfig -a<br>eth0 Link encap:Ethernet HWaddr FA:16:3E:65:16:33 <br> inet6 addr: fe80::f816:3eff:fe65:1633/64 Scope:Link<br>
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br> RX packets:18 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:5 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:1000 <br>
RX bytes:2152 (2.1 KiB) TX bytes:902 (902.0 B)<div class="im"><br><br>lo Link encap:Local Loopback <br> inet addr:127.0.0.1 Mask:255.0.0.0<br> inet6 addr: ::1/128 Scope:Host<br></div>
UP LOOPBACK RUNNING MTU:16436 Metric:1<br>
RX packets:12 errors:0 dropped:0 overruns:0 frame:0<br> TX packets:12 errors:0 dropped:0 overruns:0 carrier:0<br> collisions:0 txqueuelen:0 <br> RX bytes:1020 (1020.0 B) TX bytes:1020 (1020.0 B)<br>
<br>### route -n<div class="im"><br>Kernel IP routing table<br>Destination Gateway Genmask Flags Metric Ref Use Iface<br></div>route: fscanf<br>### cat /etc/resolv.conf<br>cat: can't open '/etc/resolv.conf': No such file or directory<br>
### gateway not found<br>/sbin/cirros-status: line 1: can't open /etc/resolv.conf: no such file<br>### pinging nameservers<br>### uname -a<br>Linux cirros 3.2.0-37-virtual #58-Ubuntu SMP Thu Jan 24 15:48:03 UTC 2013 x86_64 GNU/Linux<br>
<br>
</div></div><div><br><br></div></div><div class=""><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Nov 18, 2013 at 9:53 AM, Remo Mattei <span dir="ltr"><<a href="mailto:remo@mattei.org" target="_blank">remo@mattei.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div>You could chk if your vm has the ip do not chk from the dashboard because you may see the ip but the vm may not have it. </div>
<div><br></div><div>Let us know<br><br><div>Inviato da iPhone (<span style="background-color:rgba(255,255,255,0)">)</span></div></div><div><br>Il giorno Nov 17, 2013, alle ore 17:24, sylecn <<a href="mailto:sylecn@gmail.com" target="_blank">sylecn@gmail.com</a>> ha scritto:<br>
<br></div><blockquote type="cite"><div><div><div><div dir="ltr"><div>I have enabled namespace, but I did not use overlapping IP addresses so far.<br><br></div>Here is the result of the netns command:<br><br>root@172-17-6-68:/var/log/neutron# ip netns show<br>
qrouter-c5b513fa-6d6a-476f-bfc0-2114954a15aa<br>
qdhcp-a63f0950-cdea-4a6d-8312-1819113dc244<br>root@172-17-6-68:/var/log/neutron# ip netns exec qdhcp-a63f0950-cdea-4a6d-8312-1819113dc244 ping 10.0.1.3<br>PING 10.0.1.3 (10.0.1.3) 56(84) bytes of data.<br>From 10.0.1.2 icmp_seq=1 Destination Host Unreachable<br>
>From 10.0.1.2 icmp_seq=2 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=3 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=4 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=5 Destination Host Unreachable<br>
>From 10.0.1.2 icmp_seq=6 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=7 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=8 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=9 Destination Host Unreachable<br>
>From 10.0.1.2 icmp_seq=10 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=11 Destination Host Unreachable<br>From 10.0.1.2 icmp_seq=12 Destination Host Unreachable<br>^C<br>--- 10.0.1.3 ping statistics ---<br>13 packets transmitted, 0 received, +12 errors, 100% packet loss, time 12061ms<br>
pipe 3<br>root@172-17-6-68:/var/log/neutron# <br><br>Using ip netns exec qdhcp-* ping, I can ping 10.0.1.1 and 10.0.1.2. However, ping 10.0.1.3 still fail.<br><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Mon, Nov 18, 2013 at 12:59 AM, Kyle Mestery (kmestery) <span dir="ltr"><<a href="mailto:kmestery@cisco.com" target="_blank">kmestery@cisco.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>On Nov 17, 2013, at 2:45 AM, sylecn <<a href="mailto:sylecn@gmail.com" target="_blank">sylecn@gmail.com</a>> wrote:<br>
><br>
> Thanks for the information. Now I have configured a provider router based network, with Open vSwitch GRE tunnels.<br>
><br>
> Here is the network topology:<br>
><br>
> external network: <a href="http://172.17.0.0/16" target="_blank">172.17.0.0/16</a><br>
> external network physical router: 172.17.0.1<br>
> neutron node IP: 172.17.6.68<br>
><br>
> virtual provider router: 172.17.6.70<br>
><br>
> virtual subnet1: <a href="http://10.0.1.0/24" target="_blank">10.0.1.0/24</a><br>
><br>
> Now I can boot a vm instance and it got an IP from private IP pool (10.0.1.3). I can also associate a floating IP to it (172.17.6.71). But I can't ping the private ip nor the floating ip.<br>
><br>
> From the neutron node, I can ping 172.17.6.70, but can't ping 10.0.1.1, 10.0.1.3, 172.17.6.71.<br>
> So I can't ssh into the vm. My guess is something is wrong with the <a href="http://10.0.1.0/24" target="_blank">10.0.1.0/24</a> network, but I don't know what.<br>
><br>
</div>Are you setup to use network namespaces with overlapping IP addresses? If so, each tenant network will have it's own network namespace on the node running the Neutron L3 agent. To see these, run this command:<br>
<br>
ip netns show<br>
<br>
>From the qdhcp-* one, you can try to ping your tenant network address:<br>
<br>
ip netns exec qdhcp-* ping 10.0.1.3<br>
<br>
Let me know if that helps.<br>
<br>
Thanks,<br>
Kyle<br>
<div><div><br>
> I used the NoopFirewallDriver in OVS plugin, so icmp and tcp:22 are not blocked by security-group rules.<br>
><br>
> Here is the current setup:<br>
> (neutron) net-list<br>
> +--------------------------------------+--------+----------------------------------------------------+<br>
> | id | name | subnets |<br>
> +--------------------------------------+--------+----------------------------------------------------+<br>
> | a63f0950-cdea-4a6d-8312-1819113dc244 | net1 | 708f2a58-bd85-4493-b91c-a6d42c0db5e7 <a href="http://10.0.1.0/24" target="_blank">10.0.1.0/24</a> |<br>
> | ee318d0b-74e5-43c6-92bd-abb690df3334 | extnet | 4c111c62-50f2-4332-b635-57846cf1980c <a href="http://172.17.0.0/16" target="_blank">172.17.0.0/16</a> |<br>
> +--------------------------------------+--------+----------------------------------------------------+<br>
> (neutron) subnet-list<br>
> +--------------------------------------+---------+---------------+------------------------------------------------+<br>
> | id | name | cidr | allocation_pools |<br>
> +--------------------------------------+---------+---------------+------------------------------------------------+<br>
> | 4c111c62-50f2-4332-b635-57846cf1980c | extnet | <a href="http://172.17.0.0/16" target="_blank">172.17.0.0/16</a> | {"start": "172.17.6.70", "end": "172.17.6.75"} |<br>
> | 708f2a58-bd85-4493-b91c-a6d42c0db5e7 | subnet1 | <a href="http://10.0.1.0/24" target="_blank">10.0.1.0/24</a> | {"start": "10.0.1.2", "end": "10.0.1.254"} |<br>
> +--------------------------------------+---------+---------------+------------------------------------------------+<br>
> (neutron) port-list<br>
> +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+<br>
> | id | name | mac_address | fixed_ips |<br>
> +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+<br>
> | 234b4e76-7b7a-461f-8b61-2b7c58165fd2 | | fa:16:3e:86:95:d3 | {"subnet_id": "708f2a58-bd85-4493-b91c-a6d42c0db5e7", "ip_address": "10.0.1.1"} |<br>
> | 35a8ab42-4f1a-4f1e-b656-ab4dd0e83725 | | fa:16:3e:44:c2:0a | {"subnet_id": "708f2a58-bd85-4493-b91c-a6d42c0db5e7", "ip_address": "10.0.1.2"} |<br>
> | 85f4d2d7-c92b-4bc1-b080-2b1978bb6e17 | | fa:16:3e:cd:77:17 | {"subnet_id": "708f2a58-bd85-4493-b91c-a6d42c0db5e7", "ip_address": "10.0.1.3"} |<br>
> | 9a24c2e9-a6da-4a24-93d4-9eef8cb0bcfa | | fa:16:3e:01:a2:ef | {"subnet_id": "4c111c62-50f2-4332-b635-57846cf1980c", "ip_address": "172.17.6.70"} |<br>
> | f508b629-6e95-4be4-89c0-b37be3907231 | | fa:16:3e:7c:41:0a | {"subnet_id": "4c111c62-50f2-4332-b635-57846cf1980c", "ip_address": "172.17.6.71"} |<br>
> +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+<br>
> (neutron) port-show 234b4e76-7b7a-461f-8b61-2b7c58165fd2<br>
> +-----------------------+---------------------------------------------------------------------------------+<br>
> | Field | Value |<br>
> +-----------------------+---------------------------------------------------------------------------------+<br>
> | admin_state_up | True |<br>
> | allowed_address_pairs | |<br>
> | binding:capabilities | {"port_filter": false} |<br>
> | binding:host_id | <a href="http://172-17-6-68.yygamedev.com" target="_blank">172-17-6-68.yygamedev.com</a> |<br>
> | binding:vif_type | ovs |<br>
> | device_id | c5b513fa-6d6a-476f-bfc0-2114954a15aa |<br>
> | device_owner | network:router_interface |<br>
> | extra_dhcp_opts | |<br>
> | fixed_ips | {"subnet_id": "708f2a58-bd85-4493-b91c-a6d42c0db5e7", "ip_address": "10.0.1.1"} |<br>
> | id | 234b4e76-7b7a-461f-8b61-2b7c58165fd2 |<br>
> | mac_address | fa:16:3e:86:95:d3 |<br>
> | name | |<br>
> | network_id | a63f0950-cdea-4a6d-8312-1819113dc244 |<br>
> | status | ACTIVE |<br>
> | tenant_id | 860483f3ceeb43aab4d1f0e8f76b4064 |<br>
> +-----------------------+---------------------------------------------------------------------------------+<br>
> (neutron)<br>
> root@172-17-6-68:/etc/neutron# nova list<br>
> +--------------------------------------+------+--------+------------+-------------+----------------------------+<br>
> | ID | Name | Status | Task State | Power State | Networks |<br>
> +--------------------------------------+------+--------+------------+-------------+----------------------------+<br>
> | ec214f0b-eede-421e-9036-a1b56bff3c37 | c1 | ACTIVE | None | Running | net1=10.0.1.3, 172.17.6.71 |<br>
> +--------------------------------------+------+--------+------------+-------------+----------------------------+<br>
><br>
><br>
<br>
<br>
</div></div></blockquote></div><br></div></div></div>
!DSPAM:2,528971b5319253744511446!
</div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a></span><br>
<span>Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a></span><br><span>Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a></span><br>
<span></span><br><span></span><br><span>!DSPAM:2,528971b5319253744511446!</span><br></div></blockquote></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div>