[Openstack] 答复: 答复: instance cannot access outside network
David
yuezhou.li at hisoft.com
Fri Jun 15 10:00:54 UTC 2012
Hi
I use FlatDHCP mode actually .
And br100 as flat bridge , flat interface is eth0
Also part of iptable rule on compute-node :
-A nova-compute-inst-2 -m state --state INVALID -j DROP
-A nova-compute-inst-2 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A nova-compute-inst-2 -j nova-compute-provider
-A nova-compute-inst-2 -s 192.168.4.1/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A nova-compute-inst-2 -s 192.168.4.0/27 -j ACCEPT
-A nova-compute-inst-2 -j nova-compute-sg-fallback
-A nova-compute-local -d 192.168.4.3/32 -j nova-compute-inst-2
-A nova-compute-sg-fallback -j DROP
-A nova-filter-top -j nova-compute-local
And the output of ~#cat /proc/sys/net/ipv4/ip_forward is 1
Output of ~# ip addr :
br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0c:29:cf:ec:d7 brd ff:ff:ff:ff:ff:ff
inet 192.168.7.153/27 brd 192.168.7.159 scope global br100
inet 192.168.4.1/27 scope global br100
inet6 fe80::20c:29ff:fecf:ecd7/64 scope link
valid_lft forever preferred_lft forever
So these looks all reasonable . I really don’t know why the vm cannot get ip L.
Thank you
Best Regards
David(李跃洲)
E-MAIL: yuezhou.li at hisoft.com
发件人: emilien.macchi at gmail.com [mailto:emilien.macchi at gmail.com] 代表 Emilien Macchi
发送时间: 2012年6月15日 17:38
收件人: David
抄送: openstack at lists.launchpad.net
主题: Re: 答复: [Openstack] instance cannot access outside network
If you use VLAN, you should to be sure that your physical network interface is configured as a a trunk interface on the Switch.
On the physical switch :
switchport mode trunk
switchport trunk encapsulation dot1q
To snif the network and see if you can see tagget packets, use :
tcpdump -nnei eth1 (if eth1 is your bridge)
Regards
On Fri, Jun 15, 2012 at 11:14 AM, David <yuezhou.li at hisoft.com> wrote:
Hi
I try to add flag –routing_source_ip = 192.168.7.151 which is my controller ip with nova-network .but it didn’t work .
I found the console.log output error : when Starting network...
udhcpc (v1.18.5) started
Sending discover...
Sending discover...
Sending discover...
No lease, failing
WARN: /etc/rc3.d/S40-network failed
I doubt that the vm on compute-node cannot find dhcp-server or vm cannot get the response from dhcp-server
So I use tcpdump try to trace the dhcp request . The log as follow :
First I restart network in vm . the vm will retry discovery dhcp-server to get ip .
The MAC of vm is : fe:16:3e:50:6d:1c , and nova give the ip : 192.168.4.3 to the vm ,I use (~#nova list) to find.
BUT the ip didn’t associate to vm . it just in db as a record .
On compute-controller :
On compute-node :
~# tcpdump -i br100 -n port 67 or 68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes
01:13:46.794501 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:50:6d:1c, length 280
01:13:49.799593 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:50:6d:1c, length 280
01:13:52.803964 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:50:6d:1c, length 280
~# tcpdump -i br100 -n port 67 or 68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes
01:13:47.995389 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:50:6d:1c, length 280
01:13:47.995785 IP 192.168.4.1.67 > 192.168.4.3.68: BOOTP/DHCP, Reply, length 309
01:13:51.000454 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:50:6d:1c, length 280
01:13:51.000911 IP 192.168.4.1.67 > 192.168.4.3.68: BOOTP/DHCP, Reply, length 309
01:13:54.004840 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from fa:16:3e:50:6d:1c, length 280
01:13:54.005196 IP 192.168.4.1.67 > 192.168.4.3.68: BOOTP/DHCP, Reply, length 309
Could anyone tell me why ?
David(李跃洲)
E-MAIL: yuezhou.li at hisoft.com
发件人: emilien.macchi at gmail.com [mailto:emilien.macchi at gmail.com] 代表 Emilien Macchi
发送时间: 2012年6月14日 19:03
收件人: David
抄送: <openstack at lists.launchpad.net>
主题: Re: [Openstack] instance cannot access outside network
Don't forget to CC the mail ;-) for the community.
I think you should try to add --routing_source_ip=<IP of nova-network> flag in your nova.conf.
Regards,
On Thu, Jun 14, 2012 at 12:51 PM, David <yuezhou.li at hisoft.com> wrote:
I try
#cat /proc/sys/net/ipv4/ip_forward
The result is 1
And the iptables have wrote by nova actually .
As follow :
# Generated by iptables-save v1.4.12 on Fri Jun 15 02:58:58 2012
*nat
:PREROUTING ACCEPT [774:124753]
:INPUT ACCEPT [534:94672]
:OUTPUT ACCEPT [858:54250]
:POSTROUTING ACCEPT [919:59061]
:nova-api-OUTPUT - [0:0]
:nova-api-POSTROUTING - [0:0]
:nova-api-PREROUTING - [0:0]
:nova-api-float-snat - [0:0]
:nova-api-snat - [0:0]
:nova-network-OUTPUT - [0:0]
:nova-network-POSTROUTING - [0:0]
:nova-network-PREROUTING - [0:0]
:nova-network-float-snat - [0:0]
:nova-network-snat - [0:0]
:nova-postrouting-bottom - [0:0]
-A PREROUTING -j nova-network-PREROUTING
-A PREROUTING -j nova-api-PREROUTING
-A OUTPUT -j nova-network-OUTPUT
-A OUTPUT -j nova-api-OUTPUT
-A POSTROUTING -j nova-network-POSTROUTING
-A POSTROUTING -j nova-api-POSTROUTING
-A POSTROUTING -j nova-postrouting-bottom
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
-A nova-api-snat -j nova-api-float-snat
-A nova-network-POSTROUTING -s 192.168.4.0/27 -d 192.168.7.151/32 -j ACCEPT
-A nova-network-POSTROUTING -s 192.168.4.0/27 -d 10.128.0.0/24 -j <http://10.128.0.0/24-j> ACCEPT
-A nova-network-POSTROUTING -s 192.168.4.0/27 -d 192.168.4.0/27 -m <http://192.168.4.0/27-m> conntrack ! --ctstate DNAT -j ACCEPT
-A nova-network-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.7.151:8775
-A nova-network-snat -j nova-network-float-snat
-A nova-network-snat -s 192.168.4.0/27 -j SNAT --to-source 192.168.7.151
-A nova-postrouting-bottom -j nova-network-snat
-A nova-postrouting-bottom -j nova-api-snat
COMMIT
# Completed on Fri Jun 15 02:58:58 2012
# Generated by iptables-save v1.4.12 on Fri Jun 15 02:58:58 2012
*mangle
:PREROUTING ACCEPT [224637:108582489]
:INPUT ACCEPT [221029:107788819]
:FORWARD ACCEPT [29116:11774224]
:OUTPUT ACCEPT [187094:188512394]
:POSTROUTING ACCEPT [216210:200286618]
-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Fri Jun 15 02:58:58 2012
# Generated by iptables-save v1.4.12 on Fri Jun 15 02:58:58 2012
*filter
:INPUT ACCEPT [91273:50201087]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [84624:51049560]
:nova-api-FORWARD - [0:0]
:nova-api-INPUT - [0:0]
:nova-api-OUTPUT - [0:0]
:nova-api-local - [0:0]
:nova-filter-top - [0:0]
:nova-network-FORWARD - [0:0]
:nova-network-INPUT - [0:0]
:nova-network-OUTPUT - [0:0]
:nova-network-local - [0:0]
-A INPUT -j nova-network-INPUT
-A INPUT -j nova-api-INPUT
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A FORWARD -j nova-filter-top
-A FORWARD -j nova-network-FORWARD
-A FORWARD -j nova-api-FORWARD
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -j nova-filter-top
-A OUTPUT -j nova-network-OUTPUT
-A OUTPUT -j nova-api-OUTPUT
-A nova-api-INPUT -d 192.168.7.151/32 -p tcp -m tcp --dport 8775 -j ACCEPT
-A nova-filter-top -j nova-network-local
-A nova-filter-top -j nova-api-local
-A nova-network-FORWARD -i br100 -j ACCEPT
-A nova-network-FORWARD -o br100 -j ACCEPT
COMMIT
# Completed on Fri Jun 15 02:58:58 2012
Best Regards
David(李跃洲)
E-MAIL: yuezhou.li at hisoft.com
发件人: emilien.macchi at gmail.com [mailto:emilien.macchi at gmail.com] 代表 Emilien Macchi
发送时间: 2012年6月14日 16:51
收件人: David
抄送: openstack at lists.launchpad.net
主题: Re: [Openstack] instance cannot access outside network
Hi,
Can you try :
echo 1 > /proc/sys/net/ipv4/ip_forward on the nova-network node.
And also modify /etc/sysctl.conf to uncomment the ip_forward parameter :
net.ipv4.ip_forward = 1
Is it ok now ?
Regards
On Thu, Jun 14, 2012 at 10:24 AM, David <yuezhou.li at hisoft.com> wrote:
Hi All
I try to install openstack on multi-node .
I can boot instance and use vnc console to visit the instance .
But , I cannot access outside in instance .
I found the fixed ip range didn’t put on the br100 at compute-node only on compute-controller. The compute-node only install nova-compute
On compute-node , ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br100 state UP qlen 1000
link/ether 00:0c:29:cf:ec:d7 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:cf:ec:e1 brd ff:ff:ff:ff:ff:ff
inet 192.168.7.153/27 brd 192.168.7.159 scope global eth1
inet6 fe80::20c:29ff:fecf:ece1/64 scope link
valid_lft forever preferred_lft forever
4: br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0c:29:cf:ec:d7 brd ff:ff:ff:ff:ff:ff
inet6 fe80::20c:29ff:fecf:ecd7/64 scope link
valid_lft forever preferred_lft forever
on compute-controller :
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 169.254.169.254/32 scope link lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br100 state UP qlen 1000
link/ether 00:0c:29:88:31:51 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:88:31:5b brd ff:ff:ff:ff:ff:ff
inet 192.168.7.151/27 brd 192.168.7.159 scope global eth1
inet6 fe80::20c:29ff:fe88:315b/64 scope link
valid_lft forever preferred_lft forever
4: br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0c:29:88:31:51 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.3/27 brd 10.0.0.31 scope global br100
inet 192.168.4.33/27 brd 192.168.4.63 scope global br100
inet6 fe80::20c:29ff:fe88:3151/64 scope link
valid_lft forever preferred_lft forever
could Any one tell me why ? Or tell me how openstack set network bridge on compute-node so that I can figure out what happened .
My nova.conf :
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/run/lock/nova
--allow_admin_api=true
--use_deprecated_auth=false
--auth_strategy=keystone
--scheduler_driver=nova.scheduler.simple.SimpleScheduler
--s3_host=192.168.7.151
--ec2_host=192.168.7.151
--rabbit_host=192.168.7.151
--cc_host=192.168.7.151
--nova_url=http://192.168.7.151:8774/v1.1/
--routing_source_ip=192.168.7.151
--glance_api_servers=192.168.7.151:9292
--image_service=nova.image.glance.GlanceImageService
--iscsi_ip_prefix=192.168.4
--sql_connection=mysql://root:hisoft@192.168.7.151/nova
--ec2_url=http://192.168.7.151:8773/services/Cloud
--keystone_ec2_url=http://192.168.7.151:5000/v2.0/ec2tokens
--api_paste_config=/etc/nova/api-paste.ini
--libvirt_type=qemu
--libvirt_use_virtio_for_bridges=true
--start_guests_on_host_boot=true
--resume_guests_state_on_host_boot=true
# vnc specific configuration
--novnc_enabled=true
--novncproxy_base_url=http://192.168.7.151:6080/vnc_auto.html
--vncserver_proxyclient_address=192.168.7.151
--vncserver_listen=192.168.7.151
# network specific settings
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=eth1
--flat_network_bridge=br100
--fixed_range=192.168.4.1/27
--floating_range=192.168.7.208/28
--network_size=32
--flat_network_dhcp_start=192.168.4.33
--flat_injected=False
--force_dhcp_release
--iscsi_helper=tgtadm
--connection_type=libvirt
--root_helper=sudo nova-rootwrap
--verbose
And nova version :
#nova-manage version
2012.1 (2012.1-LOCALBRANCH:LOCALREVISION)
Thank you in advance .
David(李跃洲)
E-MAIL: yuezhou.li at hisoft.com
_______________________________________________
Mailing list: https://launchpad.net/~openstack <https://launchpad.net/%7Eopenstack>
Post to : openstack at lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack <https://launchpad.net/%7Eopenstack>
More help : https://help.launchpad.net/ListHelp
--
Emilien Macchi
SysAdmin (Intern)
www.stackops.com <http://www.stackops.com/> | emilien.macchi at stackops.com
错误！未指定文件名。
******************** ADVERTENCIA LEGAL ********************
Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley.
***************** PRIVILEGED AND CONFIDENTIAL ****************
We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law.
--
Emilien Macchi
SysAdmin (Intern)
www.stackops.com <http://www.stackops.com/> | emilien.macchi at stackops.com
错误！未指定文件名。
******************** ADVERTENCIA LEGAL ********************
Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley.
***************** PRIVILEGED AND CONFIDENTIAL ****************
We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law.
--
Emilien Macchi
SysAdmin (Intern)
www.stackops.com <http://www.stackops.com/> | emilien.macchi at stackops.com
<http://stackops.s3-external-3.amazonaws.com/STACKOPSLOGO-ICON.png>
******************** ADVERTENCIA LEGAL ********************
Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley.
***************** PRIVILEGED AND CONFIDENTIAL ****************
We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120615/a08361be/attachment.html>
More information about the Openstack
mailing list
