[Openstack] Unable to launch hadoop cluster in Sahara
varun bhatnagar
varun292006 at gmail.com
Fri Oct 16 12:36:12 UTC 2015
Hi,
Now I have tried launching one more cluster with having 1 instance (All in
one setup). When the cluster was in "Waiting" state but I saw the below
messages getting logged in sahara log file:
2015-10-16 13:21:00.045 27689 DEBUG sahara.service.engine [-] Can't login
to node mymapr-allinone-001 172.24.4.235, reason error: [Errno 113] No
route to host _is_accessible
/usr/lib/python2.7/site-packages/sahara/service/engine.py:128
2015-10-16 13:21:08.702 27689 DEBUG sahara.service.engine [-] Can't login
to node mymapr-allinone-001 172.24.4.235, reason error: [Errno 113] No
route to host _is_accessible
/usr/lib/python2.7/site-packages/sahara/service/engine.py:128
2015-10-16 13:21:17.106 27689 DEBUG sahara.service.engine [-] Can't login
to node mymapr-allinone-001 172.24.4.235, reason error: [Errno 113] No
route to host _is_accessible
/usr/lib/python2.7/site-packages/sahara/service/engine.py:128
The instance was running and was reachable by floating IP and ssh also
works fine then why is this message being thrown by Sahara?
[root at controller ~(keystone_admin)]# nova list
+--------------------------------------+---------------------+--------+------------+-------------+----------------------------------+
| ID | Name | Status |
Task State | Power State | Networks |
+--------------------------------------+---------------------+--------+------------+-------------+----------------------------------+
| ceb87326-1f84-476b-86fa-fc004a9c5744 | mymapr-allinone-001 | ACTIVE | -
| Running | internal=11.0.0.12, 172.24.4.235 |
+--------------------------------------+---------------------+--------+------------+-------------+----------------------------------+
[root at controller ~(keystone_admin)]# ping 172.24.4.235
PING 172.24.4.235 (172.24.4.235) 56(84) bytes of data.
64 bytes from 172.24.4.235: icmp_seq=1 ttl=63 time=10.6 ms
64 bytes from 172.24.4.235: icmp_seq=2 ttl=63 time=1.85 ms
64 bytes from 172.24.4.235: icmp_seq=3 ttl=63 time=1.02 ms
^C
--- 172.24.4.235 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.028/4.520/10.680/4.369 ms
[root at controller ~(keystone_admin)]# ssh -i Downloads/testkey.pem
ubuntu at 172.24.4.235
The authenticity of host '172.24.4.235 (172.24.4.235)' can't be established.
ECDSA key fingerprint is a3:cc:5c:4e:fb:c8:83:80:46:54:77:31:e7:60:c5:c2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.24.4.235' (ECDSA) to the list of known
hosts.
Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-44-generic x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Fri Oct 16 11:41:12 UTC 2015
System load: 0.77 Processes: 124
Usage of /: 4.7% of 98.30GB Users logged in: 0
Memory usage: 1% IP address for eth0: 11.0.0.12
Swap usage: 0%
Graph this data and manage this system at:
https://landscape.canonical.com/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
7 packages can be updated.
7 updates are security updates.
ubuntu at mymapr-allinone-001:~$ ifconfig
eth0 Link encap:Ethernet HWaddr fa:16:3e:46:f5:5d
inet addr:11.0.0.12 Bcast:11.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe46:f55d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:954 errors:0 dropped:0 overruns:0 frame:0
TX packets:944 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:109259 (109.2 KB) TX bytes:134857 (134.8 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ubuntu at mymapr-allinone-001:~$
ubuntu at mymapr-allinone-001:~$
After sometime the cluster entered into "Configuring" state and after some
time ran into Error and displayed the below message:
2015-10-16 14:19:10.603 27689 ERROR sahara.service.ops [-] Error during
operating on cluster myMapR (reason: SSHException: Error reading SSH
protocol banner)
2015-10-16 14:19:10.994 27689 INFO sahara.utils.general [-] Cluster status
has been changed: id=e15529e9-be16-40a2-ba87-7132713f7460, New status=Error
Can anyone please suggest some way to configure this cluster?
BR,
Varun
On Fri, Oct 16, 2015 at 9:26 AM, varun bhatnagar <varun292006 at gmail.com>
wrote:
> Hi Chris,
>
> Thanks a lot for your answer.
> I still have couple of questions.
>
> You mentioned something about configuring keys -- could you please tell me
> where & how should I do that?
> Also, you mentioned that node couldn't access the outside world -- Is this
> a mandatory thing that the nodes should contact the outside world. I have
> assigned floating IPs but these are just internal and this won't be able to
> access outside world as this is my test lab which doesn't have this outside
> connectivity. The traffic doesn't go out of the OpenStack environment.
>
> I am using neutron as my network component.
>
>
> BR,
> Varun
>
>
> On Fri, Oct 16, 2015 at 1:10 AM, Chris Buccella <
> chris.buccella at antallagon.com> wrote:
>
>> I haven't tried the mapr plugin, but here are some thoughts:
>>
>> Sahara's error reporting is pretty bad... in my experience, the plugin
>> rarely logs the true cause of an error; you'll need to dig for it.
>> Configure your cluster to use a key so you can login to the nodes. You can
>> then login to the controller node and look in log files there. For a time
>> out error, it could be that a node couldn't access another node, or
>> couldn't access the outside world. In that case, it might be as simple as
>> ensuring your security groups are permissive enough.
>>
>> On Wed, Oct 14, 2015 at 10:49 AM, varun bhatnagar <varun292006 at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I am having OpenStack Kilo single node setup and I am trying to launch
>>> Hadoop cluster using MapR plugin in Sahara.
>>>
>>> I am following the example mentioned in the below site:
>>>
>>>
>>> https://www.mapr.com/blog/tutorial-how-set-mapr-private-cloud-using-sahara-devstack#.Vg5ntfmqqko
>>>
>>> It starts off well but after certain time it reaches a point where the
>>> cluster status gets changed to "Configuring" and then after some time it
>>> runs into error due to timeout. Can anyone please help in successfully
>>> launching the cluster?
>>>
>>>
>>>
>>> *2015-10-14 16:28:37.869 30280 DEBUG sahara.context [-] Thread
>>> configure-sh-c1050100-1775-4654-93a8-05c2ba350864 failed with exception:
>>> 'Operation' timed out after 600 second(s)*
>>> *Error ID: e2f61330-2c84-482e-97d0-d3947a5e5f02 _wrapper
>>> /usr/lib/python2.7/site-packages/sahara/context.py:193*
>>> *2015-10-14 16:28:37.923 30280 ERROR sahara.service.ops [-] Error during
>>> operating on cluster LaunchCluster (reason: An error occurred in thread
>>> 'configure-sh-c1050100-1775-4654-93a8-05c2ba350864': 'Operation' timed out
>>> after 600 second(s)*
>>> *Error ID: e2f61330-2c84-482e-97d0-d3947a5e5f02*
>>> *Error ID: f533bba0-afa8-4c99-bb73-a13f8c47d4a9)*
>>> *2015-10-14 16:28:38.572 30280 INFO sahara.utils.general [-] Cluster
>>> status has been changed: id=2ee0f93f-195b-4e53-a6be-241bdeff5958, New
>>> status=Error*
>>> *2015-10-14 16:28:39.576 30280 DEBUG keystonemiddleware.auth_token [-]
>>> Removing headers from request environment:
>>> X-Service-Catalog,X-Identity-Status,X-Service-Identity-Status,X-Roles,X-Service-Roles,X-Domain-Name,X-Service-Domain-Name,X-Project-Id,X-Service-Project-Id,X-Project-Domain-Name,X-Service-Project-Domain-Name,X-User-Id,X-Service-User-Id,X-User-Name,X-Service-User-Name,X-Project-Name,X-Service-Project-Name,X-User-Domain-Id,X-Service-User-Domain-Id,X-Domain-Id,X-Service-Domain-Id,X-User-Domain-Name,X-Service-User-Domain-Name,X-Project-Domain-Id,X-Service-Project-Domain-Id,X-Role,X-User,X-Tenant-Name,X-Tenant-Id,X-Tenant
>>> _remove_auth_headers
>>> /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py:672*
>>>
>>> BR,
>>> Varun
>>>
>>> _______________________________________________
>>> Mailing list:
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>> Post to : openstack at lists.openstack.org
>>> Unsubscribe :
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20151016/c8e31403/attachment.html>
More information about the Openstack
mailing list