[Openstack] Unable to launch hadoop cluster in Sahara
varun bhatnagar
varun292006 at gmail.com
Sun Oct 18 11:22:09 UTC 2015
Hi Chris,
Thanks for the suggestion. I have posted my question on his blog but it is
not yet approved and is still in pending state.
Is it possible to provide any other suggestion to solve this problem. I am
really out of ideas at the moment and I reallt want to fix this problem as
my actual work is on hold.
Can anyone please suggest something else?
BR,
Varun
On Fri, Oct 16, 2015 at 7:07 PM, Chris Buccella <
chris.buccella at antallagon.com> wrote:
> Those "No route to host" log messages are DEBUG level; they don't
> indicate a problem per say, it may just be that the instance didn't come up
> yet. If the cluster transitioned from Waiting to Configuring, I assume the
> instance did become accessible.
>
> You might try reaching out to Abizer, the author of the blog, directly.
>
> On Fri, Oct 16, 2015 at 8:36 AM, varun bhatnagar <varun292006 at gmail.com>
> wrote:
>
>> Hi,
>>
>> Now I have tried launching one more cluster with having 1 instance (All
>> in one setup). When the cluster was in "Waiting" state but I saw the below
>> messages getting logged in sahara log file:
>>
>> 2015-10-16 13:21:00.045 27689 DEBUG sahara.service.engine [-] Can't login
>> to node mymapr-allinone-001 172.24.4.235, reason error: [Errno 113] No
>> route to host _is_accessible
>> /usr/lib/python2.7/site-packages/sahara/service/engine.py:128
>> 2015-10-16 13:21:08.702 27689 DEBUG sahara.service.engine [-] Can't login
>> to node mymapr-allinone-001 172.24.4.235, reason error: [Errno 113] No
>> route to host _is_accessible
>> /usr/lib/python2.7/site-packages/sahara/service/engine.py:128
>> 2015-10-16 13:21:17.106 27689 DEBUG sahara.service.engine [-] Can't login
>> to node mymapr-allinone-001 172.24.4.235, reason error: [Errno 113] No
>> route to host _is_accessible
>> /usr/lib/python2.7/site-packages/sahara/service/engine.py:128
>>
>> The instance was running and was reachable by floating IP and ssh also
>> works fine then why is this message being thrown by Sahara?
>>
>>
>> [root at controller ~(keystone_admin)]# nova list
>>
>> +--------------------------------------+---------------------+--------+------------+-------------+----------------------------------+
>> | ID | Name | Status |
>> Task State | Power State | Networks |
>>
>> +--------------------------------------+---------------------+--------+------------+-------------+----------------------------------+
>> | ceb87326-1f84-476b-86fa-fc004a9c5744 | mymapr-allinone-001 | ACTIVE | -
>> | Running | internal=11.0.0.12, 172.24.4.235 |
>>
>> +--------------------------------------+---------------------+--------+------------+-------------+----------------------------------+
>> [root at controller ~(keystone_admin)]# ping 172.24.4.235
>> PING 172.24.4.235 (172.24.4.235) 56(84) bytes of data.
>> 64 bytes from 172.24.4.235: icmp_seq=1 ttl=63 time=10.6 ms
>> 64 bytes from 172.24.4.235: icmp_seq=2 ttl=63 time=1.85 ms
>> 64 bytes from 172.24.4.235: icmp_seq=3 ttl=63 time=1.02 ms
>> ^C
>> --- 172.24.4.235 ping statistics ---
>> 3 packets transmitted, 3 received, 0% packet loss, time 2004ms
>> rtt min/avg/max/mdev = 1.028/4.520/10.680/4.369 ms
>> [root at controller ~(keystone_admin)]# ssh -i Downloads/testkey.pem
>> ubuntu at 172.24.4.235
>> The authenticity of host '172.24.4.235 (172.24.4.235)' can't be
>> established.
>> ECDSA key fingerprint is a3:cc:5c:4e:fb:c8:83:80:46:54:77:31:e7:60:c5:c2.
>> Are you sure you want to continue connecting (yes/no)? yes
>> Warning: Permanently added '172.24.4.235' (ECDSA) to the list of known
>> hosts.
>> Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-44-generic x86_64)
>>
>> * Documentation: https://help.ubuntu.com/
>>
>> System information as of Fri Oct 16 11:41:12 UTC 2015
>>
>> System load: 0.77 Processes: 124
>> Usage of /: 4.7% of 98.30GB Users logged in: 0
>> Memory usage: 1% IP address for eth0: 11.0.0.12
>> Swap usage: 0%
>>
>> Graph this data and manage this system at:
>> https://landscape.canonical.com/
>>
>> Get cloud support with Ubuntu Advantage Cloud Guest:
>> http://www.ubuntu.com/business/services/cloud
>>
>> 7 packages can be updated.
>> 7 updates are security updates.
>>
>>
>> ubuntu at mymapr-allinone-001:~$ ifconfig
>> eth0 Link encap:Ethernet HWaddr fa:16:3e:46:f5:5d
>> inet addr:11.0.0.12 Bcast:11.0.0.255 Mask:255.255.255.0
>> inet6 addr: fe80::f816:3eff:fe46:f55d/64 Scope:Link
>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> RX packets:954 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:944 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:1000
>> RX bytes:109259 (109.2 KB) TX bytes:134857 (134.8 KB)
>>
>> lo Link encap:Local Loopback
>> inet addr:127.0.0.1 Mask:255.0.0.0
>> inet6 addr: ::1/128 Scope:Host
>> UP LOOPBACK RUNNING MTU:65536 Metric:1
>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:0
>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>>
>> ubuntu at mymapr-allinone-001:~$
>> ubuntu at mymapr-allinone-001:~$
>>
>>
>> After sometime the cluster entered into "Configuring" state and after
>> some time ran into Error and displayed the below message:
>>
>> 2015-10-16 14:19:10.603 27689 ERROR sahara.service.ops [-] Error during
>> operating on cluster myMapR (reason: SSHException: Error reading SSH
>> protocol banner)
>> 2015-10-16 14:19:10.994 27689 INFO sahara.utils.general [-] Cluster
>> status has been changed: id=e15529e9-be16-40a2-ba87-7132713f7460, New
>> status=Error
>>
>>
>> Can anyone please suggest some way to configure this cluster?
>>
>> BR,
>> Varun
>>
>>
>> On Fri, Oct 16, 2015 at 9:26 AM, varun bhatnagar <varun292006 at gmail.com>
>> wrote:
>>
>>> Hi Chris,
>>>
>>> Thanks a lot for your answer.
>>> I still have couple of questions.
>>>
>>> You mentioned something about configuring keys -- could you please tell
>>> me where & how should I do that?
>>> Also, you mentioned that node couldn't access the outside world -- Is
>>> this a mandatory thing that the nodes should contact the outside world. I
>>> have assigned floating IPs but these are just internal and this won't be
>>> able to access outside world as this is my test lab which doesn't have this
>>> outside connectivity. The traffic doesn't go out of the OpenStack
>>> environment.
>>>
>>> I am using neutron as my network component.
>>>
>>>
>>> BR,
>>> Varun
>>>
>>>
>>> On Fri, Oct 16, 2015 at 1:10 AM, Chris Buccella <
>>> chris.buccella at antallagon.com> wrote:
>>>
>>>> I haven't tried the mapr plugin, but here are some thoughts:
>>>>
>>>> Sahara's error reporting is pretty bad... in my experience, the plugin
>>>> rarely logs the true cause of an error; you'll need to dig for it.
>>>> Configure your cluster to use a key so you can login to the nodes. You can
>>>> then login to the controller node and look in log files there. For a time
>>>> out error, it could be that a node couldn't access another node, or
>>>> couldn't access the outside world. In that case, it might be as simple as
>>>> ensuring your security groups are permissive enough.
>>>>
>>>> On Wed, Oct 14, 2015 at 10:49 AM, varun bhatnagar <
>>>> varun292006 at gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am having OpenStack Kilo single node setup and I am trying to launch
>>>>> Hadoop cluster using MapR plugin in Sahara.
>>>>>
>>>>> I am following the example mentioned in the below site:
>>>>>
>>>>>
>>>>> https://www.mapr.com/blog/tutorial-how-set-mapr-private-cloud-using-sahara-devstack#.Vg5ntfmqqko
>>>>>
>>>>> It starts off well but after certain time it reaches a point where the
>>>>> cluster status gets changed to "Configuring" and then after some time it
>>>>> runs into error due to timeout. Can anyone please help in successfully
>>>>> launching the cluster?
>>>>>
>>>>>
>>>>>
>>>>> *2015-10-14 16:28:37.869 30280 DEBUG sahara.context [-] Thread
>>>>> configure-sh-c1050100-1775-4654-93a8-05c2ba350864 failed with exception:
>>>>> 'Operation' timed out after 600 second(s)*
>>>>> *Error ID: e2f61330-2c84-482e-97d0-d3947a5e5f02 _wrapper
>>>>> /usr/lib/python2.7/site-packages/sahara/context.py:193*
>>>>> *2015-10-14 16:28:37.923 30280 ERROR sahara.service.ops [-] Error
>>>>> during operating on cluster LaunchCluster (reason: An error occurred in
>>>>> thread 'configure-sh-c1050100-1775-4654-93a8-05c2ba350864': 'Operation'
>>>>> timed out after 600 second(s)*
>>>>> *Error ID: e2f61330-2c84-482e-97d0-d3947a5e5f02*
>>>>> *Error ID: f533bba0-afa8-4c99-bb73-a13f8c47d4a9)*
>>>>> *2015-10-14 16:28:38.572 30280 INFO sahara.utils.general [-] Cluster
>>>>> status has been changed: id=2ee0f93f-195b-4e53-a6be-241bdeff5958, New
>>>>> status=Error*
>>>>> *2015-10-14 16:28:39.576 30280 DEBUG keystonemiddleware.auth_token [-]
>>>>> Removing headers from request environment:
>>>>> X-Service-Catalog,X-Identity-Status,X-Service-Identity-Status,X-Roles,X-Service-Roles,X-Domain-Name,X-Service-Domain-Name,X-Project-Id,X-Service-Project-Id,X-Project-Domain-Name,X-Service-Project-Domain-Name,X-User-Id,X-Service-User-Id,X-User-Name,X-Service-User-Name,X-Project-Name,X-Service-Project-Name,X-User-Domain-Id,X-Service-User-Domain-Id,X-Domain-Id,X-Service-Domain-Id,X-User-Domain-Name,X-Service-User-Domain-Name,X-Project-Domain-Id,X-Service-Project-Domain-Id,X-Role,X-User,X-Tenant-Name,X-Tenant-Id,X-Tenant
>>>>> _remove_auth_headers
>>>>> /usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py:672*
>>>>>
>>>>> BR,
>>>>> Varun
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list:
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>> Post to : openstack at lists.openstack.org
>>>>> Unsubscribe :
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20151018/a80ea820/attachment.html>
More information about the Openstack
mailing list