[Openstack] Controller deployment fails - any advice é help appreciated
Chris Suttles
suttles at gmail.com
Fri Feb 24 03:12:13 UTC 2017
To me, that looks like your network configuration has a problem and fuel/nailgun is bubbling up the error when trying to provision neutron on the node.
Since it is a fresh install, I would consider starting over, and validating network config in fuel before restarting your deployment.
I don't have a ton of experience with fuel, but that's my best guess based on my limited exposure to it and what I see in your logs/traceback. That highlighted line is not a neutron call, that's within fuel, presumably when it is trying to provision neutron.
-Chris
> On Feb 23, 2017, at 6:33 PM, Mario Romascanu <romar at mindtrek.dnsalias.net> wrote:
>
> Hi
>
> I was able to get to a certain point in deploying an environment on a fresh installation of Mirantis OpenStack 9.0 but I am stuck with this bizarre error in the controller deployment
> It does not seem to be a communication issue - but rather a logical one - an error 500 on a particular REST call URI to neutron and 200 OK otherwise on different neutron URI.
> This occurs after deploying Ubuntu and Openstack on the Controller node
> I am at a loss of what the issue is here - any suggestion welcome
>
> "Deployment has failed. All nodes are finished. Failed tasks: Task[connectivity_tests/4] Stopping the deployment process!"
> "Failed to deploy node 'MOS_Controller (1e:c1)': Task connectivity_tests failed on node 4"
> [root at fuel ~]# fuel node
> id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
> ---+---------+------------------------+---------+-----------+-------------------+------------+---------------+--------+---------
> 3 | stopped | MOS_Compute (46:4a) | 1 | 10.20.0.4 | 00:50:56:98:46:4a | compute | | 1 | 1
> 4 | error | MOS_Controller (1e:c1) | 1 | 10.20.0.3 | 00:50:56:98:1e:c1 | controller | | 1 | 1
>
> An in the logs I see the traceback partially included - the root cause is highlighted
>
> Date Level Message
> 2017-02-23 20:56:14 ERROR [7fad42058880] (base) Unexpected exception occured
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 297, in handle_errors
> return func(cls, *args, **kwargs)
> File "<string>", line 2, in GET
> File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 355, in validate
> return func(cls, *args, **kwargs)
> File "<string>", line 2, in GET
> File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/base.py", line 381, in serialize
> resp = func(cls, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/nailgun/api/v1/handlers/network_configuration.py", line 92, in GET
> return self.serializer.serialize_for_cluster(cluster)
> File "/usr/lib/python2.7/site-packages/nailgun/objects/serializers/network_configuration.py", line 112, in serialize_for_cluster
> result = cls.serialize_net_groups_and_vips(cluster, allocate_vips)
> File "/usr/lib/python2.7/site-packages/nailgun/objects/serializers/network_configuration.py", line 42, in serialize_net_groups_and_vips
> nets = cluster.network_groups + [
> File "/usr/lib/python2.7/site-packages/nailgun/db/sqlalchemy/models/cluster.py", line 159, in network_groups
> net_list.extend(ng.networks)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 237, in __get__
> return self.impl.get(instance_state(instance), dict_)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 583, in get
> value = self.callable_(state, passive)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/strategies.py", line 532, in _load_for_state
> return self._emit_lazyload(session, state, ident_key, passive)
> File "<string>", line 1, in <lambda>
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/strategies.py", line 602, in _emit_lazyload
> result = q.all()
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2588, in all
> return list(self)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 86, in instances
> util.raise_from_cause(err)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 71, in instances
> rows = [proc(row) for row in fetch]
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 428, in _instance
> loaded_instance, populate_existing, populators)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/loading.py", line 486, in _populate_full
> dict_[key] = getter(row)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/type_api.py", line 1030, in process
> return process_value(value, dialect)
> File "/usr/lib/python2.7/site-packages/nailgun/db/sqlalchemy/models/fields.py", line 32, in process_result_value
> value = jsonutils.loads(value)
> File "/usr/lib/python2.7/site-packages/oslo_serialization/jsonutils.py", line 229, in loads
> return json.loads(encodeutils.safe_decode(s, encoding), **kwargs)
> File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads
> return _default_decoder.decode(s)
> File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
> File "/usr/lib64/python2.7/json/decoder.py", line 381, in raw_decode
> obj, end = self.scan_once(s, idx)
> ValueError: Expecting ',' delimiter: line 1 column 320 (char 319)
> [pid: 3540|app: 0|req: 656/1262] 192.168.198.222 () {50 vars in 905 bytes} [Thu Feb 23 20:56:14 2017] GET /api/clusters/1/network_configuration/neutron?_=1487883365500 => generated 39 bytes in 11 msecs (HTTP/1.1 500) 5 headers in 223 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 607/1263] 192.168.198.222 () {50 vars in 841 bytes} [Thu Feb 23 20:56:15 2017] GET /api/clusters?_=1487883365502 => generated 627 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 608/1264] 192.168.198.222 () {50 vars in 835 bytes} [Thu Feb 23 20:56:15 2017] GET /api/tasks?_=1487883365504 => generated 286 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 657/1265] 192.168.198.222 () {50 vars in 835 bytes} [Thu Feb 23 20:56:15 2017] GET /api/nodes?_=1487883365503 => generated 2 bytes in 13 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 658/1266] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:56:29 2017] GET /api/notifications?_=1487883365506 => generated 769 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 609/1267] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:56:29 2017] GET /api/nodes/allocation/stats?_=1487883365505 => generated 30 bytes in 17 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 610/1268] 192.168.198.222 () {50 vars in 867 bytes} [Thu Feb 23 20:56:38 2017] GET /api/clusters/1/attributes?_=1487883365508 => generated 35673 bytes in 20 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 611/1269] 192.168.198.222 () {50 vars in 871 bytes} [Thu Feb 23 20:56:38 2017] GET /api/clusters/1/plugin_links?_=1487883365510 => generated 2 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 612/1270] 192.168.198.222 () {50 vars in 861 bytes} [Thu Feb 23 20:56:38 2017] GET /api/nodes?cluster_id=1&_=1487883365511 => generated 2 bytes in 12 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 613/1271] 192.168.198.222 () {50 vars in 845 bytes} [Thu Feb 23 20:56:38 2017] GET /api/clusters/1?_=1487883365507 => generated 625 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 614/1272] 192.168.198.222 () {50 vars in 861 bytes} [Thu Feb 23 20:56:38 2017] GET /api/tasks?cluster_id=1&_=1487883365512 => generated 2 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 615/1273] 192.168.198.222 () {50 vars in 871 bytes} [Thu Feb 23 20:56:38 2017] GET /api/nodegroups?cluster_id=1&_=1487883365513 => generated 67 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 659/1274] 192.168.198.222 () {50 vars in 857 bytes} [Thu Feb 23 20:56:38 2017] GET /api/clusters/1/roles?_=1487883365509 => generated 7255 bytes in 67 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 660/1275] 192.168.198.222 () {50 vars in 845 bytes} [Thu Feb 23 20:56:38 2017] GET /api/releases/2?_=1487883365515 => generated 45381 bytes in 7 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 616/1276] 192.168.198.222 () {50 vars in 905 bytes} [Thu Feb 23 20:56:38 2017] GET /api/clusters/1/network_configuration/neutron?_=1487883365514 => generated 2441 bytes in 29 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 661/1277] 192.168.198.222 () {50 vars in 887 bytes} [Thu Feb 23 20:56:39 2017] GET /api/v1/clusters/1/vmware_attributes?_=1487883365516 => generated 4068 bytes in 21 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 617/1278] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:56:49 2017] GET /api/notifications?_=1487883365518 => generated 769 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 662/1279] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:56:49 2017] GET /api/nodes/allocation/stats?_=1487883365517 => generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 618/1280] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:57:09 2017] GET /api/notifications?_=1487883365520 => generated 769 bytes in 6 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 663/1281] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:57:09 2017] GET /api/nodes/allocation/stats?_=1487883365519 => generated 30 bytes in 17 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 619/1282] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:57:29 2017] GET /api/notifications?_=1487883365522 => generated 769 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 664/1283] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:57:29 2017] GET /api/nodes/allocation/stats?_=1487883365521 => generated 30 bytes in 19 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 620/1284] 192.168.198.222 () {50 vars in 811 bytes} [Thu Feb 23 20:57:44 2017] GET /api/notifications?_=1487883365524 => generated 769 bytes in 27 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 621/1285] 192.168.198.222 () {50 vars in 839 bytes} [Thu Feb 23 20:57:44 2017] GET /api/version?_=1487883365525 => generated 110 bytes in 2 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 622/1286] 192.168.198.222 () {50 vars in 841 bytes} [Thu Feb 23 20:57:44 2017] GET /api/settings?_=1487883365526 => generated 563 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 623/1287] 192.168.198.222 () {50 vars in 841 bytes} [Thu Feb 23 20:57:44 2017] GET /api/clusters?_=1487883365527 => generated 627 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 624/1288] 192.168.198.222 () {50 vars in 835 bytes} [Thu Feb 23 20:57:44 2017] GET /api/tasks?_=1487883365529 => generated 286 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 625/1289] 192.168.198.222 () {50 vars in 835 bytes} [Thu Feb 23 20:57:44 2017] GET /api/nodes?_=1487883365528 => generated 2 bytes in 13 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 665/1290] 192.168.198.222 () {50 vars in 829 bytes} [Thu Feb 23 20:57:44 2017] GET /api/nodes/allocation/stats?_=1487883365523 => generated 30 bytes in 81 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 626/1291] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:57:44 2017] GET /api/notifications?_=1487883365531 => generated 769 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 666/1292] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:57:44 2017] GET /api/nodes/allocation/stats?_=1487883365530 => generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 667/1293] 192.168.198.222 () {50 vars in 867 bytes} [Thu Feb 23 20:57:46 2017] GET /api/clusters/1/attributes?_=1487883365533 => generated 35673 bytes in 21 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 668/1294] 192.168.198.222 () {50 vars in 871 bytes} [Thu Feb 23 20:57:46 2017] GET /api/clusters/1/plugin_links?_=1487883365535 => generated 2 bytes in 4 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 669/1295] 192.168.198.222 () {50 vars in 845 bytes} [Thu Feb 23 20:57:46 2017] GET /api/clusters/1?_=1487883365532 => generated 625 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 670/1296] 192.168.198.222 () {50 vars in 861 bytes} [Thu Feb 23 20:57:46 2017] GET /api/nodes?cluster_id=1&_=1487883365536 => generated 2 bytes in 12 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 671/1297] 192.168.198.222 () {50 vars in 861 bytes} [Thu Feb 23 20:57:46 2017] GET /api/tasks?cluster_id=1&_=1487883365537 => generated 2 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 672/1298] 192.168.198.222 () {50 vars in 871 bytes} [Thu Feb 23 20:57:46 2017] GET /api/nodegroups?cluster_id=1&_=1487883365538 => generated 67 bytes in 2 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 627/1299] 192.168.198.222 () {50 vars in 857 bytes} [Thu Feb 23 20:57:46 2017] GET /api/clusters/1/roles?_=1487883365534 => generated 7255 bytes in 70 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 628/1300] 192.168.198.222 () {50 vars in 845 bytes} [Thu Feb 23 20:57:46 2017] GET /api/releases/2?_=1487883365540 => generated 45381 bytes in 14 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 673/1301] 192.168.198.222 () {50 vars in 905 bytes} [Thu Feb 23 20:57:46 2017] GET /api/clusters/1/network_configuration/neutron?_=1487883365539 => generated 2441 bytes in 40 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 674/1302] 192.168.198.222 () {50 vars in 887 bytes} [Thu Feb 23 20:57:46 2017] GET /api/v1/clusters/1/vmware_attributes?_=1487883365541 => generated 4068 bytes in 19 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 629/1303] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:58:04 2017] GET /api/notifications?_=1487883365543 => generated 769 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 675/1304] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:58:04 2017] GET /api/nodes/allocation/stats?_=1487883365542 => generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 630/1305] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:58:24 2017] GET /api/notifications?_=1487883365545 => generated 769 bytes in 5 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 676/1306] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:58:24 2017] GET /api/nodes/allocation/stats?_=1487883365544 => generated 30 bytes in 19 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 631/1307] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:58:44 2017] GET /api/notifications?_=1487883365547 => generated 769 bytes in 3 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 677/1308] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:58:44 2017] GET /api/nodes/allocation/stats?_=1487883365546 => generated 30 bytes in 16 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3540|app: 0|req: 678/1309] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:59:04 2017] GET /api/notifications?_=1487883365549 => generated 769 bytes in 5 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 632/1310] 192.168.198.222 () {50 vars in 869 bytes} [Thu Feb 23 20:59:04 2017] GET /api/nodes/allocation/stats?_=1487883365548 => generated 30 bytes in 22 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
> [pid: 3539|app: 0|req: 633/1311] 192.168.198.222 () {50 vars in 851 bytes} [Thu Feb 23 20:59:24 2017] GET /api/notifications?_=1487883365551 => generated 769 bytes in 5 msecs (HTTP/1.1 200) 4 headers in 185 bytes (2 switches on core 0)
>
>
>
>
> This email has been checked for viruses by Avast antivirus software.
> www.avast.com
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170223/7f232383/attachment.html>
More information about the Openstack
mailing list