[Openstack] Controller deployment consistently fails - any advice / help / fix appreciated (opened bug - https://bugs.launchpad.net/fuel-plugin-vmware-dvs/+bug/1667857)

Mario Romascanu romar at mindtrek.dnsalias.net
Sat Feb 25 00:29:36 UTC 2017


After many trial and error atempts around the vc_ca_file issue reported 
earlier, still stumbling on same puppet error that aborts the controller 
deployment

Any advice / help greatly appreciated

The root cause of the issue is visible in the puppet logs on the 
(aborted) controller (highlighted) - invalid vCenter CA file

2017-02-24 23:14:03    ERR 
  /usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:271:in `block in 
validate'
2017-02-24 23:14:03    ERR 
  /usr/lib/ruby/vendor_ruby/puppet/resource.rb:502:in `validate_parameter'
2017-02-24 23:14:03    ERR     Invalid parameter vc_ca_file on 
Vmware::Compute::Ha[0] on node node-5.domain.tld

On the Fuel master only the 3 astute errors are visible ( 3 retries)

Although vCenter is using self-signed certificates (and according to the 
doc there is no need for a CA bundle if the "bypass" is checked) I have 
extracted the root cert bundle from the vCenter.

Provided the vc_ca_file file in the vMware and Glance settings and as 
visible in the attached zip containing the  .yaml files  - they are OK

The issue was reported recently on Feb 20  but no resolution in sight: 
https://answers.launchpad.net/fuel-plugin-vmware-dvs/+question/466490

I have opened a bug with the issue and in this E-Mail I am providing the 
logs and generated .yaml files in the hope the issue will be tracked 
down and a fix will be found.

Here are the alarms

"Deployment has failed. All nodes are finished. Failed tasks: 
Task[vmware-vcenter/5] Stopping the deployment process!"

"Failed to deploy node 'MOS_Controller (1e:c1)': Task vmware-vcenter 
failed on node 5"

On the Fuel Master only error is in the astute.log
2017-02-24 23:14:03    DEBUG    [1173] {"nodes"=>[{"status"=>"error", 
"error_type"=>"deploy", "uid"=>"5", "role"=>"vmware-vcenter"}]}
2017-02-24 23:14:03    DEBUG    [1173] Node 5 has failed to deploy. 
There is no more retries for puppet run.
2017-02-24 23:14:03    ERROR    [1173] Node 5(vmware-vcenter) status: error
2017-02-24 23:14:03    DEBUG    [1173] Node[5]: Node 5: task 
vmware-vcenter, task status running

Failed state:

[root at fuel ~]# fuel node
id | status   | name                   | cluster | ip        | 
mac               | roles                     | pending_roles | online | 
group_id
---+----------+------------------------+---------+-----------+-------------------+---------------------------+---------------+--------+---------
  5 | error    | MOS_Controller (1e:c1) |       1 | 10.20.0.3 | 
00:50:56:98:1e:c1 | cinder-vmware, controller |               | 1 |        1
  3 | discover | MOS_Compute (46:4a)    |         | 10.20.0.4 | 
00:50:56:98:46:4a |                           |               | 1 |
[root at fuel ~]#
[root at fuel ~]# fuel task  (contain multiple attempts)
id  | status | name                    | cluster | progress | uuid
----+--------+-------------------------+---------+----------+-------------------------------------
84  | error  | deploy                  | 1       | 100      | 
e4209052-3627-477f-9793-5f62b2be6ccf
7   | ready  | update_dnsmasq          |         | 100      | 
8156d9a1-3d77-4394-98ca-167b808f3202
63  | ready  | reset_environment       | 1       | 100      | 
7c418e63-96b6-41dd-a10f-394cab6ec796
20  | ready  | reset_environment       | 1       | 100      | 
972b7b2f-2de2-43a5-a301-f4eede2f727f
5   | ready  | update_dnsmasq          |         | 100      | 
92680b8b-2dec-4820-aed8-4b2bbe5c6f93
8   | error  | deploy                  | 1       | 100      | 
5a53e5bd-5843-4759-bc46-eecc76857c42
4   | ready  | update_dnsmasq          |         | 100      | 
b5e20ff9-bd78-47ba-a628-6bb317947e16
26  | ready  | provision               | 1       | 100      | 
ecfd6977-1fba-4dbe-afdf-69ab69810fe9
21  | ready  | reset_environment       | 1       | 100      | 
ec446920-851a-40e8-9c17-e0f4923bb520
22  | ready  | reset_environment       | 1       | 100      | 
d8b53b73-4c35-43db-a9cb-e6e38c94e034
11  | error  | deploy                  | 1       | 100      | 
de12f677-ff99-4649-a7cc-032c652f77b6
13  | error  | check_before_deployment | 1       | 100      | 
4ed191e1-dce9-4616-be69-9921d6cdcf11
23  | error  | deploy                  | 1       | 100      | 
d14e83c9-65af-43e6-b7a9-c8370ad0b448
62  | error  | deployment              | 1       | 100      | 
e28e6686-f774-4d81-915e-8bb4acd4fc17
39  | error  | deployment              | 1       | 100      | 
8983249b-df5a-469b-8713-b470c542147b
87  | ready  | provision               | 1       | 100      | 
1be33faa-c71c-48fb-9fec-01c3ff675036
73  | ready  | reset_environment       | 1       | 100      | 
b25eb9f5-c91d-4af1-b370-18293383ca40
35  | error  | deploy                  | 1       | 100      | 
779b093c-008a-43a4-af34-38fd0a9ac015
33  | ready  | reset_environment       | 1       | 100      | 
90cc1426-ead7-4062-9c4c-3fe27da69fbe
34  | ready  | reset_environment       | 1       | 100      | 
1167f01a-319d-4417-a750-22e1d280cd67
27  | error  | deployment              | 1       | 100      | 
8ca54d3e-37c7-45b0-bf47-9972b79846a0
44  | error  | deploy                  | 1       | 100      | 
b28e628c-50cf-4ba1-95a8-183517069f3b
64  | ready  | reset_environment       | 1       | 100      | 
9fa14568-76eb-4978-b64e-26c422bdccc5
38  | ready  | provision               | 1       | 100      | 
18eb78c0-8080-4668-af0b-57546fb1dc6e
32  | ready  | reset_environment       | 1       | 100      | 
462c3388-384b-4901-9948-e4e8bb0ba650
41  | ready  | reset_environment       | 1       | 100      | 
9a23dad0-f6a2-47cd-8a40-c2c153aeb300
40  | ready  | reset_environment       | 1       | 100      | 
8ab81c53-95d9-4e3e-87fb-741df32cc0ce
42  | ready  | reset_environment       | 1       | 100      | 
6e5c7cde-8df1-4fec-9a01-712f309bd03e
47  | ready  | provision               | 1       | 100      | 
208ae427-bd72-4b7c-8588-a18cd9811914
65  | ready  | reset_environment       | 1       | 100      | 
5d9d57c4-2a00-4c94-88c1-7da1a0704dcc
50  | ready  | check_dhcp              | 1       | 100      | 
becfbd76-6bd2-4269-9138-6c8529c371b8
88  | error  | deployment              | 1       | 100      | 
00d24943-79c1-419e-865d-472ed66d4ef0
104 | error  | deployment              | 1       | 100      | 
10330910-4b29-436a-b64a-c358d29eee5b
66  | error  | deploy                  | 1       | 100      | 
0e28d6f0-ad93-45c4-8804-70f219d9bf1e
57  | ready  | check_networks          | 1       | 100      | 
da1f4b24-e5e9-48bd-8753-dcce93a441a2
69  | ready  | provision               | 1       | 100      | 
4b028aea-dfe9-4cc2-a1f6-66ee82b74721
51  | ready  | check_repo_availability | 1       | 100      | 
982d7439-8c4c-453e-806b-715886c1537a
48  | error  | deployment              | 1       | 100      | 
a2dc7ce7-ae42-4e23-98d6-e81f3830485c
49  | ready  | verify_networks         | 1       | 100      | 
1087a4e6-c9ec-41d1-a21f-8010d2e72eaf
52  | ready  | reset_environment       | 1       | 100      | 
00a0ae92-2fbd-492e-88a6-06367f116435
53  | ready  | reset_environment       | 1       | 100      | 
b94531d9-12fe-4943-8d7d-825e8bae580b
54  | ready  | reset_environment       | 1       | 100      | 
670c5fb0-59de-4bc8-8a9c-0f8008e88730
90  | ready  | reset_environment       | 1       | 100      | 
e18d6c0a-eefa-4174-93f6-eecf94a0c442
58  | error  | deploy                  | 1       | 100      | 
e5b6365a-77a6-46e5-9a9c-9a8100cfd544
61  | ready  | provision               | 1       | 100      | 
f10d57b1-bc1a-4c52-bdb8-2ef6c40ba698
100 | error  | deploy                  | 1       | 100      | 
b724ec56-64f7-4ae2-aee9-cc60431e4b93
98  | ready  | reset_environment       | 1       | 100      | 
9d9c311a-1555-4264-ba6c-a2182b4e1faf
97  | ready  | reset_environment       | 1       | 100      | 
42b7d607-c3ed-4e93-924e-b76874169b6a
89  | ready  | reset_environment       | 1       | 100      | 
7a6c0277-ff32-4b78-8cd4-d048e834ee27
99  | ready  | reset_environment       | 1       | 100      | 
bc51da41-8c05-4429-a582-fff2c4d5f26b
83  | ready  | node_deletion           |         | 100      | 
bf0b47e9-2602-4636-a84a-04be41bdfcbb
80  | ready  | reset_environment       | 1       | 100      | 
64cf09bf-6150-4dda-8269-f0102782cce5
81  | ready  | reset_environment       | 1       | 100      | 
91aebf8f-b604-4ead-bc91-a599126977de
82  | ready  | reset_environment       | 1       | 100      | 
2765bf95-45b7-4c90-ad13-aaee450daa33
91  | ready  | reset_environment       | 1       | 100      | 
0a04d6d2-a06b-49c4-81f2-ecc8e1e0d0a2
70  | error  | deployment              | 1       | 100      | 
9bb5f33f-8dd2-4fef-b2de-654eaed288c4
71  | ready  | reset_environment       | 1       | 100      | 
c27f4025-3075-45a4-ab47-d633a2ed71a2
72  | ready  | reset_environment       | 1       | 100      | 
e067fbfa-ab73-4098-91f6-4d9a16411dd7
103 | ready  | provision               | 1       | 100      | 
d372f9cb-c497-42c8-9001-e87e180201c5

Here are the relevant settings








---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170224/4e66e5ce/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: abaligmpgacceobc.png
Type: image/png
Size: 75612 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170224/4e66e5ce/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ffieijdmhmcopnbm.png
Type: image/png
Size: 50625 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170224/4e66e5ce/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hgiifkjgbhpdffmc.png
Type: image/png
Size: 29175 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170224/4e66e5ce/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: failed_controller_deployment.zip
Type: application/x-zip-compressed
Size: 129870 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170224/4e66e5ce/attachment.bin>


More information about the Openstack mailing list