[Openstack] Mirantis - Fuel 9.2 Failed Deployment
Raja T Nair
rtnair at gmail.com
Wed Oct 25 01:44:49 UTC 2017
Thanks Evgeniy, for the support.
I ran the command on shell, and it immediately came out, leaving a line in
the logfile which said that all required images are present.
After this I re-ran the deployment. It went past the point, but errored
out, with the following info on the dashboard:
Error
Deployment has failed. All nodes are finished. Failed tasks:
Task[fuel_pkgs/3], Task[fuel_pkgs/2], Task[fuel_pkgs/4] Stopping the
deployment process!
3 of the nodes show `Error Deployment' and 2 of them show `Stopped'
Log tab gives the following lines for nodes:
<>
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/bin/puppet:8:in `<main>'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:92:in `execute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:146:in `run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util.rb:496:in `exit_on_fail'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application.rb:507:in `plugin_hook'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application.rb:381:in `block (2 levels) in
run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:159:in `run_command'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:198:in `main'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:236:in `block in main'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:302:in `apply_catalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:133:in `run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet.rb:246:in `override'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in `override'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:134:in `block in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:227:in `run_internal'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:119:in `apply_catalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:120:in `block in
apply_catalog'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:168:in `apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:112:in
`as_logging_destination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util/log.rb:149:in `with_destination'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:169:in `block in apply'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:138:in `evaluate'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in
`traverse'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:in `block in evaluate'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util.rb:334:in `thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util.rb:335:in `block in thinmark'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:in `block (2 levels) in
evaluate'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:147:in `call'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:221:in `eval_resource'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:19:in
`process_events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:in
`queued_events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:92:in `each'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:93:in `block
in queued_events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:20:in `block
in process_events'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/transaction/event_manager.rb:101:in
`process_callback'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:583:in `refresh'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:in `sync'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:127:in `times'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/type/exec.rb:130:in `block in sync'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/provider/exec/posix.rb:45:in `run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:in `run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:29:in `chdir'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:64:in `block in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/provider/exec.rb:68:in `block (2 levels)
in run'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:in `execute'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
/usr/lib/ruby/vendor_ruby/puppet/util/execution.rb:186:in `waitpid2'
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Command exceeded timeout
2017-10-24 17:13:43 ERR (/Stage[main]/Apt::Update/Exec[apt_update])
Failed to call refresh: Command exceeded timeout
</>
Inside individual nodes, puppet.log has these lines, which shows of some
unmet dependency:
2017-10-24 17:14:00 +0000 /Package[fuel-misc] (info): Starting to evaluate
the resource
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/dpkg-query -W
--showformat '${Status} ${P
ackage} ${Version}\n' fuel-misc'
2017-10-24 17:14:00 +0000 Package[fuel-misc](provider=apt_fuel) (debug):
Call: install
2017-10-24 17:14:00 +0000 Package[fuel-misc](provider=apt_fuel) (debug):
Call: wait_for_lock
2017-10-24 17:14:00 +0000 Package[fuel-misc](provider=apt_fuel) (debug):
Call: locked?
2017-10-24 17:14:00 +0000 Puppet (debug): Executing '/usr/bin/apt-get -q -y
-o DPkg::Options::=--force-
confold -o APT::Get::AllowUnauthenticated=1 install fuel-misc'
2017-10-24 17:14:00 +0000 Package[fuel-misc](provider=apt_fuel) (warning):
Attempt '1' of '3' has faile
d: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold
-o APT::Get::AllowUnauthenti
cated=1 install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
fuel-misc : Depends: socat but it is not installable
E: Unable to correct problems, you have held broken packages.
But dpkg shows that both are installed:
root at node-1:/var/log# dpkg -l fuel-misc socat
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==============-==============-============================================
ii fuel-misc 9.0.0-1~u14.04 all Misc Fuel library
scripts
ii socat 1.7.2.3-1 amd64 multipurpose relay
for bidirectional data tr
root at node-1:/var/log#
Please help me to get this deployment completed.
Many Thanks,
Raja.
On 24 October 2017 at 21:13, Evgeniy L <eli at mirantis.com> wrote:
> Hi,
>
> Yes, execution expired, means that it could not build an image within
> required period of time.
> The message you see can be used to build an image (copy paste into console
> "command"), also there is a file with logs /var/log/fuel-agent-env-1.log,
> which you can use to try to identify where the problem is.
>
> Thanks,
>
> On Tue, Oct 24, 2017 at 7:48 AM, Raja T Nair <rtnair at gmail.com> wrote:
>
>> Hello All,
>>
>> We are trying to deploy one environment using Fuel 9.2, and it failed at
>> around 23%.
>>
>> The error says:
>>
>> <>
>> Error
>> Provision has failed. Failed to execute hook 'shell'
>>
>> command: cd / && fa_build_image --image_build_dir /var/lib/fuel/ibp
>> --log-file /var/log/fuel-agent-env-1.log --data_driver nailgun_build_image
>> --input_data '{"image_data": {"/boot": {"container": "gzip", "uri": "
>> http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64-boot.img.gz",
>> "format": "ext2"}, "/": {"container": "gzip", "uri": "
>> http://172.22.66.2:8080/targetimages/env_1_ubuntu_1404_amd64.img.gz",
>> "format": "ext4"}}, "output": "/var/www/nailgun/targetimages", "repos":
>> [{"name": "ubuntu", "section": "main universe multiverse", "uri": "
>> http://archive.ubuntu.com/ubuntu/", "priority": null, "suite": "trusty",
>> "type": "deb"}, {"name": "ubuntu-updates", "section": "main universe
>> multiverse", "uri": "http://archive.ubuntu.com/ubuntu/", "priority":
>> null, "suite": "trusty-updates", "type": "deb"}, {"name":
>> "ubuntu-security", "section": "main universe multiverse", "uri": "
>> http://archive.ubuntu.com/ubuntu/", "priority": null, "suite":
>> "trusty-security", "type": "deb"}, {"name": "mos", "section": "main
>> restricted", "uri": "http://172.22.66.2:8080/mitaka-9.0/ubuntu/x86_64",
>> "priority": 1050, "suite": "mos9.0", "type": "deb"}, {"name":
>> "mos-updates", "section": "main restricted", "uri": "
>> http://mirror.fuel-infra.org/mos-repos/ubuntu/9.0/", "priority": 1050,
>> "suite": "mos9.0-updates", "type": "deb"}, {"name": "mos-security",
>> "section": "main restricted", "uri": "http://mirror.fuel-infra.org/
>> mos-repos/ubuntu/9.0/", "priority": 1050, "suite": "mos9.0-security",
>> "type": "deb"}, {"name": "mos9.2-updates", "section": "main restricted",
>> "uri": "http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority":
>> 1050, "suite": "mos9.0-updates", "type": "deb"}, {"name":
>> "mos9.2-security", "section": "main restricted", "uri": "
>> http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1050,
>> "suite": "mos9.0-security", "type": "deb"}, {"name": "Auxiliary",
>> "section": "main restricted", "uri": "http://172.22.66.2:8080/mitak
>> a-9.0/ubuntu/auxiliary", "priority": 1150, "suite": "auxiliary", "type":
>> "deb"}, {"name": "mos9.2-holdback", "section": "main restricted", "uri": "
>> http://mirror.fuel-infra.org/mos-repos/ubuntu/9.2", "priority": 1100,
>> "suite": "mos9.0-holdback", "type": "deb"}], "packages": ["acl", "anacron",
>> "bash-completion", "bridge-utils", "bsdmainutils", "build-essential",
>> "cloud-init", "curl", "daemonize", "debconf-utils", "gdisk", "grub-pc",
>> "hwloc", "i40e-dkms", "linux-firmware", "linux-firmware-nonfree",
>> "linux-headers-generic-lts-xenial", "linux-image-generic-lts-xenial",
>> "lvm2", "mcollective", "mdadm", "multipath-tools", "multipath-tools-boot",
>> "nailgun-agent", "nailgun-mcagents", "network-checker", "ntp",
>> "openssh-client", "openssh-server", "puppet", "python-amqp", "ruby-augeas",
>> "ruby-ipaddress", "ruby-json", "ruby-netaddr", "ruby-openstack",
>> "ruby-shadow", "ruby-stomp", "telnet", "ubuntu-minimal", "ubuntu-standard",
>> "uuid-runtime", "vim", "virt-what", "vlan"], "codename": "trusty"}'
>>
>> Task: a84a3af7-2ed4-4d5e-9911-a12dfc0f05e9: overall timeout error:
>> execution expired
>> Task timeout: 3600, Retries: 1
>>
>> </>
>>
>> I understand it was a timeout, probably because of a slow Internet
>> connection, while building a required image - please correct me if I'm
>> wrong here.
>>
>> Might be stupid question - but is there a way for me to build such images
>> before clicking the deploy button, thereby avoiding the timeout issue?
>>
>> Can anyone guide me or point me to a document to understand what is going
>> wrong here?
>>
>> Many thanks in advance.
>>
>> Regards,
>> Raja.
>>
>> --
>> :^)
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>> Post to : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>>
>>
>
--
:^)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20171025/8c039bc8/attachment.html>
More information about the Openstack
mailing list