[Openstack] [Fuel] node name issue
Jim Okken
jim at jokken.com
Mon Nov 20 19:06:44 UTC 2017
update to an old question:
I have gotten around this issue.
I'm not sure how I got around this issue, but my theory is something I
noticed quite by accident.
The servers I was having these troubles on use 80GB hard drives, but they
also have a flash drive in them, for small OS deployments. I stumbled
across a /dev/sda6 partition on these flash drives.
On this partition I found 2 files: meta-data and user-data. In those files
was the old node name I mentioned in the original post. This partition must
have been detected, and these stale files were used by fuel-agent when the
newly provisioned node first booted, even though they were booting from the
80GB drive which had its own /dev/sda6 partition.
I suspect the stale/incorrect /dev/sda6 is probably is from Fuel 8 when we
tried to deploy on some of these flash drives...
once I deleted the stale/incorrect /dev/sda6 then theprovisioning and
deployment went perfectly
thanks
-- Jim
On Thu, Sep 28, 2017 at 5:02 PM, Jim Okken <jim at jokken.com> wrote:
> I ran "fuel2 node update -H blade13 20" just to get out of the node-*
> naming convention, as someone suggested
>
>
>
> The deploy still names the node node-11 and provisioning fails.
>
> digging a little more, i see it might have to do with the fuel-agent
> cloud-init scripts.
>
> in the cloud-init.log on the new node I see the node name being set to
> node-11!
>
>
>
> this isnt node-11. this was node-20, but I renamed it to blade13 with the
> command "fuel2 node update -H blade13 20"
>
>
>
> i also noted that after the cloud-init scripts ran at the end the first
> boot of the new provisioned OS, that in the Fuel GUI, the FQDN field became
> node-11.ourdomain.com (before it was bootstrap.ourdomain.com)
>
> (in the same window Hostname still show as blade13)
>
>
>
> But FQDN in the Fuel2 CLI output still shows node-20.ourdomain.com!!!
>
>
>
> [fuel2 node show 20
>
> | id | 20
> |
>
> | name | Untitled (68:58)
> |
>
> | status | ready
> |
>
> | os_platform | ubuntu
> |
>
> | roles | [u'compute']
> |
>
> | kernel_params | None
> |
>
> | pending_roles | []
> |
>
> | hostname | node-20
> |
>
> | fqdn | node-20.dialogic.com
> |
>
> | platform_name | ProLiant BL460c Gen9
> |
>
>
>
>
>
>
>
>
>
>
>
> where can i find the cloud init settings which are deploy to new nodes?
>
> i guess this has something to do with this file:
> /usr/share/fuel-agent/cloud-init-templates/cloud_config_ubuntu.jinja2
>
> in that file I see
>
> hostname: {{ common.hostname }}
>
> fqdn: {{ common.fqdn }}
>
>
>
> please help me with an info you might have or let me know that populates
> those 2 parts of the template?
>
>
>
> Is there a database these values are all stored in on the fuel server?
>
>
>
> Thanks
>
>
>
> --Jim
>
> -- Jim
>
> On Tue, Sep 26, 2017 at 12:00 PM, Jim Okken <jim at jokken.com> wrote:
>
>> also I should add, I dont have the original hard drives in the system so
>> it isn't because it is booting the old OS where these node names were set.
>> this is definitely the newly installed OS being given the wroing hostname
>>
>>
>>
>> is there a database this is all kept in? maybe I could look around and
>> find where these old node names are being saved?
>>
>> thanks!
>>
>> -- Jim
>>
>> On Mon, Sep 25, 2017 at 6:03 PM, Jim Okken <jim at jokken.com> wrote:
>>
>>> hi all,
>>>
>>> I am using Fuel 10.
>>>
>>> i have 2 nodes I am trying to deploy as compute nodes. at one time in
>>> the past I was attempting to deploy them too. I assume back then their node
>>> names were node-11 and node-20.
>>>
>>> they were never successfully deploy and now I've worked out their
>>> hardware issues and are attempting to deploy them again. now Fuel has given
>>> them the names node-80 and node-81.
>>> (i may be at 80 in my node names but I only have 17 nodes so far)
>>>
>>> the deploy of these 2 nodes does not get past installing Ubuntu. The
>>> nodes reboot after Ubuntu is installed and come up incorrectly as node-11
>>> and node-20. After that Fuel sits for a long while and then gives an error
>>> (pasted at the end of email). I assume the nodes come up with the wrong
>>> name/ip/ssh-key and Fuel can't contact them.
>>>
>>> I'm a novice at using the fuel and fuel2 cli's but I've tried deleting
>>> these nodes and removing from database. Then re-PXE boot the nodes and
>>> start a fresh deploy just to have them named node11 and 20 again. Fuel cli
>>> does show the correct host name for these nodes, but I've tried anyway to
>>> (re)set the host name for these node with no affect.
>>>
>>> If I try to delete node-11 and node-20 I get this error
>>> 404 Client Error: Not Found for url: http://10.20.243.1:8000/api/v1
>>> /nodes/?ids=11 (NodeCollection not found)
>>>
>>> what can I do to get past this please?
>>>
>>>
>>>
>>> Errors from the Fuel Astute log:
>>> 2017-09-25 21:06:28 ERROR [1565] Error running provisioning:
>>> #<RuntimeError: Could not find any hosts in discovery data provided> ,
>>> trace: ["/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:178:in
>>> `rescue in initialize_mclient'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/mclient.rb:161:in `initialize_mclient'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:51:in
>>> `initialize'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/nailgun_hooks.rb:421:in `new'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
>>> `run_shell_without_check'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/nailgun_hooks.rb:449:in `update_node_status'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:313:in
>>> `reboot_hook'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/nailgun_hooks.rb:38:in `block in process'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
>>> `each'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
>>> `process'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/image_provision.rb:117:in
>>> `reboot'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:273:in
>>> `soft_reboot'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:240:in `provision_piece'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:126:in
>>> `block (3 levels) in provision_and_watch_progress'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:309:in
>>> `call'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:309:in
>>> `sleep_not_greater_than'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:120:in `block (2 levels) in
>>> provision_and_watch_progress'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:119:in `loop'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:119:in
>>> `block in provision_and_watch_progress'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:118:in `catch'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:118:in
>>> `provision_and_watch_progress'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:52:in `provision'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/orchestrator.rb:109:in
>>> `provision'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/dispatcher.rb:46:in `provision'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/dispatcher.rb:37:in
>>> `image_provision'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/server.rb:172:in `dispatch_message'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:131:in
>>> `block in dispatch'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/task_queue.rb:64:in `call'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:64:in
>>> `block in each'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/task_queue.rb:56:in `each'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:56:in
>>> `each'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:128:in
>>> `each_with_index'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/server.rb:128:in `dispatch'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:106:in
>>> `block in perform_main_job'"]
>>> 2017-09-25 21:06:26 ERROR [1565] Error occured while provisioning:
>>> #<Astute::MClientError: #<RuntimeError: Could not find any hosts in
>>> discovery data provided> >
>>> 2017-09-25 21:06:26 ERROR [1565] No more retries for MCollective client
>>> instantiation after exception: ["/usr/share/gems/gems/mcollec
>>> tive-client-2.8.4/lib/mcollective/rpc/client.rb:507:in `discover'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/mclient.rb:167:in
>>> `initialize_mclient'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/mclient.rb:51:in `initialize'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
>>> `new'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:421:in
>>> `run_shell_without_check'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/nailgun_hooks.rb:449:in `update_node_status'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:313:in
>>> `reboot_hook'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/nailgun_hooks.rb:38:in `block in process'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
>>> `each'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/nailgun_hooks.rb:26:in
>>> `process'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/image_provision.rb:117:in
>>> `reboot'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:273:in
>>> `soft_reboot'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:240:in `provision_piece'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:126:in
>>> `block (3 levels) in provision_and_watch_progress'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:309:in
>>> `call'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:309:in
>>> `sleep_not_greater_than'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:120:in `block (2 levels) in
>>> provision_and_watch_progress'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:119:in `loop'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:119:in
>>> `block in provision_and_watch_progress'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:118:in `catch'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/provision.rb:118:in
>>> `provision_and_watch_progress'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/provision.rb:52:in `provision'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/orchestrator.rb:109:in
>>> `provision'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/dispatcher.rb:46:in `provision'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/dispatcher.rb:37:in
>>> `image_provision'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/server.rb:172:in `dispatch_message'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:131:in
>>> `block in dispatch'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/task_queue.rb:64:in `call'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:64:in
>>> `block in each'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/task_queue.rb:56:in `each'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/task_queue.rb:56:in
>>> `each'", "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:128:in
>>> `each_with_index'", "/usr/share/gems/gems/astute-1
>>> 0.0.0/lib/astute/server/server.rb:128:in `dispatch'",
>>> "/usr/share/gems/gems/astute-10.0.0/lib/astute/server/server.rb:106:in
>>> `block in perform_main_job'"]
>>> 2017-09-25 21:06:21 ERROR [1565] Retrying RPC client instantiation
>>> after exception: #<RuntimeError: Could not find any hosts in discovery data
>>> provided>
>>> 2017-09-25 21:06:16 ERROR [1565] Retrying RPC client instantiation
>>> after exception: #<RuntimeError: Could not find any hosts in discovery data
>>> provided>
>>> Version: 10.0
>>>
>>>
>>> -- Jim
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20171120/d206bbb6/attachment.html>
More information about the Openstack
mailing list