[Openstack-operators] Problem launching instance, Kernel panic… (Scientific Linux)

Salman Toor salman.toor at it.uu.se
Wed Oct 24 12:18:28 UTC 2012


Hi, 

Thanks for your response yes it works and serves the purpose. 

Now I am getting the response by using ttyS0. 

The problem is with the disk controller, I think, not sure. But the strange thing is that the problem only appears when I try to boot an instance using nova. 

I am using nova with qemu and libvirt.

If I using qemu command on the same compute node everything works fine. 

#  qemu-system-x86_64 -k 'sv' vm-basic-2.6.0-4-1-x86_64.vdi 

or 

#  qemu-system-x86_64 -k 'sv' vm-basic-2.6.0-4-1-x86_64.qcow

instance boot without any problem 

But when I use nova boot command that internally using qemu end-up with "kernel panic .. "


Following is the boot log:

Linux version 2.6.18-308.13.1.el5 (brewbuilder at norob.fnal.gov) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-52)) #1 SMP Tue Aug 21 18:44:22 EDT 2012
Command line: BOOT_IMAGE=/boot/vmlinuz-2.6.18-308.13.1.el5 ro root=LABEL=root noresume crashkernel=1M at 16M divider=10 console=ttyS0
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000010000 - 000000000009dc00 (usable)
 BIOS-e820: 000000000009dc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000001fffe000 (usable)
 BIOS-e820: 000000001fffe000 - 0000000020000000 (reserved)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
DMI 2.4 present.
No NUMA configuration found
Faking a node at 0000000000000000-000000001fffe000
Bootmem setup node 0 0000000000000000-000000001fffe000
ACPI: PM-Timer IO Port: 0xb008
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 6:2 APIC version 17
ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
Setting APIC routing to flat
ACPI: HPET id: 0x8086a201 base: 0xfed00000
Using ACPI (MADT) for SMP configuration information
Nosave address range: 000000000009d000 - 000000000009e000
Nosave address range: 000000000009e000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000f0000
Nosave address range: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at 30000000 (gap: 20000000:dffc0000)
SMP: Allowing 1 CPUs, 0 hotplug CPUs
Built 1 zonelists.  Total pages: 127662
Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.18-308.13.1.el5 ro root=LABEL=root noresume crashkernel=1M at 16M divider=10 console=ttyS0
Initializing CPU#0
PID hash table entries: 2048 (order: 11, 16384 bytes)
Console: colour VGA+ 80x25
Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
Checking aperture...
ACPI: DMAR not present
Memory: 506056k/524280k available (2617k kernel code, 17764k reserved, 1671k data, 224k init)
Calibrating delay loop (skipped), value calculated using timer frequency.. 5594.18 BogoMIPS (lpj=2797092)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 0/0 -> Node 0
SMP alternatives: switching to UP code
Freeing SMP alternatives: 32k freed
ACPI: Core revision 20060707
Using local APIC timer interrupts.
Detected 47.462 MHz APIC timer.
Brought up 1 CPUs
WARNING: CPU#0: NMI appears to be stuck (0->0)!
time.c: Using 100.000000 MHz WALL HPET GTOD HPET/TSC timer.
time.c: Detected 2797.092 MHz processor.
checking if image is initramfs... it is
Freeing initrd memory: 2454k freed
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using configuration type 1
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI quirk: region b000-b03f claimed by PIIX4 ACPI
PCI quirk: region b100-b10f claimed by PIIX4 SMB
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKS] (IRQs 9) *0, disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
pnp: PnP ACPI: found 8 devices
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
hpet0: at MMIO 0xfed00000 (virtual 0xffffffffff5fe000), IRQs 2, 8, 0
hpet0: 3 64-bit timers, 100000000 Hz
ACPI: DMAR not present
PCI-GART: No AMD northbridge found.
NET: Registered protocol family 2
IP route cache hash table entries: 4096 (order: 3, 32768 bytes)
TCP established hash table entries: 16384 (order: 6, 262144 bytes)
TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
TCP: Hash tables configured (established 16384 bind 8192)
TCP reno registered
audit: initializing netlink socket (disabled)
type=2000 audit(1350977002.539:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Initializing Cryptographic API
alg: No test for crc32c (crc32c-generic)
ksign: Installing public key data
Loading keyring
- Added public key 904C1484EAE644E
- User ID: Red Hat, Inc. (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
Limiting direct PCI/PCI transfers.
PCI: PIIX3: Enabling Passive Release on 0000:00:01.0
Activating ISA DMA hang workarounds.
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Real Time Clock Driver v1.12ac
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
?serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
brd: module loaded
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX3: IDE controller at PCI slot 0000:00:01.1
PIIX3: chipset revision 0
PIIX3: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xc180-0xc187, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0xc188-0xc18f, BIOS settings: hdc:pio, hdd:pio
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
input: AT Translated Set 2 keyboard as /class/input/input0
ACPI: (supports S5)
Initalizing network drop monitor service
Freeing unused kernel memory: 224k freed
Write protecting the kernel read-only data: 530k
Red Hat nash version 5.1.19.6 starting
Mounting proc filesystem
Mounting sysfs filesystem
Creating /dev
Creating initial device nodes
Setting up hotplug.
input: ImExPS/2 Generic Explorer Mouse as /class/input/input1
Creating block device nodes.
Loading ehci-hcd.ko module
Loading ohci-hcd.ko module
Loading uhci-hcd.ko module
USB Universal Host Controller Interface driver v3.0
ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
ACPI: PCI Interrupt 0000:00:01.2[D] -> Link [LNKD] -> GSI 11 (level, high) -> IRQ 11
uhci_hcd 0000:00:01.2: UHCI Host Controller
uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c140
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
Loading jbd.ko module
usb 1-1: new full speed USB device using uhci_hcd and address 2
Loading ext3.ko module
Loading dm-mem-cache.ko module
Loading dm-mod.ko module
usb 1-1: configuration #1 chosen from 1 choice
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.11.6-ioctl (2011-02-18) initialised: dm-devel at redhat.com
Loading dm-log.ko module
input: QEMU 1.0.1 QEMU USB Tablet as /class/input/input2
input,hidraw0: USB HID v0.01 Pointer [QEMU 1.0.1 QEMU USB Tablet] on usb-0000:00:01.2-1
Loading dm-region_hash.ko module
Loading dm-message.ko module
Loading dm-raid45.ko module
device-mapper: dm-raid45: initialized v0.2594l
Scanning and configuring dmraid supported devices
noresume passed, not resuming...
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

I have looked at the following thread 

http://forums.openstack.org/viewtopic.php?f=15&t=523

They have used "kvm" and I have used "qemu" otherwise I think there is quite a lot of similarities. I have also check the file

/var/log/libvirt/qemu/<instance>.log

And I also have "if=none" thingy in the long command. 

I am wondering is there any way to solve this. I mean by setting up some parameters in nova's or qemu's config files? 

Regards.
Salman. 
 

Salman Toor, PhD
salman.toor at it.uu.se



On Oct 19, 2012, at 7:33 PM, Abel Lopez wrote:

> You should make 'disk_format' equal to whatever the file format of the image is.
> So, if you're using something like VirtualBox to initially create the image, and you export it as vmdk, then use that. If you're using something like qemu-img to create the image, you're probably creating something like a raw or qcow2 image.
> 
> Definitively make sure that your grub.conf has console redirection. I personally do
> echo 'ttyS0' >> /etc/securetty
> echo 'S0:12345:respawn:/sbin/agetty ttyS0 115200' >> /etc/inittab
> sed -i.backup 's/LABEL=.*/&console=ttyS0/' /boot/grub/grub.conf
> 
> This will allow you to not only see bootup dmesg stuff when you use 'nova console-log', but it would allow you to login using something like 'virsh vncdisplay'
> 
> Hope that helps
> 
> On Fri, Oct 19, 2012 at 4:46 AM, Salman Toor <salman.toor at it.uu.se> wrote:
> Hi all,
> 
> Struggling with booting an instance on compute node.
> 
> 
> I  am trying to boot a Scientific linux image but get "kernel panic" while booting the image. On status it shows instance "Active" and "Running".
> 
> I couldn't find any logs in the compute file. one get some last lines in the VNC web console.
> 
> I am using qemu with libvirtd.
> 
> The strange thing is that I can boot the image with qemu commandline tool on the same host without any problem
> 
> # qemu-system-x86_64 -k 'sv' cernvm-basic-2.6.0-4-1-x86.vmdk
> 
> I have 3 questions.
> 
> 1 - Please suggest a better logging way as in compute I am not getting any error message?
> 
> 2 - What should I do to check what is going wrong in the system?
> 
> 3 - While uploading the image using "glance", I have tried following
> 
> disk_formate=vmdk
> or
> disk_formate=vdi
> 
> both works fine. Where
> 
> container_formate=bare
> in both cases.
> 
> Which one I should really set for the disk_formate?
> 
> also there is one disk_formate vmk. What is this for?
> 
> Thanks in advance.
> 
> Regards..
> Salman.
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20121024/4c0ceb76/attachment-0001.html>


More information about the OpenStack-operators mailing list