Re: [SLUG] Xen virtual (lol) nightmare

From: Mike Branda (mike@wackyworld.tv)
Date: Wed Oct 31 2007 - 12:50:53 EST


Ian C. Blenke wrote:
> Mike Branda wrote:
>
>> Has anybody here had any experience with Xen? I subscribed to the
>> xen-users list and have posted a few times and have gotten no responses.
>
>
> Loads of experience. http://ian.blenke.com/xen
>

Thanks! I'll check that out as I continue to work through this.

> Your Dom0 kernel is responsible for talking to your hardware.
>

Good deal. I wasn't completely sure if that was the case or if it was
the hypervisor. Originally, I was booting from the raid and the
hypervisor just barfed with a "waiting for device /dev/sda to appear".
I then did a clean install on a single scsi attached directly to the
mobo and now I can at least boot the hypervisor and Dom0.

> When you run "lspci" (or "cat /proc/pci") do you see the megaraid card
> while booted under the hypervisor?
>

Are you kidding me!!!!! AAARGH... now, after 2 days of tinkering it
shows up!!

:~ # lspci
04:02.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID (rev 01

:~ # lsmod | grep -i megaraid
megaraid_mbox 34832 0
megaraid_mm 14624 1 megaraid_mbox
scsi_mod 140504 7 sr_mod, megaraid_mbox, libata, sg,
sd_mod, aic79xx, scsi_transport_spi

The only thing I can think of between the last xen boot and now is that
I added 6 modules to /etc/sysconfig/kernel under INITRD_MODULES.
Originally, scsi_transport_spi, sd_mod, sg, sr_mod, libata and
megaraid_mm only showed up in an lsmod under the default SuSE non-xen
kernel. I attempted to load these by hand last time and I would see
evidence of the card being initialized in /var/log/messages but the disk
would never appear in an fdisk. Now it does!! Oddly enough,
megaraid_mbox was already in INITRD_MODULES and I DID NOT have to add it.

Could that be it?

> Realize that the kernel you installed with is a _different_ kernel (and
> has different kernel modules) than the Xen Dom0 kernel you are booting
> with that can't see your megaraid controller.
>
> My guess here is that your Xen dom0 kernel does not have the
> /lib/modules for megaraid.
>

Just so we're on the same page here... SuSE provided via several rpms:
a bigsmp kernel (my default boot where the raid card always worked)
a xenpae kernel
a xend and hypervisor and other misc binaries

Their xen.src.rpm shows 230 patches!! and until I can make sense of
them... I'm not attempting my own builds from a vanilla source.

A quick scan through both configs (SuSE renames the .config from the
kernel build and stuffs them in /boot for reference) turns up:

:~ # grep -in "megaraid" /boot/config-2.6.22.9-0.4-*
/boot/config-2.6.22.9-0.4-bigsmp:1349:CONFIG_MEGARAID_NEWGEN=y
/boot/config-2.6.22.9-0.4-bigsmp:1350:CONFIG_MEGARAID_MM=m
/boot/config-2.6.22.9-0.4-bigsmp:1351:CONFIG_MEGARAID_MAILBOX=m
/boot/config-2.6.22.9-0.4-bigsmp:1352:CONFIG_MEGARAID_LEGACY=m
/boot/config-2.6.22.9-0.4-bigsmp:1353:CONFIG_MEGARAID_SAS=m

/boot/config-2.6.22.9-0.4-xenpae:1228:CONFIG_MEGARAID_NEWGEN=y
/boot/config-2.6.22.9-0.4-xenpae:1229:CONFIG_MEGARAID_MM=m
/boot/config-2.6.22.9-0.4-xenpae:1230:CONFIG_MEGARAID_MAILBOX=m
/boot/config-2.6.22.9-0.4-xenpae:1231:CONFIG_MEGARAID_LEGACY=m
/boot/config-2.6.22.9-0.4-xenpae:1232:CONFIG_MEGARAID_SAS=m

which would explain why I could load them by hand with a modprobe but
not why it wouldn't work with Dom0....

well, keep watching... I'm sure I'll have more after I get at least 1
DomU going. Not to mention the networking oddities....

Mike Branda Jr.
-----------------------------------------------------------------------
This list is provided as an unmoderated internet service by Networked
Knowledge Systems (NKS). Views and opinions expressed in messages
posted are those of the author and do not necessarily reflect the
official policy or position of NKS or any of its employees.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 19:49:41 EDT