IPMI

Vendor Specific

This section covers how to deal with the implementations I have stumbled on our there that really seemingly go out their way to ruin IPMI.

Supermicro

ipmitool raw

Dedicate/Shared/Failover LAN

On the X9 motherboard, the command was shortened to:

# get LAN mode
ipmitool raw 0x30 0x70 0x0c 0

# set LAN mode
ipmitool raw 0x30 0x70 0x0c X

Where X:

0: dedicated
1: share
2: failover

From Supermicro FAQ #14417.

ATEN iKVM via VNC

I did some work to noVNC to support connecting to the X9 series and earlier iKVMs. Fortunately it was adopted and improved by a number of bright people who figured out also the support for the X10 and newer iKVMs as well as the Dell iDRAC.

Remote Booting

These instructions are something I figured out at work on how to boot over the WAN a Debian Pre-seeding Install (baremetal).

Seems there are a number of oddities with booting disk images on Supermicro motherboards:

the webfrontend is horrible, but not as nearly as horrible as the Java console, which only works if you are on a particular selection of OSes, running a specific set of Java runtimes with the planets aligned against one another in a handful of combinations - I have bust my balls trying to figure out how to do this all just with curl and ipmi, so everyone owes me beer :)
to mount a CD ISO you have to make a Samba share accessible that holds the ISO; god knows why they did not just use HTTP. Fortunately you can upload to the BMC a floppy disk image to boot
the uploaded floppy disk image file must be 1440kB, everything else is ignored
the floppy image must contain some expected DOS looking magic otherwise it fails to boot, good old Supermicro (means later the make bin/ipxe.dsk image does not work)

So, the solution is to use syslinux to create a bootable floppy disk that then chainloads ipxe.lkrn loading the ipxe.pxe script which drops us into a shell.

Lets build a bootable floppy using the build-floppy.sh script, run the following as root (on a Debian Linux box):

sh build-floppy.sh

N.B. COM3 is used as the serial port of IPMI, but if you are testing this under a VM, you probably want it left set to COM1; edit the build script to do this

You should now have a bootable floppy image called floppy.fdd.

N.B. you can test this image in Parallels if you like. If you add a serial port, you can use socat to connect to the UNIX server socket created with "socat UNIX:/tmp/test -,raw,echo=0". If you are using Parallels, add to the 'Boot flags' "devices.net.force_adapter_type=rtl" as I have found that the emulated e1000 network card (when used with iPXE) seems to pretty much immediately lock up

Once you have the image, get yourself the authentication cookie from the web frontend and upload the floppy image:

curl -v -o /dev/null --cookie /tmp/cookie.txt --cookie-jar /tmp/cookie.txt --data-ascii "name=ADMIN&pwd=AHEM" http://1.2.3.4/cgi/login.cgi
curl -v -o /dev/null --cookie /tmp/cookie.txt --cookie-jar /tmp/cookie.txt --header "Expect:" -F "file=@floppy.fdd;filename=test.img;type=application/octet-stream" http://1.2.3.4/cgi/uimapin.cgi

ipmitool -H 1.2.3.4 -U ADMIN -P AHEM -I lanplus chassis bootdev floppy
ipmitool -H 1.2.3.4 -U ADMIN -P AHEM -I lanplus chassis power reset
ipmitool -H 1.2.3.4 -U ADMIN -P AHEM -I lanplus sol activate

You should see all the built in PXE loading and everything, but if you do not it might be that the buggy unreliable piece of crap that is the Supermicro IPMI implentation is unreliable. The fix is to reboot it via (you will need to re-run through the above login/upload proceedure though):

ipmitool -H 1.2.3.4 -U ADMIN -P AHEM -I lanplus mc reset cold
/usr/sbin/ipmiping 1.2.3.4     <-- will show you as it goes down and back up

If everything works out right, you should just be presented with a prompt:

Intel(R) Boot Agent GE v1.3.65
Copyright (C) 1997-2010, Intel Corporation

CLIENT MAC ADDR: 00 11 22 33 44 55  GUID: 0025907C A5D6 0607 0025 907CA5D60E0F
PXE-E51: No DHCP or proxyDHCP offers were received.

PXE-M0F: Exiting Intel Boot Agent.

SYSLINUX 4.02 debian-20101014 EDD Copyright (C) 1994-2010 H. Peter Anvin et al
Now booting the iPXE from SYSLINUX...
Loading ipxe.lkrn...
Loading ipxe.pxe...ready.
iPXE initialising devices...ok

iPXE 1.0.0+ (3fcb) -- Open Source Network Boot Firmware -- http://ipxe.org
Features: VLAN HTTP iSCSI DNS TFTP AoE SRP bzImage ELF MBOOT PXE PXEXT Menu
iPXE>

Once you have finished with iPXE (and your OS install is running) you can umount the floppy image with:

curl -v -o /dev/null --cookie /tmp/cookie.txt --cookie-jar /tmp/cookie.txt http://1.2.3.4/cgi/uimapout.cgi

So we now need to put an IP address and DNS settings on a network interface so that we may download the debseeder (if you are using a VM, all the "set" commands and the "ifopen" can be replaced with the single call to the command "dhcp"):

iPXE> ifstat
net0: 00:11:22:33:44:55 using 82579lm on PCI00:19.0 (closed)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
net1: 00:11:22:33:44:55 using 82574l on PCI02:00.0 (closed)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
iPXE> set net0/ip 1.2.3.4
iPXE> set net0/netmask 255.255.255.0
iPXE> set net0/gateway 1.2.3.1
iPXE> set dns 8.8.8.8
iPXE> ifopen net0
iPXE> chain http://stuff.digriz.org.uk/conf
http://stuff.digriz.org.uk/conf... ok
http://http.debian.net/debian/dists/squeeze/main/installer-amd64/current/images/netboot/debian-installer/amd64/linux... ok
http://stuff.digriz.org.uk/initrd... ok
hostname:HOSTNAME
domain:example.com

The call to chain downloads the conf script that iPXE runs through, where it automatically downloads all the necessary componments and primes up the kernel parameters. Once done, we boot:

iPXE> boot