Installing Debian on the Technologic Systems TS-7800
Installing Debian onto the NAND is unfortunately not a straight forward process for a number of reasons:
- Debian's stock linux-image-2.6.32-5-orion5x kernel is not configured to:
- support the TS-7800 board
- enable the 'platnand' driver
- CONFIGBLKDEVRAM is a module so initrd's do not work
- due to a bug in the kernel, the NAND is not enabled for kernels 2.6.32 through to 2.6.36 inclusive
- more recent FPGA revisions are not known to the kernel so patches need to be applied to get the NAND and other hardware working
- trivial patch should really be applied to get the NAND twice as fast
- Technologic System's SD interface is closed so un-usable
- Technologic System's NAND and SD bootloaders do not set an appropriate machine ID for the board
- the unit only has 128MB of RAM which means it will not take a compressed 96MB initrd (the size of a base Debian root filesystem), meaning we have to use the SD card to load the kernel and put a temporary copy of the root filesystem on a USB key
- the NAND has a 512 byte page size however UBI for some reason keeps setting the mininum IO size to 2048 bytes (I am guessing the controller likes to roll with 2048 byte pages) and so sets a sub-page size of 512 bytes. This would not normally cause a problem but CONFIGMTDNANDVERIFYWRITE does not work with sub-paging which is something that is enabled in Debian kernels. For completeness I will show the 'normal' ubi related commands alongside the ones you must use, so make sure you use the correct ones. This quirk also means we have to amend the kernel command line from 'ubi.mtd=rootfs' to 'ubi.mtd=rootfs,2048', '-s 2048' to ubiformat and '-O 2048' to ubiattach
- we need to use the kernel command line parameter 'mtdparts=...' so that the 'mbr' partition is writable and we can replace it with something more useful
- booting off the NAND with an initrd seems just not to work. The kernel definately loads, as 'panic=10' is triggered successfully, but for some reason the console does not function at all; you do get some output with 'console=uart,mmio,0xf1012100,115200' though. No idea if this is a kernel or bootloader bug but for now the obvious and unfortunate workaround is to not use an initrd so I have amended the MBR to not load one and we compile UBIFS and plat_nand into the final kernel image
- there is no way with the TS bootloader to specify a kernel command line (u-boot-esque with an NVRAM) so we are forced to always compile our own kernels and hardcode it in. This also unfortunately has a knock on effect that it is unlikely Debian will ever be able to 'support' the TS-7800
Preparation
Before you continue you must have a pre-built 'armel' based Debian rootfs to hand, using an approach such as the one I detail on my page titled Cross debootstrap Debian Root Filesystem with User Mode QEMU; these instructions continue from where that page leaves off. You will additionally need the following packages installed:
- mtd-utils - do not use 20100907
- kernel-package
- fakeroot
- devio
- binfmt-support
- qemu-user-static - for Ubuntu weenies this package is called 'qemu-arm-static'
- you will additionally need an armel toolchain
Root Filesystem
Well, we start off by installing two packages to the root filesystem and then do some minor tweaking:
berk:/usr/src/orion5x# cat /etc/resolv.conf > rootfs/etc/resolv.conf
berk:/usr/src/orion5x# cp /usr/bin/qemu-arm-static rootfs/usr/bin/
berk:/usr/src/orion5x# chroot rootfs
berk:/# apt-get update
berk:/# aptitude install initramfs-tools mtd-utils linux-base
[snipped, ignore qemu errors and other minor errors]
berk:/# apt-get clean
berk:/# exit
berk:/usr/src/orion5x# rm rootfs/usr/bin/qemu-arm-static
berk:/usr/src/orion5x# : > rootfs/etc/resolv.conf
berk:/usr/src/orion5x# sed -i -e '/^MODULES=/ s/^\(.*\)=.*$/\1=most/' rootfs/etc/initramfs-tools/initramfs.conf
berk:/usr/src/orion5x# #echo plat_nand >> rootfs/etc/initramfs-tools/modules
berk:/usr/src/orion5x# sed -i -e '/^update_initramfs=/ s/^\(.*\)=.*$/\1=no/' rootfs/etc/initramfs-tools/update-initramfs.conf
berk:/usr/src/orion5x# cat `<<'EOF' >`> rootfs/etc/modules
ipv6
timeriomem_rng
EOF
N.B. initramfs-tools fails to detect ubifs root filesystems and so you need to additionally patch 'rootfs/usr/share/initramfs-tools/hook-functions' for now manually as described in the bug report
Kernel
We need a copy of the stock Debian kernel source package:
berk:/usr/src/orion5x$ wget http://ftp.de.debian.org/debian/pool/main/l/linux-2.6/linux-2.6_2.6.32-41.dsc
berk:/usr/src/orion5x$ wget http://ftp.de.debian.org/debian/pool/main/l/linux-2.6/linux-2.6_2.6.32.orig.tar.gz
berk:/usr/src/orion5x$ wget http://ftp.de.debian.org/debian/pool/main/l/linux-2.6/linux-2.6_2.6.32-41.diff.gz
berk:/usr/src/orion5x$ dpkg-source -x linux-2.6_2.6.32-41.dsc
Originally, when I wrote the NAND support for the mainline kernel, I incorrectly specified some memory region causing a bug to be triggered by a patch that went into 2.6.32. This resulted in the NAND no longer being detected but fortunately was later fixed by a patch that went into 2.6.37 written by Michael Spang; unfortunately we still need to apply this patch to a stock Debian 2.6.32 kernel:
berk:/usr/src/orion5x$ wget "http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=patch;h=377304ab" -O linux-2.6-2.6.32/debian/patches/bugfix/arm/orion5x-ts78xx-nand-fix.patch
berk:/usr/src/orion5x$ echo "+ bugfix/arm/orion5x-ts78xx-nand-fix.patch" >> linux-2.6-2.6.32/debian/patches/series/base
We now apply other misc patches to support newer FPGA revisions and to speed up the NAND:
berk:/usr/src/orion5x$ # FPGA rev update
berk:/usr/src/orion5x$ wget "http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=patch;h=17718e17" -O linux-2.6-2.6.32/debian/patches/bugfix/arm/orion5x-ts78xx-update-revs.patch
berk:/usr/src/orion5x$ echo "+ bugfix/arm/orion5x-ts78xx-update-revs.patch" >> linux-2.6-2.6.32/debian/patches/series/base
berk:/usr/src/orion5x$ # faster NAND
berk:/usr/src/orion5x$ wget "http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=patch;h=e25bac96" -O linux-2.6-2.6.32/debian/patches/bugfix/arm/orion5x-ts78xx-fast-nand.patch
berk:/usr/src/orion5x$ echo "+ bugfix/arm/orion5x-ts78xx-fast-nand.patch" >> linux-2.6-2.6.32/debian/patches/series/base
berk:/usr/src/orion5x$ # support unknown FPGA rev
berk:/usr/src/orion5x$ wget "http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=patch;h=f9b1184e" -O linux-2.6-2.6.32/debian/patches/bugfix/arm/orion5x-ts78xx-unknown-rev.patch
berk:/usr/src/orion5x$ echo "+ bugfix/arm/orion5x-ts78xx-unknown-rev.patch" >> linux-2.6-2.6.32/debian/patches/series/base
berk:/usr/src/orion5x$ # fix unknown FPGA rev support
berk:/usr/src/orion5x$ wget "http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=patch;h=b3882330" -O linux-2.6-2.6.32/debian/patches/bugfix/arm/orion5x-ts78xx-fix-rev.patch
berk:/usr/src/orion5x$ echo "+ bugfix/arm/orion5x-ts78xx-fix-rev.patch" >> linux-2.6-2.6.32/debian/patches/series/base
We first need a suitable kernel for when we boot off the SD card and mount the USB key (for this example the USB key is not partitioned and we will use ext4):
berk:/usr/src/orion5x$ cd linux-2.6-2.6.32
berk:/usr/src/orion5x/linux-2.6-2.6.32$ sed -i 's/^abiname: \([0-9]\+\)$/abiname: \1+ts78xx/' debian/config/defines
berk:/usr/src/orion5x/linux-2.6-2.6.32$ fakeroot make -f debian/rules clean
berk:/usr/src/orion5x/linux-2.6-2.6.32$ fakeroot make -f debian/rules.gen setup_armel_none_orion5x
berk:/usr/src/orion5x/linux-2.6-2.6.32$ cd debian/build/build_armel_none_orion5x
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # use menuconfig to amend things so that:
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_MACH_TS78XX=y
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_CMDLINE="console=ttyS0,115200n8 earlyprintk panic=10 rw mtdparts=gen_nand:128k@0k(mbr),4096k@128k(kernel),4096k@4224k(initrd),515968k@8320k(rootfs) rootdelay=10 root=/dev/sda"
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_MTD_NAND_PLATFORM=y
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_UBIFS_FS=y (depends CONFIG_MTD_UBI=y)
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_EXT4_FS=y
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_USB_EHCI_HCD=y (depends CONFIG_USB=y)
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_USB_STORAGE=y (depends CONFIG_SCSI=y and CONFIG_BLK_DEV_SD=y)
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ make menuconfig
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ cd ../../../
berk:/usr/src/orion5x/linux-2.6-2.6.32$ CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- do_tools=false fakeroot make -f debian/rules.gen binary-arch_armel_none_orion5x DEBIAN_KERNEL_JOBS=$(grep ^processor /proc/cpuinfo | wc -l)
berk:/usr/src/orion5x/linux-2.6-2.6.32$ cd debian/build/build_armel_none_orion5x
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ cp arch/arm/boot/zImage ..
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x# CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- INSTALL_MOD_PATH=/usr/src/orion5x/rootfs make modules_install
Now to cook the 'production' kernel we will be using, this also covers the process what you will regularly be using to update your kernel (producing '/usr/src/orion5x/linux-image-2.6.32-5+ts78xx-orion5x2.6.32-41armel.deb' that can then be installed):
berk:/usr/src/orion5x/linux-2.6-2.6.32$ fakeroot make -f debian/rules clean
berk:/usr/src/orion5x/linux-2.6-2.6.32$ fakeroot make -f debian/rules.gen setup_armel_none_orion5x
berk:/usr/src/orion5x/linux-2.6-2.6.32$ cd debian/build/build_armel_none_orion5x
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # use menuconfig to amend things so that:
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_MACH_TS78XX=y
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_CMDLINE="console=ttyS0,115200n8 earlyprintk panic=10 rw ubi.mtd=rootfs,2048 root=ubi0:rootfs rootfstype=ubifs"
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_MTD_NAND_PLATFORM=y
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ # * CONFIG_UBIFS_FS=y (depends CONFIG_MTD_UBI=y)
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ make menuconfig
berk:/usr/src/orion5x/linux-2.6-2.6.32/debian/build/build_armel_none_orion5x$ cd ../../../
berk:/usr/src/orion5x/linux-2.6-2.6.32$ CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- do_tools=false fakeroot make -f debian/rules.gen binary-arch_armel_none_orion5x DEBIAN_KERNEL_JOBS=$(grep ^processor /proc/cpuinfo | wc -l)
NAND with an initrd
This is just a "for your information" section and should not be used. If we could boot off the NAND with an initrd, then the amended kernel options list instead would look like:
- CONFIGMACHTS78XX=y
- CONFIGBLKDEV_RAM=y
- CONFIGMTDNAND_PLATFORM=m
- CONFIG_CMDLINE="console=ttyS0,115200 panic=10 rw ubi.mtd=rootfs,2048 root=ubi0:rootfs rootfstype=ubifs"
Installation
We now need to create a bootable SD card and build the USB mountable root filesystem. On the USB key we also will place an amended NAND MBR and the production copy of the final root filesystem as an UBI image; we assume that the USB key, at least 512MB in size, on your workstation when plugged in is at '/dev/sdb' whilst the SD card is at '/dev/mmcblk0' and is at least 16MB in size:
berk:/usr/src/orion5x# dd if=sd-mbr.dd of=/dev/mmcblk0
berk:/usr/src/orion5x# blockdev --rereadpt /dev/mmcblk0
berk:/usr/src/orion5x# { devio 'wl 0xe3a01c06,4' 'wl 0xe3811074,4'; cat zImage; } | dd of=/dev/mmcblk0p1
berk:/usr/src/orion5x# mkfs.ext4 /dev/sdb
berk:/usr/src/orion5x# blockdev --rereadpt /dev/sdb
berk:/usr/src/orion5x# mkdir build
berk:/usr/src/orion5x# mount /dev/sdb build
berk:/usr/src/orion5x# cp -a rootfs/* build
berk:/usr/src/orion5x# cp flash-mbr.digriz.dd build/root
berk:/usr/src/orion5x# cp linux-image-2.6.32-5+ts78xx-orion5x_2.6.32-41_armel.deb build/root
berk:/usr/src/orion5x# rm -rf rootfs/lib/modules/2.6.32-5+ts78xx-orion5x
berk:/usr/src/orion5x$ cat `<<EOF >` ubi.cfg
[ubifs]
mode=ubi
image=ubifs.img
vol_id=0
vol_size=128MiB
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize
EOF
berk:/usr/src/orion5x# # WARNING: only use the 2048 related commands and *not* the 512 ones.
berk:/usr/src/orion5x# # mkfs.ubifs -v -r rootfs -m 2048 -e $(((128*1024)-(2*512))) -c $(((((512-4-4)*1024)-128)/128)) -o ubifs.img
berk:/usr/src/orion5x# mkfs.ubifs -v -r rootfs -m 2048 -e $(((128*1024)-(2*2048))) -c $(((((512-4-4)*1024)-128)/128)) -o ubifs.img
berk:/usr/src/orion5x# # ubinize -v -p 128KiB -m 2048 -s 512 -o ubi.img ubi.cfg
berk:/usr/src/orion5x# ubinize -v -p 128KiB -m 2048 -o ubi.img ubi.cfg
berk:/usr/src/orion5x# cp ubi.img build/root
berk:/usr/src/orion5x# umount build
Now place the SD card in your TS-7800, plug in the USB key and turn it on. Once you get to a root prompt type the following:
root@(none):~# # replace with the output of 'date' from your own workstation
root@(none):~# date -s "Thu Dec 30 15:58:00 GMT 2010"
root@(none):~# flash_eraseall /dev/mtd0
root@(none):~# flash_eraseall /dev/mtd1
root@(none):~# # flash_eraseall /dev/mtd2
root@(none):~# nandwrite -p /dev/mtd0 flash-mbr.digriz.dd
root@(none):~# # ubidetach /dev/ubi_ctrl -m 3
root@(none):~# # ubiformat -f ubi.img /dev/mtd3
root@(none):~# ubiformat -s 2048 -f ubi.img /dev/mtd3
root@(none):~# mkdir target
root@(none):~# ubiattach /dev/ubi_ctrl -m 3 -O 2048
root@(none):~# mount -t ubifs ubi:rootfs target
root@(none):~# mount --bind /sys target/sys
root@(none):~# echo "ubi0:rootfs / ubifs rw,relatime 0 0" > target/proc/mounts
root@(none):~# cp linux-image-2.6.32-5+ts78xx-orion5x_2.6.32-38_armel.deb target
root@(none):~# chroot target
root@(none):/# dpkg -i linux-image-2.6.32-5+ts78xx-orion5x_2.6.32-38_armel.deb
root@(none):/# exit
root@(none):~# nandwrite -p /dev/mtd1 target/boot/vmlinuz-2.6.32-5+ts78xx-orion5x
root@(none):~# # nandwrite -p /dev/mtd2 target/boot/initrd.img-2.6.32-5+ts78xx-orion5x
root@(none):~# umount target/sys
root@(none):~# rm target/proc/mounts
root@(none):~# rm target/
root@(none):~# umount target
root@(none):~# halt
If everything went okay, you should be able to unplug the SD card and USB key and find it boots in Debian off the NAND.
Application Hints
sshd
N.B. this seems to no longer be applicable but remains incase others stumble into this problem
Seems if you set 'UsePrivilegeSeparation' to 'yes', when you SSH the daemon will segfault nine times out of ten. Until I find some time to diagnose whats going on (annoyingly sshd is PIC compiled so GDB cannot be used) setting this instead to 'no' will have to remain the workaround for now. Strangely I do not see this problem on my OpenRDs and SheevaPlugs...or a custom compiled kernel.