Friday 21 September 2012

Netbooting NetBSD on a PC Engines Alix2

Alix 2d2 board
I've been looking for a small box to act as a firewall and OpenVPN server, and someone pointed me at the PC Engines Alix2 - a delightful little AMD Geode board, with up to 256M, three Ethernet ports, USB, CompactFlash, miniPCI and a serial console.

The Geode is a 486 class processor, so will just run NetBSD/i386 directly.

I picked the 2D2, which just has two Ethernet ports, primarily because the 2D3 was out of stock and I wanted something now :)

Installing NetBSD

The easiest option would be to put the Compact Flash card into a normal PC, install as normal, and set the boot blocks to use the serial console. Of course this assumes that: (a) you haven't already put the CF card into the Alix and assembled the case, and (b) you're not in the process of redecorating  your study, the contents of which have been decanted into a huge pile in the kitchen, with your only CF equipped machine at the bottom.

So, a good opportunity to test netbooting and install, which I've been doing on Suns and other machines since early 1990's, but this is possibly the first time I've done it on an x86 box. I'm going to use my T500 running NetBSD/amd64 as the server.

1) Console setup

 

The Alix 2 has a serial console defaulting to 38400 bps. As serial ports are a distant memory for most modern machines, my T500 needs a USB to serial adaptor, plus the obvious null model cable. On NetBSD the USB serial port shows up as ttyU0 (or dtyU0 for the 'direct' device), so to connect:
# cu -l dtyU0 -s 38400
Powering on the ALIX rewards me with:
PC Engines ALIX.2 v0.99h
640 KB Base Memory
261120 KB Extended Memory

01F0 Master 848A TS8GCF133                              
Phys C/H/S 15538/16/63 Log C/H/S 974/255/63 LBA

BIOS setup:

(9) 9600 baud (2) 19200 baud *3* 38400 baud (5) 57600 baud (1) 115200 baud
*C* CHS mode (L) LBA mode (W) HDD wait (V) HDD slave (U) UDMA enable
(M) MFGPT workaround
(P) late PCI init
*R* Serial console enable
(E) PXE boot enable
(X) Xmodem upload
(Q) Quit
All that should be required now is to select PXE boot and away we go.

It is at this point that I will reveal that while the NetBSD PXE boot program is more than happy to speak to the console at the speed it is given, when it goes to load the kernel it seems to switch to 9600 baud (at least as on NetBSD 6.0_RC1). Rather than relate the actual sequence of events (which involved the NetBSD boot startup, go to load the kernel, then some time of watching a non-responsive screen, followed by a "hmmm", and a retry... closely followed by another retry with tcpdump running, a quizzical expression and finally a minor baud rate epiphany, I'll continue as if I already knew about the baud rate issue (purely to make me seem more forward thinking).

So, that would be a [9], [e], [q] and a [y] to save, followed by [RETURN][~][.] to exit cu, and reconnecting at the new speed with
# cu -l dtyU0 -s 9600
This time you should be rewarded with a message ending something like:
CLIENT MAC ADDR: 00 0D B9 DD EE FE
which we need for the netboot server.

2) The netboot server


For this we need three services, conveniently all of which ship with NetBSD:
  1. A DHCP server to provide an IP address and details of where to load the boot program (dhcpd)
  2. A tftp server to serve the boot program (tftpd)
  3. An NFS server to provide the NetBSD kernel which the boot program loads (rpdbind, mountd, nfsd)
We could extract a full NetBSD/i386 distribution onto the NFS server and export it to the Alix, but a diskless network routing box which requires an NFS server to run seems a little... perverse, so we'll just use a NetBSD INSTALL kernel.

That means we need two files from a NetBSD/i386 distribution:
  1. The PXE boot program - installation/misc/pxeboot_ia32.bin
  2. An INSTALL kernel with ramdisk - binary/kernel/netbsd-INSTALL.gz
Other items to note, and potentially change in your configuration
  • The gateway to the Internet is at 192.168.2.1
  • The server is at 192.168.2.2
  • The Alix is at 192.168.2.3 and named "alix"
  • The Alix "hardware ethernet" address is copied from above
  • The directory to NFS export will be /export/alix"
So, starting with the dhcpd setup
  • Create an /etc/dhcpd.conf as below
  • Add "dhcpd=YES" to /etc/rc.conf
  • Start dhcpd with /etc/rc.d/dhcpd start
# dhcpd.conf
ddns-update-style none;
subnet 192.168.2.0 netmask 255.255.255.0 {  option routers 192.168.2.1;
  option domain-name-servers 192.168.2.1;
  host alix {
    hardware ethernet 00:0d:b9:dd:ee:fe;
    filename "pxeboot_ia32.bin";
    fixed-address 192.168.2.3;
    next-server 192.168.2.2;
    option host-name "alix";
    option root-path "/export/alix";
  }
}
(If its never been run before you will also need to touch /var/db/dhcpd.leases)

Next the PXE boot program
  • Copy pxeboot_ia32.bin into /tftpboot/pxeboot_ia32.bin
  • Edit /etc/inetd.conf, uncomment the line starting "#tftp"
  • Restart inetd with /etc/rc.d/inetd restart

Finally the kernel
  • Copy netbsd-INSTALL.gz in /export/alix as netbsd (no need to uncompress it)
  • Edit /etc/exports and add a line "/export/alix 192.168.2.3"
  • Add the following lines to /etc/rc.conf "mountd=YES", "nfsd=YES", "nfs_server=YES"
  • Start the daemons: /etc/rc.d/rpcbind start; /etc/rc.d/mountd start; /etc/rc.d/nfsd start

3) Netbooting the Alix


Once all the above is in place we can connect the console, switch on the Alix and netboot it. We need to interrupt the PXE boot program to tell it to use the serial console, but apart from that it should just boot directly into the NetBSD sysinst installer. Log below:
PC Engines ALIX.2 v0.99h
640 KB Base Memory
261120 KB Extended Memory

01F0 Master 848A TS8GCF133
Phys C/H/S 15538/16/63 Log C/H/S 974/255/63 LBA

Intel UNDI, PXE-2.0 (build 082)
Copyright (C) 1997,1998,1999 Intel Corporation
VIA Rhine III Management Adapter v2.43 (2005/12/15)

CLIENT MAC ADDR: 00 0D B9 DD EE FE
CLIENT IP: 192.168.2.3 MASK: 255.255.255.0 DHCP IP: 192.168.2.2
GATEWAY IP: 192.168.2.1


>> NetBSD/x86 PXE boot, Revision 5.1 (from NetBSD 6.0_POST_RC1)
>> Memory: 555/261120 k
Press return to boot now, any other key for boot menu
booting netbsd - starting in 5 seconds. [SPACE]
type "?" or "help" for help.
> consdev com0

>> NetBSD/x86 PXE boot, Revision 5.1 (from NetBSD 6.0_POST_RC1)
>> Memory: 555/261120 k
> boot netbsd
PXE BIOS Version 2.1
Using PCI device at bus 0 device 9 function 0
Ethernet address 00:0d:b9:dd:ee:fe
10741836+5665276+443200 [573760+563454]=0x11296ac
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009, 2010, 2011, 2012
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.

NetBSD 6.0_POST_RC1 (INSTALL)
total memory = 255 MB
avail memory = 234 MB
RTC BIOS diagnostic error 0x80<clock_battery>
mainbus0 (root)
acpi_probe: failed to initialize tables
cpu0 at mainbus0: Geode(TM) Integrated Processor by AMD PCS, id 0x5a2
pci0 at mainbus0 bus 0: configuration mode 1
pchb0 at pci0 dev 1 function 0: vendor 0x1022 product 0x2080 (rev. 0x33)
glxsb0 at pci0 dev 1 function 2: RNG AES
vr0 at pci0 dev 9 function 0: vendor 0x1106 product 0x3053 (rev. 0x96)
vr0: interrupting at irq 10
vr0: Ethernet address: 00:0d:b9:dd:ee:fe
ukphy0 at vr0 phy 1: OUI 0x0002c6, model 0x0034, rev. 3
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr1 at pci0 dev 11 function 0: vendor 0x1106 product 0x3053 (rev. 0x96)
vr1: interrupting at irq 15
vr1: Ethernet address: 00:0d:b9:dd:ee:ff
ukphy1 at vr1 phy 1: OUI 0x0002c6, model 0x0034, rev. 3
ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
gcscpcib0 at pci0 dev 15 function 0: vendor 0x1022 product 0x2090 (rev. 0x03)
gcscpcib0: Watchdog Timer via MFGPT0, GPIO
gpio0 at gcscpcib0: 32 pins
viaide0 at pci0 dev 15 function 2: AMD CS5536 IDE Controller (rev. 0x01)
viaide0: primary channel interrupting at irq 14
atabus0 at viaide0 channel 0
viaide0: secondary channel ignored (disabled)
ohci0 at pci0 dev 15 function 4: vendor 0x1022 product 0x2094 (rev. 0x02)
ohci0: interrupting at irq 12
ohci0: OHCI version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
gcscehci0 at pci0 dev 15 function 5: vendor 0x1022 product 0x2095 (rev. 0x02)
gcscehci0: interrupting at irq 12
gcscehci0: companion controller, 4 ports each: ohci0
usb1 at gcscehci0: USB revision 2.0
isa0 at gcscpcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com0: console
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
attimer0 at isa0 port 0x40-0x43
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
sysbeep0 at pcppi0
isapnp0 at isa0 port 0x279
npx0 at isa0 port 0xf0-0xff
attimer0: attached to pcppi0
uhub0 at usb0: vendor 0x1022 OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1 at usb1: vendor 0x1022 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
wd0 at atabus0 drive 0
wd0: <TS8GCF133>
wd0: 7647 MB, 15538 cyl, 16 head, 63 sec, 512 bytes/sect x 15662304 sectors
boot device: vr0
root on md0a dumps on md0b
root file system type: ffs
WARNING: clock lost 4632 days
WARNING: using filesystem time
WARNING: CHECK AND RESET THE DATE!
warning: no /dev/console
Created tmpfs /dev (1474560 byte, 2848 inodes)
Alix running sysinst (via T500)
Followed by the sysinst screen. The only thing to remember is that when prompted to install the boot blocks, select the serial console version, and your preferred speed (I went back to 38400 at that point).