Update 2015-12-03:According to a reader comment (below), NetBSD for RPi has matured significantly since I wrote this post. That sounds great to me! But I have not tested yet.
As a long time Linux user I have always had some kind of curiosity about the BSDs, especially NetBSD and its minimalistic approach to system design. For a while I have been thinking that perhaps NetBSD is the perfect operating system for turning a Raspberry Pi into a server.
I have read anti-BSD rants like this “BSD, the truth“, and I have also appreciated pkgsrc for Mac OS X. I felt I needed got get my own opinion. It is easy to have a romantic idea about “Old Real UNIX”, but my limited experience with IRIX and Solaris is not that positive. And BSD is another beast.
For the Raspberry Pi (Version 1, Model B) it is supposed to be possible to run both (stable) NetBSD 6.1.5 and (beta) NetBSD 7.0. It seemed, after all, that the beta 7.0 was the way to go.
At first it was fine
I followed the official instructions and installed NetBSD 7.0. I (first) used the (800MB) rpi.img. I set up my user:
# useradd zo0ok ... # mkdir /home # mkdir /home/zo0ok # chown zo0ok:users /home/zo0ok # usermod -G wheel zo0ok
Then it was time to configure pkgsrc and start installing packages.
The Disk Problem
I did a quick check to see how much available space I have, before installing stuff. To my surprise:
# df -h Filesystem Size Used Avail %Cap Mounted on /dev/ld0a 650M 623M -5.4M 100% / /dev/ld0e 56M 14M 42M 24% /boot kernfs 1.0K 1.0K 0B 100% /kern ptyfs 1.0K 1.0K 0B 100% /dev/pts procfs 8.0K 8.0K 0B 100% /proc tmpfs 112M 8.0K 112M 0% /var/shm
It seemed like the filesystem had not been (automatically) expanded as it should be according to the instructions above. So i followed the manual instructions to resize my root partition, with no success whatsover.
So I ran disklabel to see if NetBSD recognized my 8GB SD-card…
# /sbin/disklabel ld0 # /dev/rld0c: type: SCSI disk: STORAGE DEVICE label: fictitious flags: removable bytes/sector: 512 sectors/track: 32 tracks/cylinder: 64 sectors/cylinder: 2048 cylinders: 862 total sectors: 1766560 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # microseconds track-to-track seek: 0 # microseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize cpg/sgs] a: 1381536 385024 4.2BSD 0 0 0 # (Cyl. 188 - 862*) b: 262144 122880 swap # (Cyl. 60 - 187) c: 1766560 0 unused 0 0 # (Cyl. 0 - 862*) d: 1766560 0 unused 0 0 # (Cyl. 0 - 862*) e: 114688 8192 MSDOS # (Cyl. 4 - 59)
Clearly, NetBSD thought this SD-card was 900MB rather than 8GB, and this is why it failed to automatically resize it.
The sysinst install
I was anyway not very comfortable with getting a preinstalled/preconfigured 800MB system with swap and everything, so I formatted the 8GB SD card with my digital camera (just to be sure the partition table did not contain anything weird), downloaded (6MB) rpi_inst.img and wrote it to the SD card.
NetBSD installation started properly, and I was looking forward to install over SSH. According to the instructions I was supposed to start the DHCP somehow. But DHCP seemed on (the RPi got an IP) but SSH was off, so I installed using keyboard.
Quite immediately I was informed that NetBSD failed to recognise the “disk geometry” properly. I tried the SD card in Linux which reluctantly reported that it had 166 heads and 30 sectors per track (it sounds like nonsense). So I gave this information to the NetBSD sysinst program and now the SD card seemed to be 7.5GB.
Then followed a long and confused period of time when I tried to be smart enough to come up with any working partition scheme that NetBSD could accept. The right procedure was:
- Choose entire disk
- Confirm to delete the (required) 56MB dos partition
- Partition, pretending to be unaware of the need of a dos partition
- Magically, in the end, it added the dos partition
I am clearly stupid. There are no words for how confused I am about the a:, c: and e: partitions (that seems to reuse the DOS naming, but for other purposes), the empty space, the disk labels, the BSD partitions inside a (non existing) primary partition.
Anyway, just after I gave up and then gave it a final try I convinced sysinst to install. Then came a phase of choosing download paths, which clearly was non-trivial since I installed a Beta, and I am fine with that.
Installation went on. In the end came a nice menu where I could configure stuff. I liked it! (I wish I knew how to start it later). It managed to get my network settings from DHCP (except the GW), but it failed to configure and test the network itself (despite it had downloaded everything over the network just a few minutes ago). I configured a few other things, I restarted, network was working and I was happy… for a while.
I configured pkgsrc, and it seems ALL other systems where pkgsrc exist have been blessed with the pkgin tool, except NetBSD where you are supposed to do all the job yourself. Well, I added the PKG_PATH to the .shrc (of my user, not root) and enjoyed pkg_add.
(not) Compiling NodeJS
I want to install node.js on my NetBSD Raspberry Pi. It is not in pkgsrc (which is it for Mac OS X, but whatever) so I had to build it myself. I am used to building node.js and I was looking forward to fix all the broken dependencies. If I had ever gotten there.
I downloaded the source and started unpacking it… it is about 10000 files and 100MB of data. My SD card (a SanDisk Ultra, class 10) is not super fast, dd-ing the image to it earlier wrote at a speed of 3MB/s. The unpacking speed of node.js; roughly 1 file per second. I realised I need a (fast) USB-drive or a faster SD card, so I (literally) went out to town, bought a fast USB drive (did not find the SD card I wanted) and a few other things. When I came back more than 8000 files had been extracted and less than 2000 remained. I started reading about how to partition and format a USB drive for NetBSD, and at some point I inserted it in the Raspberry Pi. A little later I noticed my ssh sessions were dead, and the RPi had restarted. It turns out what reality was worse than the truth in “BSD, the truth”:
[…] the kernels of the BSDs are also very fault intolerant.
The best example of this is the issue with removing USBs. The problem appears when USBs are removed without unmounting them first. The result is a kernel panic. The astounding aspect of this is that this problem has been exhibited by all the major BSD variants Free, Open, Net and DragonflyBSD ever since USB support was implemented in them 5 to 6 years ago and has never ever been fixed. FreeBSD mailing lists even ban people who dare mention about it. In Linux, such things never and happen and bugs as serious as this gets fixed before a release is made.
Fact is, NetBSD 7.0 Beta for RPi, crashes, immediately, when I insert a USB drive.
This actually did not make me give up. I really restarted the system with the USB drive inserted, with the intention of treating my USB drive as a fixed disk and not inserting/removing it unless I shut the RPi down first. This was when I did give up: deleting the 16GB dos partition and creating a NetBSD filesystem was just too difficult for me. Admittedly, my patience was running out.
More on memory card performance
I found this very interesting article (linked to, by the Gentoo people, of course). Without going into details; clearly a Raspberry Pi with an SD card root filesystem needs a filesystem and block device implementation that works well with actual SD cards. This is not trivial and this means doing things very differently from rotating media.
I did the same unpacking of the node.js source on Raspbian (I installed Raspbian on exactly the same SD card as I used for NetBSD): 22 seconds (tar: 18s, sync 4s), compared to 3h for NetBSD.
Conclusion
In theory, NetBSD would be a beautiful fit for the Raspberry Pi. The ARMv6 is not supported by standard Debian. Raspbian comes with a little “too much” for my taste (it is not a real problem), and it does not have the feeling of “Debian stable”, but more some “inoffical Debian test” (sorry Raspbian people – I really appreciate your job!).
I have wondered why Noobs does not come with NetBSD… but I think I know now. And, sometimes I am surpised that Linux seems to work better than Mac OS X, perhaps now I know why.
My romantic idea that NetBSD would be perfect for the RPI was just plain wrong. Installing NetBSD today made me remember installing Slackware on a Compaq laptop in 1998.
Perhaps I will give Arch a try. Or put OpenWRT on the RPi.
NetBSD 7 was not quite ready then. Since then, things have been drastically improved. SD card access is now much faster, filesystems by default use FFSv2. If you mount with -o log (which enables WAPBL), you’ll see a significant speedup. All of these together make using NetBSD on a Raspberry Pi quite nice these days.
It’d be interesting if you were to try again to see how different things are.
Glad to hear! I will add a little comment at the top of my article, and perhaps try some day when I have time.
As of 07/22/16 it’s doing a lot better on a Pi2. I’ve been playing with graphics apps, GPIO, and have some examples for I2C that I’m about to try (based on mail list indications of success). The Netsurf-gtk browser works well, and the uSD performance is on par with Raspbian (as a matter of a wet finger in the wind kind of test).
The one thing remaining (for me) is SPI, which the mailling list shows as operational, but I find no prior beaten-the-path-already stuff on the internet 🙁