Installing NetBSD: amd64 pain ahoy
As covered in a previous post, I decided to deep dive a more UNIX'y operating system, ultimately NetBSD. And so it begins
When I want to learn something new, I go all-in. Diving into NetBSD, I wanted to go a 'total immersion' route, mostly because I'm fortunate enough to have lots of spare machines around I could test on and thought it would be best to use on real hardware. And focus on that machine with no distractions, just man pages like times past.
I also went all-in on coke cherry zero at my battlestation as you will see. This stuff is like nectar from the sugar free gods.
I picked NetBSD because I wanted a challenge and I wasn't exactly expecting smooth sailing but so far I've got challenges in all the places I was not expecting!
Sometimes I get irrationally mad or frustrated with things when in a hurry to get started, so this is more me aligning reality with expectations and documenting my progress.
Let me say this is not an attack on NetBSD, I understand that it's a smaller project compared to FreeBSD and OpenBSD and not even to mention GNU/Linux. It's not like companies are paying people to hack out drives for NetBSD.
First, here's some of the hardware I've seriously tried (and remembered to write stuff down on) so far:
- Dell Vostro 3500 laptop from 2011
- i5 3570k Desktop with ancient 2008 intel sata SSD (Spoilers!)
I decided to start with the latest regular, non-development release NetBSD 7.1.2, the newest release (5 days old at time of writing)
Getting diagnostics during installation
But first an aside: know how to dig info off a machine during installation!
This may not be obvious for newer experimenters like myself, but it didn't occur to me how difficult it would be to grab outputs of
dmesg off a machine during installation to share for help.
The easiest way is to have an USB stick formatted as FAT and insert it after dropping to shell from installer menu (if you can). I was booting off the installation USB stick, in that case I needed two. Do not have both plugged in when you boot, just installation media.
You can boot the install media and immediately bail to shell once you see the menu. Once there you can mount the USB stick like so
# mkdir /mnt/usbstick2
(plugin usb stick, at this point you might see in console, or:)
... check the latest message about mounting umass on device /dev/sd1 or such. Mine looked like this
umass1 at uhub2 port 7 configuration 1 interface 0 umass1: USB2.0 Flash Disk, rev 2.00/0.00, addr 5 umass1: using SCSI over Bulk-Only scsibus1 at umass1: 2 targets, 1 lun per target sd1 at scsibus1 target 0 lun 0: <USB2.0, Flash Disk, 2.60> disk removable sd1: fabricating a geometry sd1: 961 MB, 961 cyl, 64 head, 32 sec, 512 bytes/sect x 1968128 sectors sd1: fabricating a geometry sd1: mbr partition exceeds disk size
Now we know the device is 'sd1', so the first partition would be `sd1a':
# mount_msdos /dev/sd1a /mnt/usbstick2 # dmesg > /mnt/usbstick2/dmesg.txt # sync (or you can do umount /dev/sd1a)
If you're coming from a gnu/linux background like me, take heed and do not typo it
sda1 a million times, because the backspace key will NOT work correctly and you'll go mad.
Importantly, dont forget the sync or umount command. If you rip out the USB stick before hand, the file could be not there, or likely, present but empty.
Installation - laptop
Starting with the laptop, this one was very frustrating to figure out at first due to some red herring level issues.
In an effort to save anyone else chasing this, I basically saw this during boot of the installer:
# root device:
There was two steps to this
- Set hard drive mode from AHCI to IDE :(
- Boot from a cdrom instead of of USB stick
For reasons unclear to me booting from cdrom worked fine and successfully installed without issue. HOWEVER, booting the new installation ran into the same issue, even with the hard drive detected in either mode (IDE/AHCI), it fails to open. I never got past this and gave up on the laptop:
root device: wd0a .. vfs_mountroot: can't open root device cannot mount root, error = 6
Heres a lame picture to follow par for the course
At this point I decided to move to the desktop, considering laptop no go. RIP
Installation - desktop
I booted off USB stick for desktop, but ran into same issue. It wouldnt detect hard drive in AHCI mode, but it did report an error. The error in dmesg which seemed to indicate hardware failure:
ahcisata0 channel 0: clearing WDCTL_RST failed for drive 0
I was very skeptical because this machine ran windows 10, Linux, other BSDs without issue right before this. I clicked it over to 'IDE' mode instead and it detected drive and started the installer. Smooth sailing from here right?
I got a panic during auto configuration of a wifi card (which, by the way it detected no problem, go figure). I tried to reproduce it to report in GNATS but had no luck, so I just started over and skipped network configuration during installation.
At this point I had a working NetBSD 7.1.2 installation on the desktop. So at this point, I have NetBSD booting but unconfigured. I could technically move to configuration and usage at this point if I wanted. On fast hardware to boot!
Run current sources to test things
Bringing up some of the panics and really obnoxious issues, I looked into reporting/debugging them. I was told to try again on 'current' in NetBSD parlance, working sources. Which of course, is a good idea in case it has been fixed, and to ensure it is reproducible.
So I grabbed a recent snapshot of NetBSD 8-current and tried that installer. Mostly the same story there, no working AHCI.
One big thing I want to capture somewhere is that the installer failed to run due to some drivers. You'll get a panic something like this booting current:
After trying this tip from IRC:
14:02:38 <medfly_> you can drop to boot prompt 14:02:41 <medfly_> userconf disable i915* 14:02:44 <medfly_> userconf disable radeondrm* 14:02:47 <medfly_> userconf disable nouveau* 14:02:48 <medfly_> boot
Which by the way, the IRC channel (#NetBSD on freenode) had a lot of very patient and helpful people. The machine has an nvidia 780Ti so i try the last one,
userconf disable nouveau* and boot. Bam Current installer off to the races!
I was able to install current at this point. One thing I need to figure out is how to make disabling nouveau permanent because I have to drop to boot prompt every reboot to do so. I could now use the machine. I'll probably just ssh into it moving forward.
After doing some cross-checking, the old intel SSD in the desktop is in fact actually starting to fail just now! This is no surprise considering its from 2008. With that in mind, I'll consider it non usable even tho it's running current doing not much right now.
I decided to just learn how to use the system in VM on my main desktop in the meantime. NetBSD mostly works great on the emulated hardware, and my primary desktop is an extremely fast machine which can build NetBSD in minutes. My beefy desktop has a 4k monitor, if you use it on 4k or similar make sure to set display scaling in virtualbox or else you will go blind:
Eventually, I'll move some stuff around and put a good hard drive in the spare desktop and SSH into it. Now I can dig into usage, maybe run and build some software on it for starters.
NetBSD is very likely going to be difficult to install. Don't think amd64 will be smooth sailing just because its the comptemporary hardware of choice. While it's likely got the best coverage there are lots of open issues and lacking support for newer things like GPT/UEFI boot.
At some points I figured hacking custom u-boot code to get NetBSD to boot on a SBC would have been easier in some ways. But we'll see about that later.
Once it gets going, its pretty good. it's the consistent, rock solid UNIX experience I was hoping for. Coming from someone who primarily uses windows, but knows the basics of linux server administration, the UNIX experience equals I have no idea how to do anything in an otherwise familiar environment. I really like the simple rc.conf and rc.d init system and the sane directory hierarchies for third party software so far.