Archive for July, 2009

Best newspaper article ever!

Keri Rand has written a good article that has been published in the Advertiser News, one of the regional papers where I live. She sent out a questionnaire via facebook message to a bunch of people, and I was one who responded. She decided to use my input, and now I’m featured on their website.

I may sound like an absolute tool, but I do like the one quote that was made. They highlighted it in the actual paper printing, but not online. Quote:

“This recession has sort of made me go ‘back to the basics’ (by American standards), and appreciate the simpler things in life a whole lot more. I think it’s also impacted my health positively as well – I cook now instead of buying prepared foods, I exercise more for lack of anything ‘more exciting’ to do. Honestly, I kind of like the new lifestyle I’ve adopted because of it.”

So what does everyone else here think, has the latest economic recession affected you enough to make some lifestyle changes? It is an interesting thing to consider.

Link: [Teens and 20-somethings reflect on the state of economy]

A better Nagios SNMP plugin

The nagios plugin that you find in the package nagios-plugins-snmp was insufficient for my needs in a new nagios deployment. The biggest reason that it was insufficient was that it gets integer values, and then can only issue a warning or alert if that integer value is GREATER than what value you gave it. The deployment I’m setting up required that the values could be checked against a range, which is necessary when receiving SNMP data from a thermometer or hygrometer. This plugin supports receiving a range of values to check against, and then it returns the appropriate exit code.

I’ve written a tad bit of documentation in the top of the file, but here it is again, in block quotes! Download link is just below the block quote area.

# This script written with haste by Benjamin Rose, July 8th 2009 @ 11:45:42 AM
# It was written because the check_snmp plugin provided by the nagios package
# does not support range matching. It can check if the snmp value is greater than
# a given number but not less than, nor a range consisting of either a high value
# or a low value. Hence, this script, given a mode of 1 and a good range with
# which to work, will report back appropriately.
# Modes:
# 1 = Number comparison, reports on a given range. Argument order given in
# the usage statement.
# 2 = String comparison, which for now is just “Open” or otherwise.
# 1) Change the order of the arguments, putting mode in front of the
# variables, and then change the usage based on the given mode.
# 2) Allow the user to configure which strings are “good” and which are “bad”.

Plugin link: [snmp_plugin_wrapper]

64-bit Adobe Flash Player

It exists! It’s not in the Adobe yum repos yet, as it’s still an alpha build, but you can download it here:

[Native Linux 64-bit Flash Player from Adobe Labs]

It works very well. No longer do you need to get nspluginwrapper working, and then configure pulseaudio (if that’s your distro) to listen to 32-bit apps, blah blah blah. Simply unpackage, throw the .so file into the firefox plugins directory, and restart firefox. Great stuff. I’ve also noticed there’s a performance gain to be seen, my flash videos used to lag and flicker on youtube when I made them fullscreen, and this doesn’t happen anymore. Sound worked out of the box with pulse.

And the best part, although it’s supposedly an alpha build, it crashes a heck of a lot less than it used to. No more npviewer.bin segfaults in the /var/log/messages file! My guess is that this is because of the lack of a hacked wrapper. But, that being said, if it crashes on you a lot after trying it, don’t be too upset. At least they’re trying.

I’m the fool. I took the plunge this week and installed a fresh clean copy of Fedora 11, but I made my root partition btrfs instead of something more… well, for lack of better words, stable and mature. This should serve as a little documentation of my findings, in the hopes that others don’t make the hasty call to dedicate their root partition to the new filesystem.

From the dawn of my days with Linux, I have been told to heed warnings that this new system gives. This isn’t Windows any more, the warnings that you receive are no longer benign. My first experience with this was when I had a hard drive that was dying, so I decided to do a fsck. The only problem was that I did a fsck on the root partition – while it was still mounted — read/write. In all fairness, it was my own arrogance that made me add the flag to manually override that protection system. I figured, “what could get worse? The drive is already dying”. I did this with no backups. Three days and 20-something manual superblock hex-edits later, I finally got my data back. The valuable thing from that incident is the experience, though. I now knew how to do manual superblock recreations, which always comes in super handy. I also had learned to heed the warnings of the core developers very well.

Until Saturday. My friend had told me before about how funny the new anaconda argument is: “icantbelieveitsnotbtr”. And now who says these hardcore developers don’t have a sense of humor? Well, I’ve learned a heck of a lot over these last few years, and sometimes I even feel guilty that I don’t give back as much as I should to the open source community. So I figured I’d give their hard work a shot. I’ve heard nothing but glowing reviews about ZFS, and btrfs is supposed to be the up-and-coming big thing in the filesystem world. I figured, at the very least, I could do some benchmarks or something.

So here goes nothing…

I fired up the installer, adding the appropriate flags. I had /dev/sda1 already formatted 50 gig ntfs for Windows. This was to be a dual-booting system. I added a 512-meg /boot, sda2, and a 2-gig swap, sda3. sda4 was to be a pointer to an extended partition table, where sda5 resides, filling up all of the remaining space of 59 gigs. I picked my packages and started the installer. Uh oh, the format of sda5 as btrfs failed. Oh well. Reboot, repeat. Same results, but this time it just died with some non-helpful error code 1. Uhhh. So I tried to boot again. My partitioning table was simply gone. WTF.

I remade the partition table, and figured, “third time’s the charm!”. I selected btrfs as the filesystem for /. If this didn’t work, I was going to give up and use the (now-default) ext4 filesystem. Alas, it worked. I was actually looking forward to using the new filesystem. The install finished normally, and I booted into my new Linux system.

First thing’s first, I ran a full yum upgrade. I noticed something just then. The seek times with btrfs were amazing, read speeds were okay, but the write speeds were flat-out terrible. I’m guessing that’s simply because of all of the metadata btrfs stores, but I honestly have no idea why that is. Oh well, I do far more reads than writes, so this works out well. I then proceeded to use the system for a while. After my used disk space started to get above 25%, I started to get really unexpected results – kernel failures, corrupted filesystem blocks, half-missing extent entries, system bus errors, the whole nine yards.

I did the next logical thing, I strace’d the processes that were dying unexpectedly on me. Worst of all was yum, it would just die with a “Bus Error”. Strace revealed it was hitting massive ENOSPC (no space left) errors trying to open files on the hard drive. df reported the filesystem as exactly 27% full. A google search revealed this as a known bug, but no one else has really had the problem until their filesystem reached over 75% full. So why was mine being the opposite? I theorized that it may have been because of loads and loads of useless metadata across the hard drive. I rebooted (the only thing that fixed the ENOSPC errors) and installed the btrfs-progs package. This package includes userspace utilities needed to create, check, modify and correct any inconsistencies in the btrfs filesystem. I ran a btrfsck, which reported no problems. I read that it may be useful to re-examine my metadata by forcing a balance of it across all drives, even though it only existed on one partition. This is done with the command “btrfs-vol -b /”. Kicker – this operation, after about 5 minutes of heavily accessing the drive, segfaulted. I was left with a half-balanced btrfs system, ENOSPC errors everywhere, and a feeling like this was the beginning of the end.

I didn’t quite want to give up yet, but I also wanted a usable machine. I had read somewhere that kernel 2.6.30 had an insane number of btrfs improvements. On a personal wish, I downloaded the kernel source for 2.6.31 and compiled. Booting into the new kernel was like a breath of fresh air.

SIDE NOTE: kernel > 2.6.30 is awesome. readahead is now merged into the kernel source for MUCH faster booting. There have been a ludicrous amount of improvements, check the changelog on for a good list of these changes.

Anyway, the new kernel loaded up just fine. The errors held off for a bit, long enough to make me think they were gone. But then they came back full-force. The balance segfaulted again. The fsck said nothing was wrong. No one had any idea what was going on. I do like to help the open source community, but I simply don’t have time at the present to dig through hundreds of thousands of C code source lines in order to fix a bug this serious. After filing 4 bug reports, submitting probably close to 50 kernel errors, and genuinely giving it my best college try, I was done with btrfs. I rebooted into anaconda and formatted my system as ext4 and went along my way. Yes, I know you can *theoretically* convert a btrfs volume to ext4, but with the structure so far destroyed, I didn’t even bother wasting my time.

Moral? I guess just know what you’re getting into if you decide to give btrfs a try. When the kernel developers say that a feature is experimental, take that literally. There is no warning here, only some sound advice – proceed at your own risk, make plenty of backups, and just generally don’t expect too much of the filesystem at this early stage of it’s development.