Graphing SSH dictionary attacks with HighCharts

After my 10-year-old basement Linux server died this week from a power outage, I took the sad step of giving up on it. It’s died before and I’ve patched it back together with a new power supply here or an addon PCI SATA card there, but I finally decided to throw in the towel since I had a newer old computer that had been idle for several years. The one that died was an Athlon K7 750 MHz with 512 MB ram. The new one is an Athlon 2 GHz (3200+) with 1 gig. For my uses, specs don’t really matter that much, but it’s nice to have more power for free.

I put CentOS 6 on it and configured Samba and copied all the data off the old machine and was back up and running within a few hours. Since I forward ports through my FiOS router to this box I did my standard lockdown procedure, including adding myself to the AllowUsers in sshd_config. Afterwards I took a look in /var/log/secure and saw the typical flood of dictionary attacks trying to get in as root or bob or tfeldman or jweisz. I have iptables configured to rate-limit SSH connections to 2 per 5 seconds per IP so the box doesn’t get DoSed out of existence, but some stuff does make it through to sshd.

Looking through /var/log/secure, I got to thinking it would be interesting if there was some way to visualize the attacks in a handy graph. Then I remembered, oh, wait, I can do that.

I wrote a perl script to parse out the attacks from /var/log/secure and insert them into a Postgres DB. This turned out to be pretty easy. Then I thought it would be more interesting to tie the IP of each attack to its originating country. I’ve used MaxMind’s GeoIP DB pretty extensively before, but I was looking something free. That’s when I remembered that MaxMind has a free GeoIP DB: GeoLiteCity. I grabbed it and yum-installed the Perl lib and added the geo data to the attack DB. Rather than worry about normalizing the schema I just shoved the info into the same table. Life is easier this way, and it’s just a for-fun project.

So I got that all working and parsed it against the existing /var/log/secures via

[root@lunix2011 ~]# zcat /var/log/secure-20111117.gz | perl 

I wrote ssh.php to see what’s in the table:

ssh.php list of hacking attempts
ssh.php list of hacking attempts

So now that the data was all in place, time to move on to the graphs, which is what I really wanted to do. Last time I wanted to graph data programmatically I used JPGraph, which does everything in PHP and is super versatile. But I wanted something… cooler. Maybe something interactive. A little Googling turned up Highcharts which is absolutely awesome, and does everything in JavaScript. I basically modified some of their example charts and pumped my data into them and got the charts below.

Pie chart of attacks grouped by country for the past 30 days:

Pie chart by country
Pie chart by country

Bar graph of attacks per day:

Bar graph of daily attacks
Bar graph of daily attacks

So, that’s that. Code is in github if anyone wants to play around with it. I’ve cronned to run every 5 minutes so the data gets updated automatically.


Teaching myself node.js: Part 3

I’ve been playing around with node, JavaScript & MongoDB a bit with this project and one of the things I ran into was MongoDB’s lack of a sequential ID. MongoDB defaults to ObjectIDs for primary keys in collections, and they have good reasons for doing so, but in writing this app, I’d rather have URLs look like “/question/423” than “/question/3001024e521e9c6500000000”.

Fortunately, there’s a relatively convenient workaround for this problem, detailed here. Essentially, create a collection in which each document is a counter. The ID for the document is its “name” and the property “next” contains the next value. This is similar to creating and using a sequence in PostgreSQL, though hopefully in the future they’ll provide an easier way to do this – perhaps a built-in sequence object.

First, in the mongo shell, I created the counter(name) function as specified in the Mongo wiki:

> counter
function counter(name) {
    var ret = db.counters.findAndModify({query:{_id:name}, update:{$inc:{next:1}}, 'new':true, upsert:true});

Then I created the counter document for the “questions” collection (so questions can have numeric IDs):

db.counters.insert({_id:"questions", next: 1});

This is optional – the counter() function will create the document in the counters collection if needed, with an initial value of 1.

That’s it. Each call to counter() will now return an incrementing number:

> db.counters.find();
{ "_id" : "questions", "next" : 3 }
> counter("joe");
> db.counters.find();
{ "_id" : "questions", "next" : 3 }
{ "_id" : "joe", "next" : 1 }
> counter("joe");
> counter("joe");

Cool. So how do we save the counter() function in the DB so we can use it in queries without having to define it every time? Fortunately Mongo makes saving functions on the server easy:

> { _id : "counter" , value : counter });
> db.system.js.find();
{ "_id" : "counter", "value" : function cf__2__f_counter(name) {
    var ret = db.counters.findAndModify({query:{_id:name}, update:{$inc:{next:1}}, 'new':true, upsert:true});
} }

Calling the stored procedure directly in the shell is sort of strange, but it works:

> db.eval("return counter('question')");
> db.eval("return counter('question')");
> bye
[Wed Jun 22 13:14:10 evan@EvanMBP 3 ~]$ ~/Downloads/mongodb-osx-x86_64-1.8.1/bin/mongo questionsMongoDB shell version: 1.8.1
connecting to: questions
> db.eval("return counter('question')");

Unfortunately, I was unable to figure out how to call the stored function from within the mongoskin driver, and ended up writing analogous code, calling findAndModify() within the app code. I finally got it:

                {$inc : {next: 1}},
                function(err, counter) {
                        if (err) { throw new Error(err); }
                        var ins = { date: new Date(),
                                author:, body: req.body.body,
                                tags: tags, tag_count: tagCount,
                                answers: [], votes: 0,
                        db.collection('questions').insert(ins, {});
                        res.end('Added new question: '+req.body.body);

Question detail
Question detail

I tried throwing some code in there to query by either ObjectId() or integer but it didn’t really work (Edit: I tried this again on Linux and it worked fine), so I just deleted all the documents with ObjectIds and everything seems to work fine now. I think the next step is going to have to be some sort of authentication/session stuff, because typing your name in every time kind of sucks (and makes it hard to do things like list questions by user).

Full code for this revision is here.

JavaScript regex for stripping leading & trailing whitespaces

I’m sure there are a bunch of libraries that do this, but sometimes wrestling with regexes is fun.

> var str = "           This string has some mighty fine whitespace.       ";
> str
'           This string has some mighty fine whitespace.       '
> str.replace(/^(\s*)((\S+\s*?)*)(\s*)$/,"$2");
'This string has some mighty fine whitespace.'

Teaching myself node.js: Part 2

I did a little more work today on my little node.js project and added the ability to click on a tag and have the app list all questions with that tag, and a super basic method for adding new questions. Neither of these were really complicated; the most “challenging” part was figuring out how to create links and form inputs with jade. In the end I was surprised at how easy it really was.

For the tag search I just copied the app.get('/questions') route to a new app.get('/tags/:tag') route in app.js and modified the mongo find() query to search for the tag. I created a new Jade template for this, but I suppose the same list.jade template would have worked, since the only thing that was changed was the contents of the questions array.

For adding a new question I created an app.get('/question/new') route that loaded a jade template with the form (addquestion.jade), and which did a POST back to /question and the corresponding'/question'). Right now the form just takes text input fields, but eventually there’ll need to be some notion of users, and other fun functionality. Also after doing the insert into Mongo, it just spits back some plaintext response, but it gets the job done.

I should add that this is my first time writing a webapp using “routes.” It’s been a while since I really did any front-end coding at all and I’m used to the “old school” method of one script per function, where the “add a new question” function would be handled by “addQuestion.php” for instance. Using routes and MVC are part of what I’m trying to learn in doing this.

Commit for this version is here

Teaching myself node.js: Part 1

I’ve been meaning for a while to learn node.js. I read a couple of books and plenty of blog posts but in the end I find, as usual, I can only learn by doing. I have most of the fundamentals to get started, I just need to have a project. The typical tutorials I’ve found focus on creating a blog; I’m thinking of something more along the lines of a Quora question/answer site. I’ll be trying to write this as I go, which will hopefully prod me into actually doing it this time. (I have a tendency to bounce around different tenses and I expect this will be worse than usual – stuff will likely be written in past/present/future at various points).
Continue reading “Teaching myself node.js: Part 1”

JavaScript: The Good Parts

I just finished reading JavaScript: The Good Parts, one of the best programming books I’ve read. The ending is fantastic:

We see a lot of feature-driven product design in which the cost of features is not properly accounted. Features can have a negative value to consumers because they make the products more difficult to understand and use. We are finding that people like products that just work. It turns out that designs that just work are much harder to produce than designs that assemble long lists of features.

Features have a specification cost, a design cost, and a development cost. There is a testing cost and a reliability cost. The more features there are, the more likely one will develop problems or will interact badly with another. In software systems, there is a storage cost, which was becoming negligible, but in mobile applications is becoming significant again. There are ascending performance costs because Moore’s Law doesn’t apply to batteries.

Features have a documentation cost. Every feature adds pages to the manual, increasing training costs. Features that offer value to a minority of users impose a cost on all users. So, in designing products and programming languages, we want to get the core features—the good parts—right because that is where we create most of the value.

We all find the good parts in the products that we use. We value simplicity, and when simplicity isn’t offered to us, we make it ourselves. My microwave oven has tons of features, but the only ones I use are cook and the clock. And setting the clock is a struggle. We cope with the complexity of feature-driven design by finding and sticking with the good parts.

It would be nice if products and programming languages were designed to have only good parts.

Continue reading “JavaScript: The Good Parts”