rgrep on windows 7 for emacs

I lost an hour configuring this, so seems worthy of a blog post.

A fresh install of emacs for windows will have functionality that does not work because it depends on unix style utilities.

One very useful example is the command rgrep, which searches files recursively through subfolders looking for a regular expression in those files.

Under the hood it uses the unix command line tools find, and egrep. Unfortunately the windows version of find takes entirely different parameters and will not function. In fact you will get an error that looks like this:

FIND: Wrong parameter format

Using the set of native ports of Unix command line tools UnxUtils you can easily fix this:

1. Download the zip file and extract it to c:\unxutils
2. Add the following path to the very front of your path by editing your system environment variables C:\unxutils\usr\local\wbin\;

That's it. You can run rgrep now and hopefully you're up and running.

If not make sure you have restarted emacs so it picks up the new setting of PATH. Open a shell in emacs and type 'find --version'. You should see something like this if your path is configured correctly:

c:\find --version

find --version

GNU find version 4.1

and if not you will see:

C:\Windows\system32>find --help

FIND: Parameter format not correct

Couple of mysql tips

Wondering what the heck is going on on your mysql DB?

show processlist ;

Shows you all the active threads, which is very handy indeed. It shows what each thread is doing, and how long it has been doing it for.

Thread safe record insertion

When doing a record insert, another thread could be trying to insert a record that would violate any DB contraints, and if it gets there before you, then your insert will fail.

There may be cases where you want don't want to have it fail and try again; you may want to take a different action if the record already exists. For example lets say you have some data like this, where the Name is a unique key...

Name, Number of products purchase

Bob, 1,

Ben, 2,

Now if two threads try to insert a user 'Roger' who has just bought an item, then the one that arrives at the server last will fail.

INSERT INTO purchase (Name, PurchaseCount) VALUES ('Roger', '1') ON DUPLICATE KEY UPDATE PurchaseCount=PurchaseCount+1;

This will handle the key collision and execute the update clause instead, in the event that somebody beat you to it. This makes it much easier to write thread safe code that adds records.

Adjusting server logging level at runtime

log4j is an awesome logging library for Java. One thing I found out today is that you can change the logging level (debug, warn, info) at runtime, with a few lines of code.

static
{
       // This monitors the log4j for changes over a specified period of milliseconds
       PropertyConfigurator.configureAndWatch("./resources/log4j.properties", 60000);
}

What this function does is to tell log4j to check the configuration file every minute (60000 ms) for changes. So you can deploy a production server with only warn and error logging, but if something goes wrong you can enable debug and info logging (for example) simply by editing the configuration file and waiting a minute.

(source code formatted by this handy website http://formatmysourcecode.blogspot.com)

Growling Mac

(pic from flickr)

Last year I wrote a blog post about how to make growl notifications (little pop ups driven by the application Growl) appear from emacs in windows. Since then I've become a total Mac-head. Here's an update on how you get that working from emacs on a mac...

Everything is the same except you need to install Growl for Mac (obv) and also make sure you copy the folder called Extras somewhere.

Open up a terminal and then cd to where you put the Extras directory. Find a sub-directory called growlnotify. Enter that and run 'sudo ./install.sh' which will prompt you for your admin password, and then go ahead and install growlnotify.

You can now run growlnotify from a terminal window. Try it out, if it doesn't work you won't be able to growl from emacs.

Now follow the instructions in my blog post above but ignore all the windows parts. You'll need to customize the growl command (M-x customize group, todochiku) and replace the full path I used for windows with just 'growlnotify'.

If everything is cool you can now do M-x todochiku-in and type a message, a time in minutes (zero for now), and pop, you should have a notification.

Using tar and gzip

Picture from flickr by Windwirral

Here are the basic steps to use tar and gzip to package up some files in a compressed archive, so that you can move them to another computer or back them up.

First you want to create and add files to the archive. Go to the target directory and enter individual filenames and directories you want to have included.

tar -vzcf mystuff.tar.gz file1.txt file2.doc mypictures/

Now you can test if that worked by listing the files in the archive.

tar -tvf mystuff.tar.gz

When you want to extract the files, go to the folder where you want to extract them, in this case 'targetfolder'.

cd targetfolder

tar -vxf mystuff.tar.gz

Nice Code

Just wanted to point out a couple of very nice pieces of source code I came across recently.

ipin.py

Firstly check out this Python code ipin.py, by Axel E. Brzostowski, which converts png files from an iphone application into a format that you can read on any computer.

The python code is clean, easy to read, almost like literate programming. Very useful if you want to write a similar application that processes every file in a directory.

transmission

Another really nice program, is really a website. When you run transmission-daemon (linux bittorrent program), you can connect to it with a command line app called transmission-remote. In addition you can connect via a built in web server, which is a really excellently designed application. The html and javascript is so well documented and tidy it's as beautiful as the actual web page. Check it out here.

http://trac.transmissionbt.com/browser/trunk/web

Securing an ssh server with fail2ban

There are all kinds of ways to secure an ssh server, with varying degrees of increasing security and decreasing flexibility. For example by limiting your server to only accept connections from certain known IP's, you are secure from random hackers on the internet, but you lose the ability to connect to your machine from anywhere you want to. Perhaps while travelling, for example.

Port knocking and listening on a high numbered non-standard port. make it harder for an attacker to even start trying to hack your connection. But this also requires you connect with a machine that you have the knock program installed on. Again, less convenient, more secure.

However, once an attacker does find your port there's nothing to stop brute force password hacking. If you look in your log file, you should see people connecting to your ssh port quite frequently and trying password attacks.

cat /var/log/auth.log

If you have a secure password then it would require days of brute force hacking to gain access to your ssh account, but even so, if you don't watch your logs then it's perfectly possible somebody will gain access eventually.

Brute force attacks can be limited using fail2ban. There's a great article on setting it up here. This program will scan your auth.log for you, using a regular expression to find failed password attempts. On a specified number of failures from a given IP, it will then modify the iptables on your machine (the firewall), to lock that IP out for a specified time.

Now instead of watching your auth.log fill up with reams of failed passwords, you'll see a greatly reduced amount of brute force attacks, and you can watch your fail2ban log file fill up with the IP addresses of hackers.