Simple tools produce powerful results

Tools

Sometimes I really love the Linux world. Sometimes it all fits so neatly together and lets you just get stuff done. Sure it might take a bit of digging around, but when it all comes together it really is beautiful!

I’m waxing lyrical today because I have just managed to fit all the pieces together for a certain aspect of a large project we’re working here at ibrow towers. This particular problem was a three parter:

  1. Users can FTP into a server into their own directory.
  2. These users should be created automatically (i.e. sign up on site, get details, ftp) and have the same login credentials as the rest of the site
  3. Uploads to the FTP server should be monitored and each file processed

To solve this three part problem, I utilised the following:

  • VSFTPD FTP Server using MySQL as the credentials database
  • incron as the directory monitoring
  • bash as the glue

My server is running Debian, and I found the on Falko Timme tutorial for VSFTPD and MySQL at How To Forge incredibly easy to follow and it worked pretty much “out of the box”. It did take a bit of tweaking, but I was able to use my existing data structure to be used for the FTP login credentials.

The next task was directory monitoring. After some searching around I found incron. On a Debian system installation was a breeze:

$ sudo apt-get install incron

Now, for my VSFTP user I had to create an incrontab. First I had to add the user into the incron.conf file:

$ sudo nano /etc/incron.allow

This was blank, so I just added the line:

vsftpd

No to test this was working I created a small bash script that esentially appends to a text file. I created this in /home/vsftpd/bin/incron.sh

!/bin/bash
echo "start: $1 || $2" >> /home/vsftpd/bin/log.txt

The $1 and $2 variable are to store command line arguments which I’ll pass via the incrontab. Make this executable and quickly test this is working:

$ chmod 0755 /home/vsftpd/bin/incron.sh
$ /home/vsftpd/bin/incron.sh test hello
$ cat /home/vsftpd/bin/log.txt
start: test || hello

The test script is all working so now we need to call it whenever a file is uploaded. You need to edit the incrontab passing it the directory to monitor, which actions to look out for and then what script to call, passing any arguments.

$ incrontab -e

You’ll be taken to an editor where you should enter something like:

/home/vsftpd/cam1/ IN_CLOSE_WRITE /home/vsftpd/bin/incron.sh 123 $@$#

The IN_CLOSE_WRITE basically means, “once finished uploading”. “123″ is the user id, as I didn’t want the directory name to reflect the user id. Finally the $@$# is translated into the full path of the directory (as typed by you, so keep an eye on trailing slashes) and the file name that was caught by the incron, e.g: /home/vsftpd/cam1/my-file.txt

Now simply test this by FTPing and uploading something to the user’s directory. It’s always fun to do:

$ tail -f /home/vsftpd/bin/log.txt

to watch the log being updated in real time. Now that’s working you can change the incron.sh script to do anything you want!

The final thing to do is to automate the creation of the users’ FTP directories. When adding the directories we also want to ensure that they are correctly monitored. To solve this I wrote a small bash script that is saved on the FTP server, /home/vsftpd/bin/add-user.sh:

#!/bin/bash
 
# user id = $1
# user directory = $2
 
# Create the directory
mkdir /home/vsftpd/$2
 
# Add monitoring
incrontab -l > /tmp/$1.in
echo "/home/vsftpd/$2/ IN_CLOSE_WRITE /home/vsftpd/bin/incron.sh $1 $@$#" >> /tmp/$1.in
incrontab /tmp/$1.in
rm /tmp/$1.in

The important part of this is that we keep monitoring intact. And this is one of the reasons why I love Linux so much. By using incrontab -l to send the output to a tmp file, then echoing the new incrontab line to append that tmp file, then finally calling incrontab again, this time passing the tmp file we have just added a new directory to monitor, whilst keeping all the existing ones! Simple, elegant and it just works. One thing to not here is to make sure you escape the $@ and $# variables in the echo.

Make this executable:

$ chmod 0755 /home/vsftpd/bin/add-user.sh

Test this on the server:

$ /home/vsftpd/bin/add-user.sh 456 testdir

and you should see that the testdir has been created and monitored, associating with the user id 456. This script should be invoked during the user creation phase in your web app.

That’s it!

Through using bash, VSFTPD with MySQL credentials and the incrontab program I can now automatically create a new user, keeping it linked in with my website database, create the user’s FTP directory and monitor it for any uploads.

Using some faily simple tools together (here ftp, bash and incron) you can create some very powerful solutions.