NodeJS: Experiments with Middle End Part 3

In this post, we’re going to make that example slightly more complex by introducing dependencies within the modules. Now the code must be able to handle these dependencies for both the client side and the server side.

In the last post, we built a simple module consisting of a single function that returned a greeting:

var Greetings = function() { }
Greetings.prototype.hello = function(who) {
   return "Hello "+who;
}
module.exports = Greetings;

At this point, it would be very useful to sanitize and validate the parameter passed to the “hello” function. We could obviously build this functionality directly into the code, but this is exactly the sort of thing that should be abstracted into its own module(s).

For the purpose of this example, we’ll simply include the functionality to give the parameter passed a “trim”; after some hacking and snippet-borrowing from node-validator by Chris O’Hara, we can build another simple module called Tidy:

var Tidy = {
   whitespace: '\\r\\n\\t\\s',
 
   trim: function(str) {
      var whitespace =  '\\r\\n\\t\\s';
      str = str.replace(new RegExp('^['+this.whitespace+']+|['+this.whitespace+']+$', 'g'), '')
      return str;
   }
}
module.exports = Tidy;

We can now improve our Greetings module, including the dependency and functionality:

var Tidy = require('./tidy');
 
var Greetings = function() { }
Greetings.prototype.hello = function(who) {
   who = Tidy.trim(who);
   return this.hello(who);
}
module.exports = Greetings;

To see if this will work as expected, let’s add some spaces to the parameter passed in the NodeJS script:

var Greetings = require('./greetings');
var greetings = new Greetings();
console.log(greetings.hello('  Node   ')+"!");

When we run it, this is what we see:

$ node runner.js
Hello Node!

Good, so this works on the server side, but let’s test its behaviour on the client side. Again, we should add some spaces to the parameter passed to ensure that it is working:

<script type="text/javascript"> </script>
<script src="greetings.js" type="text/javascript"> </script>
<script type="text/javascript"> </script>

Unfortunately, we now see the following error:

require is not defined
[Break On This Error] var greetings = new Greetings();

Our attempt to “require” the dependency in the Greetings module has broken the client side. To fix this, we need to define our own require() function, much like we had to for the module object. We do this by creating a new front end-only script, which we’ll call “middle-end.js”.

var require = function(module) {
   document.write('<script src="'+module+'.js" type="text/javascript"> </script>');
}
 
var module = {
   exports: undefined
};

(Obviously we should include much more error checking, but this will be enough for our example).

We modify our html as follows:

<script src="middle-end.js" type="text/javascript"> </script>
<script type="text/javascript"> </script>
....

Now when we reload the page in the browser, all should work as expected. Huzzah!

This is just a very simple example of what can be done when JavaScript code is shared between the client and server sides—the “Middle End”. This concept becomes really exciting when you start thinking about potential approaches to form validation, session management, and even display logic. This is definitely something worth playing around with.

All of the code is available as a Gist.

NodeJS: Experiments with Middle End Part 2

In my previous post, I talked about Kyle Simpson’s concept of a Middle End—a layer between the front end and back end. Today, I’m going to sketch out my very simple experiments with constructing such a layer.

What I am focusing on here is simply a means of sharing JavaScript code between the client side and the server side.

Let’s start with a very simple NodeJS Module, greetings.js:

var Greetings = function() { }
Greetings.prototype.hello = function(who) {
   return "Hello "+who;
}
module.exports = Greetings;

Next, let’s create a very simple NodeJS script that imports this module, and uses it:

var Greetings = require('./greetings');
var greetings = new Greetings();
console.log(greetings.hello('Node')+"!");

When we run it, this is what we get:

$ node test.js
Hello Node!

All very simple stuff.

Now, suppose we want to use the Greetings module in our front end web app. 

<script src="greetings.js" type="text/javascript"> </script>

Well, it runs and we get the desired output. However, Firebug reports an error within the greetings.js file:

module is not defined
[Break On This Error] module.exports = Greetings;

The solution to this issue is to “mock up” the module object so the error isn’t thrown:

var module = {
   exports: undefined
};

Run it again, and now we see no errors.

Great, so it’s working. We can share simple modules between the client and the server sides.

But what if we want to have a more complex module? One that has some dependencies?

That will be discussed in the next post.

All of the code is available as a Gist.

NodeJS: Experiments with Processes

I’ve been hacking about with NodeJS a lot recently, and lately my attention has turned towards spawning child processes. Luckily NodeJS makes this very easy to do, and also the documentation is pretty good.

What I have been working on is having a single parent process that runs constantly in the background, and this parent is responsible for child processes which actually do the work. One thing I found particularly interesting was how the parent can handle a child crashing, and also what happens to the children when the parent is killed. Below is the culmination of my hackings, it comprises of two simple scripts, a child script, and a parent. I have committed them to my GitHub account, so feel free to clone, fork, hack or whatever.

Read the rest of this entry »

Amazon S3: easily bulk delete buckets

With Jabbakam we save a lot of images. All these images are stored in Amazon’s S3. Recently I thought I’d delete some of the test buckets.

Not as easy as I first thought. It turns out that you have to empty buckets before you can delete them. Fair enough, but there didn’t seem to be a way to easily bulk delete hundreds of images.

After some Googling I found Robert LaThanh’s S3Nukem which looked like it would do the job. All I needed to do now was fire up an EC2 instance, install and run. The steps were as follows:

1. Create an EC2 instance on AWS (I used Ubuntu on a medium instance)
2. make sure it is all up-to-date

   apt-get update
   apt-get upgrade

3. install ruby (if not already installed)

   apt-get install ruby1.8 ruby1.8-dev

   (not sure about ruby1.8-dev, but added just in case)

4. install Ruby Gems

   apt-get install rubygems

5. install right_aws and s3nukem

   gem install right_aws
   cd /tmp
   wget http://github.com/lathanh/s3nukem/raw/master/s3nukem --no-check-certificate
   chmod +x s3nukem

6. run

   ./s3nukem -t 20 -a  -s

Several hours and over 15 million deleted images later it was all done and I shut down the instance.

Awesome WM Clicky Widget

As some of you know, whilst Emacs is my editor (et al.) of choice my favourite window manger is Awesome. Awesome is, according to the website,

.. extremely fast, small, dynamic and heavily extensible using the Lua programming language

I’ve never delved into Lua before, but since using Awesome I’ve had to play around with a bunch of config files. They have always looked very clean and simple so, this weekend, with a spare couple of hours to kill I thought I’d have a play around with Lua and see if Awesome really is as “heavily extensible” as the claims make out.

A couple of weeks ago I set up real time web analytics on a number of sites with Clicky. Clicky have an incredibly simple API which I thought would be perfect to play around with.

I decided to try and build a widget that displays the total visitors and actions for a specified website. Really simple, but also quite useful. I had never written anything in Lua before, or even played around with widgets too much in Awesome. Fortunately both Awesome and Lua made the whole thing fantastically easy and fun. What follows is the code I wrote, use it if you want to, it’s just an example of what I did with my first attempt, so feel free to extend it as you want.

Read the rest of this entry »

NodeJS Tutorial – Part 2, Routing

In the first part of this tutorial we looked at reading a document from CouchDB and keeping our markup separate from our code using Haml templates. We arrived at the point where we clicked on a link, but nothing happened. Today we are going to put that right with some very simple routing.

The URL Module

The basis of all routing is the URL. Fortunately, Node has some basic functions which handle reading the URL so we can then decide what the user is actually trying to do.

First, we want to make sure we have included the URL module.

var sys = require('sys'),
fs = require('fs'),
http = require('http'),
url = require('url');

The next step is to extrapolate the path name from the URL that the user has visited:

Read the rest of this entry »

NodeJS Tutorial with CouchDB and Haml – ErdNodeFlips

The aim of this tutorial is to create a Node powered web app using a CouchDB database and Haml based templates. I wanted something relatively simple that I could do in a couple of sittings, but complex enough to be of some actual use. In the end I opted for a “top 10 list” type application. The functionality being as follows:

1. To list a number of items
2. To remove an item from the list
3. To add an item to the list

Obviously, before getting cracking with this tutorial I had to think up a name for this “killer app” of the Node world.

So I give you ErdNodeFlips! In honour of erdnussflips – the strangely addictive Wotsit type things that taste of peanut butter and stick to the top of your mouth. Yum! (OMG – they even have their own Facebook page!)

All code is available on GitHub.

Read the rest of this entry »

jQuery Validate URL, adding “http://”

A project we have going on here at ibrow towers involves quite a bit of jQuery Validation. Whilst the Validation plugin is great with the default functions it offers, what makes it truly excellent is the fact that it is easily extendable.

For example, one thing that has been bugging me is to validate a URL the user must type in the initial http://. But why? Can the validation not just assume that if, for example, www.robsearles.com is typed in, that the user actually meant http://www.robsearles.com?

Well, after a quick read of the documentation and a hack around this little irritant is now gone. It’s all down to addMethod() which lets you add a custom validation method. Lets create one called complete_url().

jQuery.validator.addMethod("complete_url", function(val, elem) {
    // will contain our validation code
}, 'You must enter a valid URL');

A quick word about return values. If the method returns true, everything is considered OK. However, if false is returned then validation is assumed to have failed.

Our new method needs to do three things:

1. If no url, don’t do anything
2. Check that the user had entered http:// in their URL, if not, add it in for them.
3. Check that the URL is now valid. For this I just used the source for the original url validation method.

Lets translate this into some code:

jQuery.validator.addMethod("complete_url", function(val, elem) {
    // if no url, don't do anything
    if (val.length == 0) { return true; }
 
    // if user has not entered http:// https:// or ftp:// assume they mean http://
    if(!/^(https?|ftp):\/\//i.test(val)) {
        val = 'http://'+val; // set both the value
        $(elem).val(val); // also update the form element
    }
    // now check if valid url
    // http://docs.jquery.com/Plugins/Validation/Methods/url
    // contributed by Scott Gonzalez: http://projects.scottsplayground.com/iri/
    return /^(https?|ftp):\/\/(((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:)*@)?(((\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5]))|((([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])*([a-z]|\d|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])))\.)+(([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])*([a-z]|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])))\.?)(:\d*)?)(\/((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|@)+(\/(([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|@)*)*)?)?(\?((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|@)|[\uE000-\uF8FF]|\/|\?)*)?(\#((([a-z]|\d|-|\.|_|~|[\u00A0-\uD7FF\uF900-\uFDCF\uFDF0-\uFFEF])|(%[\da-f]{2})|[!\$&'\(\)\*\+,;=]|:|@)|\/|\?)*)?$/i.test(val);
});

Validating the form with this new method is very straight forward:

$("#form1").validate({
    rules: {
        url: "complete_url"
    }
});

Feel free to use/copy/modify as you see fit. If you do use it, why not let me know, or drop me a trackback?

Have fun

Other jQuery Specific Posts

• IE 7 indexOf() Replacement
• Fixed – jQuery DatePicker not working in Thickbox
• jQuery: Link override with search box

Easily Add Current File to SVN Within Emacs

As I’ve now been using Emacs for quite a long time, I have picked up a few tips along the way. One thing I’ve been doing more and more is using the shell within Emacs. (For more details see here and here.)

Some of my most used shell commands are using SVN, such as add, revert etc. However, I couldn’t find anywhere to just add/revert the current file I am working on. Fortunately there is Stack Overflow! Someone with a very similar issue to me has had a couple of good answers, which I have “borrowed” and now adding files to my SVN repository is really easy.

First, add the following to your .emacs file

(define-key minibuffer-local-map [f3]
(lambda() (interactive) (insert (buffer-file-name (nth 1 (buffer-list))))))

Now either open up Emacs, or reload your .emacs file:

M-x load-file RET ~/.emacs RET

Now, when you are working on a new file you can easily add it to SVN with the following command:

M-! svn add F3 RET

Nice!

My Perfect Desktop Day 1 – The Base System

So today is the day I start building my perfect system. Before getting going,  I had a good think about which distro to use for the base install. I’ve been using Ubuntu for about 2 years now and love the ease of package management with apt and aptitude. I had a bit of a play around with Ubuntu Karmic yesterday and I must say it looks fantastic, and with Lucid Lynx only a few weeks away it might come as a bit of a surprise to discover that my choice is a distro I’ve never used before: Arch.

Why Arch?

There are three main reasons I’ve picked Arch over Ubuntu, or even Debian.

1. I really like the idea of Arch’s rolling release system. My laptop is currently running Jaunty and in order to get the latest packages I would have to upgrade to Karmic. With Arch this is less of an issue.
2. As mentioned in my initial post, the base system should be small, quick and light. With Arch you can pick and choose exactly what you want at installation time, keeping it bloat free.
3. Finally, Arch will be something new to play with!

After downloading and burning a Net Install ISO I followed the excellent installation guide in the Arch Wiki.

Installation was relatively painless, the only two slight issues being:

I chose Auto prepare for partitioning, but I didn’t really know what filesystem to choose. Eventually I opted for Ext4, with so far, no ill effects.

During Boot Device Selection, again I wasn’t really sure what to do, but after some Googling I found this page, whose advice I followed, but just pressing enter.

I restarted and was greeted with a shell prompt. I quickly pinged Google and my network was still up and running through the ethernet on DHCP.

The next step up a non-root user and X. Again, the Arch documentation is excellent.

Setting up X was relatively simple. I originally decided to not opt for hotplugging, which was a mistake because I only have a USB keyboard and mouse, neither of which I could use in X (muppet of the day award goes to me). After turning off the machine and setting up hotplugging, all was fine!

I have been using the Awesome window manager for about a month now. Awesome is a tiling window manager which is heavily geared towards being keyboard friendly, and as it’s name suggests, it is pretty awesome.

Unlike Jaunty, installing the latest stable version of Awesome was a breeze, I simply ran:

# pacman -S awesome

and edited my ~/.xinitrc file adding:

exec awesome

at the bottom. When I started X again, Awesome was there without any problem.

Sound was equally as simple to install, which was a nice surprise.

So my base system is now setup, with my window manager of choice in place and sound ready and waiting for my music collection. And all this running under 100MB of RAM!

After my first foray into the world of Arch I’m pretty impressed. Pacman is very straight forward to use, after a bit of tweaking here and there it has just worked.

Next time, file management tools.