NodeJS: Experiments with Processes

I’ve been hacking about with NodeJS a lot recently, and lately my attention has turned towards spawning child processes. Luckily NodeJS makes this very easy to do, and also the documentation is pretty good.

What I have been working on is having a single parent process that runs constantly in the background, and this parent is responsible for child processes which actually do the work. One thing I found particularly interesting was how the parent can handle a child crashing, and also what happens to the children when the parent is killed. Below is the culmination of my hackings, it comprises of two simple scripts, a child script, and a parent. I have committed them to my GitHub account, so feel free to clone, fork, hack or whatever.

Note: Node version at time of writing was v0.4.9 and running off a Debian machine.

Child Worker

For this experiment we need a really simple child script. It doesn’t have to do anything apart from tell us it is alive when it is running, and then kill itself so we can see how the parent handles this event.

The main bulk of this script is the loop:

var child_loop = function() {
   // report the current loop
   console.log("Child loop: "+NUMBER_OF_LOOPS);
 
   // to simulate the child exiting itself without just running
   // out of program, if we have completed the desired number of
   // loops, exit this process
   if( NUMBER_OF_LOOPS < 0 ) {
      console.log("Child finished loop");
      process.exit();
   }
 
   // sleep for a bit, then continue through the loops
   setTimeout(function() { child_loop() }, LOOP_SLEEP_TIME);
   NUMBER_OF_LOOPS--;
}
 
// start the loop
child_loop();

As you can see this loops for NUMBER_OF_LOOPS, outputting what loop it is currently on. At the end of the loop it kills itself, informing us whilst it does it.

So, this is fine whilst the child is running, but we also need to try and get the child to clean up after itself if it is killed by an external process – i.e. the parent. After some playing around and reading of the documentation, I ended up with this:

// detect and report if this child exited
process.on("exit", function() {
   console.log("Child exiting");
});
// detect and report if this child was killed
process.on("SIGTERM", function() {
   console.log("Child SIGTERM detected");
   process.exit();
});

I need to go more in depth, but whilst testing, this seemed to correctly handle the child being terminated externally.

So, that’s the child script. You can run this directly from the command line and it does what is expected. Good, so we’re all ready to build the parent so it can spawn multiple children.

Parent

This was the main focus of the experiment. The chief concerns were:

  • Ensuring the process ran indefinitely as a daemon
  • Spawning the children
  • Handling a child that crashed or exited
  • Cleaning up the children when the parent was exited

The first point is relatively simple, all we need is an indefinite loop:

loop = setInterval(function() {
   // functionality here
}, LOOP_SLEEP_TIME);

Handling the children was a little more complex. In the end I created an array that would contain the children. Within the main loop, I then ran through that array, checking to see if the child existed, if not then spawning it. Notice how the loop is returning itself into the variable “loop”. This will be important later.

var spawn = require('child_process').spawn;
var loop = setInterval(function() {
   for(i=0; i

This is fine for creating the children, but then what if a child dies? Simple really. The parent needs to listen for the exit event on the child. When detected, ensure that the array element for that child is cleaned out, and then in the next loop, the child will be respawned.

children[i].on('exit', function (code) {
   children[i] = undefined;
});

Next, what if the parent is killed. We need to make sure we don’t leave any orphaned children behind. After some playing around, I found that two events needed to be listened for. Again, the exit event, this time on parent, but also the SIGTERM event on the parent.

When a SIGTERM event is detected, we simply tell the script to exit cleanly:

process.on("SIGTERM", function() {
   console.log("Parent SIGTERM detected");
   // exit cleanly
   process.exit();
});

Now, when the exit event is triggered, we can clean up the children. One thing we must also do, is stop the Interval of the main loop. If we didn’t do this, when the children were killed, the next “tick” of the loop, then they would all be created again. It’s possible that this could happen before the main script has exited, orphaning the children. We can eaily do this as we have the variable that the setInterval call has been saved into.

process.on("exit", function() {
   // ensure no more "ticks"
   clearInterval(loop);
 
   // kill all children!
   for(i=0; i<TOTAL_CHILDREN; i++) {
      if(children[i] != undefined) {
         console.log("Kill child "+i);
         children[i].kill();
      }
   }
});

That is it for the main bulk of the script. As I’m testing and running it on Debian I am using start-stop-daemon to run the parent as a daemon. As such I’ve had to add the functionality for creating a .pid file. One issue I had with this, and I have no idea why, is when I tried to write the process.pid value to a file it filled the file up with nonsense. However, if I converted it to a string (which from what I can tell it already was) before writing it to the file, then it worked fine.

var pid = ""+process.pid; // need to turn into a string
fs.writeFile('/tmp/parent.pid', pid, function (err) {
   if (err) throw err;
});

You can see the complete parent and child scripts on my Github repository.

Running

Now we have our scripts, we need to run them to see if they actually behave as expected. In Debian (and derivatives) this is pretty simple:

To start

start-stop-daemon --start --exec /home/rob/workspace/NodeJS/process/parent.js --pidfile /tmp/parent.js.pid

And to stop

start-stop-daemon --stop --pidfile /tmp/parent.js.pid

As we are not running the process in the background then you will need to have two terminal windows open. Also, you will be able to see the output which explains what is going on.

You should be able to easily convert the script to log to a file (I recommend Vision Media’s Log.js) and then write an init.d script using the start-stop-daemon functionality to create a background daemon process .

You should be able to see that we now have the ability to run a Node.JS daemon process which can spawn multiple children, ensuring it can keep a consistent number of children available, and then handle the cleanup if that daemon process is killed. This opens up lots of possibilities!

Again, feel free to clone, fork, hack, improve, etc the code for this.