6 More Ways to Cut Web Development Time

I’ve just been browsing through Digg and I found two excellent blog posts about reducing web development time.

Firstly there was “10 Dirty Little Web Development Tricks” by Yong Fook. I’m pleased to say that I’m doing most of these in my normal day-to-day development roles, which at least means I’m doing something right!

Secondly there was “10 Ways to Cut Down Web Development Time” on Six Revisions. It too offers some good tips, even if the last one looks like an attempt to make the list items up to a nice, round 10!

Whilst both these posts are excellent, and I strongly recommend anyone who has even a passing interest at developing websites competently to scan through them, I thought I’d add on a few pointers I’ve learned over the years.

Firstly, without wanting to repeat any of the items in the two posts, I want to emphasize a couple of the main points:

1. I could not agree with Yong Fook’s first point more: SVN checkout as the production site. The amount of time this simple act saves is staggering, especially when you are developing multiple sites that may have multiple branches and you’re trying to keep everything up-to-date.

2. Use an IDE. Whilst agreeing with this point wholeheartedly, I thought I’d try and answer Tom’s comment: “IDEs waste as much time as they save. Terrible tip”. It’s not a terrible tip, it is just that IDEs do take some time to set up and get used to. They are not just like a normal text editor, there are normally quite a few things to set up and get working before they really begin to pull their weight and start being something more than, well, just a normal text editor. There is the debugging functionality to get sorted, the “code help” may have to be turned off, the auto complete might need tweaking (but not normally) and the structure of the projects might have to be re-jigged. But once this is all done, these tools become invaluable and indispensable.

Right, now on with my list

1. It must be simple and quick to change what you’re working on

This is related to using SVN as a production site, but slightly different. It must be really simple and quick to switch from working on a development site, to then working on and deploying to the live site. Some of the projects I work on have 3 servers – live, staging and development. The live and staging generally use the same code base, but the dev site is normally a branch which is then merged into the main trunk when it is feature complete. It must be simple for you to switch working from one site to the next.

This means that any configuration files, or caching/uploading directories set up must be outside the SVN so no conflicts happen. If it is important that the staging server and live sites are pretty much mirrors of each other, only with the staging site having a more active code base, then a script should be written to drag any images off the live site to populate the staging site (or visa versa) so when working on it any image problems etc. can easily be picked up on. For instance, if you have the functionality that resizes images, you need the source images on the staging server to check that it is working. (OK, pretty poor example, but something along those lines)

2. Use a build script, on all servers.

Yes svn update is great, but there are many times when it isn’t enough. For instance, the development site is password protected, so the .htaccess file needs a directive in there to initiate authentication. Or whilst the live site will always need to make sure it is displayed as http://www.example.com in the browser, not http://example.com then the .htaccess file needs to have a Mod Rewrite directive in there, but not so for the development site.

For this there should be a very simple build script (I normally use a quick and dirty shell script) customised for each server. My build scripts normally do the following:

  1. Remove the .htaccess file (to stop conflicts)
  2. Update from the SVN
  3. Modify the .htaccess file, adding the authentication, or removing the redirect
  4. Report that it’s done

3. Develop for the server you’re deploying to

This may sound obvious, but it is important to actually develop for the machine that the website is going to be deployed to, hosted on, uploaded to, whatever you want to call it. There have been countless times that a perfectly working site on a local WAMP machine has been uploaded to a Linux development (or even live) server and it has broken due to the different ways the two operating systems handle case sensitivity in file names. Or if the live server is running an older version of PHP (or whatever) than the development server, you find that your whizzy new function doesn’t work. It is such a fantastic waste of time, so know what you’re developing for.

4. Use Virtualisation

In web development you have to do a lot of testing. Yes you can use automated testing scripts like Selenium, and frankly you should, but you should still look with your own eyes to make sure it’s all working correctly – especially with the number of browsers that have to be developed for. Using virtualisation software like Virtualbox, Parallels or VM Ware you can set up a number of virtual computers that between them have all the different browsers that sites have to work on these days, and without having loads of physical machines cluttering up your office.

5. Enable keyboard short cuts

My main operating system now is Xubuntu and it allows for the creating of user defined keyboard short cuts. I can open up my browser, my mail reader, my IDE without having to point and click. Obviously this doesn’t save a massive amount of time, but it is really handy to whip open a terminal and start typing without having my hands leave the keyboard.

6. No distractions!

And finally, if you really want to speed up your development time the there is one very simple thing you can do that will bring untold benefits: Turn off your email, or Twitter, or Facebook, or GTalk, or mobile phone, or whatever. If you allow yourself to work without any distractions, then you will see your productivity rise, and development time decrease.

So stop reading this post and get on with some work!

Timesheet Next Gen has a new Home!

The Timesheet Next Gen team have just been putting the initial touches to their new website. Whilst it is a bit bare at the moment, we will be populating it with content for both user and developer over the coming days, weeks, months and hopefully years!

We’ve also created a new development wiki and moved the bug tracker.

Pop by and say hi.

Timesheet NG: Experimental Web Installer

I’ve just committed a new very basic web installer for Timesheet Next Gen which is the Timesheet and job tracking open source web app that I’m trying to contribute to when I have a spare moment.

Only in the SVN repository at the moment, but if you have a chance you can get check it out anonymously from SourceForge using your favourite SVN app here:

https://tsheetx.svn.sourceforge.net/svnroot/tsheetx/branches/exp_installer

More details in the news page

Warning – in no way is this stable yet and should not be used for mission critical situations!

WordPress Plugin – Registered Only – Fix

In my “real life” I manage a number of blogs for both clients and friends, mostly WordPress. One of the blogs had a requirement that was to completely password protect the entire site – i.e. only registered users could view any page/post.

For this I initially set up .htaccess password protection – quick and simple.

However, we found out that this prevented the WordPress iPhone App from login in and doing it’s stuff. So we had to look for another solution.

After doing some Googling I found a post all about password protecting wordpress on the Cre8d Design blog. This pointed me to the “Registered Only” plugin. Whilst it is quite old (4 years since last commit) it should have been just what I was looking for, so I downloaded it, installed and activated (taking care to follow the readme and remove the RSS feeds)

When testing it out it displayed the login box on first visit to the site – prefect. However, I logged in, but it still showed the login box. I assumed I typed in the wrong password, so I tried again. Same result.

Argh, a bug!

Probably because it was meant for a much older version of WordPress (this blog was using 2.6.3 – now upgraded to2.7)

There was only one thing for it, I had to view the source! Which turned out to be relatively simple:

function carthik_bouncer() {
    if (substr($_SERVER['SCRIPT_NAME'], -12) != "wp-login.php") {
        auth_redirect();
    }
}
add_action('init', 'carthik_bouncer');

The problem was clearly the auth_redirect() function. After some searching, I found this thread in the WordPress Support forums, after which I dutifully modified the code to this:

function carthik_bouncer() {
    if (substr($_SERVER['SCRIPT_NAME'], -12) != "wp-login.php") {
	    if (!is_user_logged_in()) { auth_redirect(); }
    }
}
 
add_action('init', 'carthik_bouncer');

Success! It works prefectly.

If you need to know, here is a patch:

--- registered-only-old.php	2008-12-12 14:00:09.000000000 +0100
+++ registered-only-new.php	2008-12-12 14:01:21.000000000 +0100
@@ -26,7 +26,7 @@
 
 function carthik_bouncer() {
     if (substr($_SERVER['SCRIPT_NAME'], -12) != "wp-login.php") {
-        auth_redirect();
+	    if (!is_user_logged_in()) { auth_redirect(); }
     }
 }

SVN client is too old to work with working copy? Solution using Rsync

Continuing my trials and tribulations of SVN and Rsync, I came across a new problem today.

This time the client does have Subversion installed, but it is an old version and I can’t update it as I don’t have sudo access.

The problem now is that the working copy has been rsync’d from an existing server, with all the .svn directories intact. If I try and run

$ svn update

on the working copy I get this error:

svn: This client is too old to work with working copy 'xxxx'; please get a newer Subversion client

Using my heightened Google-fu I found this article on Williamo’s Blog. Clearly he’d been having the same problems. The solution was to do a fresh SVN checkout.

All well and good, but this is a production site with extra components (such as stats, a blog which I have no control over cached imaged and other content) that I can’t simply remove just to do a fresh checkout.

So what to do.

Huzzah – Rsync to the rescue again!

As it turns out it’s a simple 3 step process.

  1. Checkout a fresh copy of the subversion repository in a temp directory
  2. Remove all the existing .svn directories held in the directory containing the live site
  3. Rsync the updates from the temp directory to the live directory, this will in effect re-add in the .svn directories, but this time they’ll be the correct version

So here goes.

Step 1. Checkout a fresh copy

$ mkdir ./tmp_dir
$ svn co http://domain/path/to/svn ./tmp_dir

Step 2. Remove existing .svn directories from live site

$ cd ./live_site
$ rm -rf `find . -type d -name .svn`

Please note: be very careful when running a rm -rf command. Make sure you do a backup. If you are unsure, DON’T DO IT!

Step 3. Rsync the temp directory to the live directory

$ cd ../

(to get back into our root directory)

$ rsync -auv ./tmp_dir/ ./live_site/

If all worked out ok, you should be able to go into the live iste and run an SVN update (which shouldn’t update anything as it would have copied over the fresh checkout)

$ cd ./live_site
$ svn update

That’s it! Simple really, but spent a few hours pulling my hair out over this one!

Weekend Reading 2008-12-07

I’ve had some time today to sit down and browse some of the web. Below are a few articles which I’ve been reading and are worth linking to.

How I learnt to love Perl by David McLaughlin. This is a great article in which David talks about his experience of “rediscovering” Perl and re-finding it’s power. Especially interesting for me as I started out in Perl 11 years ago or so, but moved over to PHP. Maybe it’s time to have look again at Perl, and in particularity Moose – a Post Modern Object System. Definitely a blog to add to your feeder.

Interesting article by Guy Kawasaki – How to use Twitter as a Twool – interesting because I like twitter (me on Twitter) and it’s always interesting to see how other people use it. As for me, I don’t use it as a business tool, but more to keep up with what’s going on in the web.

Do you Debug Your Website? Good article by Patrick McKenzie (of Bingo Card Creator fame) highlighting one of those silly little mistakes that go unnoticed but have a heavy impact. A lesson well learned.

And finally, amusing comments on Reddit: I HATE websites that won’t let me use the browser I want to use! If this is for real then it’s incredible that it is instructing you to view in IE. As a web developer I’m amazed! If, however, it’s a cunning plan to get on the front page of Reddit, then whoops, I’m contributing.

Want to SVN, but can’t? Rsync to the Rescue!

Today I had a little problem deploying a new update to a site for one of my clients.

Normally the way I deploy is by writing a build script that updates the code from an SVN repository. This is easy peasy and means that once setup, the site doesn’t have to be touched apart from running the build script when a new update has been released.

However, this client has just moved their site to a new server, and for reasons that are beyond me, it doesn’t have Subversion installed, nor is it going to. So I have a problem. My method of deployment is SVN, but I can’t deploy using SVN. Also, the location the website is to be deployed to contains lots of cached images and web pages that I don’t want to wipe.

So what do I do?

Fortunately Rsync is here to save the day!

The solution I settled for was a 2 step process.

1. Checkout a brand new copy of the latest from the SVN repository onto my workstation.

2. Copy the code from this checkout up to the server using rsync. I need to make sure I don’t delete any files on the server that don’t exist on the workstation, and also make sure I don’t overwrite any newer files on the server than in the repository.

So this is how I did it.

Step One – SVN Checkout.

To create a clean checkout I did the following

1. Create a new directory

$ mkdir LIVE

2. Checkout the repository

$ svn co http://svnserver/reportistory/trunk/ ./LIVE --username rob

It should then pump out a load of info.

Now I could go into the newly checked out repository and update database settings etc to reflect the live server, plus chmod any directories I needed to (e.g. for writing cache files to etc)

The directory is now all ready to copy to the server

Step Two – Rsync to the Server

To rsync to the server I was connecting via SSH, and the command is as follows:

$ rsync -avzue ssh ./LIVE/ user@liveserver:/path/to/live/httpdocs/ --progress

If we take a little time to analyse the above command because there is quite a lot going on there.
If we look at the switches first:

The -a switch ensures the files are transfered in “archive” mode, which essentially keeps the permissions etc.

The -v switch is for verbosity

-z compresses the transfers, reducing bandwidth

-u is very important, and it skips files that are newer on the live server. I needed this as the client can upload image and the system create cached files etc that should not be overwritten

And finally the -e switch allows you to specify a remote shell, effectively allows us to use SSH

After these switches we specify that we will be using SSH to conduct the transfer. If you don’t have the -e switch and the SSH command, you can use Rsync to copy from one file location to another, great for backing up onto an external hard drive.

After the SSH, you specify the source to transfer the files from, in this case the local directory ./LIVE. This is followed by the destination. Here it is a path on an server accessed by SSH. Note the colon (“:”) to separate the login details from the path details.

Finally I added the –progress switch just so it gives me a bit more information whilst uploading the files.

And that’s it. I still keep the site up-to-date with SVN, but I have another little step to deploy to the live server.

Hope someone finds this useful!

Useful Links

Rsync Man Page

Subversion Homepage

Notes: I’m running Ubuntu Intrepid, and the server I was trying to deploy to is some version of Redhat. But this should work for all flavours of Linux that have rsync installed and possibly even Mac OS X.

jQuery: Link override with search box


Just been working on a little something for a client involving Google Custom Search Engines.

The app we’re building uses 4 different search engines which are all specialised in a certain area. There is a central “Overview” page that gets the top results for the search from each of the search engines, plus there are normal links above the search box that, when clicked on should take the user directly to the Custom Search Engine results page for the string in the search box.

Sounds quite straightforward, but the links also save the current search (which may be different to the search box if the user has just entered a search into the box) so in this case, the old search needs to be stripped and the new search carried forward to the Custom Search Engine.

So it’s a little more tricky, but fortunately using jQuery and the URL Parser plugin by Mark Perkins I was able to cut out most of the tricky stuff.

Below is a little demo to illustrate the main part of the jQuery usage. Before you conduct a search, you should be able to click on the links to go and search Google for those keywords (jQuery, CakePHP, Ruby On Rails and No Search conducted). If you enter a string in the box and hit the “Search” button, you’ll go to Google and search for the string. However, if you enter a string in the search box but then click on a link, you’ll be taken to Google and search for the string.

This isn’t quite how it works on the client site, but is enough to show how the jQuery is being used.

Search Google

 


Javascript code in the <head> tags is:

<script src="/javascript/jquery.1.2.6.js" type="text/javascript"></script> 
<script src="/javascript/jquery.url.js" type="text/javascript"></script>
<script type="text/javascript"><!--
$(function(){
	$("#jQuerySearchExample a").click(function () {
		var path = "http://www.google.co.uk/";
		path += jQuery.url.setUrl($(this).attr("href")).attr("path");
		path += "?"+$("#jQuerySearchExampleInput").serialize();
		$(this).attr("href", path);
	});
});
// --></script>

And the HTML is as follows:

<ul id="jQuerySearchExample">
	<li><a href="http://www.google.co.uk/search" target="_blank">No Search</a></li>
	<li><a href="http://www.google.co.uk/search?q=jquery" target="_blank">jQuery</a></li>
	<li><a href="http://www.google.co.uk/search?q=CakePHP" target="_blank">CakePHP</a></li>
	<li><a href="http://www.google.co.uk/search?q=ruby+on+rails" target="_blank">Ruby On Rail</a></li>
</ul>
Search Google
<form action="http://www.google.com/search" method="get"> 
<input id="jQuerySearchExampleInput" name="q" type="text" /> 
<input type="submit" value="Search" />
</form>

As you can see, I use

jQuery.url.setUrl($(this).attr("href")).attr("path");

To strip out the current query string from the hard coded URL and then

"?"+$("#jQuerySearchExampleInput").serialize();

To replace it with the string within the search box.

Took a bit of faffing about, but works quite well

Xubuntu: Nautilus overtakes XFCE on startup

Just had a nightmare hour trying to get my lovely clean Xubuntu desktop back to normal.

For some strange reason it was loading in the Gnome desktop over the top. All very odd, loaded up as XFCE for a second or two, but then flashed into Gnome – very irritating.

Finally managed to track down the problem which was Nautilus was automatically loading, which was then replacing the desktop. After about an hour of Googling and 15 minutes of going through all the session config variables to try and work out where it was loading, I finally came across this thread in the brilliant Ubuntu Forums which mali2297 recommended removing the .cache directory. I did, and it worked!

For example’s sake, I did this:

$ rm /home/rob/.cache -Rf

I just thought I’d share this in case someone else is having a similar problem.

Welcome to the Blog of Rob Searles

Hi, and welcome to the new Blog by me, Rob Searles

I’ll be writing a brief “about me” soon, but for the time being, I’ll just let you know what to expect from me and when to expect it.

I’m a web developer, in fact I run my own web development company based on Berlin and London, you can have a look and see what we do at ibrow.com. Yes, I know what you’re thinking: the what kind of a rubbish site is that for a web development company to have. And you’re right. But this is one of the things we are going to be sorting out over Christmas and New Year, ready for 2009.

This year has been pretty hectic, we’ve been up to our eyeballs in work and trying to set up the office in Berlin, but that’s all coming to an end now which means I’ll be making sure I dedicate some time to writing this blog. In fact, I’m going to set aside at least 2 days a month that I can focus on this blog, and I’m hoping to make it up to 4 – but I am not the world’s best at getting “me stuff” done when there is client work to do!

So what can you expect to see on this blog? I’m a web developer and to be honest I love it! Some may call me a geek, and I don’t really want to admit they are right! As such, you can expect to see quite a few articles on here about web development and general geekyness. I’ve been using PHP for ever and I have a couple of things I know I want to right about. I’m playing with rails, so expect something or other on that to crop up. I’m quickly falling in love with Ubuntu (in fact Xubuntu, but there you go) so I want to scream from the roof tops that it is just so lovely. I’m a big advocate of Open Source, and will hopefully be talking about moving the TimesheetNG forward over the next year and beyond. And of course, I’ll be talking about the business of the web, and my particular views on it.

So if you like the sound of that, please bookmark me, or whack me in your RSS Reader and tune in

Have Fun