I’ve just been browsing through Digg and I found two excellent blog posts about reducing web development time.
Firstly there was “10 Dirty Little Web Development Tricks” by Yong Fook. I’m pleased to say that I’m doing most of these in my normal day-to-day development roles, which at least means I’m doing something right!
Whilst both these posts are excellent, and I strongly recommend anyone who has even a passing interest at developing websites competently to scan through them, I thought I’d add on a few pointers I’ve learned over the years.
Firstly, without wanting to repeat any of the items in the two posts, I want to emphasize a couple of the main points:
1. I could not agree with Yong Fook’s first point more: SVN checkout as the production site. The amount of time this simple act saves is staggering, especially when you are developing multiple sites that may have multiple branches and you’re trying to keep everything up-to-date.
2. Use an IDE. Whilst agreeing with this point wholeheartedly, I thought I’d try and answer Tom’s comment: “IDEs waste as much time as they save. Terrible tip”. It’s not a terrible tip, it is just that IDEs do take some time to set up and get used to. They are not just like a normal text editor, there are normally quite a few things to set up and get working before they really begin to pull their weight and start being something more than, well, just a normal text editor. There is the debugging functionality to get sorted, the “code help” may have to be turned off, the auto complete might need tweaking (but not normally) and the structure of the projects might have to be re-jigged. But once this is all done, these tools become invaluable and indispensable.
Right, now on with my list
1. It must be simple and quick to change what you’re working on
This is related to using SVN as a production site, but slightly different. It must be really simple and quick to switch from working on a development site, to then working on and deploying to the live site. Some of the projects I work on have 3 servers – live, staging and development. The live and staging generally use the same code base, but the dev site is normally a branch which is then merged into the main trunk when it is feature complete. It must be simple for you to switch working from one site to the next.
This means that any configuration files, or caching/uploading directories set up must be outside the SVN so no conflicts happen. If it is important that the staging server and live sites are pretty much mirrors of each other, only with the staging site having a more active code base, then a script should be written to drag any images off the live site to populate the staging site (or visa versa) so when working on it any image problems etc. can easily be picked up on. For instance, if you have the functionality that resizes images, you need the source images on the staging server to check that it is working. (OK, pretty poor example, but something along those lines)
2. Use a build script, on all servers.
Yes svn update is great, but there are many times when it isn’t enough. For instance, the development site is password protected, so the .htaccess file needs a directive in there to initiate authentication. Or whilst the live site will always need to make sure it is displayed as http://www.example.com in the browser, not http://example.com then the .htaccess file needs to have a Mod Rewrite directive in there, but not so for the development site.
For this there should be a very simple build script (I normally use a quick and dirty shell script) customised for each server. My build scripts normally do the following:
Remove the .htaccess file (to stop conflicts)
Update from the SVN
Modify the .htaccess file, adding the authentication, or removing the redirect
Report that it’s done
3. Develop for the server you’re deploying to
This may sound obvious, but it is important to actually develop for the machine that the website is going to be deployed to, hosted on, uploaded to, whatever you want to call it. There have been countless times that a perfectly working site on a local WAMP machine has been uploaded to a Linux development (or even live) server and it has broken due to the different ways the two operating systems handle case sensitivity in file names. Or if the live server is running an older version of PHP (or whatever) than the development server, you find that your whizzy new function doesn’t work. It is such a fantastic waste of time, so know what you’re developing for.
4. Use Virtualisation
In web development you have to do a lot of testing. Yes you can use automated testing scripts like Selenium, and frankly you should, but you should still look with your own eyes to make sure it’s all working correctly – especially with the number of browsers that have to be developed for. Using virtualisation software like Virtualbox, Parallels or VM Ware you can set up a number of virtual computers that between them have all the different browsers that sites have to work on these days, and without having loads of physical machines cluttering up your office.
5. Enable keyboard short cuts
My main operating system now is Xubuntu and it allows for the creating of user defined keyboard short cuts. I can open up my browser, my mail reader, my IDE without having to point and click. Obviously this doesn’t save a massive amount of time, but it is really handy to whip open a terminal and start typing without having my hands leave the keyboard.
6. No distractions!
And finally, if you really want to speed up your development time the there is one very simple thing you can do that will bring untold benefits: Turn off your email, or Twitter, or Facebook, or GTalk, or mobile phone, or whatever. If you allow yourself to work without any distractions, then you will see your productivity rise, and development time decrease.
So stop reading this post and get on with some work!
The Timesheet Next Gen team have just been putting the initial touches to their new website. Whilst it is a bit bare at the moment, we will be populating it with content for both user and developer over the coming days, weeks, months and hopefully years!
I’ve just committed a new very basic web installer for Timesheet Next Gen which is the Timesheet and job tracking open source web app that I’m trying to contribute to when I have a spare moment.
Only in the SVN repository at the moment, but if you have a chance you can get check it out anonymously from SourceForge using your favourite SVN app here:
In my “real life” I manage a number of blogs for both clients and friends, mostly Wordpress. One of the blogs had a requirement that was to completely password protect the entire site – i.e. only registered users could view any page/post.
For this I initially set up .htaccess password protection – quick and simple.
However, we found out that this prevented the Wordpress iPhone App from login in and doing it’s stuff. So we had to look for another solution.
After doing some Googling I found a post all about password protecting wordpress on the Cre8d Design blog. This pointed me to the “Registered Only” plugin. Whilst it is quite old (4 years since last commit) it should have been just what I was looking for, so I downloaded it, installed and activated (taking care to follow the readme and remove the RSS feeds)
When testing it out it displayed the login box on first visit to the site – prefect. However, I logged in, but it still showed the login box. I assumed I typed in the wrong password, so I tried again. Same result.
Argh, a bug!
Probably because it was meant for a much older version of Wordpress (this blog was using 2.6.3 – now upgraded to2.7)
There was only one thing for it, I had to view the source! Which turned out to be relatively simple:
function carthik_bouncer(){if(substr($_SERVER['SCRIPT_NAME'],-12)!="wp-login.php"){
auth_redirect();
}}
add_action('init','carthik_bouncer');
The problem was clearly the auth_redirect() function. After some searching, I found this threadin the Wordpress Support forums, after which I dutifully modified the code to this:
function carthik_bouncer(){if(substr($_SERVER['SCRIPT_NAME'],-12)!="wp-login.php"){if(!is_user_logged_in()){ auth_redirect(); }}}
add_action('init','carthik_bouncer');
Success! It works prefectly.
If you need to know, here is a patch:
--- registered-only-old.php 2008-12-1214:00:09.000000000 +0100
+++ registered-only-new.php 2008-12-1214:01:21.000000000 +0100@@ -26,7 +26,7 @@
function carthik_bouncer(){
if (substr($_SERVER['SCRIPT_NAME'], -12) != "wp-login.php"){- auth_redirect();+ if (!is_user_logged_in()){ auth_redirect(); }}}
Continuing my trials and tribulations of SVN and Rsync, I came across a new problem today.
This time the client does have Subversion installed, but it is an old version and I can’t update it as I don’t have sudo access.
The problem now is that the working copy has been rsync’d from an existing server, with all the .svn directories intact. If I try and run
$ svn update
on the working copy I get this error:
svn: This client is too old to work with working copy 'xxxx'; please get a newer Subversion client
Using my heightened Google-fu I found this article on Williamo’s Blog. Clearly he’d been having the same problems. The solution was to do a fresh SVN checkout.
All well and good, but this is a production site with extra components (such as stats, a blog which I have no control over cached imaged and other content) that I can’t simply remove just to do a fresh checkout.
So what to do.
Huzzah – Rsync to the rescue again!
As it turns out it’s a simple 3 step process.
Checkout a fresh copy of the subversion repository in a temp directory
Remove all the existing .svn directories held in the directory containing the live site
Rsync the updates from the temp directory to the live directory, this will in effect re-add in the .svn directories, but this time they’ll be the correct version
Step 2. Remove existing .svn directories from live site
$ cd ./live_site
$ rm-rf`find . -type d -name .svn`
Please note: be very careful when running a rm -rf command. Make sure you do a backup. If you are unsure, DON’T DO IT!
Step 3. Rsync the temp directory to the live directory
$ cd ../
(to get back into our root directory)
$ rsync -auv ./tmp_dir/ ./live_site/
If all worked out ok, you should be able to go into the live iste and run an SVN update (which shouldn’t update anything as it would have copied over the fresh checkout)
$ cd ./live_site
$ svn update
That’s it! Simple really, but spent a few hours pulling my hair out over this one!
I’ve had some time today to sit down and browse some of the web. Below are a few articles which I’ve been reading and are worth linking to.
How I learnt to love Perl by David McLaughlin. This is a great article in which David talks about his experience of “rediscovering” Perl and re-finding it’s power. Especially interesting for me as I started out in Perl 11 years ago or so, but moved over to PHP. Maybe it’s time to have look again at Perl, and in particularity Moose – a Post Modern Object System. Definitely a blog to add to your feeder.
Interesting article by Guy Kawasaki – How to use Twitter as a Twool – interesting because I like twitter (me on Twitter) and it’s always interesting to see how other people use it. As for me, I don’t use it as a business tool, but more to keep up with what’s going on in the web.
Do you Debug Your Website? Good article by Patrick McKenzie (of Bingo Card Creator fame) highlighting one of those silly little mistakes that go unnoticed but have a heavy impact. A lesson well learned.
And finally, amusing comments on Reddit: I HATE websites that won’t let me use the browser I want to use! If this is for real then it’s incredible that it is instructing you to view in IE. As a web developer I’m amazed! If, however, it’s a cunning plan to get on the front page of Reddit, then whoops, I’m contributing.
Today I had a little problem deploying a new update to a site for one of my clients.
Normally the way I deploy is by writing a build script that updates the code from an SVN repository. This is easy peasy and means that once setup, the site doesn’t have to be touched apart from running the build script when a new update has been released.
However, this client has just moved their site to a new server, and for reasons that are beyond me, it doesn’t have Subversion installed, nor is it going to. So I have a problem. My method of deployment is SVN, but I can’t deploy using SVN. Also, the location the website is to be deployed to contains lots of cached images and web pages that I don’t want to wipe.
1. Checkout a brand new copy of the latest from the SVN repository onto my workstation.
2. Copy the code from this checkout up to the server using rsync. I need to make sure I don’t delete any files on the server that don’t exist on the workstation, and also make sure I don’t overwrite any newer files on the server than in the repository.
So this is how I did it.
Step One – SVN Checkout.
To create a clean checkout I did the following
1. Create a new directory
$ mkdir LIVE
2. Checkout the repository
$ svnco http://svnserver/reportistory/trunk/ ./LIVE --username rob
It should then pump out a load of info.
Now I could go into the newly checked out repository and update database settings etc to reflect the live server, plus chmod any directories I needed to (e.g. for writing cache files to etc)
The directory is now all ready to copy to the server
Step Two – Rsync to the Server
To rsync to the server I was connecting via SSH, and the command is as follows:
If we take a little time to analyse the above command because there is quite a lot going on there.
If we look at the switches first:
The -a switch ensures the files are transfered in “archive” mode, which essentially keeps the permissions etc.
The -v switch is for verbosity
-z compresses the transfers, reducing bandwidth
-u is very important, and it skips files that are newer on the live server. I needed this as the client can upload image and the system create cached files etc that should not be overwritten
And finally the -e switch allows you to specify a remote shell, effectively allows us to use SSH
After these switches we specify that we will be using SSH to conduct the transfer. If you don’t have the -e switch and the SSH command, you can use Rsync to copy from one file location to another, great for backing up onto an external hard drive.
After the SSH, you specify the source to transfer the files from, in this case the local directory ./LIVE. This is followed by the destination. Here it is a path on an server accessed by SSH. Note the colon (”:”) to separate the login details from the path details.
Finally I added the –progress switch just so it gives me a bit more information whilst uploading the files.
And that’s it. I still keep the site up-to-date with SVN, but I have another little step to deploy to the live server.
Notes: I’m running Ubuntu Intrepid, and the server I was trying to deploy to is some version of Redhat. But this should work for all flavours of Linux that have rsync installed and possibly even Mac OS X.
