Online testing was an idea we stumbled upon when we were looking for an efficient and scalable way to select people who had good programming capabilities. Our earlier approach to selection of programmers was to shortlist based on the resume and call the person for an interview where we gave them a small programming task. And then we had the technical and HR interviews as well. But, in our view, this approach had a couple of weaknesses and we think these have effectively been taken care of by the new online testing approach.

One, the programming task had to be a fairly simple one since the entire process could not take more than an hour or so. So the test was not so good a filter. Second, it was not really scalable because there was the limitation of programming infrastructure and the limitation of time. As a result, we had to rely on the technical interview to guage the technical expertise to a large extent. We were not quite happy with this method as it was not very good at testing for a person's ability to understand requirements, develop a programme in line with the requirements, adherence to coding standards and most importantly, actual programming skill.

It is to address some of these concerns that we came up with the Online Test as a part of our selection process. We have done only one complete iteration as of now with the first set of offer letters being issued last week. While it is too early to say with definitiveness, we would say we are pretty happy with this method and would like to stick to it for hiring programmers.

So now this is how our selection process looks like.

  • Step 1: Short list based on resume
  • Step 2: Short listed candidates participate in the Online Test
  • Step 3: Personal face to face interview for successful candidates
  • Step 4: Offer letters to successful candidates

The Online Test process is as follows. We intimate the short listed candidates of the Online Test a few days prior to the weekend on which it is scheduled. Their usernames and passwords are sent to them on email. The candidates can access the site and take the test any time between 10:00 AM Saturday and 6:00 PM Sunday. All they need is a computer with an internet connection. The programming task is described on the site with as much of instructions as required. They can contact us on email or phone for clarifications and in case of any technical problems with the site. Normally, the programming task should take anywhere between 6 to 8 hours for a person.

Once the site is closed, we get down to the business of evaluating. This includes looking at whether the necessary outputs are generated, the efficiency of the programme, adherence to coding standards and documentation. Based on this, we shortlist the candidate for a personal interview. The technical part of the interview generally revolves around the programming task that was part of the Online Test. And then we decide.

The results of the first iteration which has been completed and the second one which is nearing completion are encouraging. We are able to filter in people who seem to be good programmers. The one disadvantage from the point of the potential candidate is the time that he or she is expected to put in for the Online Test. We hope to iterate this a few times and gather some feedback from the candidates and further fine tune the process.