He makes a brings up an interesting point about the idea checking the effectiveness of interview methods. It seems that lots of people have intricate hacks for divining the talented programmers in an interview, but NO ONE can back their methods up with hard data based on performance and non-performance of candidates over project time-spans.
This a good point. It seems somewhat unavoidable, since no one will ever be in a position to evaluate the effectiveness of both those who did well on their metric and those who did poorly (since they will only hire those who do well).
Perhaps there are companies out there with a large enough sample size that are recording metrics on a "programming concepts verbal interview", "analytical thinking verbal interview", "programming test" and "resume match", such that they can evaluate what are the best predictors.
Speaking as someone who's thought a lot about this (mostly during my psychology studies), here's what I would do, if someone let me.
1) Establish the competencies of your employees. Think small programming exercises, IQ tests, personality tests etc. Use this to establish a minimum score for hiring.
2) Apply these same tests to all prospective employees.
3) Of all the candidates that meet the minimum criteria, hire a random set of them.
4) Follow up the progress of all these employees over 1-2 years.
Now, this would never fly in many legal senses, and I doubt that anyone will ever let me do it, but it would provide useful data to improve hiring processes. I suspect that this would need to be done for each individual company at least 100+ times before you would start to be able to derive useful patterns.
Some hypotheses:
1) non linear relationship between IQ and performance
2) Increasing performance and longetivity of employment based on the similarity between personality of candidate and personality of team.
