No, you're crazy! (or, the problem with assuming that computer programmers have all that much in common)

CropperCapture[96]Five Non-Essential Phone Screen Questions Which Tell Me Nothing

Jeff Atwood recently linked Steve Yegge's post on Five Essential Phone-Screen Questions. My first impression was that it was horrible advice. For reference, here's the five areas which Steve considers essential in eliminating poor job candidates:

  1. Coding. The candidate has to write some simple code, with correct syntax, in C, C++, or Java.
  2. OO design. The candidate has to define basic OO concepts, and come up with classes to model a simple problem.
  3. Scripting and regexes. The candidate has to describe how to find the phone numbers in 50,000 HTML pages.
  4. Data structures. The candidate has to demonstrate basic knowledge of the most common data structures.
  5. Bits and bytes. The candidate has to answer simple questions about bits, bytes, and binary numbers.

He goes on to give some example questions, including the following:

  • What's the worst-case insertion performance of a hashtable? Of a binary tree?
  • What are some options for implementing a priority queue?
  • Format an RGB value (three 1-byte numbers) as a 6-digit hexadecimal string.
  • Tell me how to test whether the high-order bit is set in a byte.
  • Write a function to count all the bits in an int value; e.g. the function with the signature int countBits(int x)

Now, in fairness, both Jeff and Steve were careful to clarify that these aren't go/no-go questions. The idea is to use these to determine if an applicant has a total void of knowledge in a fundamental area.

However, I still believe that Steve's questions would be useless in hiring someone to work on my current projects. I don't care if a candidate can check if a high-order bit is set, to the point that I might be a little turned off by an applicant whose answer revealed they were a bit-twiddler. I've hired bit-twiddlers before, and while they interviewed well, they weren't much help in shipping applications. Worse, we're wasting valuable time talking about hexadecimal formatting when we should be covering things like database access and rudimentary knowledge of HTML and HTTP. An applicant could ace the above questions and be clueless on where to start with constructing a web application.

You see my error, don't you?

Steve Yegge works for Google. Steve and I are both professional programmers; in fact we both work on web applications. Yet, our jobs are no more related than if we were both doctors in different fields - say, a podiatrist and an orthodontist. For example, if you work for Google, you're going to be concerned with how to make your applications run against huge data sets using large, unreliable computer clusters. None of those are priorities in applications I work on, since I'm generally working against a relational database which aren't running at high volume or scale. So, Steve is thinking about how to build Google-scale applications running on Map/Reduce, while I'm working on comparatively small projects running against relational databases.1

As Dare points out, it's incorrect to view that dichotomy and try to pick a winner in a fight between Map/Reduce and Relational Databases. Both are tools, used to solve very different problems. Problems which are so different that those who solve them are essentially in different lines of work.

Different enough that Steve Yegge isn't qualified to interview people who will work on my projects (nor I, his).

Computer Programming: A Field, Not A Job Description

The field of computer programming is a wide one, a fact we seem to ignore when blogging about "the way computer programming ought to be done." It's amusing to read comments on popular blog posts (e.g. just about anything Jeff writes) and watch the arguments:

  • You're crazy! There's no way programmers should [insert topic here]! I've never had to do that in fifteen years of professional development!
  • No, you're crazy! I just coded it up in Lisp last week, and anyone who can't isn't a real programmer!

Of course, the real answer is that both people are crazy for assuming that they're colleagues.

Information Is Useless Without Context

Steve Yegge is the guy to listen to - especially if you want to know about writing Lisp in Emacs, or managing a 500KLOC Java game you wrote in your spare time. Joel Spoelsky is an undisputed expert - on running a business that produces wonderfully usable bug tracking software (on classic ASP and PHP).

And since you're here, I'll mention that my field of expertise: I specialize in ASP.NET web applications running on SQL Server, generally slanted towards the bleeding edge of Microsoft technologies. It's been a decade since I shipped production C++ code, and I haven't been paid to write PHP code in seven years. I spend an inordinate amount of time reading about what's going on outside of my speciality, but I know the limits of my expertise, and so should you. When you're reading my opinions you be aware of my background for three reasons:

  1. You should know when I'm speaking from experience (as opposed to just making stuff up)
  2. You should be aware of any bias or unspoken assumptions that I bring to an opinion, such as assuming that most non-academic applications will work with relational databases

I don't believe that we should require authors to bury their opinions in weasel words and disclaimers. Rather, as readers, we need to appreciate the author's context and bias, and intelligently translate their opinions to our context.

And It's Not Just About Differing Fields Of Expertise, Either...

I've noticed that unspoken bias and context often explains a lot of long-running arguments. For example, I had a long running argument with another developer over WinMerge vs. Beyond Compare. I'm talking about a blood feud spanning years, raging in the battleground of inexpensive file merge applications. Finally, we got on the phone and spent an hour, taking turns explaining why we thought our application was superior. When we were done, we both understood the real issue: he primarily used Beyond Compare to deploy files to a webserver, while I used WinMerge to handle source merges in Subversion. We'd come up with different answers to the question of "What's the best file diff/merge utility" based on different, unspoken assumptions as to what the utility would be used for.

The same sort of solution seems to be useful in just about any seemingly intractable argument in our field. Is there a case in which your views on Test Driven Development wouldn't apply? Your thoughts on dynamic SQL vs. stored procedures? Project management methodologies? Your decisions to use Microsoft software or GPL software (or both, or neither)? Managed code environments vs. C and C++?

1 Just for context, I think I could adequately answer just about every question on Steve's list. I'm not being defensive, I'm arguing that these questions don't come close to defining the successful developers I work with.

20 Comments

  • Excellent post, Jon! Your point about the breadth of specialties in our profession is well taken. I doubt that I could effectively interview a candidate for a position in different department at the company I'm at now, let alone someone working for another company using a different set of technologies and methodologies.

  • Phenomenal post, Jon. I'd say it's the best blog post I've read so far this year.

    After Jeff Atwood published that interview post, I actually wrote up half a rant about how the technical things important to Steve Yegge (he was working for Amazon at the time he wrote his phone screen post) are so far removed from the things important to people working for consulting companies doing custom .NET apps for businesses.

    Why would a company like Vertigo want to hire developers who spend their valuable time learning arcane facts about bit manipulation? A person like that obviously does not value pragmatism. But for a company like Amazon or Google, where developers face hardcore issues because of the scale of those companies' operations, then hell yes, their candidates should be familiar and comfortable with stuff like that.

    It's so important to understand that there are many different, largely independent, "worlds" in software development, and each one should naturally have its own best practices for interviewing.

  • Jon, is there a typo in the last line of your footnote? "I'm arguing that these questions come close to defining the successful developers I work with." It would seem that your are arguing that those questions don't define the successful developers you work with.

    ++Alan

  • I really wish some of my co-workers had been asked to actually code during their interviews. Because maybe then I wouldn't have to work with people who can't write a basic SQL query, but list SQL and Database Administration as an area of "expertise" on their resume.

  • This is probably the most awesome common-sense response to the "every developer should know" posts that crop up from time to time. Right on!

  • @Alan Stevens - Great catch, thanks. I've corrected the footnote.

  • i love rob's answer! that's my take on the whole thing as well. great post. :)

  • Something developers are very guilty of is ignoring reality.

    People have personalities, the personality of an individual and how they'd fit into a team is vital.

    If they have the right personality, then the rest of the interview is just dertermining what level of technical expertise they have, this can be asscertained in a few ways really simply.

    Showing them some simple code, and talking them through it.

    Showing them some slightly more complex code and asking them to talk you through it.

    Asking them how they lay out their applications (acrhitecture) gives you a good feel for both the quality of their code and also the levels of maintenance required.

    Asking them about certain exploits or hacks gives you an insight into how much perifery reading an individual does.

    But for me, it's personality that wins. People who are enthusiastic and passionate about their work can do the job. They might take a bit longer at times, but they'll do it and do it well.

    Do not underestimate Passion.

  • Great post

    I've been on both sides of job interviews and there are always places that I don't get.
    I was once handed a 10 page written test, un like a previous commenter, I don't work with an open google window, but I do open one whenever I forget something and I do use intellisense instead of filling my brain with too much information.

    My favorite question to ask a potential employee is "what did you discover recently that you're excited about" the answer can be a new technology, some simple solution or trick to a problem, or anything else. This question let's me know if you enjoy learning new things and find solutions to problems.

    I think the best developers are those who know how to open google and find an answer to their problem quickly.
    In my field of expertise (which also happens to be .Net and web application developemnt) the most important thing is being able to learn new technologies, because it seems like there's something new coming out all the time

    btw, a lot of companies use standard questions in their tests, all these questions are available online with the answers, so even if it's good questions, it's meaningless

  • Thanks for putting that out there. I've been to several interviews where the employer would have me do some test that was either way too simple, or so frustratingly hard because it really isn't what you do, and/or the equipment they put aside for testing is the 'bottom of the barrel' in the office.

    The best developers may not be the ones with Google sitting open (you would be quite silly to not use such a great resource like that anyway) but rather the ones that have a desire to learn.

    I taught myself BASIC back in 1983 when I was 13 and just kept learning. Can I use the info I learned way back then, hardly. The entire landscape of system, web, app development has changed year after year. So much so that one has to pick a technology and hang on, else they might be left behind.

    One will never know everything, but with a desire to learn, and google, they can figure out anything. Anything.

    Thanks for the post!

  • But what Steve mentions are really the programming basics. You've got to know at least first four of them if you want to work with ASP .NET, for example. Maybe one doesn't have to know how to count set bits in a 32-bit integer variable if he or she wants to create a e-commerce site but one has to know that the value is stored as a bit array in the memory. I agree with Steve and Jeff on that a programmer has to know some fundamental stuff to be called a programmer.

    I do agree with you that there are many programming roles and application areas and that the interviewer has to pay attention to the skills needed for their particular projects.

  • I couldn't agree more. In fact, I would go as far to make the statement that, assuming the goal of employment greater than one project, willingness to learn and a being good fit to the culture of the company is more important than current skills.

    IMO, the frameworks should handle the bit pushing and architecture to enable efficiency in effectively tackling the business problem. There are certain things you can do based on assumptions to enhance performance, but you never really know until the app is in production. Since I can pretty much guarantee I will revisit code I write today, my main goal is to make it readable.

    The question I have is - why hasn't Google come up with a solid framework that does the bit-pushing? Or do they always hire at that level?

  • You just gained a reader.

  • @Wayne - Like you I taught myself BASIC in the late 70's on an HP2000 mini-computer (mini in that it took up half the room instead of the entire room.) Over the years I've taught myself dBase/xBase, Clipper, FoxBase, FoxPro, Databus, RPG, VB, C#, and HTML. I still have most of the books and magazines from all those languages. I still refer to the old stuff from time-to-time because they taught me the fundamentals of logical thinking and breaking a large problem into more manageable bite-sized pieces. I would imagine you still use concepts you learned 20 years ago without even realizing it.

  • Excellent post Jon.

    It's a great reminder to _think_ about what you're trying to achieve when hiring. While these questions might be included to better gauge the candidates full background, it's not the be all and end all of the interview.

    Just as relevant in my book would be:
    * Code elegance & their reasons for using certain conventions
    * Team fit & personality
    * See if you can learn something from the candidate as well - little things can count for a lot, especially if you consider yourself well seasoned

    Steve still has some valid points in his article. I guess the moral of the story is: read lots, challenge it, take what you need and make up your own mind.

    Answer wise, I'd be in the same boat as Rob up there, but maybe a little more nervous sounding if it was verbatim - and the text-to-speech program from the Commodore 64 running the interview :)

  • All those guys who think this is an excellent post should shed their engineering degree.

    Steve and Joel are the true engineers and software engineers.....

  • I request the writer of the post and all those who agree with him -- Try to be a true programmer and not a glue programmer.

  • @Jon: Thanks for this post. I am glad that one of the blogs I read is addressing this misunderstanding.

    @Rob Conery: Thanks for being transparent. I would not be able to answer the majority of those questions with anything but a blank stare.

  • I donot mind learning. But is it required for the course?. Really required?

    Most times, it could be a show off from the person who is interviewing.

    But then, I do not want any one not to learn. It is like carrying loads of junk data in my computer hard drive and the machine stops performing and you complain in the end.

    Well for one second, do you really want to carry that load. I mean if required i do not mind. But then, please make sure, if it is required.

  • Bottom line is, despite being a professional web app developer, I can't answer those questions without looking things up. I'd imagine thats true for the majority of people who can code a usable web app from start to finish. So reading them is a little disconcerting but as you say, it should be taken in context.

Comments have been disabled for this content.