What Your Interviewer Wants: Mastering the Software Engineering Interview

Xanda Schofield


What matters in a software engineering interview? It's a question I've asked myself a lot, both when looking for jobs and interviewing others for positions. It led to a lot of very interesting conversations with other engineers about what they valued in interviews, what got on their nerves, and what they wished they could tell their interviewees not to get nervous about.

After wondering about this for a long time, I finally decided to see if I could get some answers. I built up a survey asking people who have interviewed people for software engineering positions to score a bunch of things about resumes and interviews. First, they were to rate each item on a scale of importance from 0 to 3 (or N/A if irrelevant or difficult to answer). Then, then they picked their top three most important things from each list. Finally, they would give some personal feedback about what they had seen go poorly in interviews and what advice they had. The results of these surveys are in bar plots scattered throughout the post below, with N/As omitted from the counts.

Demographics

How many years have you worked in tech? How many years have you interviewed others? What is your current position? What is your gender?


In the interest of full disclosure, it seems fair to say that nearly all of the respondents, to my knowledge, are either my friends or friends of my friends; while I have reason to believe that they represent a variety of companies and demographics, there is, as with any survey with data gathered this way, a significant possibility of sample bias (for instance, there are more female respondants in my sample than there would probably be in a normal sample of software engineers). However, I still believe that this sample has good coverage of a variety of industry demographics, and that the responses I got can help debunk a few old career center myths for those on the job market about what matters. Hopefully, this can also start a discussion about whether the things valued by interviewers make sense as things to value in an interview.

I've gathered some of the lessons I thought were interesting and useful from the data, with a few scattered quotes in purple from answers to the free-response questions and what insights I can provide from my brief experience as a software engineering candidate. To lead us off, here's the simple piece of advice from one of the participants:

Show that you can be a productive, motivated addition to the project/team.


I love this advice. Obviously, it doesn't explain much about how you do an interview well, but it contains an important fact to remember: your interview is to show you will fit well on a particular team, not that you are the smartest person in the world or a skilled conversationalist or smooth in programming and puzzle-solving competitions. How do you do that? Well, let's start by talking about what makes a good resume. And let's start simple.

Your cover letter doesn't (usually) matter.


"Candidate has a cover letter" was one of the statements with the most consensus among the respondents. 73.8% of responses listed this as completely unimportant, with fewer than 10% of responses rating it at least "Important" and a single response listing it as "Very Important". I thought this might reflect a generational gap, but upon investigation, all the different scores had a reasonable cross-section of the demographic groups represented (Less than 1 year, 1-5 years, 5+ years). This might still raise concerns about having the 1 in 10 resume screener who still cares a lot about your resume; note, however, that nobody, not even the single person who marked it as a 3 (Very Important), ranked it as among their top 3 most important parts of a resume submission. It may worth asking a recruiter before you submit if cover letters are customary, but don't spend too long worrying about how it's going to read.

If you are writing a cover letter, target it to a recruiter or a manager, not an engineer. In the survey data, the average software engineer importance score is a whopping 0.1. This reflects that engineers are not likely to see your cover letter and, even if they do, probably don't care much about the content. Cover letters are for communicating to people coordinating recruiting what interests you have: it's a good place to describe at a high level your relevant achievements and interests so they can target the correct group for you. Don't bother going into detail about your technical skills in it: that's what your resume and your interview are for.

Ratings of resume traits (0 - unimportant, 3 - very important)


Keep your resume tight.


Figure out your narrative, and talk with that as your guide. Be able to connect everything on your resume to that narrative.

I had participants score several statements about resume length related to standard advice about resumes, reported here with their average scores: We can see from this that the majority of the respondants did not care much at all about whether candidates had more experience than could fit on one page. However, it did matter that you had enough to fill a page, and that your resume was not overly long (more than two pages). You can see below that not all interviewers will necessarily not care so much about each of these traits - each one was rated a "Very important" at least once - but you'll get more bang for your buck if you can fill one page with impressive achievements than if you can successfully chronicle everything you've done.

The quote above is one of those tips that's hard to get used to as a younger software engineering candidate: you shouldn't list everything you've done, just the stuff that shows you're good for the job. Oftentimes, this means cutting mentions of outside interests (0.69 average), non-software experience (0.62), and academic honors (0.49) in favor of class projects or personal projects (1.87), a complete list of skills (2.15), and, most importantly, your software engineering experience (2.33).

Top 3 resume trait choices


Don't fake your skills.


Knooooow the language you say you do!

It seems like an innocent thing at the time: you list all the programming languages you know and then add in the one you used that one time or that toolkit that you used for one homework assignment. Alternately, you list yourself as fluent or proficient in a language that you've used a lot in school but not particularly for full-fledged software projects. The thing is, when interviewers look at this list (and they do - with an average score of 2.15 and 19 of the participants ranking it in their top 3), they're not counting to see how long it is. They want to see if your list includes the tools and languages used by the company, or, failing that, they will test your proficiency in one of the better-known things you list yourself as knowing well to see if you're a conscientious programmer. Once you hit the interview, you might get a question that you can't understand or answer, but to avoid the embarrassment of not knowing, you might make an answer up that seems impressive enough. This can lead to some of the cringe-iest moments for you and your interviewers.

Story time: one of the roughest interviews I ever conducted was with a very promising student looking for an internship. This student started talking about how he had recently gotten into programming in a language I had used somewhat and also thought was quite cool. So, I was pretty impressed and excited when he decided to code up his technical problem up in this language. After coming up with a rough algorithm, he paused for a while. When I asked him why, he told me he didn't know how to write a while loop in the language. He kept trying to make it work anyhow, but the damage had been done - because he wanted to use a hip language to show off, he had shown that he was incapable of programming in what he had asserted was his favorite language. I am convinced that, had he switched languages to something he was more comfortable with and fess'd up to not knowing the language that well, he would have passed his interview, but the information I had to evaluate him did not give any evidence that he had real technical skills. And, in the real world, my sympathy does not mean that he gets to try again. There are too many people to be interviewed and too few hours in the day for people to get a do-over.

This advice was echoed in a variety of forms by the surveyed interviewers:

Don't be afraid to say 'I don't know.' If the interviewer asks you to code something in a language you're not super-familiar with, let them know and ask to use a different language.

I'd rather you say you don't know something off of the top of your head and tell me how you'd figure it out that give me an answer you're not confident about.

State the limits of [your] knowledge - I care about integrity and the ability to learn new technologies, not unsubstantiated mastery.

Don't say you are proficient in a language unless you are familiar with its standard libraries. It is better to write simple correct pseudo-code for a technical question than to say you can write C++ or Java and fail to use their respective libraries correctly.

It's a simple conclusion: it's okay not to know something, just don't try to pretend you know something you don't. Interviewers will figure it out.
Let's move right on in to stuff about interviews, then:

Ratings of interview traits (0 - unimportant, 3 - very important)


Be ready to talk about past projects.


Practice talking about your projects in a professional manner. I think most of the people I screen have never actually even thought about describing their projects to anyone, let alone being evaluated on how they were able to convey their thoughts.

This is one of the things people often forget about when preparing for an interview in favor of prepping for working technical questions. However, three of the five things rated most important (at average rating 2.88, 2.88, and 2.79, respectively) are "Candidate is capable of discussing detail of past projects", "Candidate can describe contributions on these projects", and "Candidate can concisely describe prior projects." Note that these average ratings are higher than any of the statements about getting a solution to your technical question! 11 out of the 38 free responses to "What sorts of things do software engineering candidates most often mess up in the screening/interviewing process?" mentioned an inability to talk about past projects.

You know those people who tell you over and over again to have your "elevator pitch" ready? They're totally right. You need to be able to tell anyone with limited technical background about your project in 30 seconds. You don't have to say how, but by the end, they should have a vague idea of what you did and, critically, why. From there, you should be able to expand more as interviewers are interested: what specifically did you do? What went wrong? What tools did you use? Why did you do what you did? Your answers don't have to make you and your team out to be perfect - no one is. It's much more valuable to be able to paint a clear picture of what you did and what you learned so that you interviewer can see that you have real understanding of what it means to work on a big project.

And again, be honest about your contributions: it is not that embarrassing to not have done everything on a project you list. It is extremely embarrassing when someone starts asking you technical questions that you can't answer and you have to explain that you overstated your contribution.

Top 3 interview trait choices



In technical questions, your thinking matters more than your answer.


There's a lot of parts to a technical or programming question. You have to understand the question quickly, ask all the clarifying questions you should, develop an efficient algorithm to solve it, implement it perfectly (often on a whiteboard, so without any way to check your work besides eyeballing it), show why it's correct, compute complexity, and adapt to whatever changes in the problem might be thrown at you once you get something close, all while talking constantly about what's going on in your head without saying anything that might be perceived as silly. That's perhaps why so much of the attention on software engineering interview preparation is on getting this part right. That's also why one of my main goals in this survey was to find out which parts of this were important to interviewers.

The most important part of this for the surveyed interviewers is being able to communicate your thinking. Note that I'm not just saying "an important part": this is more important both in the overall ratings and in the top-3 rankings than even getting a correct solution by the end of the technical question. Now, it should be noted that getting a correct solution (if not the optimal one) is something that most interviewers both rated highly and mentioned in their feedback as important, so don't read this as "it doesn't matter what you do so long as you talk." However, as stated by a participant,

Technical questions should be designed to see how someone solves problems, not to see if the interviewee comes up with a single "right" solution.


The point of a technical question is to see how you think. Unfortunately, interviewers cannot read minds, so to do a technical question well, you have to explain how you think as you do it. Fortunately, it is okay if, while illustrating your thought process, you take a little longer, or fail to get the best solution in time. Interviewers already know the best solution. They are there to hear how you would try to approach a problem, and often, watching you work out the beginning of a pretty-good solution and hearing all of your questions as you go is more than enough without finishing a perfect solution. In fact, while finding some solution is important, finding the "best solution" ranks well under politeness, friendliness, enthusiasm about the company, and even "writes code with few bugs", which suggests a useful strategy: it is better to settle for implementing something you know you can implement well than to use up your time getting the best algorithm without demonstrating your coding ability.

Practice.

If you leave this page and only retain one thing, it's this: practice for your interview. That doesn't just mean figuring out how to solve every question in Cracking the Coding Interview, though a lot of the respondents recommended that and Glassdoor.com to get a hold of questions. It means that you need to practice explaining those solutions as you write them - with a friend, with your cat, or with your mirror, just as long as you actually think out loud. Practice explaining your projects in the same way. It's easy to get a hold of some of the classic questions like "What is a hard problem you've encountered in a project and how did you overcome it?" and say your answers to your bedroom ceiling. An interview is almost exclusively about talking, and it's important that you know how to do that part. And, when it comes to explaining things well, all you can do is practice.

A longer response explains the how and why of this better than I could:

Fundamentally, a job interview is a sales meeting. You are trying to sell yourself for the job. It's important, therefore, to leave your interviewers with the feeling that you really could be a great fit for the job. You do that by establishing personal rapport with them, by asking them interesting questions about their projects and the company's direction, and looking for opportunities to demonstrate your desire to help be a part of the company's future. Technical questions aren't like the SAT, and you can't quite so easily "prepare" for them. You can, however, prepare for the rest of the interview, which involves discussing the company, the work, the projects, and insinuating yourself in a gentle and friendly way into the picture. There will always be another candidate who can also find a good solution to my technical question. There will not always be another candidate who looks like he or she will make an amazing addition to my team.


This advice to me seems spot-on. As the first piece of advice in this piece said, the goal of an interview is to impress the people you're meeting with, not just as a smart person, but as someone they would want to work with. There are a lot of parts to that, and it's important to be ready to show off all of that stuff that makes you a great employee.

In Conclusion


I've gone through in this some of the stories told by my data that I think speak to some common problems and misconceptions I've seen with interviewing for software engineering positions. To review:
  1. Don't sweat the cover letter too much,
  2. Keep your resume focused on what makes you a good employee for this company,
  3. Don't try to misrepresent your skills,
  4. Prepare to talk about past projects,
  5. Explain your thinking on technical problems (and make sure to get a naive solution), and
  6. Practice!
This is obviously not everything you need to do in order to interview well, and I have a bunch more advice about little other things that I'd love to share (and might try to prepare in a more formal format when I get the chance). But hopefully, if you're in the market for a software engineering job and are new to the game, it will help you pick some things to focus on and provide a bit of insight into what your interviewers see on the other side of the table. After all, your interviewers are just people, and they often have a lot in common with you. If you're an interviewer, then maybe this can give you some food for thought about what matters to you and your fellow interviewers versus what you think should matter. Either way, I hope you enjoy finding these insights as much as I did. Happy interviewing.

Have feedback? Check out my website or email me at [my first name]@cs.cornell.edu. I'd love to talk more about this data and my own experience.