Five Interview Questions for Hiring DevOps Staff

Over the past seven or eight years I have developed a list of five key interview questions for recruiting staff to software development teams. These five questions have come to stand out as being highly indicative of the candidate’s aptitude for approaching software in [what is now called] a “DevOps” manner, namely, seeing software as the running, evolving system in the Production environment.

[Hello, recruiters! I know you find this post useful, so please donate £5 to the charity Shelter; Shelter provides support to homeless people  in the UK]

See also: Questions to ask when applying for DevOps jobs

During these past eight year, I have interviewed perhaps 150 people for software development roles (mostly software developers/testers, with some technical project managers and business analyst types). Of these, I hired (or recommended) around 15-20 people, so a “hit rate” or about 10%. The five questions are taken from a longer list of 20-30 questions, partially refined from lists by the likes of Matt Inman of SEOMoz and Joel Spolsky, which together address (hopefully) everything I need to know about the candidate’s approach to software systems engineering. However, the five key questions have never yet let me down when selecting those who “get it” from those who do not (with “it” being web-based software systems engineering).

Update: it’s my assumption that you’d want to ask these questions in person as a way of starting an exploratory dialogue with the candidate, rather than use them as pre-screening questions by email (thanks to Steve Freeman for pointing this out!)

Knocking on the door - interview questions for DevOps

As we’ll see, the questions are not traditional “software developer” questions (weaker candidates should be weeded out with an initial technical test), but rather focussed on eliciting what might be called “full stack” or “roadmap” responses which require the ability to think at many different levels of abstraction or at many different points in time.

The “DevOps” mentality interview questions

The questions I have found useful for hiring people with a DevOps-y mentality are:

  1. How does HTTP work, and specifically how does a web page appear in my browser?
  2. How would you prepare for a migration from one [CMS/CRM/database/hosting] platform to another?
  3. What different types of testing need to be carried out on a software system, and what tools would you use to achieve this?
  4. How would you make the key aspects of a software system traceable?
  5. How would you assess how “deployable” a software system is?

We’ll dig into the reasons why I think these questions work well next.

What about these questions makes them useful?

1. How does HTTP work…?

This ought to be just another of the 20-30 technical questions which every second-interview candidate should be able to answer with ease, but very few candidates can answer this question; those than can are among those that “get it”. The answers I look for include a discussion of the OSI model, TCP connections (preferably with a mention of Keep-Alive), web server technology, MIME types, and so on.

In my experience, any candidate who fails at this hurdle is likely to be too “lightweight” to be of great value, because good answers require a “top to bottom” understanding of what is really happening with web-based software.

2. How would you prepare for migration…?

This question evaluates the candidate’s experience of real projects with all the awkwardness and complexity they bring. Developing greenfield systems with little or no existing technology in place is always easier than having to deal with legacy components and configuration, and candidates who appreciate that any interesting software system will in effect be under constant migration are well-suited for DevOps roles.

I like to hear discussion of cut-over, dress rehearsals, roll-back and roll-forward, DNS solutions, feature toggles, branch by abstraction, and automation.

3. What types of testing are needed…?

Too often, members of software teams (usually developers) will look for the “fair weather” path to system completion; that is, they start from an assumption that software will usually work and only occasionally fail. I like to find team members who practise defensive programming in a pragmatic way, which often means assuming that the code will fail and planning for those failures.

This question also exposes the level of experience the candidate has, and their exposure to real-world failure modes. I look for mentions of: unit test strategy; use of test harnesses; early load testing; network simulation; A/B and multi-variate testing; etc.

4. How would you ensure traceability…?

The ability to trace or track behaviour of a software system is sometimes bolted-on as an after-thought. This question probes the candidate’s attitude to metrics, logging, transaction journeys, and reporting; the ideal candidate will identify that metrics, monitoring and logging need to be a core part of the software system, and that without them, the software is essentially unmaintainable and undiagnosable.

The words SysLog, Splunk, error tracking, Nagios, SCOM, Avicode and so on are what I’m looking for here.

5. How would you make software deployable…?

The ability to script the installation and reconfiguration of software systems is essential to controlled, automated change. Although (rightly) there is an increasing trend for new software to enable this, older systems and products suffer from the assumption that changes would be infrequent and minor, and so make automated changes difficult.

Software team members who appreciate the need to expose configuration and settings in a manner accessible to automation will talk about Inversion of Control (IoC) and Dependency Injection, scripted installation, test harnesses, separation of concerns, command-line tools, and “infrastructure as code“.

Summary

There may be other questions which are more specific or equally good at driving out misconceptions or a too-focussed view of software systems engineering, but these five have worked very well for me over the past seven or eight years; the great thing about them is that, short of the web moving away from HTTP, all five should still be completely relevant in another seven or eight years’ time.

2 responses to “Five Interview Questions for Hiring DevOps Staff

  1. Good points for interview. I’d only add that this should be done face-to-face as a dialogue, to make sure that you get everything the candidate knows. People often don’t know how much they know until the need arises. Asking “all about HTTP” and then sitting back in silence (happened to me) begs the question “in what context?”.

  2. Steve, you’re absolutely right: I should have made clear in the post that these are questions to ask people *in person* in order to start an exploratory dialogue, rather than some kind of pre-screening questions by email.
    I have added an update to clarify this – thanks for bringing this up!
    Matthew

Join the discussion...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s