Over the past seven or eight years I have developed a list of five key interview questions for recruiting staff to software development teams. These five questions have come to stand out as being highly indicative of the candidate’s aptitude for approaching software in [what is now called] a “DevOps” manner, namely, seeing software as the running, evolving system in the Production environment.
[Hello, recruiters! I know you find this post useful, so please donate £5 to the charity Shelter; Shelter provides support to homeless people in the UK]
See also: Questions to ask when applying for DevOps jobs
During these past eight year, I have interviewed perhaps 150 people for software development roles (mostly software developers/testers, with some technical project managers and business analyst types). Of these, I hired (or recommended) around 15-20 people, so a “hit rate” or about 10%. The five questions are taken from a longer list of 20-30 questions, partially refined from lists by the likes of Matt Inman of SEOMoz and Joel Spolsky, which together address (hopefully) everything I need to know about the candidate’s approach to software systems engineering. However, the five key questions have never yet let me down when selecting those who “get it” from those who do not (with “it” being web-based software systems engineering).
The “DevOps” mentality interview questions
The questions I have found useful for hiring people with a DevOps-y mentality are:
- How does HTTP work, and specifically how does a web page appear in my browser?
- How would you prepare for a migration from one [CMS/CRM/database/hosting] platform to another?
- What different types of testing need to be carried out on a software system, and what tools would you use to achieve this?
- How would you make the key aspects of a software system traceable?
- How would you assess how “deployable” a software system is?
We’ll dig into the reasons why I think these questions work well next.
What about these questions makes them useful?
1. How does HTTP work…?
This ought to be just another of the 20-30 technical questions which every second-interview candidate should be able to answer with ease, but very few candidates can answer this question; those than can are among those that “get it”. The answers I look for include a discussion of the OSI model, TCP connections (preferably with a mention of Keep-Alive), web server technology, MIME types, and so on.
In my experience, any candidate who fails at this hurdle is likely to be too “lightweight” to be of great value, because good answers require a “top to bottom” understanding of what is really happening with web-based software.
2. How would you prepare for migration…?
This question evaluates the candidate’s experience of real projects with all the awkwardness and complexity they bring. Developing greenfield systems with little or no existing technology in place is always easier than having to deal with legacy components and configuration, and candidates who appreciate that any interesting software system will in effect be under constant migration are well-suited for DevOps roles.
I like to hear discussion of cut-over, dress rehearsals, roll-back and roll-forward, DNS solutions, feature toggles, branch by abstraction, and automation.
3. What types of testing are needed…?
Too often, members of software teams (usually developers) will look for the “fair weather” path to system completion; that is, they start from an assumption that software will usually work and only occasionally fail. I like to find team members who practise defensive programming in a pragmatic way, which often means assuming that the code will fail and planning for those failures.
This question also exposes the level of experience the candidate has, and their exposure to real-world failure modes. I look for mentions of: unit test strategy; use of test harnesses; early load testing; network simulation; A/B and multi-variate testing; etc.
4. How would you ensure traceability…?
The ability to trace or track behaviour of a software system is sometimes bolted-on as an after-thought. This question probes the candidate’s attitude to metrics, logging, transaction journeys, and reporting; the ideal candidate will identify that metrics, monitoring and logging need to be a core part of the software system, and that without them, the software is essentially unmaintainable and undiagnosable.
The words SysLog, Splunk, error tracking, Nagios, SCOM, Avicode and so on are what I’m looking for here.
5. How would you make software deployable…?
The ability to script the installation and reconfiguration of software systems is essential to controlled, automated change. Although (rightly) there is an increasing trend for new software to enable this, older systems and products suffer from the assumption that changes would be infrequent and minor, and so make automated changes difficult.
Software team members who appreciate the need to expose configuration and settings in a manner accessible to automation will talk about Inversion of Control (IoC) and Dependency Injection, scripted installation, test harnesses, separation of concerns, command-line tools, and “infrastructure as code“.
Summary
There may be other questions which are more specific or equally good at driving out misconceptions or a too-focussed view of software systems engineering, but these five have worked very well for me over the past seven or eight years; the great thing about them is that, short of the web moving away from HTTP, all five should still be completely relevant in another seven or eight years’ time.
Matthew Skelton 
Good points for interview. I’d only add that this should be done face-to-face as a dialogue, to make sure that you get everything the candidate knows. People often don’t know how much they know until the need arises. Asking “all about HTTP” and then sitting back in silence (happened to me) begs the question “in what context?”.
Steve, you’re absolutely right: I should have made clear in the post that these are questions to ask people *in person* in order to start an exploratory dialogue, rather than some kind of pre-screening questions by email.
I have added an update to clarify this – thanks for bringing this up!
Matthew
Would you have any suggestions on pre-screening questions that could be asked and not googled etc?
Would you have any suggestions on pre-screening questions that could be asked and not googled?
medoix, I think all five questions (and similar ‘core’ questions) are fairly hard to google effectively – the candidate needs to show a deep knowledge and experience of these kinds of situations in order to answer the question effectively.
We’d probably update the list for 2015 to include some fundamental questions relating to cloud, build/CI, deployment, and so on.
I don’t think these are particular for devops role.
@Andy it very much depends on how you define ‘DevOps’ 🙂 These questions would be useful for finding switched-on ‘pure’ developers, forward-thinking sysadmin types, as well as for infrastucture developers and build/release engineers.