Over the past seven or eight years I have developed a list of five key interview questions for recruiting staff to software development teams. These five questions have come to stand out as being highly indicative of the candidate’s aptitude for approaching software in [what is now called] a “DevOps” manner, namely, seeing software as the running, evolving system in the Production environment.
[Hello, recruiters! I know you find this post useful, so please donate £5 to the charity Shelter; Shelter provides support to homeless people in the UK]
During these past eight year, I have interviewed perhaps 150 people for software development roles (mostly software developers/testers, with some technical project managers and business analyst types). Of these, I hired (or recommended) around 15-20 people, so a “hit rate” or about 10%. The five questions are taken from a longer list of 20-30 questions, partially refined from lists by the likes of Matt Inman of SEOMoz and Joel Spolsky, which together address (hopefully) everything I need to know about the candidate’s approach to software systems engineering. However, the five key questions have never yet let me down when selecting those who “get it” from those who do not (with “it” being web-based software systems engineering).
The “DevOps” mentality interview questions
The questions I have found useful for hiring people with a DevOps-y mentality are:
- How does HTTP work, and specifically how does a web page appear in my browser?
- How would you prepare for a migration from one [CMS/CRM/database/hosting] platform to another?
- What different types of testing need to be carried out on a software system, and what tools would you use to achieve this?
- How would you make the key aspects of a software system traceable?
- How would you assess how “deployable” a software system is?
We’ll dig into the reasons why I think these questions work well next.
What about these questions makes them useful?
1. How does HTTP work…?
This ought to be just another of the 20-30 technical questions which every second-interview candidate should be able to answer with ease, but very few candidates can answer this question; those than can are among those that “get it”. The answers I look for include a discussion of the OSI model, TCP connections (preferably with a mention of Keep-Alive), web server technology, MIME types, and so on.
In my experience, any candidate who fails at this hurdle is likely to be too “lightweight” to be of great value, because good answers require a “top to bottom” understanding of what is really happening with web-based software.
2. How would you prepare for migration…?
This question evaluates the candidate’s experience of real projects with all the awkwardness and complexity they bring. Developing greenfield systems with little or no existing technology in place is always easier than having to deal with legacy components and configuration, and candidates who appreciate that any interesting software system will in effect be under constant migration are well-suited for DevOps roles.
I like to hear discussion of cut-over, dress rehearsals, roll-back and roll-forward, DNS solutions, feature toggles, branch by abstraction, and automation.
3. What types of testing are needed…?
Too often, members of software teams (usually developers) will look for the “fair weather” path to system completion; that is, they start from an assumption that software will usually work and only occasionally fail. I like to find team members who practise defensive programming in a pragmatic way, which often means assuming that the code will fail and planning for those failures.
This question also exposes the level of experience the candidate has, and their exposure to real-world failure modes. I look for mentions of: unit test strategy; use of test harnesses; early load testing; network simulation; A/B and multi-variate testing; etc.
4. How would you ensure traceability…?
The ability to trace or track behaviour of a software system is sometimes bolted-on as an after-thought. This question probes the candidate’s attitude to metrics, logging, transaction journeys, and reporting; the ideal candidate will identify that metrics, monitoring and logging need to be a core part of the software system, and that without them, the software is essentially unmaintainable and undiagnosable.
5. How would you make software deployable…?
The ability to script the installation and reconfiguration of software systems is essential to controlled, automated change. Although (rightly) there is an increasing trend for new software to enable this, older systems and products suffer from the assumption that changes would be infrequent and minor, and so make automated changes difficult.
Software team members who appreciate the need to expose configuration and settings in a manner accessible to automation will talk about Inversion of Control (IoC) and Dependency Injection, scripted installation, test harnesses, separation of concerns, command-line tools, and “infrastructure as code“.
There may be other questions which are more specific or equally good at driving out misconceptions or a too-focussed view of software systems engineering, but these five have worked very well for me over the past seven or eight years; the great thing about them is that, short of the web moving away from HTTP, all five should still be completely relevant in another seven or eight years’ time.