Here are some criteria which should be useful when trying to cut through the tangled mess of “DevOps” job titles in order to work out if a prospective role will actually lead to working within a DevOps culture, or whether you’ll be stuck in an old-school IT silo with a fancy “DevOps” name tag and the same crappy us-and-them mentality.
There are an increasing number of “DevOps” jobs being advertised, many of which would have been called “Sys Admin”, “Application Support”, or “Infrastructure Engineer” in the recent past, indicating a plethora of meaning being attached to “DevOps” (Ops people who develop code? Dev people who magically ‘do Ops’? A funky name which means we can hire more server grunts?). Talking this over recently with a friend in the US who is looking for a new role, I realised that there are some handy criteria to use when assessing whether the role will indeed likely involve the Four Pillars of DevOps (CAMS), or whether the job advert is (mis-)using the term DevOps as a draw for a role which leaves Dev and Ops just as far apart as ever.
1. A low jerk count
One of the most important things about any organisation for me is the “jerk count” – how many jerks work for the organisation? Working with or for jerks is debilitating and brain-numbing, and best avoided (one of the great things about working for thetrainline.com is the people). When evaluating an organisation, ask yourself which one “feels” to have the best culture or working environment, and whether there is any evidence for an effective, collaborative culture. Effective DevOps needs lots of trust & respect between individuals and teams; jerks and rock-stars don’t really fit in with this culture.
2. Does the organisation ”get” web operations?
Web-scale software systems do not run themselves; they need people with operations experience to help build, operate, and diagnose problems. Some organisations have convinced themselves that “we don’t need web operations“, as if cloud infrastructure somehow makes all the pain of full storage, power outages, and hardware failures just magically go away.
If the people in the organisation believe they do not need operational experience in order to be successful commercially, then they are unlikely to be successful commercially until they do! The model cited blithely by many – Netflix’s “NoOps” – is so hugely far from “no operations experience”, but many organisations (typically driven by people with limited Ops experience) seem to have missed the point.
Make sure the people in the organisation you intend to work for really understand web operations: DNS, load-balancing, virtualisation, clustering, database tuning, workload management, cloud provisioning, infrastructure automation, Cisco firewalls, HTTP, monitoring & alerting, and so on. This stuff is not “bolt-on”; it’s a core component of properly-engineered web-scale software.
3. Are the teams aligned to the value stream, or silo’d?
Perhaps the most devious (and dispiriting) use of “DevOps” in job titles is for roles which used to be called “SysAdmin” or “Infrastructure Engineer”. Many organisations – unable to cope with the cultural change needed to enable real cross-functional collaboration between Ops and Dev – resort to setting up a separate team (that is, another silo) for the “DevOps” people to write infrastructure code. With this pattern, the “DevOps” team site between Ops and Dev, keeping these two warring factions at bay (and further apart than ever), and smoothing out the worst types of operational failures without actually getting Dev and Ops to align on value delivery.
So ask how/whether the Ops people and the infrastructure developers are aligned to the value streams in the business, or even whether the business understand the concept of value streams and why aligning teams with the value streams is A Good Thing. Make sure that Operations is also part of these value delivery teams, not a separate, silo’d unit.
Further questions to ask at your DevOps job interview
In my recent post Three organisational changes to encourage DevOps I identified three key steps for encouraging a DevOps culture which you can use when assessing organisations’ readiness for rapid, reliable product delivery:
1. Establish and nurture a shared goal for Development and IT Operations
2. Have developers on-call for responding to Priority 1 incidents
3. Make the product owner responsible for the operational success of the product, not just feature delivery
So, when you consider the companies on your job shortlist, ask if there is a shared goal between Dev and Operations and what this goal is and how the goal is realised on the ground. For instance, “rapidly deliver reliable value to our customers” covers functionality and operational concerns. If the organisation has different goals for development (“FEATURES, NOW!”) and Ops (“STABILITY!”) then there will be friction which no amount of ground-up “DevOps” will fix.
Ask developers when they last were woken up for a P1 incident, or when they last paired with an Ops person to help fix a bug in Production. How close are the developers to problems in Production, and how much do they care?
Finally, ask how operational requirements are scheduled into the product development flow: are operational aspects only addressed after a production incident, or is the operability of the software system a first-class concern for the product owner?