Here are some criteria which should be useful when trying to cut through the tangled mess of “DevOps” job titles in order to work out if a prospective role will actually lead to working within a DevOps culture, or whether you’ll be stuck in an old-school IT silo with a fancy “DevOps” name tag and the same crappy us-and-them mentality.
See also: Five Interview Questions for Hiring DevOps Staff
There are an increasing number of “DevOps” jobs being advertised, many of which would have been called “Sys Admin”, “Application Support”, or “Infrastructure Engineer” in the recent past, indicating a plethora of meaning being attached to “DevOps” (Ops people who develop code? Dev people who magically ‘do Ops’? A funky name which means we can hire more server grunts?). Talking this over recently with a friend in the US who is looking for a new role, I realised that there are some handy criteria to use when assessing whether the role will indeed likely involve the Four Pillars of DevOps (CAMS), or whether the job advert is (mis-)using the term DevOps as a draw for a role which leaves Dev and Ops just as far apart as ever.
Photo credit – http://www.flickr.com/photos/skellywag/5876972814/in/photostream/
1. A low jerk count
One of the most important things about any organisation for me is the “jerk count” – how many jerks work for the organisation? Working with or for jerks is debilitating and brain-numbing, and best avoided (one of the great things about working for thetrainline.com is the people). When evaluating an organisation, ask yourself which one “feels” to have the best culture or working environment, and whether there is any evidence for an effective, collaborative culture. Effective DevOps needs lots of trust & respect between individuals and teams; jerks and rock-stars don’t really fit in with this culture.
2. Does the organisation “get” web operations?
Web-scale software systems do not run themselves; they need people with operations experience to help build, operate, and diagnose problems. Some organisations have convinced themselves that “we don’t need web operations“, as if cloud infrastructure somehow makes all the pain of full storage, power outages, and hardware failures just magically go away.
If the people in the organisation believe they do not need operational experience in order to be successful commercially, then they are unlikely to be successful commercially until they do! The model cited blithely by many – Netflix’s “NoOps” – is so hugely far from “no operations experience”, but many organisations (typically driven by people with limited Ops experience) seem to have missed the point.
Make sure the people in the organisation you intend to work for really understand web operations: DNS, load-balancing, virtualisation, clustering, database tuning, workload management, cloud provisioning, infrastructure automation, Cisco firewalls, HTTP, monitoring & alerting, and so on. This stuff is not “bolt-on”; it’s a core component of properly-engineered web-scale software.
3. Are the teams aligned to the value stream, or silo’d?
Perhaps the most devious (and dispiriting) use of “DevOps” in job titles is for roles which used to be called “SysAdmin” or “Infrastructure Engineer”. Many organisations – unable to cope with the cultural change needed to enable real cross-functional collaboration between Ops and Dev – resort to setting up a separate team (that is, another silo) for the “DevOps” people to write infrastructure code. With this pattern, the “DevOps” team site between Ops and Dev, keeping these two warring factions at bay (and further apart than ever), and smoothing out the worst types of operational failures without actually getting Dev and Ops to align on value delivery.
So ask how/whether the Ops people and the infrastructure developers are aligned to the value streams in the business, or even whether the business understand the concept of value streams and why aligning teams with the value streams is A Good Thing. Make sure that Operations is also part of these value delivery teams, not a separate, silo’d unit.
Further questions to ask at your DevOps job interview
In my recent post Three organisational changes to encourage DevOps I identified three key steps for encouraging a DevOps culture which you can use when assessing organisations’ readiness for rapid, reliable product delivery:
1. Establish and nurture a shared goal for Development and IT Operations
2. Have developers on-call for responding to Priority 1 incidents
3. Make the product owner responsible for the operational success of the product, not just feature delivery
So, when you consider the companies on your job shortlist, ask if there is a shared goal between Dev and Operations and what this goal is and how the goal is realised on the ground. For instance, “rapidly deliver reliable value to our customers” covers functionality and operational concerns. If the organisation has different goals for development (“FEATURES, NOW!”) and Ops (“STABILITY!”) then there will be friction which no amount of ground-up “DevOps” will fix.
Ask developers when they last were woken up for a P1 incident, or when they last paired with an Ops person to help fix a bug in Production. How close are the developers to problems in Production, and how much do they care?
Finally, ask how operational requirements are scheduled into the product development flow: are operational aspects only addressed after a production incident, or is the operability of the software system a first-class concern for the product owner?
Matthew Skelton 
Another thing to watch out for is when a “DevOps” role is a re-branded release manager or release engineer role. i.e. a third silo, between the development and operations silo.
Good point, Kief – re-branding the release engineer as “DevOps” is particularly unhelpful; needing a special person or team to release software is another anti-pattern.
This is a difficult thing to discuss. I’d much rather walk into successful company A and teach them devops culture and break down the silos and barriers than I would walk into a fully developed devops type shop mostly because i’m not a developer, i’m an operations / data dude. I also see the “DevOps” role as an incubator role – as soon as the tools we use that define devops beyond our cultural definition mature, the “Dev” role won’t be needed and quite frankly, if it is perpetually needed – “Devops” has failed. Devops should be a cultural alignment for as everything matures, the silos are less role silos and process silos, but knowledge silos that no one person can nor should break.
Also, I can’t stand anyone looking for a devops role. Have you seen some of the interviews that these “Devops” managers put people through? They’re embarrassing and completely miss the “C.A.M.S” value center of “devops” grass roots efforts. Testing people on how much arcane commands they can remember, whether nor not they use the book or wrote the book.. and oddly enough, every single on of these devops managers that seeks perfection and “just like me” type people are the ones I see CONTINUALLY hopping jobs because that perfection ends up being absolutely elusive or a giant douchebag behind it.
I like to “dig people up”. I see value in cultural shifts. I see value in attempting to make a difference even if their small differences because the end goal isn’t having something my way or the “devops” way, but rather making a business successful because it has a strong culture, it can automate the things, it can measure the things and it can share the things. The “CAMS” value add is just as important in desktop support as it is corporate, as it is helpdesk through to development or any “tech” resource chain wherever it may be. And to be quite honest, it needs to grow beyond “Dev” and into “Ops” and i’m betting we will see “Agile Ops” win in the end. did I blab on too much 🙂
Thanks for the comment, Byron; I largely agree with you. I’m certainly not advocating looking only for a fully-formed “DevOps” culture, but rather suggesting ways for people to avoid being conned into a so-called DevOps role when the organisation has no intention of becoming more collaborative in its culture or recognising the importance of web operations. The douchbag managers you refer to probably map to the “jerks” I mention in point #1 🙂
Matthew