The way we think about data and databases must adapt to fit with dynamic cloud infrastructure and Continuous Delivery. The need for rapid deployments and feedback from software changes combined with an increase in complexity of modern distributed systems and powerful new tooling are together driving significant changes to the way we design, build, and operate software systems. These changes require new ways of writing code, new team structures, and new ownership models for software systems, all of which in turn have implications for data and databases.
Originally posted on Experience DevOps:
One of the exercises we do during the Experience DevOps workshop is a ball flow scenario inspired by the Ball Point game. We set up different combinations and topologies of Dev teams and Ops teams, and then see how many balls the teams can pass in (say) 60 seconds.
When we ran the workshop recently in Bangalore, we had a large number of participants, which enabled some interesting experimentation with the topology of the teams. In the exercise, the ‘Dev’ team takes balls from the ‘backlog’, and eventually passes the balls to the ‘Ops’ team, who must ‘make the features live’ under some constraints designed to simulate real-world physical constraints.
With the large group of participants in Bangalore, we experimented with multiple value streams (or products). After ‘warm-up’ runs using a single ‘product’ (value stream) and therefore a single Dev team and a single Ops team, we split people into two separate…
View original 256 more words
I was recently asked to contribute to an eBook from Zend about moving to Continuous Delivery (CD). The 29 authors in the book share a wide range of experience with CD, and there is plenty of useful advice; the contributions from Mathias Meyer (@roidrage), Kate Matsudaira (@katemats), and Jamie Ingilby (@jamiei) are particularly worth reading, I think.
In my section of the book I explain how using ThoughtWorks GO to model the testing and release steps (effectively part of the value stream) we won trust from several different people and teams during a move to CD. Using a prototype also helped us to validate the activities undertaken:
We tried to empathize with their situation and, using role-based security in the deployment pipeline, uncovered enough information to give them a sense of visibility and control.
Without being able to visualise and communicate easily the activities we were automating, progress would have been slow or even blocked.
Get a free copy of the eBook here: http://bit.ly/ZendCDbook
I have recently read (and re-read) several books on Chef in order that I can recommend books to clients who are starting with infrastructure automation (and to remind myself of the more obscure uses of knife, encrypted databags, and so on). In this post I comment on these books:
- Chef Infrastructure Automation Cookbook by Matthias Marschall
- Managing Windows Servers with Chef by John Ewart
- Test-Driven Infrastructure with Chef (2nd Edition) by Stephen Nelson-Smith
- Automation Through Chef Opscode by Navin Sabharwal and Manak Wadhwa
Summary: read Chef Infrastructure Automation Cookbook for a good introduction to Chef on both Linux and Windows; read Managing Windows Servers with Chef if you manage many Windows machines; but most of all read Test-Driven Infrastructure with Chef because without a test-driven approach your infrastructure code will rapidly become tangled, unsupported, and obsolete.
I wrote an article recently for the Simple Talk website called Common database deployment blockers and Continuous Delivery headaches, where I outline some of the common problems preventing databases from being deployable – a major blocker to Continuous Delivery.
Deployability is now a first-class concern for databases, and there are several technical choices (conscious and accidental) which band together to block the deployability of databases. Can we improve database deployability and enable true Continuous Delivery for our software systems? Of course we can, but first we have to see the problems.
The recommendations include:
- Minimize changes in Production
- Reduce accidental complexity
- Archive, distinguish, and split data
- Name things transparently
- Source Business Intelligence from a data warehouse
- Value more highly the need for change
- Avoid Production-only tooling and config where possible [I mention this in my talk How to choose tools for DevOps and Continuous Delivery]
To address [these things] individually perhaps doesn’t seem too challenging, but to tackle deployability requires close, effective collaboration between developers, DBAs, and operations teams to achieve the right balance between rapid deployment and access to data.
I was interviewed recently by the folks at Ranger4 for their #DevOpsFriday5 question series. Since June 2014 (when I was interviewed) I have published a couple of things which expand on the original answers, so I have outlined these here. The questions were:
- What’s your preferred definition of DevOps?
- When people ‘do’ DevOps, what’s the most common mistake you see them make?
- How do you recommend an organisation new to DevOps start?
- What’s your prediction for what DevOps will look like in 2020?
- Where do you like to go to get a DevOps hit?
I am very pleased that the first version of the Build Quality In book has been published on LeanPub, with contributions from Chris O’Dell and Dave Farley (co-author of the book Continuous Delivery). The book is edited by me and Steve Smith.
In the spirit of ‘lean’, we’re publishing a new version of the book whenever one or two additional contributions are ready; you can see the expected publication schedule on the LeanPub page. Buyers of the book receive free updates for life, so buy your copy now at the early bird price!