The transformational power of Continuous Delivery

I was recently asked to answer some questions about Continuous Delivery for someone’s undergraduate university research. The questions were interesting, so here are my answers 🙂

  1. What do you feel are the benefits of adopting Continuous Delivery?
  2. How do you feel adopting Continuous Delivery has affected your development cycle?
  3. Do you think Continuous Delivery is an important approach for a company to pick up, If so Why?
  4. How do you think Agile compares to more traditional models like Waterfall?
  5. What is the biggest change you’ve noticed since adopting Agile?
  6. What do you foresee in the future for these models in the industry?

1. What do you feel are the benefits of adopting Continuous Delivery?

The benefits of Continuous Delivery are huge:

  • Greater focus on finishing and shipping.
  • Increased awareness of need for setting up the work to enable feedback and learning.
  • Sense of ‘flow’ within teams.
  • Decisions made using actual data rather than opinions alone.
  • Higher quality software.
  • More joy in work.

(Can I stop yet?)

2. How do you feel adopting Continuous Delivery has affected your development cycle?

(Answering on behalf of our clients) Continuous Delivery has helped us to increase the ownership over software and focus on the value-add things that our organisation produces, rather than ceremonies around testing and releasing.

3. Do you think Continuous Delivery is an important approach for a company to pick up, If so Why?

Yes: adopting CD properly can be truly transformative for the organisation as a whole. IT becomes a means to receive rapid feedback on product/marketing/service offerings, allowing the business to invest more wisely and do more with less risk and lower costs.

4. How do you think Agile compares to more traditional models like Waterfall?

Agile is woefully misunderstood (as was Waterfall) so in that regard, they are similar (!). Truly agile organisations are rare because agility challenges entrenched, comfortable positions within an organisation. Agile done well really makes the nature of an organisation transparent.

5. What is the biggest change you’ve noticed since adopting Agile?

(Answering for our clients) We’re able to improve the software delivery part of the process, but this has highlighted the lack of clarity and vision in the Business.

6. What do you foresee in the future for these models in the industry?

We’re starting to see a backlash against Agile and DevOps already, because people misunderstand or misrepresent what’s going on. Things like SAFe seem to be rebranded PRINCE2 which is a shame. Essentially, many organisations are going to fail because their management does not see the need to change.


At Skelton Thatcher Consulting we have put together a handy Continuous Delivery checklist template (on Trello) to help you assess the things you need to address within your organisation:

2017-03-20--cdchecklist.info

See cdchecklist.info 

 

How and why to run internal tech conferences – InfoQ article

In an environment of rapidly-changing technology and approaches, an internal tech conference can be a powerful and effective way of spreading new ideas and practices and sharing learning & experience. Having organised and run several internal tech conferences (at different organisations), Victoria Morgan-Smith and I decided to write about our experiences in an article for InfoQ: Internal Tech Conferences – How and Why. We also interviewed several other people from various organisations who have also run internal tech conferences in order to give a broader perspective.

Our aim was to inspire and enable other people to develop and run internal tech conferences in their own organisations, building on the experiences of the teams and organisations in the article.

In this article we draw on our personal experience of running internal tech events at companies we’ve worked with, along with reflections and advice from people at Paddy Power Betfair, Callcredit Information Group, ING and others. You’ll find further reading & listening material at the end of the article – there is so much inspirational work happening in so many organisations.

1

Key points from the article are:

  • Software engineering today is as much about people as the technology itself: an internal tech conference can give a huge boost to your organisation’s social capital – that currency by which relationships flourish.
  • The format you choose for your internal tech conference depends on what you want to achieve from it: it can be “by the people for the people”, or a showcase to celebrate achievement. You can keep the audience or speakers to just a single department, or invite other divisions, or even invite external speakers and/or audience.
  • Making the event a success takes effort: choose your speakers well, and mentor themas they prepare their talks. Work on the logistics – it’s the little things that count.
  • Remember to have fun: ‘death by PowerPoint’ will mean people remember the event for the wrong reasons!
  • Follow through: for a lasting impact, keep sight of the outcomes you seek and be ready to work with others to keep the momentum going.

We hope that the article is useful for people thinking of running or improving their own internal tech conferences!

Thanks to everyone involved: people we interviewed, the amazing InfoQ team, and to my co-author Victoria Morgan-Smith.

New eBook on Continuous Delivery with Windows and .NET

Back in 2010 when Jez Humble and Dave Farley wrote their ground-breaking book Continuous Delivery, the Windows and .NET platforms lagged behind the Linux/Mac world in terms of automation capability. That is no longer the case – every core feature in Windows and .NET now has a PowerShell API and all the core tooling needed for Continuous Delivery – package management, artifact repositories, build servers, deployment pipelines tools, infrastructure automation, monitoring,and logging – are all now available natively on Windows/.NET.

Chris O’Dell (@ChrisAnnODell) and I decided we should explain how to make Continuous Delivery work with Windows and .NET, and thanks to the great editorial team at O’Reilly, we’ve published a short eBook:

CD with Windows - cover

The dedicated book website is at CDwithWindows.net and O’Reilly have published the first chapter of the book online as an article: Introduction to Continuous Delivery with Windows. We’d love your feedback: book@cdwithwindows.net

UPDATE: we’ll be at both PIPELINE Conference (March 23 2016) and WinOps Conference (May 24 2016) with printed copies of the book.

Note: we began writing the book in August 2015, and it’s astonishing (and exciting!) how much has changed in the 8 months since then, with Windows Nano, Azure and Windows support for Docker and containers, .NET Core, SQL Server on Linux, and even SSH for Windows. These and more recent developments do not feature in the book – perhaps we’ll do an updated version soon. 

Notes on ‘team responsibilities in cloud-native operations’ (Pete Mounce)

Summary:  Pete Mounce (@petemounce) from Just Eat gave a compelling talk at the London Continuous Delivery meetup group on ‘team responsibilities in cloud-native operations’. I found the talk hugely engaging, with loads of detail applicable to many organisations. Here are my notes from the meetup.

I captured my notes as slides:

Update: the video of Pete’s talk is here on Vimeo:

Pete Mounce video frame

There were several specific points made by Pete that were interesting for me:

Continue reading

HP is trying to patent Continuous Delivery – here is how you can help block this madness

Summary: Hewlett-Packard (HP) has filed several patents covering standard Continuous Delivery (CD) practices. You can help to have these patents revoked by providing ‘prior art’ examples on Stack Exchange.

On 1st March 2015 I discovered that in 2012 HP had filed a patent (WO2014027990) with the USPO for ‘Performance tests in a continuous deployment pipeline‘ (the patent was granted published in 2014). The exact search I used was https://www.google.co.uk/webhp?q=performance+testing++in+a+pipeline and the patent grabbed my attention almost immediately as it was around the 5th result (as I write this, it still is):

Search results for performance testing in a pipeline

I immediately tweeted and @-mentioned Jez Humble (@jezhumble) and Dave Farley (@davefarley77), co-authors of the foundational book Continuous Delivery, to alert them (their book was published in 2010, two years before the HP patents were filed).

My friend and colleague Steve Smith (@agilestevesmith) quickly created a ‘prior art’ request on the Ask Patents Stack Exchange site to coordinate the collection of references to prior art, so that we have a coordinated place to document the many existing examples of running performance tests in a deployment pipeline prior to the patent being filed. This will help us to refute this patent (WO2014027990). As of today (6th March 2015) the Ask Patents page and various Twitter threads had contributions from several early proponents of CD including Chris Read, Dan North, Jez Humble, Dave Farley, Pat Kua, Andreas Grabner, Erik Doernenburg, and Martin Fowler (amongst others).

The plot thickens

As the “WTF?!” spread on Twitter, Marco Abis (@capotribu) pointed out that not only were the authors almost literally unknown anywhere else on the internet, but that the same HP authors had filed patents for many more standard CD practices:

Here is a screenshot of the search results as of 6th March:

Patents by Inbar SHANI

For the record, here are the patent reference identifiers for these HP patents covering basic CD practices (alternate references given, and hyperlinks to the Ask Patents pages – I will update this page with links to new Ask Patent pages as they appear):

There may of course be other patents filed by HP covering standard CD practices (see below, Okay, what can I do to help?)

WT-actual-F is HP doing?!

There are two explanations I can think of for what HP is doing here. Either the authors of these patents – collectively Adam SPEKTOR, Inbar SHANI, Yaron Burg, Amichai Nitsan, Sigal MAON, Ilan Shufer, Eli Mordechai, and Lior Manor, all of Hewlett-Packard Development Company, L.P. – have been locked away in a cupboard for the last 10 years and have independently invented terms like ‘deployment pipeline’ and are filing what they believe to be genuine applications for true innovations (in which case, I kind of get it, as I think that deployment pipelines are awesome), or (the more likely explanation in my view) HP is patent trolling in a frankly stupid and disrespectful way.

So what’s the problem?

The ‘inventions’ in the HP patents have been practiced, implemented, and documented by advocates of Continuous Integration and Continuous Delivery for many years prior to 2012, particularly by people at ThoughtWorks but also by people from Atlassian, The Guardian, Dynatrace and many others (details in the Ask Patents page for WO2014027990A1).

For instance, here is extract from an article in Dr. Dobb’s (sadly now moribund) from February 2008 by Steve Haines called Continuous Integration and Performance Testing (examples use the first CI server, CruiseControl):

Now you can add performance tests into the continuous integration process. The most straightforward method is to create a new virtual project in CruiseControl, one that uses the same build script but executes a performance unit-testing target. The following shows the key changes you could make to a copy of Listing One’s project definition:

<project name=”ant-junit-performance-unit-tests”>…<schedule interval=”600″><ant anthome=”apache-ant-1.6.5″buildfile=”projects/${project.name}/build.xml”target=”performance-unit-tests.execute” /></schedule><publishers>

<onsuccess>

<artifactspublisher dest=”artifacts/${project.name}”

dir=”projects/ant-junit/profiling-reports”/>

</onsuccess>

In this case, CruiseControl is configured to check for source updates every five minutes—because performance tests take longer to run, they cannot and should not be run as frequently. In a real-world application, checking hourly is more reasonable.

Not only does the article describe the concept of running performance tests as part of CI, but it also shows how; we are also shown how to collect the results, using JMeter:

Once integrated, you define a JMeter Ant task by adding the following to the Ant script:

 <taskdef name=”jmeter”classname=”org.programmerplanet.ant.taskdefs.jmeter.JMeterTask”/>

This is a clear example of prior art. For nearly 40 years Dr. Dobb’s Journal was one of the foremost software magazines in the world (I remember in the late 1990s tentatively reading my first Dr. Dobb’s articles on sorting algorithms and mutexes, and realising how relevant Dr. Dobb’s seemed). There is no excuse for the HP people for missing this kind of prior art when it was so readily available.

To me the patents filed by HP feel like a ‘land grab’ by a company that is struggling to be relevant in the world of Continuous Delivery, with old-skool application release automation (ARA) tools simply rebranded as ‘Continuous Delivery’ (incidentally, may the gods help anyone who pays for these soul-destroying HP tools).

Okay, so what can I do to help?

For each of the patents above we need examples of prior art before 2012: articles, blog posts, conference talks, book chapters, etc. If a patent does not have an Ask Patent page, please create an Ask Patents page (the easiest way to do this is via Google – search for the patent code). Then post the relevant details on the Ask Patents page for the specific patent.

You can also search for other patents from HP relating to CD – perhaps using a different author name (the patents above all relate to Inbar Shani only).

Together let us get the patents overturned through decent prior art details and stop this madness. From the start Continuous Delivery has been a practitioner- and community-led endeavour (with generous guidance from ThoughtWorks). CD is not the place for large behemoth corporations like HP to muscle in and take ownership. As Dan North puts it:

“Identification of a failed code change” in a build pipeline. As a patent. Really @HP? #getoffmylawn

Footnote

Thanks to everyone who helped to spread the word about this, in particular Steve Smith, Dan North, Marco Abis, Pushpak Singh, and all who have added prior art details to the Ask Patents page and contributed to the Twitter WTF stream.

Updates

  1. Corrected ‘granted’ to ‘published’ in first main paragraph. (2015-03-08)

Continuous Delivery for databases: microservices, team structures, and Conway’s Law

The way we think about data and databases must adapt to fit with dynamic cloud infrastructure and Continuous Delivery. The need for rapid deployments and feedback from software changes combined with an increase in complexity of modern distributed systems and powerful new tooling are together driving significant changes to the way we design, build, and operate software systems. These changes require new ways of writing code, new team structures, and new ownership models for software systems, all of which in turn have implications for data and databases.

Read the full article on Simple Talk: Continuous Delivery for Databases: Microservices, Team Structures, and Conway’s Law.

(These slides were presented in a talk I gave at develop:BBC 2014 conference on 13th November in London)

Continuous Delivery eBook from Zend – views from 29 authors

I was recently asked to contribute to an eBook from Zend about moving to Continuous Delivery (CD). The 29 authors in the book share a wide range of experience with CD, and there is plenty of useful advice;  the contributions from Mathias Meyer (@roidrage), Kate Matsudaira (@katemats), and Jamie Ingilby (@jamiei) are particularly worth reading, I think.

In my section of the book I explain how using ThoughtWorks GO to model the testing and release steps (effectively part of the value stream) we won trust from several different people and teams during a move to CD. Using a prototype also helped us to validate the activities undertaken:

We tried to empathize with their situation and, using role-based security in the deployment pipeline, uncovered enough information to give them a sense of visibility and control.

Without being able to visualise and communicate easily the activities we were automating, progress would have been slow or even blocked.

Get a free copy of the eBook here: http://bit.ly/ZendCDbook

Zend eBook CD