Continuous Delivery for databases: microservices, team structures, and Conway’s Law

The way we think about data and databases must adapt to fit with dynamic cloud infrastructure and Continuous Delivery. The need for rapid deployments and feedback from software changes combined with an increase in complexity of modern distributed systems and powerful new tooling are together driving significant changes to the way we design, build, and operate software systems. These changes require new ways of writing code, new team structures, and new ownership models for software systems, all of which in turn have implications for data and databases.

Read the full article on Simple Talk: Continuous Delivery for Databases: Microservices, Team Structures, and Conway’s Law.

(These slides were presented in a talk I gave at develop:BBC 2014 conference on 13th November in London)

Deployability for databases for Continuous Delivery – article on Simple Talk

I wrote an article recently for the Simple Talk website called Common database deployment blockers and Continuous Delivery headaches, where I outline some of the common problems preventing databases from being deployable – a major blocker to Continuous Delivery.

Deployability is now a first-class concern for databases, and there are several technical choices (conscious and accidental) which band together to block the deployability of databases. Can we improve database deployability and enable true Continuous Delivery for our software systems? Of course we can, but first we have to see the problems.

The recommendations include:

  1. Minimize changes in Production
  2. Reduce accidental complexity
  3. Archive, distinguish, and split data
  4. Name things transparently
  5. Source Business Intelligence from a data warehouse
  6. Value more highly the need for change
  7. Avoid Production-only tooling and config where possible [I mention this in my talk How to choose tools for DevOps and Continuous Delivery]

To address [these things] individually perhaps doesn’t seem too challenging, but to tackle deployability requires close, effective collaboration between developers, DBAs, and operations teams to achieve the right balance between rapid deployment and access to data.

Deployability for Databases

Read the full article here

Oracle on Windows Seminar

I recently attended an “Oracle on Windows” seminar in London, organised by Oracle and sponsored by Quantix.

There were four speakers:

  1. Mark Whitehorn – independent DB consultant
  2. Julian Boneham – Quantix
  3. Paul Brankin – Oracle
  4. Jules Lane – Oracle

The talk from Quantix was unfeasably dull; full of sales-speak (“…all vertical markets…”) and delivered by a bloke who clearly wasn’t interested.

Paul Brankin talked about data silos and how this architecture arises (it’s simple to manage initially), how it becomes limiting or fragmenting in business terms (due to inaccessible data, and under- or over-utilization of hardware resources dedicated to a single silo) and how Oracle’s Real Application Clustering (RAC) can help solve the problem. Specifically, Oracle has an Active-Active database failover solution, whereas Microsoft’s SQL Server has only Active-Passive (in the form of mirroring).

Jules Lane gave an overview of the Oracle middleware application stack. Interestingly, he said …we are not really expecting people to write Java code any more…, but instead to rely on component configuration and code-gen tools alone when building middleware applications. The Oracle BPEL Process Manager is akin to BizTalk, as a business process orchestrator, although seems more advanced. No mention of WF/WCF in Jules’s talk, though of course this technology is still fairly new. Also interesting was the Oracle Webservices Manager, which allows policy-based access control to web services, including ASP.NET web services.

In general, the talks by Oracle and Quantix were disappointing; they were generally too sales-focussed, and their “Oracle on Windows” pitch was somewhat embarassed, as if Windows was something they only supported grudgingly. Far more engaging was the first session, by the independent consultant Mark Whitehorn.

Mark – much to the later ire of the Oracle speakers – said categorically that all three major database engines (DB2, Oracle and SQL Server) are extremely competant database engines, and that debates about their relative merits are pretty arcane and irrelevent. He noted that many people choose databases on religious grounds, proffering out-of-date evidence for why one engine is superior to another (e.g. “it’s a poor man’s Sybase fork” or “it cannot even row-lock”).

He stressed the need to look at the other kinds of tools and features available for these engines as more important reasons to choose one over the other:

  • Analysis (e.g. Business Intelligence [BI] tools)
  • Middleware connectivity
  • Server/Database Management

Mark then went on to give some lucid examples of real-world BI analysis (on 150 year old plant specimen records, collected by Charles Darwin!) to demonstrate how useful this kind of analysis can be, in combination with human domain experts.

He finshed by commenting on the new Spatial data types offered by the three database engines, and how some amazing results can be had using “mashups” (think Google Maps).

Mark was an extremely entertaining speaker, who clearly is an outstanding specialist in his field, and it was a pleasure to listen to what he had to say. By contrast, the other speakers seemed rather awkward and apologetic! It was clear from this seminar that Oracle is still well-placed for extremely high-end database applications, but similar resiliency CAN be implemented using SQL Server, at a reduced cost. All three vendors, but especially Oracle and Microsoft, are increasingly competing head to head for the Enterprise AND medium-size database markets, and this trend is set to continue for the next five years at least.