The Practical DevOps playbook

Schedule a demo

What is DevOps?

DevOps is a set of practices that help companies innovate faster on software applications by ensuring that releases are frequent, predictable, and error free.

Every company today is under pressure to be great at developing and delivering software. Most companies have figured out how to accelerate software development through agile principles. However, the bottleneck is now software delivery. What is the point of producing code at a faster rate if you can't ship it as quickly?

DevOps is all about removing bottlenecks from the software delivery workflow. This means minimizing manual handoffs, creating transparency across the people involved in software delivery, and measuring the effectiveness in moving business value quickly through the pipeline and into the hands of customers.

The reality of DevOps today

Everyone talks about 'DevOps culture' but culture is just the symptom. The underlying problems with DevOps are a general lack of clarity around what success looks like, a fragmented toolset without end to end visibility, and ad-hoc implementations.

DevOps done right will transform your organization, but the path to getting there is riddled with challenges. Most organizations who've tried to adopt DevOps practices are nowhere close to realizing the benefits of this approach in spite of pouring a lot of time, money and effort into it. There is a lot of frustration around this and every vendor pretends to offer a silver bullet, even if their solution addresses only a small portion of the overall end to end workflow.

DevOps guides today focus extensively on 'DevOps Culture', which is mainly about Devs and Ops getting along and trying to work better together. This can be misleading since the problems with DevOps go far beyond culture. Until you fix the root causes, trying to change culture is like putting lipstick on a pig. It makes the pig pretty, but IT’S STILL A PIG.

Culture builds over a period of time due to ground realities. Ops teams don’t hate change because they are irrational or want to be blockers. Over the years, they’ve taken the heat every time an over-enthusiastic Dev team tried to fast track changes to production without following every step along the way. Seating them with the Devs might help make the work environment a little friendlier but it doesn’t address the root cause, no matter how many beers they have together.

When you address the underlying problems, the culture in your organization will change automatically. You will be able to ship software as fast as it is developed and Ops teams will feel confident that software releases will be predictable and error free. Slowly but surely, Dev and Ops teams will develop a mutual respect which will go a long way in helping you innovate faster and win.

Remember: The only way to fix culture is to address the problems that led to it in the first place.

The underlying causes behind most failed DevOps implementations are the following:

No clear definition of success

Most organizations do not know where they are or where they need to be in terms of DevOps maturity. How do you know you’re ‘Done’? Is it when you ship daily? Weekly? Do you think you’re done just because you think your current release frequency is the best you can achieve? Or are you really proud of your ability to ship frequently and without errors?

Fragmented DevOps tools

The DevOps tools available today are fragmented point solutions that only help you automate specific tasks in particular stages of the DevOps workflow. You can string them together to create your pipeline, but you lose fidelity along the way since the information surfaced by each tool is specific to the task it performs. Since each team uses their own tools in a silo, other teams don't have visibility unless they are included in every conversation. Most teams create manual processes to communicate changes that affect others, but this is error prone and inefficient.

Non-scalable approach

DevOps efforts are usually ad-hoc and are not handled holistically across the entire organization. Each application needs its own custom pipeline, so you spend a lot of time writing pipelines instead of your real applications. Microservices architectures make this problem exponentially worse. All this leads to complex and fragmented spaghetti automation which, when combined with poor cross team communication, eventually leads to chaos.

DevOps Done Right

Doing DevOps right means approaching DevOps in a systematic, platform-centric fashion and letting the culture fall into place naturally. Fix the root causes and the symptom will go away.

The only way to address the challenges described above is approach the problem in a systematic way. You need a standard playbook, a map, that you can follow step by step to achieve DevOps maturity for each application. This playbook needs to be developed as a platform that standardizes how DevOps is adopted across all applications across your organization.

Your playbook should include the following key aspects:

A quantitative approach to answering the questions: Have I achieved all I can with DevOps? Am I ‘Done’? We call this the DevOps Reality Score.

A standard approach to implemented DevOps Automation, i.e. a platform that can connect the various tools available to give you end to end pipelines that preserve fidelity and visibility in a standard fashion.

Seven stages of software delivery

Stages of software delivery

DevOps Reality Score

What does ‘DevOps maturity’ mean? We believe that your application has reached DevOps maturity when you have complete automation, aka Zero Touch Automation, across all stages of your software delivery workflow. The closer you get to this ideal, the faster you can deliver value to your customers while also improving quality.

Measuring DevOps reality with a quantitative score

The DevOps Reality Score Matrix

Achieving Zero Touch Automation across all applications across your organization will get you to Full DevOps maturity. You can easily gauge where you currently are by creating the DevOps Reality Score matrix for each of your applications. The matrix measures your level of DevOps maturity by estimating what portion of your software delivery workflow is automated, i.e. how far you are from achieving Zero Touch Automation for the application. In the example matrix above, the DevOps maturity is 6/16, i.e. 37.5%.

Complete this exercise for all applications in your organization and average the results. This gives you a DevOps Reality Score, which tells you where you are with your DevOps efforts across the organization. Adopting a standard quantitative model like this will help you standardize how DevOps is understood and tackled across your organization. You are DONE when your score is 100%.

DevOps Automation Platform

Just like the DevOps Reality Score standardizes your understanding of DevOps, your automation efforts should also leverage a common platform that standardizes how DevOps pipelines are implemented. This is crucial to doing DevOps right since it will prevent the creation of one-off custom pipelines that are difficult to manage and scale. The Automation platform should be adopted as a standard across your organization so that all DevOps pipelines follow a standard approach that leverages the platform.

Our systematic approach models the ideal DevOps platform as a Cube. The top face of the cube is the "Automation Engine" which helps you configure Zero Touch Automation across all stages of your DevOps workflow. This uber goal is supported by the four sides of the cube, namely “State”, “Visibility”, “Operations” and “Runtime”. These 4 sides, or pillars, are supported by the bottom layer, a native "Integrations" layer that acts as the foundation. This layer lets you use any tools you want for parts of the DevOps workflow without needing to rip-and-replace.

Let us take a deeper look at what you need for each side of this DevOps Cube.

Practical DevOps playbook

The DevOps Cube

Automation engine

The Automation engine is a declarative workflow engine that makes it easy to automate DevOps pipelines across all environments, projects, and tools. The requirements for this pillar are:

  • Your pipelines can be configured in a declarative language that is easy to learn and master
  • You can reuse and recombine portions of your pipeline config to easily create new pipelines.
  • You can templatize frequently used tasks to ensure they can be repeated in a consistent manner.
  • You can create pipelines for any application, regardless of language and toolchain without needing to rip-and-replace.

Job Runtime

This is the infrastructure and environment preparation required for executing the workflow configured using Automation Engine. The requirements for this pillar are:

  • Your infrastructure is scalable on-demand and can be anywhere: in the public cloud, inside your VPC, in your data center, or a mix of all three.
  • You can control the build environment for each application and have an automated way to update and refresh build images.
  • You can use environment variables and avoid hard coding code anything. You can make changes in one place and all pipelines adapt magically.
  • Your build environment can be dynamically prepped ,configured and refreshed for every build depending on context.

State

The storage of stateful information related to the DevOps activities is critical for your success. This includes service discovery, secrets/config, artifacts, and DevOps metadata. The requirements for this pillar are:

  • Sensitive information like passwords, tokens, keys is encrypted abstracted from automation code for security and maintainability.
  • You need out of the box Service discovery to enable your services to find each other and communicate.
  • You need complete history of the state of every job and resource in your pipelines. This is a must for audit purposes.
  • Critical information generated at runtime should be versioned and made available to following tasks with high fidelity.

Visibility

All stakeholders need complete visibility into your DevOps pipelines. Your platform need to offer reporting, audit and analytics on the status and performance of all DevOps activities. The requirements for this pillar are:

  • You need a single, consistent view of the status of your DevOps pipelines across all applications with the ability to dig in deeper if required.
  • You need a way to identify how business value flows through your pipelines and where the bottlenecks are.
  • You need to track performance metrics at a application, team, and individual level.
  • Your platform needs to track your progress against achieving DevOps maturity with zero-touch automation across your entire organization.

Operations

Your platform should allow for operational management of all DevOps activities like monitoring and auto-scaling of DevOps infrastructure, or rolling back deployments. The requirements for this pillar are:

  • You should be able to deploy new application versions with zero downtime. You can easily implement release strategies like canary or blue-green deployments.
  • You need the ability to roll back your applications with a single click in case you discover problems with your release.
  • Your platform needs to monitor and surface data about the utilization and performance of your automation infrastructure.
  • Your infrastructure needs to be scaled up on-demand as load increases or scaled down during periods of low utilization.

Integrations

Your platform needs to natively support all tools, languages, and services used across all applications. It should be flexible enough to support new technologies with minimal to no rewrites. The requirements for this pillar are:

  • Your platform needs to integrate in a consistent fashion with all third party tools, languages, and services required by all applications across all stages.
  • You need to have complete visibility into each stage of your pipeline regardless of the tool used.
  • Adding new integrations should be plug and play and should not require any major re-writes of the platform.
  • Your platform needs to be future proof allowing you to add modern tools with ease.

Build or Buy?

You can achieve DevOps success by approaching it in a systematic fashion with a standard playbook and a powerful Automation platform. Your platform needs to have all the capabilities discussed in the section above to help you achieve the DevOps nirvana of Zero Touch Automation for all applications in your organization.

You need to decide whether it makes sense for you to build or buy this platform. Companies who have successfully implemented DevOps, like Netflix, Amazon and Facebook, have built the automation platform in-house over several years with large teams and millions of dollars of investment. Even if you don’t need the scale they do, you will still need to invest time and money into hiring a DevOps team and building all this yourself.

Your other choice is to buy an out of the box platform with which you can start automating your DevOps workflows immediately. Shippable was built to be exactly this platform for organizations that want to get a head start on DevOps and systematically adopt it across their applications. We enable Zero Touch Automation for all types of applications, from traditional Java applications being deployed to a VM to modern Docker-ized applications deployed as microservices. Shippable already includes all faces of the DevOps Cube, including native integrations with all popular third party tools you might already be using for parts of your end to end software delivery workflow.

Learn how Shippable's DevOps Platform can get you started today!

Learn More