Flex your Agile Software Development and Delivery with Containers and Flash Storage

Containers and containerization have ushered in a sea change in the way in which applications are both developed and deployed, but what are containers and what value can modern all-flash storage can add to this story?

Containers and Their Lineage

Containers are a form of virtualization, but they should not be confused with virtual machines. In essence, containers are like sand boxes, which run on top of an operating system without having to run a full-blown operating system image or use a hyper-visor. Simply put, containers provide a form of process isolation that is much more lightweight than virtualisation.

The concept of a container can be traced all the way back to chroot in Unix 7 in 1979 when isolating a process to its own execution environment was first introduced. Fast forward to 2013 and Docker® was born, sowing the seeds of the explosion in container usage we see today. There are other means of running containers such as LXC on Unix or rkt by CoreOS, however Docker is far and away the most popular container engine.

Who Is Using Containers?

Google™ probably stands out as the “Container poster child” in that they run virtually their entire infrastructure on containers, however Netflix®, Spotify® and eBay® are also notable container users:

What Applications Are People Using Containers For?

DockerHub is the public Docker registry used to store container images, think of a container image as a “Cookie cutter” like template used to create containers. A glance at the top ten most pulled images provides some colour as to Docker’s usage:

Note that mysql, mongo and postgres all appear in list of the top ten most pulled images; this alludes to the fact that people are using containers to store data.

Why Are Containers So Popular?

By virtue of their lightweight nature, container usage leads to better use of compute resources, also consider this simple Jenkins build pipeline:

This performs the following functions:

  • It checks out code from a local GIT repository
  • The code is built into a deploy-able artifact
  • A container using the Linux® SQL Server image is spun up
  • The code is then deployed to the container
  • There is a stub in which some tests can be run
  • Finally the container is stopped and removed

Take the step of starting the container, were this to be a virtual machine deployed in any IT environment that has some rigour around processes and procedures, this might involve:

  • Talking to an infrastructure team about getting the virtual machine created
  • Talking to a networking team about the assignment of an ip address and port numbers the virtual machine can use
  • Talking to whoever manages the domain about being able to join this virtual machine to the domain
  • Talking to the infrastructure team again about the requirements of the virtual machine to be patched, backed up be scanned for viruses etc . . .

This is anything but an agile process, then compare this to the two lines of code in the script excerpt above that are required to start the container, stop and  remove it:

The value that containers add to continuous integration and build processes does not end here, it’s common practice for Jenkins build slaves (‘Slaves’ being processes that can participate in a build) to be run as containers.

Another attraction of containers is that irrespective of where a container is run, it always executes the same way, whether this is on a laptop, on premises server, public cloud, Linux or Windows®. Consider a problem that most developers have faced in their time:

This is the “The code worked in test but went bang in production problem” and the main reason for this that there is some environment difference between test/staging and the production environment. To distil this problem down further, this exists due the tight coupling that can exist between the code and the environment it is running on. There are a slew of desired state configuration tools now available such as Ansible®, Puppet®, Chef® and Salt, however, the beauty of containers is the consistency you get without having to go to any effort to manage the underlying environment the container engine is running on:

Software Architectures

Up until the turn of the millennium, the overriding trend in software design was to structure software was multi-tiered, typically there would be a data tier, a business logic tier, a presentation logic tier and perhaps an orchestration tier. It was common practice to implement each tier as a monolithic slab of code. Then service oriented architectures (SOA) came along. A service-oriented architecture is a technology and platform agnostic way of building out a software architecture from components (services) exposed via APIs. Wikipedia defines the key characters of a service as:

  1. It logically represents a business activity with a specified outcome.
  2. It is self-contained.
  3. It is a black box for its consumers.
  4. It may consist of other underlying services.

The ultimate goal of this style of software architecture is to be able to compose systems from different components irrespective of the underlying technology used to implement these services and a whole middle-ware industry emerged off the back of this. The focus of a service-oriented architecture was exposing software via service APIs and having a homogeneous API layer across all the software assets in the architecture.

Enter Micro Services

Consider a simple on-line bookstore application which consists of order processing, inventory management and payment handling functionality, designed using monolithic layer approach:

Software designed in this way presents several challenges, a single change to either the order processing, inventory management or payment processing modules, requires that the whole business logic tier is redeployed. Secondly, the tight coupling between the three different functional areas means that they cannot be developed interdependently of one another and a defect in one area may impact the whole business logic tier.

Micro-services are the evolution of service-oriented architectures, there is no formal definition of what a “Micro-service” is per se. However, in this recording from the GOTO 2014 conference, Martin Fowler outlines nine common characteristics of micro services:

  1. Componentization via services
  2. Organized around business capabilities
  3. Products not projects
  4. Smart endpoints and dumb pipes
  5. Decentralized governance
  6. Decentralized data management
  7. Infrastructure automation
  8. Design for failure
  9. Evolutionary design

Lets take a look at what the online book store application looks like when designed using micro-services:

The tight coupling between the three functional areas is removed and along with this the inability to develop components interdependently of one another and the lack of isolation between functional areas from bugs / defects elsewhere in the software. The point to all of this is that containers are a perfect fit for the deployment of micro services, in that a container is ideal for encapsulating a micro service:

For those wishing to learn more about Micro services, this presentation by Sam Newman, a thought leader in this space, from Devvox 2015 and “The State of Art In Microservices” by former Netflix architect Adrian Cockcroft are highly recommended.

Where Does Storage Come Into This Picture?

The need to persist data within containerized applications is clear, containers with persistence lend themselves extremely well to continuous integration pipelines and finally we have increase in popularity of micro-services oriented architectures. All of these factors mandate the requirement for agile storage. Consider the steps involved in provisioning storage the conventional way:

With Pure’s Docker certified volume plugin, freely available from the Docker store here, these five steps become a single command line that integrates into any scripted or automated build pipeline with ease:

Along with the Pure Docker volume manager plugin, a number of other plugin’s and code samples can be found at code.purestorage.com.

The Bottom Line

2016 marked the shift to a new order dominated by organisations whose lifeblood is software, a fact illustrated by the top five publicly traded companies by market capitalisation over the last fifteen years:

Agility in developing and delivering software is a key requirement for competitive and disruptive companies. In this new digital order, agile software development and delivery via containerization will become the new normal. Pure’s data platform will continue to evolve along with this new landscape and deliver the simplicity, reliability, and performance that our customers love.