Automation in the Cloud with Terraform

In this post, we explain the background and advantages of automating deployments in public cloud environments and explain automated provisioning using Terraform.

Why automate deployments

Cloud technologies come with a fairly new approach in design and deployment compared to traditional IT infrastructure. For instance, generic hardware replaces proprietary hardware. For the gain of more agile deployments, one has to accept new pricing models and loses control over the hardware in the case of public cloud concepts. These new preconditions lead to benefits on the one hand, but also to drawbacks on the other hand.

Although both AWS and Azure provide an all-supporting UI for deploying business cases one observes a poor operability and usability relatively quickly. Both cloud providers lack a UI, which should enhance the controllability and overview of an operator. Instead, rolling out instances via mouse-clicking, tasks become tedious and error prone. In fast growing environments with dozens of instances this circumstance potentiates accordingly.

This is not or not only caused by a poor UI design, but rather by the way how cloud computing is designed and deployed. The design of a cloud is different and also the way its deployed needs adjustments and should be leaned to the concept of clouds.

At this point, automation comes into play. Automation represents a key element in modern IT industry for deploying and controlling state of the art technologies. Automating systems is reached by mapping manual tasks to code-based sequences of tasks. The buzzword for automated cloud environments is Infrastructure as Code (IaC).

Infrastructure as Code (IaC)

With IaC, an infrastructure in the cloud can be initialized and declared on a code-basis in a first step and can be deployed in a second step instantly. IaC does not eliminate the complexity of modern systems, but shifts the complexity to another level of abstraction, which makes the deployment handier and more manageable. The shift leaves the path from mouse-click tasks to editing simple text files with a predefined syntax.

Besides configuration-management tools like Ansible, Chef or Puppet there are also tools available for provisioning the infrastructure itself automatically, like CloudFormation, Heat or Terraform. Casting the infrastructure into code is often applied in cloud environments, but other areas of deployment are possible.

Benefits of automated provisioning of infrastructures

Consistency:
An automated spawn of instances gives an increased control over the consistency of the infrastructure. Properties like names, descriptions or the size of instances are declared clearly and follow a predefined structure with every new roll out.
Reusability:
Once a template is ready and executable, it can be reused easily or can serve as an example-template for other scenarios. This helps engineers in saving time and costs during development and deployment. The reusability makes every element of a cloud infrastructure reproducible instantly.
Version Control:
Writing code is usually interrelated with version control. By keeping track of changes in the code one obtains a comfortable possibility of stepping forward and back of changes. This rollback option avoids trouble by erroneous deployments.
Error resistance:
By shifting recurring manual tasks to automated tasks the so-called tedious monkey work is eliminated, which leads to better control and reduces human errors.
Scaling:
The instant reproducibility of every part of the infrastructure makes services available continuously on high traffic peaks and saves costs on low traffic peaks, when unnecessary parts of the infrastructure can be terminated on demand.

Terraform

Terraform is an open-source tool developed by HashiCorp for orchestrating infrastructures. Cloud environments as well as bare metal infrastructure can be provisioned, but here the focus will be on cloud technologies. Xantaro utilizes Terraform for an automated roll in/out of instances or for destroying instances in cloud environments.

Properties of Terraform

Vendor agnostic:
The big advantage towards all the proprietary solutions from cloud providers like Cloud Formation (AWS), Azure Resource Manager or Cloud Deployment Manager (Google) is the compatibility with multiple cloud providers. One tool for all platforms means less effort in training and knowledge build up.
Immutable infrastructure:
A configuration drift within IT-Systems is a well-known issue and can be tedious to handle. It often leads to incompatibility between multiple instances. To avoid this drawback, the workflow of Terraform does not update instances, instead instances are deleted and new instances with an inserted update are spawned again, when it comes to changes. This decreases the configuration drift in the whole system of instances.
Declarative code style:
A declarative code is a more high-level language, which specifies the desired end state of a system. Logical structures like loops or if-statements, which are usual in procedural languages are not foreseen. The declarative code consists of instructions and it’s the task of the system behind, how to complete the request. These instructions are written in a key value format.
Ease of use:
Once the terraform-code is developed for a certain system, the deployment is comfortable with concise commands. A workflow with the most common terraform commands are presented in the picture below.
Documentation:
Terraform comes with extensive documentation for development and operations. The documentation is peppered with minimal examples which facilitates the entry in writing own code. Terraform is based on a big community, which pushes development and bug fixing faster than proprietary IaC solutions.

Terraform Workflow

The core Terraform workflow has three steps and a fourth step for termination:

1: Write – Author writes infrastructure as code.

2: Plan – Preview changes before applying.

3: Apply – Provision reproducible infrastructure.

4: Destroy – Terminate infrastructure

Write Terraform Code

The first code snippet shows an example of how to declare the provider Terraform shall connect to. In this case, an access-key, secret-key and a region need to be specified to connect to AWS. The access-key and secret-key have to be generated manually via the AWS web UI once. In this example, the keys are substituted by global variables, which is recommended regarding data privacy.

provider "aws" {
  access_key = "ACCESS_KEY"
  secret_key = "SECRET_KEY"
  region = "eu-central-1"
}

The next code snippet describes a minimal example of declaring the resources needed to spawn a Virtual Private Cloud (VPC) inside the AWS cloud. A VPC is an AWS specific term, which provides an enclosed environment for private use cases. The equivalent part in the Azure Cloud is called Virtual Network.

The keyword “resource” is a mandatory keyword of Terraform and initiates a construction for listing attributes, which describe the properties of an instance. Some attributes are essential, others are optional. The VPC needs at least an IP-prefix, all other attributes are optional, but its beneficial to assign a suitable name. A VPC is an essential requirement for spawning any server instances.

resource "aws_vpc" "test_netz" {
  cidr_block       = "10.10.0.0/16"

  tags = {
    Name = "Test_Netz"
  }
}

For spawning an instance (ubuntu) inside the VPC, three attributes are necessary. First, one has to declare the desired image. Therefore, the name of the AWS specific Amazon Machine Image (AMI) is needed. Secondly, the instance type needs to be declared. This specifies the size of the instance. The type “t2.micro” describes a size of 1 vCPU and 1 GB of memory. The bigger the size, the more powerful the instance, but also the higher the costs arise. Finally, a subnet is needed. A subnet is declared in a separate resource. The ID of the subnet is then referenced as an attribute in the instance section. The subnet itself needs a reference to the VPC-ID, a suitable IP-prefix to subnet the VPC IP-prefix and finally an availability zone.

resource "aws_instance" "ubuntu" {
  ami           = "ami-0093cac2bf998a669"
  instance_type = "t2.micro"
  subnet_id     = aws_subnet.ubuntu_subnet.id

  tags = {
    Name = "Ubuntu_Client"
  }
}

resource "aws_subnet" "ubuntu_subnet" {
  vpc_id            = aws_vpc.test_netz.id
  cidr_block        = "10.10.10.0/24"
  availability_zone = "eu-central-1a"

  tags = {
    Name = "Ubuntu_Subnet"
  }
}

This Terraform code example is a way of putting a desired infrastructure into code (IaC) and defines in this case a scenario for spawning one ubuntu instance in the AWS cloud. The result can now be applied via the Terraform workflow and the corresponding commands.