Automate your Infrastructure by Reusing Terraform Definitions
Terraform is the tool for infrastructure as code that we use at Ona to automate the AWS resources we setup. Terraform can also be used with other cloud infrastructure providers. Find the full list of supported providers here.
Terraform allows us to reuse infrastructure definitions through modules. However, there’s little documentation on how this can be done across environments. As an example, we want to use the same set of definitions to provision OnaData’s production and staging environments. This post will take you through how we were able to achieve this.
1. Create a Reusable Module
All our Terraform work shares a single parent directory that we refer to as terraform in this post.
Keep the Terraform resource blocks for your setup in terraform/modules/<Name of setup>. For instance, the OnaData setup’s resource blocks are kept in terraform/modules/onadata. You can then isolate the Terraform variables, for each of the environments your setup is to be deployed in, in terraform/<Name of environment>/<Name of setup>/terraform.tfvars.
In OnaData’s case, from the above explanation we get the below directory structure:
terraform
├── modules
│ └── onadata
├── production
│ └── onadata
│ └── terraform.tfvars
└── staging
└── onadata
└── terraform.tfvars
2. Define Your Resource Blocks
You should now have a directory terraform/modules/<Name of setup> for what will be your setup’s Terraform module. As a requirement, the directory should have a main.tf file (considered the entry point). You should be able to, at this point, put all your variable definitions and resource blocks in the main.tf file. However, we anticipate that the main.tf file will grow in size (mainly due to how verbose resource blocks can get), to the point where it would be hard to maintain. Therefore, we recommend you split your resource blocks into the following files, and leave the main.tf file for shared data blocks:
compute.tf: Put resource blocks that define compute resources (like EC2 servers) in your setup here. Blocks for resources that are tightly linked to the compute resources should also be put here. Resources for CloudWatch alarms tied to compute or null resources that trigger ansible scripts to run on hosts are examples or tightly linked resource blocks.network.tf: Put resource blocks for networking resources here. This includes (but is not limited to) load balancer, DNS, and VPC resources.storage.tf: Put resource blocks for storage resources here. This includes (but is not limited to) S3 bucket, ElastiCache, and RDS resources.variables.tf: Put variable definitions here. A variable definition is its name, type and an optional default value. All variables, regardless of which file they are used in, should be defined here.
Terraform will automatically include all the .tf files in the module directory when run. The order of resource creation is dictated by resource dependencies and not the name of the file the resource definitions have been put in.
In OnaData’s case, the directory structure now looks like this:
terraform
├── modules
│ └── onadata
│ ├── compute.tf
│ ├── main.tf
│ ├── network.tf
│ ├── storage.tf
│ └── variables.tf
├── production
│ └── onadata
│ └── terraform.tfvars
└── staging
└── onadata
└── terraform.tfvars
3. Import Reusable Module In Environment Working Directories
Now that you’ve defined a Terraform module (or at least a shell of it) for your setup in terraform/<Name of module>, you can import it into the directories you had created for the different environments in step 1. You do this by creating a main.tf file in each of the environment directories (terraform/<Name of environment>/<Name of setup>/main.tf). In the main.tf file add:
provider "aws" {
region = "<AWS region deployment should be done>"
}
module "<Name of setup>" {
source = "../../modules/<Name of setup>"
variable1 = vars.variable1
variable2 = vars.variable2
.
.
.
variableN = vars.variableN
}
The environment directories are technically now referred to as working directories. You should theoretically now be able to run terraform init when in any of these directories.
However, you now need to define the variables vars.variable1, vars.variable2 etc used in the main.tf file you’ve just created. You can, of course add the variable definitions in the main.tf file but if we’re to follow the structure defined in step 2, the best place to do this is in a new variables.tf file in the same directory as the main.tf file. This would look something like:
variable "variable1" {
type = "list"
}
variable "variable2" {
type = "string"
default = "some default value"
}
.
.
.
variable "variable2" {
type = "map"
}
The variables.tf file should be very similar to the variables.tf file in the shared module you created in step 2 (terraform/modules/<Name of setup>/variables.tf).
Now add the values for the variables defined in the variables.tf file in the terraform.tfvars files created in step 1. The newly created main.tf and variables.tf files should be in the same directory as the terraform.tfvars file (called the working directory). The terraform.tfvars file will look something like:
variable1 = ["value a", "value b"]
variable2 = "some value"
.
.
.
variableN = {
"keyA" = "value A"
"keyB" = "Value B"
}
Terraform will prompt for values during execution for all variables defined in the variables.tf file that you don’t put values for in the terraform.tfvars file.
At this point, the directory structure for the OnaData setup now look like this:
terraform
├── modules
│ └── onadata
│ ├── compute.tf
│ ├── main.tf
│ ├── network.tf
│ ├── storage.tf
│ └── variables.tf
├── production
│ └── onadata
│ ├── main.tf
│ ├── terraform.tfvars
│ └── variables.tf
└── staging
└── onadata
├── main.tf
├── terraform.tfvars
└── variables.tf
4. Use Shared Remote Terraform States
By default, Terraform stores the state for your setup locally (in a terraform.tfstate file), on the host you’re running Terraform on. However, we strongly recommend that you store Terraform states in remote, shared stores like Amazon S3. You could also track the state files using Git, however, this is not recommended.
Add support for storing Terraform states for you setup in an Amazon S3 bucket by adding the following line at the top of each for the main.tf files for your setup’s per-environment working directories (terraform/<Name of environment>/<Name of setup>/main.tf):
terraform {
backend "s3" {
"bucket" = "terraform-states"
"key" = "<Name of your setup>-<Name of environment>.tf"
"region" = "eu-central-1"
}
}
Read more on Terraform state here.
5. Run Terraform
You should now be able to bring up or tear down resources for your setup in the different environments. To run Terraform against an environment, first make sure that you’re in the right per-environment working directory:
cd terraform/<Environment>/<Name of setup>
It is always good practice to run terraform plan before provisioning, or tearing down resources. plan will ensure that you are able to access the shared Terraform state and the AWS API. Most importantly, it will list out tainted resources for your setup. A resource is considered tainted if it doesn’t match up to what is defined in the Terraform files.
Other Terraform commands that will be useful to you at this point are:
terraform init: Initializes the Terraform working directory by creating initial files (in the .terraform directory), loading any remote state, and downloading modules. You only need to run terraform init once per working directory. Run terraform plan in the directory first if you’re not sure the working directory has been initialized.
terraform apply: Creates or modifies resources based on whether they are tainted or not. Non-tainted resources are not touched. However, tainted resources might be either recreated or modified depending on the type of change needed.
terraform fmt: Use this command to format the Terraform files you are working on.
terraform taint: Use this command to forcefully taint (to force recreation of) a resource. Since resource blocks are defined in the shared Terraform module you created in step 2, you will need to add a -module=<Name of setup> flag for the command to run.
terraform destroy: Use this command to tear-down all the provisioned resources.
References
We used the following resources as inspiration: