We keep all our Terraform files under the terraform directory.
Terraform allows to reuse definitions across environments. We thus don't have to have a set of definitions for provisioning Onadata resources in the production environment and a different set of definitions for provisioning Onadata resources in the staging environment. How we go about this is by creating a shared Terraform module for our setup, then including this module in Terraform files corresponding to each of the environments we plan to deploy the setup.
The following subsections will take you through how to codify your setup using Terraform, the Ona way. As a reference, we will use Onadata's Terraform files:
Keep the Terraform resource blocks for your setup in ../terraform/modules/<Name of setup>. For instance, the Onadata setup's resource blocks are kept in terraform/modules/onadata. Then isolate the Terraform variables for each of the environments your setup is to be deployed in in ../terraform/<Name of environment>/<Name of setup>/terraform.tfvars.
In Onadata's case, from the above explanation we get the directory structure:
terraform
├── modules
│ └── onadata
├── production
│ └── onadata
│ └── terraform.tfvars
└── stage
└── onadata
└── terraform.tfvars
You should now have a directory (../terraform/modules/) for what will be your setup's Terraform module. As a requirement, the directory should have a main.tf file (considered the entry point). You should be able to, at this point, put all your variable definitions and resource blocks in the main.tf file. We however anticipate that the main.tf file will grow in size (mainly due to how verbose resource blocks can get) to the point where it would be hard to maintain. We, therefore, recommend you split your resource blocks into the following files, and leave the main.tf file for shared data blocks:
- compute.tf: put resource blocks that define compute resources (like EC2 servers) in your setup here. Blocks for resources that are tightly linked to the compute resources should also be put here. Resources for Cloudwatch alarms tied to compute or null resources that trigger ansible scripts to run on hosts are examples or tightly linked resource blocks.
- network.tf: put resource blocks for networking resources here. This includes (but not limited to) load balancer, DNS, and VPC resources.
- storage.tf: put resource blocks for storage resources here. This includes (but not limited to) S3 bucket, ElastiCache, and RDS resources.
- variables.tf: Put variable definitions here. A variable definition is its name, type and an optional default value. All variables, regardless of which file they are used in should be defined here.
Terraform will automatically include all the .tf files in the module directory when run. The order of creation of the resources is dictated by resource dependencies and not the name of the file the resource definitions have been put in.
In Onadata’s case, the directory structure now looks like this:
terraform
├── modules
│ └── onadata
│ ├── compute.tf
│ ├── main.tf
│ ├── network.tf
│ ├── storage.tf
│ └── variables.tf
├── production
│ └── onadata
│ └── terraform.tfvars
└── stage
└── onadata
└── terraform.tfvars
Now that you’ve defined a Terraform module (or atleast a shell of it) for your setup in ../terraform/<Name of module>, you can import it into the directories you had created for the different environments in step 1. You do this by creating a main.tf file in each of the environment directories (../terraform/<Name of environment>/<Name of setup>/main.tf). In the main.tf file add:
provider "aws" {
region = "<AWS region deployment should be done>"
}
module "<Name of setup>" {
source = "../../modules/<Name of setup>"
variable1 = vars.variable1
variable2 = vars.variable2
.
.
.
variableN = vars.variableN
}
The environment directories are technically now referred to as working directories. You should theoretically now be able to run terraform init when in any of these directories.
You, however, now need to define the variables vars.variable1, vars.variable2 e.t.c used in the main.tf file you’ve just created. You can, ofcourse add the variable definitions in the main.tf file but if we’re to follow the structure defined in step 2, the best place to do this is in a new variables.tf file in the same directory as the main.tf file. This would look something like:
variable "variable1" {
type = "list"
}
variable "variable2" {
type = "string"
default = "some default value"
}
.
.
.
variable "variable2" {
type = "map"
}
The variables.tf file should be very similar to the variables.tf file in the shared module you created in step 2 (../../modules/<Name of setup>/variables.tf).
Now add the values for the variables defined in the variables.tf file in the terraform.tfvars files created in step 1. The newly created main.tf and variables.tf files should be in the same directory as the terraform.tfvars file (called the working directory). The terraform.tfvars file will look something like:
variable1 = ["value a", "value b"]
variable2 = "some value"
.
.
.
variableN = {
"keyA" = "value A"
"keyB" = "Value B"
}
Terraform will prompt for values during execution for all variables defined in the variables.tf file that you don’t put values in the terraform.tfvars file.
At this point, the directory structure for the Onadata setup now look like this:
terraform
├── modules
│ └── onadata
│ ├── compute.tf
│ ├── main.tf
│ ├── network.tf
│ ├── storage.tf
│ └── variables.tf
├── production
│ └── onadata
│ ├── main.tf
│ ├── terraform.tfvars
│ └── variables.tf
└── stage
└── onadata
├── main.tf
├── terraform.tfvars
└── variables.tf
By default, Terraform stores the state for your setup locally (in a terraform.tfstate file), on the host you’re running Terraform in. We, however, strongly recommend that you store Terraform states in remote, shared stores like Amazon S3. You could also track the state files using Git, however, this is not recommended.
Add support for storing Terraform states for you setup in an Amazon S3 bucket by adding the following line at the top of each for the main.tf files for your setup’s per-environment working directories (../../modules/<Name of setup>/main.tf):
terraform {
backend "s3" {
"bucket" = "terraform-states"
"key" = "<Name of your setup>-<Name of environment>.tf"
"region" = "eu-central-1"
}
}
Read more on Terraform state here.
You should now be able to bring up or tear down resources for your setup in the different environments. To run Terraform against an environment, first make sure that you’re in the right per-environment working directory:
cd terraform/<Environment>/<Name of setup>It is always good practice to run terraform plan before provisioning, or tearing down resources. plan will ensure that you are able to access the shared Terraform state and the AWS API. It will most importantly list out tainted resources for your setup. A resource is considered tainted if it doesn’t match up to what is defined in the Terraform files.
Other useful Terraform commands that would be most useful to you at this point are:
terraform init: Initializes the Terraform working directory by creating initial files (in the .terraform directory), loading any remote state, and downloading modules. You only need to run terraform init once per working directory. Run terraform plan in the directory first if you’re not sure the working directory has been initialized.
terraform apply: Creates or modifies resources based on whether they are tainted or not. Non-tainted resources are not touched. However, tainted resources might be either recreated or modified depending on the type of change needed.
terraform fmt: Use this command to format the Terraform files you are working on.
terraform taint: Use this command to forcefully taint (to force recreation) a resource. Since resource blocks are defined in the shared Terraform module you created in step 2, you will need to add a -module=<Name of setup> flag for the command to run.
terraform destroy: Use this command to tear-down all the provisioned resources.
For mainly the directory structure defined here, we used the following resources for inspiration: