Terraform - Module Meta-Arguments

In Terraform, a meta-argument is a special kind of argument that can be used to customize the behavior of a resource block, Module block, and Data block.

Meta-arguments are the way to give you more control over how Terraform creates, updates, and destroys your infrastructure. Using them, you will be able to build complex and flexible configurations of the infrastructure.

Terraform provides several meta-arguments that can be used to customise Module behaviour. The meta-arguments used are as follows:

depends_on

count

for_each

provider

depends_on is a metaargument in Terraform that explicitly specifies dependencies of Modules. It is used where Terraform cannot, by itself, automatically infer the dependency between modules.

Terraform does an outstanding job automatically inferring dependency relationships between modules via their access of each other's data. That is to say: if module B uses module A's output to configure itself, terraform will automatically create A before B.

However, there are scenarios when one module depends on the behaviour of another module, where it does not use any data of that other module in its configuration. In such cases, Terraform can't automatically compute the dependency between the two modules. That's where depends_on comes in place.

Now, suppose you have two modules, A and B. Module B depends on the behaviour of module A, but it doesn't, in fact, use any of the data or output from module A in its configuration. That is to say, B requires A to have been created, or otherwise configured, prior to its own creation/configuration, but it does not reference A's data in any way.

In such a case, you will want to use depends_on to tell Terraform that B depends on A's behavior. This says, in effect, even though B did not use any data from A, Terraform needs to make sure A is created before B is created.

You have two modules: one is security_group (Module A), which creates a security group with a certain set of rules, and another is server (Module B), which will create an EC2 instance and will be associated with the security group created by Module A.

Here, Module B acts like the server or client depends on the behavior of Module A, which is, in this case, security_group. This is because an instance wants to be associated with a particular security group upon creation. However, the configuration of an EC2 instance in Module does use any output from the Module of a security group, for instance, its ID or ARN.

In order to ensure that Terraform will create the security group before attempting to create the EC2 instance, you would make sure depends_on is used to indicate Module B depends on the behavior of Module A:

// File: security/main.tf (Child Module)
resource "aws_security_group" "my_sg" {
    name = "Example Security Group"
    vpc_id = aws_vpc.example.id
    ingress {
        from_port = 80
        to_port = 80
        protocol = "tcp"
        cidr_blocks = ["0.0.0.0/0"]
    }
    egress {
        from_port = 0
        to_port = 0
        protocol = "-1"
        cidr_blocks = ["0.0.0.0/0"]
    }
}
// File: instance/main.tf (Child Module)
resource "aws_instance" "example_ec2_instance" {
    ami = "ami-0123456789"
    instance_type = "t2.micro"
    tags = {
        Name = "my-ec2-instance"
    }
}
// File: main.tf (Root Module)
module "security_group" {
    source = "./security/main.tf"
}
module "server" {
    source = "./instance/main.tf"
    depends_on = [module.security_group]
}

In Terraform, a module block in Terraform commonly creates one set of infrastructure resources. Sometimes you want to manage more than one identical set of infrastructure resources without writing or calling a separate module block for each set.

For this purpose, Terraform provides two meta-arguments: count and for_each. In this explanation, we will explain the count meta-argument.

Adding a count argument to a module informs Terraform to create several instances of this module. The value of count must be a whole number and indicates how many instances to create. Each instance has an associated infrastructure object, and each is created, updated or destroyed separately as this configuration is applied.

Here is an example using count on an AWS EC2 instance resource:

// File: instance/main.tf (Child Module)
resource "aws_instance" "example_ec2_instance" {
    ami = "ami-0123456789"
    instance_type = "t2.micro"
}
// File: main.tf (Root Module)
module "server" {
    count = 4
    source = "./instance/main.tf"
}

Here, Terraform is going to create four instances of module. Each of them will have an infrastructure object associated with a distinct one, and each will separately be created, updated, or destroyed upon applying the configuration.

When using the count meta-argument with a module, Terraform also exposes an additional object, count, in expressions. This object can be used to customize the configuration of each instance that the count meta-argument creates.

The count object has only one attribute:

count.index: This attribute returns the distinct index number starting with 0 for each instance that the count meta-argument creates.

For instance, consider a resource block using the count meta-argument to create three instances of the compute instance.

// File: instance/main.tf (Child Module)
resource "aws_instance" "server" {
    ami = "ami-0123456789"
    instance_type = "t2.micro"
    tags = {
        Name = "Ec2-${index}"
    }
}
variable "index" {
    type = string
}
// File: main.tf (Root Module)
module "server" {
    count = 4
    source = "./instance/main.tf"

    index = "${count.index}"
}

In the above example, the count.index attribute would take the values 0, 1, and 2 for each of the three instances created and would then pass those values to the child module in order to create machines with different tags "Ec2-0", "Ec2-1", and "Ec2-2".

In Terraform, the count argument is used to create a module multiple times. Terraform identifies multiple modules by assigning an index number for each instance, relying on the starting index 0.

Modules can be referred in Terraform in the following two ways:

Entire Module Block: To refer to the entire module block, it should be in the format <module>.<NAME>. For instance, module.server refers to the whole module block.

Individual Instances: An individual instance is referenced by adding the index number in square brackets. Syntax is <module>.<NAME>[<INDEX>]. For example, module.server[0] refers to the first instance, while module.server[1] refers to the second, and so on.

The for_each metaargument is a way to create multiple instances of a module block according to a collect of values. If a module block includes a for_each argument whose value should be either a map or a set of strings, then Terraform will create an instance for each member of that map or set.

Each instance is handled independently, with its own set of infrastructural object associated with it. This means each instance will be independently created, updated, or destroyed based on the changes you've made when applying your Terraform configuration.

NOTE: A given module block can't use the count and for_each at the same time.

// File: s3-bucket/main.tf (Child Module)
resource "aws_s3_bucket" "buckets" {
    bucket = var.bucket_name
    region = var.bucket_region
}
variable "bucket_name" {
    type = string
}
variable "bucket_region" {
    type = string
}
// File: main.tf (Root Module)
module "bucket" {
    for_each = tomap ({
        "my-bucket-1" = "us-east-1"
        "my-bucket-2" = "us-west-2"
        "my-bucket-3" = "eu-east-1"
    })
    source = "./s3-bucket/main.tf"

    bucket_name = "${each.key}"
    bucket_region = "${each.value}"
}

The tomap() here is a Terraform function that converts a list of key-value pairs into a map. The for_each meta-argument will step through the map, creating one instance of module for each key-value pair. The each.key and each.value expressions are used to access the key and value of each pair, respectively.

Here is an example where for_each is used on a set of strings to create many AWS RDS:

// rds/main.tf (Child Module)
resource "aws_db_instance" "database" {
    for_each = toset ([ "database-1", "database-2", "database-3" )]
    allocated_storage = 10
    engine = "mysql"
    instance_class = "db.t2.micro"
    db_name = var.db_name
    username = "myuser"
    password = "mypassword"
}
variable "db_name" {
    type = string
}
// File: main.tf (Root Module)
module "rds_instance" {
    for_each = toset ([ "database-1", "database-2", "database-3" )]
    source = "./rds/main.tf"

    db_name = "${each.key}"
}

Here, toset() is a Terraform function that takes a list of strings in and returns a set. The for_each meta-argument will iterate over that set and create one instance of the module for every string in that set. Finally, the each.key expression is used to access the string value of each item of that set using the db_name variable as the database name.

When you use the for_each inside of a Terraform block, you get this special object called each inside that block. So, it can be used to modify the configuration of each instance.

Each object has only two attribute:

each.key: The identifier of each entity within the collection. If you are working with a map, that is a collection of key-value pairs, then each.key will return the key in the pair. If you're working with a set, meaning a collection of just unique values, then each.key yields the actual value.

each.value: This is the value associated with your key in the map. In case of a set, each.value is same as each.key as sets do not have key-value pairs.

Following is an example using AWS EC2:

// File: server/main.tf (Child Module)
resource "aws_instance" "instance" {
    ami = "ami-1234567890"
    instance_type = var.server_type
    tags = {
        Env = "${var.env}"
    }
}
variable "env" {
    type = string
}
variable "server_type" {
    type = string
}
// File: main.tf (Root Module)
module "rds_instance" {
    for_each = tomap ({
        "dev" = "t2.micro"
        "prod" = "t2.large"
    })
    source = "./server/main.tf"

    env = "${each.key}"
    server_type = "${each.value}"
}

In the above example, Terraform will create two instances of the rds_instance module: one for "dev" and one for "prod". Within this block, the each object is available, whose properties can be used to configure each instance individually.

In the "dev" instance, each.key would be "dev" and each.value would be "t2.micro". Thus, the env variable is to be set to "dev", and the server_type is to be "t2.micro". Wherein, for an instance "prod", each.key will be "prod" and each.value will be "t2.large". So, env is set to "t2.large" and server_type will set to "t2.large".

When you attach a for_each to a Terraform block, Terraform will make several repetitions of that module. In order to distinguish between them, Terraform uses the map key (or set of members) from the value provided to for_each.

There are two ways to reference modules in Terraform:

Entire Module Block: If you want to refer to the block itself, you can use the following syntax <module>.<NAME>. Example - module.bucket will refer to the whole module block.

Individual Instances: In order to address individual Instances, you need to add the key of the map or the set member respectively to the reference. The syntax is the following: <module>.<NAME>[<KEY>]. Example: module.bucket["my-bucket-1"] refers to the first instance and so forth.

The for_each meta-argument is a powerful feature in Terraform. It allows one to create modules multiple times, based on a map or set of values. When using module, the for_each returns a map of objects where each key represents an individual instance and the respective value represents the attributes of that instance.

You could establish one-to-one relationships among modules by using the output from one for_each block as the input for another for_each block. This approach is referred to as "Chaining for_each between resources."

Consider a scenario in which we need to create multiple EC2 instances and assign an Elastic IP address uniquely to each of the created instances. One can use for_each for creating EC2 instances, then make the output of that block as an input for another for_each block creating Elastic IP addresses.

// File: server/main.tf (Child Module)
resource "aws_instance" "instance" {
    ami = var.ami
    instance_type = var.instance_type
    tags = {
        Name = "Instance with type ${var.instance_type}"
    }
}
variable "ami" {
    type = string
}
variable "instance_type" {
    type = string
}
output "id" {
    value = aws_instance.instance.id
}
// File: elasticip/main.tf (Child Module)
resource "aws_eip" "example" {
    instance = var.id
}
variable "id" {
    type = string
}
// File: main.tf (Root Module)
module "server" {
    for_each = tomap ({
        "t2.micro" = {
            "ami" = "ami-abc123"
            "instance_type" = "t2-micro"
        },
        "t2.small" = {
            "ami" = "ami-def456"
            "instance_type" = "t2-small"
    }
    })
    source = "./server/main.tf"

    ami = "${each.value.ami}"
    instance_type = "${each.value.instance_type}"
}
module "elastic" {
    for_each = module.server
    source = "./elasticip/main.tf"

    id = "${each.value.id}"
}

In this example, a server module creates EC2 instances using the aws_instance resource. The for_each meta-argument is used to create multiple instances, with each instance having a unique ami and instance_type coming from the map provided. An elasticip module creates Elastic IP addresses using the aws_eip resource. The for_each meta-argument is used once again, this time the input coming from the output of the other module, server, in the form of the id of each EC2 instance:

In other words, by setting the server output of the module as input for the elasticip module, you relate each of the two kinds of resources, namely the EC2 instances and Elastic IP addresses, one to one. That's what is meant by "chaining for_each between modules".

When using for_each with a map or set of values, the keys of the map, or all the values in case of a set of strings, must be known values. That is, values present in for_each can't be empty during the Terraform configuration apply phase. If you try to use unknown values, you get an error message indicating that for_each has dependencies that cannot be determined before apply.

Sensitive input variables, sensitive outputs, sensitive resource attributes are particularly sensitive values that cannot be used as an argument for for_each. The value used in for_each is to identify the resource instance and will always be disclosed in UI output during plan which can potentially reveal sensitive information. Using sensitive values as an argument to for_each will result in an error.

The providers meta-argument in a Terraform module call is used to specify which provider configurations are available to the child module from the parent module. By default, a child module inherits all default provider configurations from its parent. Therefore, this means that if no providers argument is given, the child module will contain all the default provider configurations from its parent.

NOTE: Non-default provider configurations are never inherited automatically from the parent module into the child module.

If you need to customize the provider configurations available to a child module, you can specify the providers argument. The value is a map where:

Keys are the provider configuration names used inside a child module.

Values are provider configuration names from the parent module.

For example, let's assume we have one child module, instance/main.tf, and it has used one aws_instance resource. In the parent module, main.tf, we have defined two AWS provider configurations. One is for the us-east-1 region, which is the default provider by default, and another for the us-west-2 region, a non-default provider with alias west-region. No provider has been specified for the child module; hence, it will by default inherit all the default providers of the parent module. In the child module, to use the west-region provider configuration we need to specify the providers argument in the module call:

// File: instance/main.tf (Child Module)
resource "aws_instance" "instance" {
    ami = "ami-0123456789"
    instance_type = "t2.micro"
    tags = {
        Name = "my-ec2-instance"
    }
}
// File: main.tf (Root Module)
provider "aws" {
    region = "us-east-1"
}
provider "aws" {
    alias = "west-region"
    region = "us-east-1"
}
module "server" {
    source = "./instance"
    providers = {
        aws = aws.west-region
    }
}

This would instruct Terraform to make use of the west-region provider configuration in the parent module to create an aws_instance resource in the child module.

Sometimes, we may want to provide many configurations of the same provider to a child module. For instance, suppose we have a child module called database that creates a database in AWS. However, this time we want to create databases in two different regions, us-east-1 and us-west-2. In this case, the child module database requires two different configurations of the AWS provider, one for each region.

// File: rds/main.tf (Child Module)
resource "aws_db_instance" "database1" {
    provider = aws.west
    allocated_storage = 10
    engine = "mysql"
    instance_class = "db.t2.micro"
    db_name = var.db_name
    username = "myuser"
    password = "mypassword"
}
resource "aws_db_instance" "database2" {
    provider = aws.east
    allocated_storage = 10
    engine = "mysql"
    instance_class = "db.t2.micro"
    db_name = var.db_name
    username = "myuser"
    password = "mypassword"
}
// File: main.tf (Root Module)
provider "aws" {
    alias = "east-region"
    region = "us-east-1"
}
provider "aws" {
    alias = "west-region"
    region = "us-west-2"
}
module "rds_instance" {
    source = "./rds/main.tf"
    providers = {
        aws.west = aws.west-region
        aws.easts = aws.east-region
    }
}