Backend block

The backend in Terraform is a place where the statefile are stored. Terraform uses this state data for tracking the resources it manages. The state data is a record of the resources Terraform has created or modified. This data is used for tracking the current state of the infrastructure to make some decisions for further changes.
A remote backend is a common way to store the state data in every complex Terraform configuration. This will allow multiple people to share access to the state data and collaborate with others on infrastructure resources. Also, changes to state data are stored in one place, making it easier to manage and track changes. Common examples of remote backends are Amazon S3, Azure Blob Storage, and Google Cloud Storage.

Backend block syntax

The basic syntax of a Backend block is:
backend "backend_type" {
    # Backend configuration options 
}
Let's break this syntax apart:
NOTE: The Backend block in Terraform is a nested block, which must be used inside the top-level Terraform block.
Here is an example of the backend block in Terraform using an AWS S3 backend:
terraform {
    backend = "s3" {
        bucket = "my-terraform-state-bucket"
        key = "prod/us-east-1/app/terraform.tfstate"
        region = "us-east-1"
        dynamodb_table = "my-lock-table"
        access_key = "aws key"
        secret_key = "aws secret"
    }
}
Above configuration will allow Terraform to store the state data in an S3 bucket and use a DynamoDB table for state locking. In this way, multiple team members can collaborate on the same infrastructure by having the same state data coming from the S3 bucket. This encourages teamwork and maintains consistency in infra management within the team. But this setup is insecure because access and secret keys are given directly inside the backend block; that is, they're stored in the .terraform/terraform.tfstate file. This leads to sensitive credentials exposure.
{
      "version" : 4
      "terraform_version" : 1.5.0
      "serial" : 8
      "lineage" : 60dd9c0a-caa1-de0e
      "outputs" : {}
      "backend" : {
          "type" : "s3"
          "hash" : 1585194947
          "config" : {
              "bucket" = "my-terraform-state-bucket"
              "key" = "prod/us-east-1/app/terraform.tfstate"
              "region" = "us-east-1"
              "dynamodb_table" = "my-lock-table"
              "access_key" = "aws key"
              "secret_key" = "aws secret"
              "workspace_dir" = null
        }
    }
}
In order to avoid this, it is generally recommended to set these credentials via environment variables or other secure methods, rather than hardcoding them directly in the configuration.
There are some important limitations on the backend configuration:

Backend Types

The following types are supported:

1. local

Local backend in Terraform stores the state of your infrastructure on the local file system. It uses the system's APIs to lock the statefile, allowing only one process to have access and modify the state at any given time. All operations, including planning, applying, and destroying of resources, are performed natively on the machine running Terraform.
If you don't define a backend in your configuration, Terraform uses the "local" backend by default.
terraform {
    backend "local" {
        path = "./statefile/ownstate.tfstate"
    }
}
Local backend in Terraform supports the following configurable options:
You can access information from a Terraform state file using the terraform_remote_state data block. This allows accessing the values in that state file and using them in your Terraform configuration. For example
data "terraform_remote_state" "local_statefile" {
    backend = "local"
    config = {
        path = "./statefile/ownstate.tfstate"
    }
}
resource "null_resource" "local" {
    provisioner "local-exec" {
        command = "echo 'The terraform statefile located in ${data.terraform_remote_state.local_statefile.path}'"
    }
}
data.terraform_remote_state.local_statefile: Reading...
data.terraform_remote_state.local_statefile: Read complete after 0s
Terraform will perform the following actions:
    # null_resource.local will be created
    + resource "null_resource" "local" {
      + id = (known after apply)
    }
Plan: 1 to add, 0 to change, 0 to destroy.

Enter a value: yes

null_resource.local: Creating...
null_resource.local: Provisioning with 'local-exec'...
null_resource.local: Executing: ["/bin/sh" "-c" "echo 'The terraform statefile located in ./statefile/ownstate.tfstate'"]
null_resource.local: No file found
null_resource.local: Creation complete after 0s [id=4425172414869377883]

2. consul

The Consul backend allows you to store Terraform state in a Consul Key-Value (KV) store. The backend supports state locking, which ensures that only one user can modify the state at one time. To use it, your Terraform settings need to be configured to use the Consul backend. Here's an example:
NOTE: Partial configuration for access credentials is highly recommended due to security reasons.
terraform {
    backend "consul" {
        address = "stage.consul.com"
        scheme = "https"
        path = "stage/app/tfstate"
        access_token = "SECRET-TOKEN"
    }
}
The following configuration options and environment variables are supported by the Consul backend:

Required Configuration Options:

Argument
Description
path
The path in the Consul KV store where the state will be stored.
access_token or CONSUL_HTTP_TOKEN
The access token required to authenticate with Consul.

Optional Configuration Options:

Argument
Description
scheme
The protocol to use when communicating with Consul (either http or https).
address or CONSUL_HTTP_ADDR
DNS name and port of your Consul endpoint (e.g. dnsname:port). Default: local agent HTTP listener.
datacenter
The datacenter to use. Default: the datacenter of the agent.
http_auth
HTTP Basic Authentication credentials for communicating with Consul (format: user or user:pass).
gzip
Compress state data using gzip (true or false). Default: false (no compression).
lock
Disable state locking (false to disable, true to enable). Default: true (locking enabled).
If you want to access the same state stored in Consul from another Terraform configuration, you can use a terraform_remote_state data source. Here's an example:
data "terraform_remote_state" "local_statefile" {
    backend = "consul"
    config = {
        path = "stage/app/tfstate"
    }
}

3. HTTP

The HTTP backend allows you to store your Terraform state using a simple REST client. This backend supports the following operations.
The HTTP backend also supports state locking that prevents more than one user from modifying the state. When such option is enabled, the backend will use the following additional operations.
terraform {
    backend "http" {
        address = "http://stage.terraform.com/statefile"
    }
}
The http backend supports the following configuration options and environment variables.

Required Configuration Options:

Argument
Description
address or TF_HTTP_ADDRESS
The URL of the REST endpoint where the state will be stored.

Optional Configuration Options:

Argument
Description
update_method or TF_HTTP_UPDATE_METHOD
The HTTP method to use when updating the state (default: POST).
lock_address or TF_HTTP_LOCK_ADDRESS
The URL of the lock REST endpoint (default: disabled).
lock_method or TF_HTTP_LOCK_METHOD
The HTTP method to use when locking (default: LOCK).
unlock_address or TF_HTTP_UNLOCK_ADDRESS
The URL of the unlock REST endpoint (default: disabled).
unlock_method or TF_HTTP_UNLOCK_METHOD
The HTTP method to use when unlocking (default: UNLOCK).
username or TF_HTTP_USERNAME
The username for HTTP basic authentication.
password or TF_HTTP_PASSWORD
The password for HTTP basic authentication.
You could access the state stored using the HTTP backend from another Terraform configuration using a terraform_remote_state data source:
data "terraform_remote_state" "local_statefile" {
    backend = "kubernetes"
    config = {
        address = "http://stage.terraform.com/statefile"
    }
}

4. kubernetes

The Kubernetes backend stores Terraform's state in a Kubernetes secret. It supports state locking using an Lease resource.
terraform {
    backend "kubernetes" {
        secret_suffix = "statefile"
        config_path = "~/.kube/config"
    }
}
The user/service account running Terraform requires permissions to read/write secrets inside the namespace that state is being stored within. Either one of config_path, config_paths or in_cluster_config must be set. If more than one is set, configuration at config_path will be used.
The following configuration options and environment variables are supported by the kubernetes backend.

Required Configuration Options:

Argument
Description
secret_suffix
The suffix used when creating secrets (e.g., "tfstate-{workspace}-{secret_suffix}").

Optional Configuration Options:

Argument
Description
config_path or KUBE_CONFIG_PATH
Path to the kube config file.
config_paths or KUBE_CONFIG_PATHS
List of paths to kube config files.
config_context or KUBE_CTX
Context to choose from the config file.
config_context_auth_info or KUBE_CTX_AUTH_INFO
Authentication info context of the kube config.
in_cluster_config or KUBE_CONFIG_PATHS
List of paths to kube config files.
host
The hostname (in URI form) of the Kubernetes master (defaults to https://localhost).
username or KUBE_USER
The username for HTTP basic authentication.
password or KUBE_PASSWORD
The password for HTTP basic authentication.
insecure or KUBE_INSECURE
Whether to access the server without verifying the TLS certificate.
labels
Additional labels to apply to the secret and lease.
exec
Configuration block to use an exec-based credential plugin.
api_version
API version to use when decoding the ExecCredentials resource. It must be used inside exec block.
command
Command to execute. It must be used inside exec block.
args
List of arguments to pass when executing the plugin. It must be used inside exec block.
env
Map of environment variables to set when executing the plugin. It must be used inside exec block.

5. pg

The Pg backend stores Terraform's state in a Postgres database. It has state locking, so only one process is capable of modifying the state at any one time.
terraform {
    backend "pg" {
        conn_str = "postgres://admin:1234@[email protected]/terraform_backend"
    }
}
To keep sensitive credentials secure, it is good to use environment variables in the configuration of the Postgres backend. This would avoid writing credentials to disk and committing them to source control. We have two Configuration Options.
// Shell Commands
export PG_CONN_STR=postgres://admin:1234@[email protected]/terraform_backend
// Terraform Configuration
terraform {
    backend "pg" {}
}
// Shell Commands
export PGUSER=admin
export PGPASSWORD=1234@root
// Terraform Configuration
terraform {
    backend "pg" {
        conn_str = postgres://database.com/terraform_backend
    }
}
The following are the configuration options and environment variables supported by pg backend.
Argument
Description
conn_str or PG_CONN_STR
Postgres connection string (a postgres:// URL).
skip_schema_creation or PG_SKIP_SCHEMA_CREATION
If true, the Postgres schema must already exist (default: false).
skip_table_creation or PG_SKIP_TABLE_CREATION
If true, the Postgres table must already exist (default: false).
skip_index_creation or PG_SKIP_INDEX_CREATION
If true, the Postgres index must already exist (default: false).

6. s3

The Amazon S3 backend in Terraform stores the statefile as a given key in a specified S3 bucket. This backend supports the following features:
Here's an example configuration:
terraform {
    backend = "s3" {
        bucket = "my-terraform-state-bucket"
        key = "prod/us-east-1/app/terraform.tfstate"
        region = "us-east-1"
        dynamodb_table = "my-lock-table"
        access_key = "aws key"
        secret_key = "aws secret"
    }
} 
In this example, the state file would be kept under env:stage/prod/us-east-1/app/terraform.tfstate if the stage workspace was selected.
If no workspaces are used, or if the default workspace is used, Terraform requires the following permissions on the target S3 bucket:
{
    "version" : "2024-8-17"
    "Statement" : [
        {
            "Effect" = "Allow"
            "Action" = "s3:ListBucket"
            "Resource" = "arn:aws:s3:::<BUCKET_NAME>"
        },
        {
            "Effect" = "Allow"
            "Action" = ["s3:GetObject", "s3:PutObject"]
            "Resource" = "arn:aws:s3:::<BUCKET_NAME>/<KEY>"
        }
    ]
}
If using workspaces, following permissions are required for Terraform:
{
    "version" : "2024-8-17"
    "Statement" : [
        {
            "Effect" = "Allow"
            "Action" = "s3:ListBucket"
            "Resource" = "arn:aws:s3:::<BUCKET_NAME>"
        },
        {
            "Effect" = "Allow"
            "Action" = ["s3:GetObject", "s3:PutObject"]
            "Resource" = ["arn:aws:s3:::<BUCKET_NAME>/<KEY>", "arn:aws:s3:::<BUCKET_NAME>/<workspace_key_prefix>/*/<KEY>"]
        },
        {
            "Effect" = "Allow"
            "Action" = "s3:DeleteObject"
            "Resource" = "arn:aws:s3:::<BUCKET_NAME>/<workspace_key_prefix>/*/<KEY>"
        }
    ]
}
When state locking is used, Terraform requires the additional permissions on the DynamoDB table (identified by the resource arn:aws:dynamodb:::table/<TABLE NAME>)
{
    "version" : "2024-8-17"
    "Statement" : [
        {
            "Effect" = "Allow"
            "Action" = ["dynamodb:DescribeTable", "dynamodb:DescribeTable", "dynamodb:PutItem", "dynamodb:DeleteItem"]
            "Resource" = "arn:aws:dynamodb:*:*:table/<TABLE_NAME>"
        }
    ]
}
The following configuration options and environment variables are supported by the S3 backend.

Required Configuration Options:

Argument
Description
bucket
The name of the S3 bucket where the Terraform state file will be stored.
key
The path to the state file within the S3 bucket. When using a non-default workspace, the state path will be workspace_key_prefix/workspace_name/key.
region or AWS_DEFAULT_REGION or AWS_REGION
The AWS region where the S3 bucket and DynamoDB table (if used) are located.

Optional Configuration Options:

Argument
Description
acl
The canned Access Control List (ACL) to be applied to the state file.
encrypt
Enables server-side encryption of the state file.
endpoint
Custom endpoint URL for the AWS S3 API. Use endpoints.s3 instead.
workspace_key_prefix
The prefix applied to the state path inside the bucket when using a non-default workspace. Defaults to env:.
access_key or AWS_ACCESS_KEY_ID
The AWS access key. If configured, the secret_key must also be configured.
secret_key
The AWS secret key. If configured, the access_key must also be configured. Can also be sourced from the AWS_SECRET_ACCESS_KEY environment variable, AWS shared credentials file (e.g., ~/.aws/credentials), or AWS shared configuration file (e.g., ~/.aws/config).
profile
The name of the AWS profile in the AWS shared credentials file (e.g., ~/.aws/credentials) or AWS shared configuration file (e.g., ~/.aws/config) to use for credentials and/or configuration. Can also be sourced from the AWS_PROFILE environment variable.
shared_config_files
A list of paths to AWS shared configuration files. Defaults to ~/.aws/config.
shared_credentials_files
A list of paths to AWS shared credentials files. Defaults to ~/.aws/credentials.
dynamodb_table
This option specifies the name of the DynamoDB table to use for state locking and consistency. The table must have a partition key named LockID with a data type of String. If this option is not configured, state locking will be disabled.
You could access the state stored using the S3 backend from another Terraform configuration using a terraform_remote_state data source:
data "terraform_remote_state" "s3_statefile" {
    backend = "s3"
    config = {
        bucket = "my-terraform-state-bucket"
        key = "prod/us-east-1/app/terraform.tfstate"
        region = "us-east-1"
    }
}

7. azurerm

The Azurerm backend stores Terraform state as a blob in Azure Blob Storage. You can then manage and version your infrastructure state in a centralized and secure location. This backend will support the following features:
terraform {
    backend = "azurerm" {
        resource_group_name = "my-resource-group"
        storage_account_name = "my-storage-account"
        container_name = "my-container"
        key = "prod.terraform.tfstate"
    }
} 
The AzureRM backend supports three methods for authenticating to the storage account:
The AzureRM backend supports the following configuration options and environment variables.

Required Configuration Options:

Argument
Description
storage_account_name
The name of the Storage Account where the state file will be stored.
container_name
The name of the Storage Container within the Storage Account where the state file will be stored.
key
The name of the Blob that will be used to store Terraform's State file.

Optional Configuration Options:

Argument
Description
endpoint or ARM_ENDPOINT
The Custom Endpoint for Azure Resource Manager.
encrypt
Enables server-side encryption of the state file.
The terraform_remote_state data source retrieves the state of a Terraform configuration whose state is stored in an Azure Blob Storage to access it from another configuration.
data "terraform_remote_state" "s3_statefile" {
    backend = "s3"
    config = {
        resource_group_name = "my-resource-group"
        storage_account_name = "my-storage-account"
        container_name = "my-container"
        key = "prod.terraform.tfstate"
    }
}

8. gcs

The Terraform GCS backend stores the state as an object in a configurable prefix in a pre-existing Google Cloud Storage (GCS) bucket. This backends support the following features:
terraform {
    backend = "gcs" {
        bucket = "terraform-stage-statefiles"
        prefix = "application/stage/us-east-1"
    }
} 
The Gcs backend supports the following configuration options and environment variables.

Required Configuration Options:

Argument
Description
bucket
The name of the GCS bucket, which must be globally unique.

Optional Configuration Options:

Argument
Description
credentials or
GOOGLE_BACKEND_CREDENTIALS
or GOOGLE_CREDENTIALS
Local path to Google Cloud Platform account credentials in JSON format. If unset, it uses Google Application Default Credentials.
impersonate_service_account
or
GOOGLE_BACKEND_IMPERSONATE_SERVICE_ACCOUNT
The service account to impersonate for accessing the State Bucket. You must have the roles/iam.serviceAccountTokenCreator role on that account for impersonation to succeed.
access_token
A temporary OAuth 2.0 access token obtained from the Google Authorization server. This is an alternative to credentials.
prefix
GCS prefix inside the bucket. Named states for workspaces are stored in an object called /.tfstate.
encryption_key or GOOGLE_ENCRYPTION_KEY
A 32-byte base64-encoded customer-supplied encryption key used when reading and writing state files in the bucket.
kms_encryption_key or
GOOGLE_KMS_ENCRYPTION_KEY
A Cloud KMS key used when reading and writing state files in the bucket. Format: projects//locations//keyRings//cryptoKeys/.
storage_custom_endpoint or
GOOGLE_BACKEND_STORAGE_CUSTOM_ENDPOINT
A URL containing the protocol, DNS name pointing to a Private Service Connect endpoint, and path for the Cloud Storage API.
If your Terraform state is stored in a GCS bucket, you can share the state of one Terraform configuration with another using the terraform_remote_state data source.
data "terraform_remote_state" "gcs_statefile" {
    backend = "gcs"
    config = {
        bucket = "terraform-stage-statefiles"
        prefix = "application/stage/us-east-1"
    }
}

Behaviour of Backend

If you edit your backend configuration, you need to run terraform init again in order to update and validate the new backend settings. This is necessary before you are able to do any Terraform commands against it, such as plan, apply, or state operations.
Once you run the terraform init, Terraform creates a directory called .terraform/ in your local machine. This directory keeps the most recent backend configuration in a file called terraform.tfstate, which contains any authentication credentials you supplied when you ran init. Please do not commit this directory to Git. This directory might contain sensitive credentials.
NOTE: There are two different terraform.tfstate: one in the folder .terraform/terraform.tfstate, which stores the backend configuration, and another one in the root module that stores information about your real-world infrastructure. However, when using a remote backend, the infrastructure statefile (terraform.tfstate) is stored in the remote backend, not locally, whereas the backend configuration file remains in .terraform/terraform.tfstate.

Partial Configuration in Terraform

A partial configuration in Terraform is when you have not specified all the required arguments in the backend configuration. This can be useful when some of the arguments are provided automatically by an automation script, or when you want to keep sensitive information from going into version control.
When using a partial configuration, you must provide the rest of the configuration arguments during initialization. You can do this in either of these two ways:

1. File

A Terraform backend configuration file is a plain text file with a .tfbackend extension (e.g., configurable.tfbackend) that contains the backend configuration settings as top-level attributes. You don't need to wrap the settings in a terraform or backend block. Here's an example of what a backend configuration file might look like:
// s3_configurable.tfbackend
access_key = "aws key"
secret_key = "aws secret"
terraform {
    backend = "s3" {
        bucket = "my-terraform-state-bucket"
        key = "prod/us-east-1/app/terraform.tfstate"
        region = "us-east-1"
        dynamodb_table = "my-lock-table"
    }
} 
In order to use this configuration file with Terraform, one can run terraform init with the -backend-config=PATH option and specify the path to the user's configuration file. Here, PATH refers to the path where the file is placed.
terraform init -backend-config=s3_configurable.tfbackend

2. Command-line Key/Value Pairs

In addition to supplying a backend configuration file, it is also possible to specify single configuration settings using command-line flags when running terraform init. This allows you to configure inputs directly without an additional file. To supply a single key-value pair, use the -backend-config="KEY=VALUE" option at the time of running terraform init. For example:
terraform init -backend-config="access_key=aws_key"
This is useful for supplying or overriding certain configuration settings quickly. However, care must be taken when using this approach, as command-line flags can be stored in the shell's history file and may lead to access key or secret leakage.
If you need to provide several key-value pairs, use -backend-config option several times. For example:
terraform init -backend-config="access_key=aws_key" -backend-config="secret_key=aws_secret"

Multi-Location configuration

Backend settings can be provided in multiple locations, such as the main configuration and command-line options. The top-level settings are merged, where command-line options override the settings in the main configuration. For example
terraform {
    backend = "local" {
        path = "./statefile/ownstate.tfstate"
    }
} 
// CommandLine Option
terraform init -backend-config="path=./commandlinestatefile.tfstate"
{
    "version" : 4
    "terraform_version" : 1.5.0
    "backend" : {
        "type" : "local"
        "hash" : 1585194947
        "config" : {
            "path" = "./commandlinestatefile.tfstate"
            "workspace_dir" = null
      }
   }
}
The above example configuration has been saved in the .terraform directory. Now, the path setting has been changed to ./commandlinestatefile.tfstate as the command-line option overrides the main configuration.
When multiple command-line options are available, Terraform processes them in the order they come down, and the later options override the values set by the earlier ones. Let's take the very same example above.
// CommandLine Option
terraform init -backend-config="path=./commandlinestatefile.tfstate" -backend-config="path=./secondstatefile.tfstate"
{
    "version" : 4
    "terraform_version" : 1.5.0
    "backend" : {
        "type" : "local"
        "hash" : 1585194947
        "config" : {
            "path" = "./secondstatefile.tfstate"
            "workspace_dir" = null
      }
   }
}
In this example, Final configuration has the path setting set to ./secondstatefile.tfstate as the second command-line option has overridden the first one and also main configuration.

Flexible Backend Configuration with Terraform's Partial Configuration

When employing Terraform's "partial configuration" feature, which allows you to define the backend configuration separately from the main Terraform configuration, at a minimum, Terraform requires that an empty backend configuration be specified in one of the root Terraform configuration files. This is necessary to explicitly specify the backend type Terraform should use.
Suppose you have a Terraform configuration that manages resources in a cloud provider, and you want to store the statefile in a Consul backend. You do not, however, want to hardcode the Consul backend configuration in your main Terraform configuration file but instead provide the Consul backend configuration separately, via command-line options or environment variables. Your Terraform configuration, in this case, would appear something like
terraform {
    backend = "consul" {}
} 
Here, the terraform block contains the empty backend configuration with "consul" as the backend type. This is used to tell Terraform that the actual configuration of the backend will be provided somewhere else, and not directly in the main configuration file.
Now, when you run the terraform init command, Terraform will expect you to provide Consul backend configuration via either command-line options or environment variables. For instance:
// CommandLine Option
terraform init -backend-config="address=https://consul.example.com" -backend-config="path=my-state-path"
This approach will keep the main Terraform configuration file clean and maintainable, yet it also provides the backend configuration separately in a configuration file. This can come in really handy when you deal with sensitive information, such as Consul access credentials, which you do not want to put in your main Terraform configuration file.

Related Pages

Feedback

Was this page helpful?