OFFICIAL DOCUMENTATION Flashcards
Terraform cloud
Terraform Cloud runs Terraform operations and stores state remotely, so you can use Terraform without worrying about the stability of your local machine, or the security of your state file.
Terraform Cloud (free for up to five users), you can securely share your state with your teammates, provide a stable environment for Terraform to run in, and prevent race conditions when multiple people make configuration changes at once.
The remote state backend feature allows for collaboration and state sharing, provides a stable environment for Terraform to run in, and prevents race conditions when multiple people make config changes simultaneously of Terraform Cl.
Terraform Statefile
Source of truth about your environment.
It uses the statefile to determine/calculate what changes are required to match your configuration
Terraform Providers
Same as Terraform Plugins
provider maintainers
Terraform providers manage resources by communicating between Terraform and target APIs. Whenever the target APIs change or add functionality, provider maintainers may update and version the provider.
Collaboration notes
When multiple users or automation tools run the same Terraform configuration, they should all use the same versions of their required providers.
Managing Provider consistency
There are two ways for you to manage provider versions in your configuration.
1. provider ** version constraints** in your configuration’s terraform block.
2. Use the dependency lock file
If you do not scope provider version appropriately, Terraform will download the latest provider version that fulfills the version constraint. This may lead to unexpected infrastructure changes. By specifying carefully scoped provider versions and using the dependency lock file, you can ensure Terraform is using the correct provider version so your configuration is applied consistently.
TF Cloud Variable sets
Terraform Cloud lets you define input variables and environment variables using either workspace-specific variables(config or user/workspace variable), or sets of variables that you can reuse in multiple workspaces(global/environment variables).
One common use case for variable sets is for provider credentials. By defining a variable set for your credentials, you can easily reuse the same variables across multiple workspaces and efficiently and securely rotate your credentials.
You can define both input variables and environment variables in variable sets
Input Variables vs Environment variables
Input variables aka Terraform variables define the values for variables you reference in your configuration,
while environment variables aka global variables typically store provider credentials or modify Terraform’s behavior, such as logging verbosity
variable sets and security concerns
You can apply a variable set to all workspaces in your organization, or scope it to specific ones. When using variable sets with credentials, reuse the variable set with care and avoid the global option, since it does not follow least privilege best practices.
Sensitive Variable
Marking a variable as sensitive prevents Terraform from displaying it in the Terraform Cloud UI and makes the variable write-only.
Lock workspace
the lock icon indicating whether the workspace is locked, which prevents others from triggering new runs.
-var flag
When using the CLI-driven workflow for Terraform Cloud, any variables passed using the -var flag will override workspace-specific variables. However, Terraform Cloud will not save the new value in your workspace unless you update the variable in the UI. This feature can be useful when you want to test or temporarily apply a change that you expect to revert on the next apply, so your stored variable values continue to reflect the desired eventual configuration.
aka per/run variables
VCS vs CLI driven workflow
The CLI-driven workflow allows you to quickly iterate on your configuration and work locally, while the VCS-driven workflow enables collaboration within teams by establishing your shared repositories as the source of truth for infrastructure configuration.
Sentinel Enforcement Level
Enforcement levels establish whether or not an operation can proceed if a policy fails. Sentinel provides three enforcement levels:
Hard-mandatory requires that the policy passes. If a policy fails, the run stops. You must resolve the failure to proceed.
Soft-mandatory lets an organization owner or a user with override privileges proceed with the run in the event of failure. Terraform Cloud logs all overrides.
Advisory will notify you of policy failures, but proceed with the operation.
Sentinel Enforcement Level
Enforcement levels establish whether or not an operation can proceed if a policy fails. Sentinel provides three enforcement levels:
Hard-mandatory requires that the policy passes. If a policy fails, the run stops. You must resolve the failure to proceed.
Soft-mandatory lets an organization owner or a user with override privileges proceed with the run in the event of failure. Terraform Cloud logs all overrides.
Advisory will notify you of policy failures, but proceed with the operation.
Creating TFC workspace, CLI
When using the CLI-driven Terraform Cloud workflow, running terraform init on configuration with a cloud block creates the Terraform Cloud workspace specified in the block, if it does not already exist.
Terraform variables(console)
This feature can be useful when you want to test or temporarily apply a change that you expect to revert on the next apply, so your stored variable values continue to reflect the desired eventual configuration.
Entering terraform variables on the GUI can be effective in quick testing
Speculative plan logs(VCS integration)
Speculative plans are non-destructive, plan-only runs that show you the changes Terraform will make to your infrastructure if you merge a pull request. The runs will not appear in your Terraform Cloud logs and you can only access them through a direct link, which Terraform Cloud will attach to your pull request.
Policy sets vs Policy
Definition:
Policy sets are a named grouping of policies and their enforcement levels.
To apply a policy to a workspace and it’s run, you must first add it to a policy set. Each policy set can apply to specific workspaces, or to all workspaces within an organization. Policy sets are the mapping between policies and workspaces.
sentinel file format
<policy>.sentinel
</policy>
terraform init
When you initialize a Terraform workspace, Terraform configures the backend, installs all providers and modules referred to in your configuration, and creates a version lock file if one doesn’t already exist. In addition, you can use the terraform init command to change your workspace’s backend and upgrade your workspace’s providers and modules.
Open Policy Agent (OPA).
Configuration-level validation such as variable constraints and preconditions let you socialize standards from within your written configuration. However, module authors and users must voluntarily comply with the standards. Module authors must include conditions in module definitions, and users must consume those modules to provision infrastructure. To enforce infrastructure standards across entire workspaces or organizations, you can use OPA policies, which work without requiring your users to write their infrastructure configuration in a specific way.
Lifecycle pre-conditions
lifecycle {
precondition {
condition = data.aws_ec2_instance_type.bastion.default_cores <= 2
error_message = “Change the value of bastion_instance_type to a type that has 2 or fewer cores to avoid over provisioning.”
}
}
the above block of code restricts instance to a maximum of 2cores.
Drift Detection and Healthcheck
Drift Detection and Health checks are only available on Enterprise tf cloud
Drift detection only reports on changes to the resource attributes defined in your configuration.
Fixing drift is a manual process, because you need to understand whether you want to keep the infrastructure changes made outside of Terraform, or overwrite them(terraform apply)
Shortly after enabling health assessments, the first assessment runs on the workspace. After the first assessment, following assessments run once every 24 hours.
required_providers block
The terraform required_providers block contains the r, which specifies the provider
1. local name,
2. the source address,
3. and the version.
When you initialize this configuration, Terraform will download:
Minimum provider version
The >= version constraint operator specifies the minimum provider version that is compatible with the configuration.
terraform.lock.hcl
When you initialize a Terraform configuration for the first time with Terraform 1.1 or later, Terraform will generate a new .terraform.lock.hcl file in the current working directory.
Terraform Dependency Lock File
The -upgrade flag
Upgrade the AWS provider version
The -upgrade flag will upgrade all providers to the latest version consistent within the version constraints specified in your configuration.
Terraform lock file and VCS
When you initialize a Terraform configuration for the first time with Terraform 1.1 or later, Terraform will generate a new .terraform.lock.hcl file in the current working directory. You should include the lock file in your version control repository to ensure that Terraform uses the same provider versions across your team and in ephemeral remote execution environments.
~ terraform show
to Inspect the current state, use terraform show
~ terraform state
use terraform state for advanced state management. Use the list subcommand to list of the resources in your project’s state.
~terraform state list
~ terraform output
this command can print out all output parameters in the statefile
provider version and version constraints
Each provider plugin has its own set of available versions, allowing the functionality of the provider to evolve over time. Each provider dependency you declare should have a version constraint given in the version argument so Terraform can select a single version per provider that all modules are compatible with.
The version argument is optional; if omitted, Terraform will accept any version of the provider as compatible. However, we strongly recommend specifying a version constraint for every provider your module depends on.
To ensure Terraform always installs the same provider versions for a given configuration, you can use Terraform CLI to create a dependency lock file and commit it to version control along with your configuration. If a lock file is present, Terraform Cloud, CLI, and Enterprise will all obey it when installing providers.
Special note
A Terraform configuration may refer to two different kinds of external dependency that come from outside of its own codebase:
Providers, which are plugins for Terraform that extend it with support for interacting with various external systems.
Modules, which allow splitting out groups of Terraform configuration constructs (written in the Terraform language) into reusable abstractions.
Both of these dependency types can be published and updated independently from Terraform itself and from the configurations that depend on them. For that reason, Terraform must determine which versions of those dependencies are potentially compatible with the current configuration and which versions are currently selected for use.
Providers and Module versions
Version constraints within the configuration itself determine which versions of dependencies are potentially compatible, but after selecting a specific version of each dependency Terraform remembers the decisions it made in a dependency lock file so that it can (by default) make the same decisions again in future.
At present, the dependency lock file tracks only provider dependencies. Terraform does not remember version selections for remote modules, and so Terraform will always select the newest available module version that meets the specified version constraints. You can use an exact version constraint to ensure that Terraform will always select the same module version.
Terraform state lock file format
The dependency lock file uses the same low-level syntax as the main Terraform language, but the dependency lock file is not itself a Terraform language configuration file. It is named with the suffix .hcl instead of .tf in order to signify that difference.
Dependecy Lockfile creation
The lock file is always named .terraform.lock.hcl, and this name is intended to signify that it is a lock file for various items that Terraform caches in the .terraform subdirectory of your working directory.
Terraform automatically creates or updates the dependency lock file each time you run the terraform init command. You should include this file in your version control repository so that you can discuss potential changes to your external dependencies via code review, just as you would discuss potential changes to your configuration itself.
Dependencies lockfile scope
The dependency lock file is a file that belongs to the configuration as a whole, rather than to each separate module in the configuration. For that reason Terraform creates it and expects to find it in your current working directory when you run Terraform, which is also the directory containing the .tf files for the root module of your configuration.
dependency version choice
When terraform init is working on installing all of the providers needed for a configuration, Terraform considers both the version constraints in the configuration and the version selections recorded in the lock file.
If a particular provider has no existing recorded selection, Terraform will select the newest available version that matches the given version constraint, and then update the lock file to include that selection.
If a particular provider already has a selection recorded in the lock file, Terraform will always re-select that version for installation, even if a newer version has become available. You can override that behavior by adding the -upgrade option when you run terraform init, in which case Terraform will disregard the existing selections and once again select the newest available version matching the version constraint.
Understanding Lock File Changes
Because the dependency lock file is primarily maintained automatically by Terraform itself, rather than being updated manually by you or your team, your version control system may show you that the file has changed.
There are a few different types of changes that Terraform can potentially make to your lock file, which you may need to understand in order to review the proposed changes.
Note on module versions
A module intended to be used as the root of a configuration — that is, as the directory where you’d run terraform apply — should also specify the maximum provider version it is intended to work with, to avoid accidental upgrades to incompatible new versions. The ~> operator is a convenient shorthand for allowing the rightmost component of a version to increment.
Do not use ~> (or other maximum-version constraints) for modules you intend to reuse across many configurations, even if you know the module isn’t compatible with certain newer versions. Doing so can sometimes prevent errors, but more often it forces users of the module to update many modules simultaneously when performing routine upgrades. Specify a minimum version, document any known incompatibilities, and let the root module manage the maximum version.
Terraform folder structure
tree
.
├── LICENSE
├── README.md
├── main.tf
├── modules
│ └── aws-ec2-instance
│ ├── main.tf
│ └── variables.tf
├── terraform.tf
└── variables.tf
The example repository includes the following:
LICENSE - includes the text of the Mozilla Public License under which HashiCorp distributes the example configuration.
README.md - describes the example configuration.
main.tf - includes the resources and data sources used by the example configuration.
the modules/aws-ec2-instance - directory includes a Terraform module to provision an EC2 instance on AWS.
terraform.tf - defines the terraform block, which defines the providers, remote backend, and the Terraform version(s) to be used with this configuration.
variables.tf - defines the variables used in this configuration.
After ~terraform init
├── LICENSE
├── README.md
├── main.tf
├── modules
│ └── aws-ec2-instance
│ ├── main.tf
│ └── variables.tf
├── terraform.tf
└── variables.tf
├──.terraform
│ └── modules
│ └── providers
CAUTION !!!
├──.terraform
│ └── modules
│ └── providers
Terraform automatically manages the .terraform directory. Do not check it into version control, and do not directly modify this directory’s contents. Exploring the .terraform directory in this tutorial is meant to deepen your understanding of how Terraform works, but the contents and structure of this directory are subject to change between Terraform versions.
When to run terraform init
Initialize your Terraform workspace with terraform init when:
You create new Terraform configuration and are ready to use it to create a workspace and provision infrastructure.
You clone a version control repository containing Terraform configuration, and are ready to use it to create a workspace and provision infrastructure.
You add, remove, or change the version of a module or provider in an existing workspace.
You add, remove, or change the backend or cloud blocks within the terraform block of an existing workspace.
Terraform state migration
Migrate the state file
To migrate your existing state file to Terraform Cloud, you must reinitialize your configuration to update the backend.
Reinitialize your configuration. Terraform detects your updated backend and confirms that you wish to migrate your state file to Terraform Cloud. Type yes to confirm the migration.
This is all assuming you have your cloud Block within your configuration
save terraform plan
to save a terraform plan to a binaryfile called replan:
~ terraform plan -out “tfplan”
to print on screen:
~terraform show “tfplan”
Convert the saved plan into JSON, pass it to jq to format it, and save the output into a new file:
terraform show -json “tfplan” | jq > tfplan.json
then use jq for analysis:
- The configuration section further organizes your resources defined in your top level root_module:
~ jq ‘.configuration.root_module.resources’ tfplan.json
- The module_calls section contains the details of the modules used, their input variables and outputs, and the resources to create:
jq ‘.configuration.root_module.module_calls’ tfplan.json
~ terraform apply replace
~ terraform apply -replace “aws_instance.main[1]”
when a resource has become unhealthy or stops working in ways that are outside of Terraform’s control. For instance, an error in your EC2 instance’s OS configuration could require that the instance be replaced. There is no corresponding change to your Terraform configuration, so you want to instruct Terraform to reprovision the resource using the same configuration.
The -replace argument requires a resource address. List the resources in your configuration with terraform state list.
Alternative variable injection
- CLI
~ terraform apply -var ec2_instance_type=t2.micro - FILE
*.auto.tfvars or terraform.tfvars file
Description
While the description argument is optional, you should include it in all output declarations to document the intent and content of the output.
trouleshooting terraform version issues
sometimes removing the terraform version specification can solve version problems. However, this has to be done cautiously
NOTE
In general, we encourage you to use the latest available version of Terraform to take advantage of the most recent features and bug fixes. However, it is unnecessary to upgrade your Terraform projects to the latest version every time you use Terraform unless you need a specific feature or bug fix.
As a best practice, consider using ~> style version constraints to pin your major and minor Terraform version. Doing so will allow you and your team to use patch version updates without updating your Terraform configuration. You can then plan when you want to upgrade your configuration to use a new version of Terraform, and carefully review the changes to ensure that your project still works as intended.
For example, if you write Terraform configuration using Terraform 1.0.0, you would add required_version = “~> 1.0.0” to your terraform { } block. This will allow you and your team to use any Terraform 1.0.x, but you will need to update your configuration to use Terraform 1.1.0 or later.
.tftpl
.terraform template file
terraform module sources
Modules can either be loaded from the local filesystem, or a remote source. Terraform supports a variety of remote sources, including the Terraform Registry, most version control systems, HTTP URLs, and Terraform Cloud or Terraform Enterprise private module registries
Understand how modules work
When using a new module for the first time, you must run either terraform init or terraform get to install the module. When you run these commands, Terraform will install any new modules in the .terraform/modules directory within your configuration’s working directory. For local modules, Terraform will create a symlink to the module’s directory. Because of this, any changes to local modules will be effective immediately, without having to reinitialize or re-run terraform get.
GIT IGNORE
The files mentioned below will often include secret information such as passwords or access keys, which will become public if those files are committed to a public version control system such as GitHub.
There are also some other files to be aware of, and ensure that you don’t distribute them as part of your module:
terraform.tfstate and terraform.tfstate.backup: These files contain your Terraform state, and are how Terraform keeps track of the relationship between your configuration and the infrastructure provisioned by it.
.terraform: This directory contains the modules and plugins used to provision your infrastructure. These files are specific to a specific instance of Terraform when provisioning infrastructure, not the configuration of the infrastructure defined in .tf files.
.tfvars: Since module input variables are set via arguments to the module block in your configuration, you don’t need to distribute any *.tfvars files with your module, unless you are also using it as a standalone Terraform configuration.
If you are tracking changes to your module in a version control system, such as git, you will want to configure your version control system to ignore these files. For an example, see this .gitignore file from GitHub.
Terraform Statefile
The Terraform state file is a record of all resources Terraform manages. **You should not make manual changes to resources controlled by Terraform, because the state file will be out of sync, **or “drift,” from the real infrastructure. If your state and configuration do not match your infrastructure, Terraform will attempt to reconcile your infrastructure, which may unintentionally destroy or recreate resources
terraform Refresh
by default, Terraform compares your state file to real infrastructure whenever you invoke terraform plan or terraform apply. The refresh updates your state file in-memory to reflect the actual configuration of your infrastructure. This ensures that Terraform determines the correct changes to make to your resources.
If you suspect that your infrastructure configuration changed outside of the Terraform workflow, you can use a -refresh-only flag to inspect what the changes to your state file would be. This is safer than the refresh subcommand, which automatically overwrites your state file without displaying the updates.
-refresh-only Flag
A refresh-only operation does not attempt to modify your infrastructure to match your Terraform configuration – it only gives you the option to review and track the drift in your state file.
If you ran terraform plan or terraform apply without the -refresh-only flag now, Terraform would attempt to revert your manual changes. Instead, you will update your configuration to associate your EC2 instance with both security groups.
-refresh-only
In Terraform, refreshing your state file updates Terraform’s knowledge of your infrastructure, as represented in your state file, with the actual state of your infrastructure. Terraform plan and apply operations run an implicit in-memory refresh as part of their functionality, reconciling any drift from your state file before suggesting infrastructure changes. You can also update your state file without making modifications to your infrastructure using the -refresh-only flag for plan and apply operations.
Though Terraform will continue to support the refresh subcommand in future versions, it is deprecated, and we encourage you to use the -refresh-only flag instead. This allows you to review any updates to your state file. Unlike the refresh subcommand, -refresh-only mode is supported in workspaces using Terraform Cloud as a remote backend, allowing your team to collaboratively review any modifications.
terraform error management
- Language error (HCL Syntax)
- State error (statefile logical check)
- Core error: The Terraform core application contains all the logic for operations. It interprets your configuration, manages your state file, constructs the resource dependency graph, and communicates with provider plugins. Errors produced at this level may be a bug. Later in this tutorial, you will learn best practices for opening a GitHub issue for the core development team.
- Provider error: The provider plugins handle authentication, API calls, and mapping resources to services. Later in this tutorial, you will learn best practices for opening a GitHub issue for the provider development team.
