The first Terraform codebase I inherited was a single main.tf file, two thousand lines long, with the production state file sitting in a shared Dropbox folder. Applying a change was an act of faith. You'd run terraform apply, watch it propose to destroy a database you were certain you hadn't touched, cancel in a panic, and spend the afternoon working out which teammate's laptop held the “real” state. Nobody wanted to touch it, so nothing got fixed, so it got worse.
Terraform is brilliant at turning infrastructure into code you can version, review, and reproduce. It is equally brilliant at letting you build something nobody dares to run. The difference between those two outcomes isn't the tool — it's a handful of practices, most of which cost almost nothing to adopt early and a fortune to retrofit late. Here's the set that actually matters.
Remote state with locking, from day one
Terraform's state file is the map between your code and the real resources it created. Lose it, corrupt it, or fork it across two machines, and Terraform loses track of what exists. This is the single most important thing to get right, and it's the one people skip because the default — a local terraform.tfstate file — just works on day one.
State belongs in remote storage with locking. An S3 bucket with DynamoDB locking, an Azure Storage account, a GCS bucket, or Terraform Cloud — the backend matters less than the two properties you need: one shared source of truth, and a lock that stops two people applying at once.
terraform {
backend "s3" {
bucket = "acme-tfstate-prod"
key = "network/terraform.tfstate"
region = "eu-west-1"
dynamodb_table = "tf-locks" # the lock this is the part people forget
encrypt = true
}
}
The dynamodb_table line is the whole game. Without it you have remote state but no locking, and the failure mode is nasty: two applies run at once, both read the same starting state, and the second overwrites the first's changes silently. No error, just infrastructure that doesn't match your code and a state file that's quietly wrong. Turn on locking before you turn on teamwork.
Build modules, not copy-paste
The instinct when you need a second environment is to copy the folder and change some names. Do that three times and you have four subtly different definitions of “our network,” each drifting independently, each with its own bugs.
Modules are the fix. Wrap a coherent pattern — a VPC, a database, a service and its supporting bits — behind clear inputs and outputs, and call it wherever you need that pattern. The module is defined once, tested once, and improved once for everyone.
module "web_service" {
source = "../../modules/service"
name = "checkout"
instance_size = "t3.medium"
min_replicas = 2
max_replicas = 10
vpc_id = module.network.vpc_id
}
A good module hides complexity behind a small interface. The caller says instance_size = "t3.medium"; they don't see the twelve resources that implode into existence behind it. One warning worth heeding: resist the urge to make modules infinitely configurable. A module with forty optional variables trying to cover every case is harder to use and understand than two focused modules. Design for the patterns you actually have, not every pattern you can imagine.
Keep environments genuinely separate
Dev, test, and production should use the same modules with different variables — and be isolated enough that a change aimed at dev physically cannot reach prod. Same code, different state, different credentials.
modules/ <- shared, reusable definitions network/ service/ database/ environments/ dev/ main.tf --> calls modules with dev.tfvars, dev state test/ main.tf --> calls modules with test.tfvars, test state prod/ main.tf --> calls modules with prod.tfvars, prod state (separate backend)
Each environment has its own state file and, ideally, its own backend and cloud credentials. Promotion becomes predictable: the same module version rolls from dev to test to prod, and the only thing that changes is the variables. I've watched teams try to do this with Terraform workspaces instead — one state, many workspaces — and regret it, because a single fat-fingered terraform workspace select runs a prod apply while you think you're in dev. Separate directories with separate backends make that mistake impossible rather than merely unlikely. The physical separation is the safety feature.
Make plan a required review gate
terraform plan shows you exactly what will change before anything real happens — what gets created, modified, and, most importantly, destroyed. Treating that output as a review artifact, not a formality, is what catches disasters.
Run plan automatically in your pipeline on every pull request, post the output where a human can read it, and require someone to approve before apply runs. The reviewer is looking for one thing above all: unexpected destroys. A plan that says it will replace a database when your change was supposed to add a tag is Terraform telling you something you got wrong — maybe a change that forces resource recreation. Catch it in review and it's a conversation. Catch it in production and it's an incident and possibly a restore from backup.
Two eyes on a plan is cheap. A recreated production database is not.
Handle secrets like they're radioactive
Credentials do not belong in your .tf files, and they don't belong in version control. That part everyone agrees with. The subtler trap is state: Terraform writes resource attributes into the state file, and that can include generated passwords and keys in plaintext. Anyone who can read your state can read those secrets, which is another reason remote state must be encrypted and access-controlled.
Pull secrets from a vault or your cloud's secret manager at apply time rather than passing them in as plain variables. Reference the secret by identity, let Terraform fetch it during the run, and keep the raw value out of your code entirely. Where a resource would otherwise dump a secret into state, prefer patterns that store only a reference — a secret manager ARN, not the secret itself. You won't eliminate every secret from state, but you can get most of them out, and you can lock down what remains.
Enforce policy as code
Human reviewers miss things, especially the boring, repetitive things — an untagged bucket, a database left publicly accessible, storage created without encryption. Policy-as-code tooling checks every plan against rules automatically and blocks the apply when something violates them.
Wire a policy engine into the same pipeline that runs your plan, and encode the rules your reviewers would otherwise have to remember: no unencrypted storage, no public databases, every resource carries a cost-centre tag. The point isn't to replace review — it's to let reviewers focus on intent and logic while the machine enforces the checklist tirelessly. Guardrails don't get tired at 6pm on a Friday, which is exactly when the risky changes tend to land.
Pin your versions
Terraform and its providers move, and an unpinned setup means a run six months from now might use different provider versions and behave differently — or an accidental upgrade slips into a routine apply and changes behaviour you didn't sign up for.
terraform {
required_version = "~> 1.9.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.60"
}
}
}
Pinning makes runs reproducible and upgrades deliberate. When you want to move to a new provider version, you bump the constraint on purpose, run a plan, and see what changes — rather than discovering it because Tuesday's apply behaved unlike Monday's. Commit your dependency lock file too, so everyone and every pipeline resolves to identical provider versions.
Where it goes wrong anyway
Even with the practices in place, a few failure modes recur, and they're worth naming.
State drift is the quiet one. Someone makes a change in the cloud console — a quick fix during an incident — and now reality and state disagree. Terraform doesn't know until the next plan, which suddenly proposes to “correct” the manual fix, sometimes destructively. The discipline is boring but real: changes go through Terraform, not the console, and when an emergency forces a manual change, you reconcile it back into code promptly.
Giant blast radius is the other. One enormous state file holding your whole estate means every apply risks everything, and the plan takes forever. Split state along boundaries that change at different rates — networking separate from applications separate from data — so a routine app change can't threaten your VPC and a small change doesn't require reasoning about the entire world.
And the slow killer: modules that grow a new variable for every edge case until nobody understands the interface. A module should get simpler to use over time as you learn the real patterns, not accumulate configuration barnacles. When a module sprouts its fortieth optional input, that's a signal to split it, not extend it.
The short version
Remote state with locking, reusable modules, cleanly separated environments, plans reviewed by a human, secrets kept out of code and mostly out of state, policy enforced by machine, and versions pinned on purpose — that's the set that turns Terraform from a liability into a durable foundation. None of it is exotic. All of it is far cheaper to adopt while your codebase is small than to bolt on once it's the two-thousand-line file in a Dropbox folder that nobody wants to run.
Invest early. The practices feel like overhead on day one and like the only reason you're still sane on day three hundred.
Want your infrastructure codified this way? See our Infrastructure as Code with Terraform services.
