Terraform and Ansible Collaboration for AWS Cloud Deployment

In modern IT, automation and efficiency are crucial. Terraform and Ansible are leading Infrastructure as Code (IaC) tools, each with unique strengths. Terraform excels at provisioning cloud infrastructure across multiple providers using a declarative language, while Ansible focuses on configuration management and application deployment through agentless automation.

Terraform Overview

Terraform, developed by HashiCorp, allows you to define and manage cloud resources with code. It supports various cloud providers, making it ideal for multi-cloud strategies. Key features include:

• IaC and Multi-cloud Support: Define infrastructure with code and manage it across different cloud environments.
• Provisioning: Automate the creation and management of infrastructure components.

Ansible Overview

Ansible, by Red Hat, automates configuration management, application deployment, and orchestration. It uses YAML-based playbooks and does not require any agents, simplifying setup and management. Key features include:

• Configuration Management: Ensures consistency across systems by automating their configuration.
• Agentless Operation: No need for agents on managed systems, reducing overhead.

Benefits of Integrating Terraform and Ansible

Combining Terraform and Ansible leverages their respective strengths:

1. Complete Automation: Terraform handles infrastructure provisioning, while Ansible manages configuration, allowing for full automation from setup to deployment.
2. Efficiency and Consistency: Automate both provisioning and configuration to reduce errors and ensure consistency across environments.
3. Scalability and Flexibility: Manage multi-cloud and hybrid environments efficiently by using Terraform for infrastructure and Ansible for uniform configuration.
4. Enhanced Collaboration: Use version-controlled code for infrastructure and configuration, enabling better team collaboration and transparency.
5. Simplified Management: Automate complex setups with Terraform and use Ansible for ongoing configuration tasks, ensuring seamless operation.
6. Security and Compliance: Terraform provisions secure resources while Ansible enforces security policies, enhancing overall compliance.

Prerequisites for Setting Up Terraform and Ansible Integration

• AWS Account: You need an active AWS account to provision resources on AWS.
• AWS Access and Secret keys: If you do not have these keys, you can create them by following this AWS IAM guide
• Terraform Installed: Ensure Terraform is installed on your local machine. You can download it from the Terraform website.
• Ansible Installed: Install Ansible on your local machine. Instructions can be found in the Ansible documentation.
• SSH Key Pair: Generate an SSH key pair through AWS Console if you don’t have one, as you’ll need it to access your AWS EC2 instance. Ensure you have the private key file locally. Learn more about how to generate an SSH key pair here.
• Basic Understanding of Terraform and Ansible: Familiarity with basic Terraform and Ansible concepts will help you follow along with the setup and configuration processes.
• Text Editor: Use a text editor or IDE to create and modify Terraform and Ansible files. Popular choices include VS Code, Sublime Text, or Atom or if you are using Linux machine then you can choose either vi editor or nano.

Deploying Infrastructure on AWS with Terraform and Ansible

Let’s explore how to use Terraform and Ansible together to deploy and configure infrastructure on AWS. Terraform provisions the infrastructure as code, and Ansible automates the configuration and deployment on the created resources. In this article, I’ll demonstrate how to provision an Amazon Linux instance on AWS using Terraform. Afterward, I’ll use the remote-exec provisioner to run an Ansible playbook that installs the latest patches, sets up the Apache HTTP server, and configures the web service.

1. Directory Structure and Files

My project directory, terraform_ansible_collaboration, contains the following files:

.

├── ansible.cfg

├── main.tf

├── patch_apache_install.yml

├── README.md

├── terraform.tfstate

├── terraform.tfstate.backup

└── terraform.tfvars

Each file plays a specific role in the deployment process:

–  ansible.cfg: Ansible configuration file that defines default settings for Ansible, such as the remote user and host key checking behavior.

– main.tf: The main Terraform configuration file where AWS infrastructure is defined and provisioned.

– patch_apache_install.yml: An Ansible playbook that performs server configuration tasks like installing Apache, patching the system, and setting up a simple webpage.

– terraform.tfstate and terraform.tfstate.backup: Terraform state files that track the state of the infrastructure.

– terraform.tfvars: A variables file used by Terraform to input sensitive data such as AWS credentials and SSH key paths.

– README.md: A file that typically contains documentation or instructions for using the project.

2. Terraform Configuration (main.tf)

The main.tf file is the heart of my infrastructure deployment, defining the resources needed to launch an AWS EC2 instance and the security group configurations required for my web server.

Here’s a breakdown of its contents:

terraform {

required_providers {

aws = {

source  = “hashicorp/aws”

version = “~> 5.0”

}

}

}

 

provider “aws” {

region     = var.aws_region

access_key = var.aws_access_key

secret_key = var.aws_secret_key

}

–  terraform block: Specifies the required providers and their versions. Here, I am using the AWS provider version ~> 5.0.

– provider block: Defines AWS as the cloud provider and sets up access using variables for the AWS region, access key, and secret key.

resource “aws_instance” “web_server” {

ami           = “ami-02b49a24cfb95941c” # Replace with your preferred AMI ID

instance_type = “t2.micro”

key_name      = var.key_pair_name

 

vpc_security_group_ids = [aws_security_group.web_sg.id]

 

tags = {

Name = “WebServer”

}

 

provisioner “remote-exec” {

inline = [

“echo ‘Hello World! I am cj-web01 and I am alive.'”

]

 

connection {

type        = “ssh”

user        = “ec2-user”

private_key = file(var.private_key_path)

host        = self.public_ip

}

}

 

provisioner “local-exec” {

command = <<EOT

ansible-playbook -i ${self.public_ip}, –private-key ${var.private_key_path} patch_apache_install.yml

EOT

}

}

 

–  aws_instance resource: Creates an EC2 instance using a specified AMI ID and instance type (t2.micro). It also specifies an SSH key for access and associates the instance with a security group.

–  remote-exec provisioner: Executes a simple shell command to ensure the EC2 instance is accessible via SSH right after provisioning.

– local-exec provisioner: Runs an Ansible playbook (patch_apache_install.yml) after the EC2 instance is up and running, using its public IP.

resource “aws_security_group” “web_sg” {

name        = “web_sg”

description = “Allow HTTP and SSH access”

 

ingress {

from_port   = 22

to_port     = 22

protocol    = “tcp”

cidr_blocks = [“0.0.0.0/0”]

}

 

ingress {

from_port   = 80

to_port     = 80

protocol    = “tcp”

cidr_blocks = [“0.0.0.0/0”]

}

 

egress {

from_port   = 0

to_port     = 0

protocol    = “-1”

cidr_blocks = [“0.0.0.0/0”]

}

}

 

– aws_security_group resource: Defines a security group allowing inbound SSH (port 22) and HTTP (port 80) traffic from any IP and outbound traffic to any IP.

3. Ansible Playbook (patch_apache_install.yml)

The Ansible playbook patch_apache_install.yml is executed after Terraform provisions the EC2 instance. It performs several tasks to configure the web server:

– hosts: all

become: yes

tasks:

– name: Set the Server Hostname

hostname:

name: cj-web01

 

– name: Install patches

dnf:

name: “*”

state: latest

 

– name: Install Apache

dnf:

name: httpd

state: present

 

– name: Start Apache service

service:

name: httpd

state: started

enabled: yes

 

– name: Configure webpage

copy:

content: “Welcome to \”{{ ansible_hostname }}\””

dest: /var/www/html/index.html

–  Set the Server Hostname: Sets the hostname of the instance to cj-web01.

–  Install patches: Uses of dnf ansible module to update all packages to the latest version.

–  Install Apache: Installs the Apache HTTP server.

–  Start Apache service: Ensures the Apache service is started and enabled to run on boot.

–  Configure webpage: Copies a simple HTML file to the web server’s root directory, displaying a welcome message that includes the server’s hostname.

4. Terraform Variables (terraform.tfvars)

The terraform.tfvars file provides the sensitive data and configurations required by Terraform to provision resources:

aws_access_key   = “DCBA8ZABCDEFD0ABC6”

aws_secret_key   = “GOABGlv90iooa9REDFmbP9qPA900c1zsYHS5n6QJ”

key_pair_name    = “testkeypair”

private_key_path = “/home/foobar/key/testkeypair.pem”

 

– aws_access_key and aws_secret_key: Credentials for AWS access.

– key_pair_name: Name of the SSH key pair used for accessing the EC2 instance.

– private_key_path: Local path to the private key file for SSH access.

5. Ansible Configuration (ansible.cfg)

Since I’m using a different login ID to access my Linux machine for deploying and configuring AWS EC2 instances with Terraform and Ansible, I’ve configured the ansible.cfg file to specify ec2-user as the default remote user for running playbooks. Additionally, I’ve disabled SSH host key checking to prevent Ansible from getting stuck due to mismatched host keys.

Here’s how the ansible.cfg file is configured:

[defaults]

remote_user = ec2-user

host_key_checking = False

Deployment Process

1. Provisioning Infrastructure with Terraform: Run terraform init to initialize the project and terraform apply to provision the resources. Terraform uses the configurations defined in main.tf to create an EC2 instance and a security group.

2. Configuring Server with Ansible: Once the EC2 instance is created, Terraform’s local-exec provisioner triggers the Ansible playbook patch_apache_install.yml, which configures the server by updating packages, installing Apache, and setting up a basic webpage. Below is the output of terraform apply command:

charanjit@charanjit-ubuntu:~/terraform_ansible_collaboration$ terraform apply –auto-approve

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create

Terraform will perform the following actions:

# aws_instance.web_server will be created
+ resource “aws_instance” “web_server” {
+ ami = “ami-02b49a24cfb95941c”
+ arn = (known after apply)
+ associate_public_ip_address = (known after apply)
+ availability_zone = (known after apply)
+ cpu_core_count = (known after apply)
+ cpu_threads_per_core = (known after apply)
+ disable_api_stop = (known after apply)
+ disable_api_termination = (known after apply)
+ ebs_optimized = (known after apply)
+ get_password_data = false
+ host_id = (known after apply)
+ host_resource_group_arn = (known after apply)
+ iam_instance_profile = (known after apply)
+ id = (known after apply)
+ instance_initiated_shutdown_behavior = (known after apply)
+ instance_lifecycle = (known after apply)
+ instance_state = (known after apply)
+ instance_type = “t2.micro”
+ ipv6_address_count = (known after apply)
+ ipv6_addresses = (known after apply)
+ key_name = “testkeypair”
+ monitoring = (known after apply)
+ outpost_arn = (known after apply)
+ password_data = (known after apply)
+ placement_group = (known after apply)
+ placement_partition_number = (known after apply)
+ primary_network_interface_id = (known after apply)
+ private_dns = (known after apply)
+ private_ip = (known after apply)
+ public_dns = (known after apply)
+ public_ip = (known after apply)
+ secondary_private_ips = (known after apply)
+ security_groups = (known after apply)
+ source_dest_check = true
+ spot_instance_request_id = (known after apply)
+ subnet_id = (known after apply)
+ tags = {
+ “Name” = “WebServer”
}
+ tags_all = {
+ “Name” = “WebServer”
}
+ tenancy = (known after apply)
+ user_data = (known after apply)
+ user_data_base64 = (known after apply)
+ user_data_replace_on_change = false
+ vpc_security_group_ids = (known after apply)
}

# aws_security_group.web_sg will be created
+ resource “aws_security_group” “web_sg” {
+ arn = (known after apply)
+ description = “Allow HTTP and SSH access”
+ egress = [
+ {
+ cidr_blocks = [
+ “0.0.0.0/0”,
]
+ from_port = 0
+ ipv6_cidr_blocks = []
+ prefix_list_ids = []
+ protocol = “-1”
+ security_groups = []
+ self = false
+ to_port = 0
# (1 unchanged attribute hidden)
},
]
+ id = (known after apply)
+ ingress = [
+ {
+ cidr_blocks = [
+ “0.0.0.0/0”,
]
+ from_port = 22
+ ipv6_cidr_blocks = []
+ prefix_list_ids = []
+ protocol = “tcp”
+ security_groups = []
+ self = false
+ to_port = 22
# (1 unchanged attribute hidden)
},
+ {
+ cidr_blocks = [
+ “0.0.0.0/0”,
]
+ from_port = 80
+ ipv6_cidr_blocks = []
+ prefix_list_ids = []
+ protocol = “tcp”
+ security_groups = []
+ self = false
+ to_port = 80
# (1 unchanged attribute hidden)
},
]
+ name = “web_sg”
+ name_prefix = (known after apply)
+ owner_id = (known after apply)
+ revoke_rules_on_delete = false
+ tags_all = (known after apply)
+ vpc_id = (known after apply)
}

Plan: 2 to add, 0 to change, 0 to destroy.
aws_security_group.web_sg: Creating…
aws_security_group.web_sg: Creation complete after 2s [id=sg-0e50aa60f433f26be]
aws_instance.web_server: Creating…
aws_instance.web_server: Still creating… [10s elapsed]
aws_instance.web_server: Still creating… [20s elapsed]
aws_instance.web_server: Still creating… [30s elapsed]
aws_instance.web_server: Provisioning with ‘remote-exec’…
aws_instance.web_server (remote-exec): Connecting to remote host via SSH…
aws_instance.web_server (remote-exec): Host: 13.126.205.105
aws_instance.web_server (remote-exec): User: ec2-user
aws_instance.web_server (remote-exec): Password: false
aws_instance.web_server (remote-exec): Private key: true
aws_instance.web_server (remote-exec): Certificate: false
aws_instance.web_server (remote-exec): SSH Agent: false
aws_instance.web_server (remote-exec): Checking Host Key: false
aws_instance.web_server (remote-exec): Target Platform: unix
aws_instance.web_server (remote-exec): Connecting to remote host via SSH…
aws_instance.web_server (remote-exec): Host: 13.126.205.105
aws_instance.web_server (remote-exec): User: ec2-user
aws_instance.web_server (remote-exec): Password: false
aws_instance.web_server (remote-exec): Private key: true
aws_instance.web_server (remote-exec): Certificate: false
aws_instance.web_server (remote-exec): SSH Agent: false
aws_instance.web_server (remote-exec): Checking Host Key: false
aws_instance.web_server (remote-exec): Target Platform: unix
aws_instance.web_server (remote-exec): Connected!
aws_instance.web_server (remote-exec): Hello World! I am cj-web01 and I am alive.
aws_instance.web_server: Provisioning with ‘local-exec’…
aws_instance.web_server (local-exec): Executing: [“/bin/sh” “-c” ” ansible-playbook -i 13.126.205.105, –private-key /home/foobar/key/testkeypair.pem patch_apache_install.yml\n”]

aws_instance.web_server (local-exec): PLAY [all] *********************************************************************

aws_instance.web_server (local-exec): TASK [Gathering Facts] *********************************************************
aws_instance.web_server (local-exec): [WARNING]: Platform linux on host 13.126.205.105 is using the discovered Python
aws_instance.web_server (local-exec): interpreter at /usr/bin/python3.9, but future installation of another Python
aws_instance.web_server (local-exec): interpreter could change the meaning of that path. See https://docs.ansible.com
aws_instance.web_server (local-exec): /ansible/2.10/reference_appendices/interpreter_discovery.html for more
aws_instance.web_server (local-exec): information.
aws_instance.web_server (local-exec): ok: [13.126.205.105]

aws_instance.web_server (local-exec): TASK [Set the Server Hostname] *************************************************
aws_instance.web_server (local-exec): changed: [13.126.205.105]

aws_instance.web_server (local-exec): TASK [Install patches] *********************************************************
aws_instance.web_server: Still creating… [40s elapsed]
aws_instance.web_server: Still creating… [50s elapsed]
aws_instance.web_server (local-exec): ok: [13.126.205.105]

aws_instance.web_server (local-exec): TASK [Install Apache] **********************************************************
aws_instance.web_server: Still creating… [1m0s elapsed]
aws_instance.web_server (local-exec): changed: [13.126.205.105]

aws_instance.web_server (local-exec): TASK [Start Apache service] ****************************************************
aws_instance.web_server (local-exec): changed: [13.126.205.105]

aws_instance.web_server (local-exec): TASK [Configure webpage] *******************************************************
aws_instance.web_server (local-exec): changed: [13.126.205.105]

aws_instance.web_server (local-exec): PLAY RECAP *********************************************************************
aws_instance.web_server (local-exec): 13.126.205.105 : ok=6 changed=4 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

aws_instance.web_server: Creation complete after 1m5s [id=i-011713f9707417fd1]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
charanjit@charanjit-ubuntu:~/terraform_ansible_collaboration$

3. Verification: After deployment, you can verify the configuration by accessing the EC2 instance via SSH and checking the Apache service status or by visiting the public IP of the instance in a web browser to see the welcome page.:

Screenshot shows provisioned AWS EC2 instance status in AWS Console

Screenshot of installed Apache verification

Website working status screenshot

Note:

You can get this Terraform and Ansible code from my below GitHub link:
https://github.com/cjcheema/terraform_ansible_collaboration

You can also use an Ansible playbook to invoke a Terraform module and perform the same tasks. Learn more about this module at the link below:

https://docs.ansible.com/ansible/latest/collections/community/general/terraform_module.html

Wrap up!

By combining Terraform for infrastructure provisioning and Ansible for configuration management, you can automate the entire lifecycle of your cloud resources, from creation to configuration. This integration streamlines the deployment process, reduces manual effort, and ensures consistency across environments. By following the steps and using the configurations provided in this article and my GitHub link, you can set up a fully automated AWS deployment workflow with Terraform and Ansible.

• Author
• Recent Posts

Follow him

Charanjit Singh

Charanjit is working as a Linux Subject Matter Expert in Kyndryl, he has 17 Years of professional experience in IT Infrastructure Projects implementation and support service delivery. Rich work experience in Data Centers and MSCs (Mobile Switching Centers). He possesses good knowledge in Linux, VMware, AWS, and Openstack Cloud.
He is a lazy System Administrator who likes to Automate his task through Shell scripts and Ansible playbooks. He has contributed his many Automations work in Kyndryl and also maintain his own internal company Github repo.
His hobby is researching on new technologies and sharing this through his blog but if he is not researching or writing blog then you will find him listening music while sipping cup of coffee and enjoying the nature view.
You can find him on Twitter, Facebook, Linkedin or send him an email at charanjit.cheema@cjcheema.com

Follow him

Latest posts by Charanjit Singh (see all)

• Terraform and Ansible Collaboration for AWS Cloud Deployment - August 28, 2024
• Automating AWS Infrastructure Provisioning in Your CI/CD Pipeline with Terraform and GitHub Actions - July 7, 2024
• Persistent Disk Mapping in Linux – Udev Rules vs. Multipath Configuration - June 4, 2024