Martin Buchleitner, Senior IT-Consultant

About the author

Martin Buchleitner is a Senior IT-Consultant for Infralovers and for Commandemy. Twitter github LinkedIn

See all articles by this author

Automated Cloud Templates with HashiCorp Packer

In our previous post about Packer and azure, we used Azure to introduce a HashiCorp Packer definition in HCL Format which can easily be adapted to create any custom machine configuration. The next step is to use the same provisioning configuration also for other cloud providers and to have the same outcoming result each time - independent from the infrastructure your virtual machine is running.

Recap: Azure ARM Templates

A short recap on what we defined last time for azure, is this configuration item in HashiCorp Packer.

source "azure-arm" "core" {

  client_id       = var.client_id
  client_secret   = var.client_secret
  subscription_id = var.subscription_id
  tenant_id       = var.tenant_id

  managed_image_name                = "UbuntuDocker"
  managed_image_resource_group_name = "images"

  os_type         = "Linux"
  image_publisher = "Canonical"
  image_offer     = "0001-com-ubuntu-server-hirsute"
  image_sku       = "21_04"
  image_version   = "latest"

  location = "westeurope"
  vm_size  = "Standard_F2s"
}

AWS AMI Template

Now we gonna redefine the same definition for AWS to create an AWS AMI Template. This template is going to have the same custom configuration as our previous Azure VM. So we also gonna use Ubuntu 21.04 base image as the starting point for our customizing process.


source "amazon-ebs" "core" {
  ami_description             = "Ubuntu Docker AMI"
  ami_name                    = "UbuntuDocker"
  ami_regions                 = ["us-east-1"]
  ami_virtualization_type     = "hvm"
  associate_public_ip_address = true
  instance_type               = "t3.medium"
  profile                     = var.aws_profile
  region                      = "us-east-1"
  ssh_clear_authorized_keys = true
  ssh_timeout               = "5m"
  ssh_username              = "ubuntu"

  source_ami_filter {
    filters = {
      architecture        = "x86_64"
      name                = "ubuntu/images/hvm-ssd/ubuntu-hirsute-21.04-amd64-server*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"] # canonical
  }
}

Recap: Build/Customize the Image

Once again, a small recap on our build configuration for customizing the image. We use ansible to run the actual customizing and we are using a variable on the Packer template to define which ansible playbook is used within the virtual machine.


variable "playbook" {
  type    = string
  default = "docker.yml"
}

build {
  sources = [ ]

  provisioner "shell" {
    inline = ["while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done"]
  }

  provisioner "shell" {
    execute_command = "echo 'packer' | {{ .Vars }} sudo -S -E bash '{{ .Path }}'"
    script          = "packer/scripts/setup.sh"
  }

  provisioner "ansible-local" {
    clean_staging_directory = true
    playbook_dir            = "ansible"
    galaxy_file             = "ansible/requirements.yaml"
    playbook_files          = ["ansible/${var.playbook}.yml"]
  }

  provisioner "shell" {
    execute_command = "echo 'packer' | {{ .Vars }} sudo -S -E bash '{{ .Path }}'"
    script          = "packer/scripts/cleanup.sh"
  }
}

Full Combined Packer Definition

And finally here is the full definition to build 2 Virtual machines - one for use within Azure, the other within AWS. Both images will run the same provisioning process by ansible. In this case, we have to set all those variables for each of the infrastructures we are using and referencing within this build process, otherwise, we will experience errors from Packer, that the images cannot be built or some sources cannot be found.

Also, the full template gets quite messy if adding all your infrastructure within one single Packer definition.


variable "playbook" {
  type    = string
  default = "docker.yml"
}

variable "aws_profile" {
  type    = string
  default = "${env("AWS_PROFILE")}"
}

variable "subscription_id" {
  type    = string
  default = "${env("ARM_SUBSCRIPTION_ID")}"
}

variable "tenant_id" {
  type    = string
  default = "${env("ARM_TENANT_ID")}"
}

variable "client_id" {
  type    = string
  default = "${env("ARM_CLIENT_ID")}"
}

variable "client_secret" {
  type    = string
  default = "${env("ARM_CLIENT_SECRET")}"
}

source "amazon-ebs" "core" {
  ami_description             = "Ubuntu Docker AMI"
  ami_name                    = "UbuntuDocker"
  ami_regions                 = ["us-east-1"]
  ami_virtualization_type     = "hvm"
  associate_public_ip_address = true
  force_delete_snapshot       = true
  force_deregister            = true
  instance_type               = "t3.medium"
  profile                     = var.aws_profile
  region                      = "us-east-1"
  ssh_clear_authorized_keys = true
  ssh_timeout               = "5m"
  ssh_username              = "ubuntu"

  source_ami_filter {
    filters = {
      architecture        = "x86_64"
      name                = "ubuntu/images/hvm-ssd/ubuntu-hirsute-21.04-amd64-server*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"] # canonical
  }
}

source "azure-arm" "core" {

  client_id       = var.client_id
  client_secret   = var.client_secret
  subscription_id = var.subscription_id
  tenant_id       = var.tenant_id

  managed_image_name                = "UbuntuDocker"
  managed_image_resource_group_name = "images"

  os_type         = "Linux"
  image_publisher = "Canonical"
  image_offer     = "0001-com-ubuntu-server-hirsute"
  image_sku       = "21_04"
  image_version   = "latest"

  location = "westeurope"
  vm_size  = "Standard_F2s"
}

build {
  sources = ["source.amazon-ebs.core", "source.azure-arm.core"]

  provisioner "shell" {
    inline = ["while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done"]
  }

  provisioner "shell" {
    execute_command = "echo 'packer' | {{ .Vars }} sudo -S -E bash '{{ .Path }}'"
    script          = "packer/scripts/setup.sh"
  }

  provisioner "ansible-local" {
    clean_staging_directory = true
    playbook_dir            = "ansible"
    galaxy_file             = "ansible/requirements.yaml"
    playbook_files          = ["ansible/${var.playbook}.yml"]
  }

  provisioner "shell" {
    execute_command = "echo 'packer' | {{ .Vars }} sudo -S -E bash '{{ .Path }}'"
    script          = "packer/scripts/cleanup.sh"
  }
}

Final thoughts

These definitions can be adapted to any further cloud definition - e.g. Google Cloud, VMWare, Vagrant, …

The outcome of this process should be identical provisioned virtual machines for the infrastructure you define as sources. It should be the same, but is it really the same? That’s the next topic we gonna cover - how to ensure that those created virtual machines behave the same. This will open the possibility to separate some definitions and generate the template on the fly with very basic tooling.

You want to learn more about this topic?

You don't learn pure knowledge from books and it is not available in capsule form. The most effective form of exercise is still with sparring partners and a guide. Therefore, our "Commandemy" brand offers training for the IT experts of tomorrow.

Become the undisputed king of code and take a look at our current courses now!

See current courses