In this guide I go over how to use Vagrant with AWS and in the process have an automated way to install Puppet Enterprise.
I am separating data and code by having a generic Vagrantfile with the code and have a servers.yaml file with all the data that will change from user to user.
For installing the Puppet Enterprise server I am including the automated provisioning script I am using with Vagrant and using AWS Tags to set the hostname of the launched server.

Pre-requisites:

  • Vagrant
  • vagrant-aws plug-in:
  • $ vagrant plugin install vagrant-aws
    
  • AWS pre-requisites:
  • While you can add your AWS credentials to the Vagrantfile, it is not recommended. A better way is to have the AWS CLI tools installed and configured

    $ aws configure
    AWS Access Key ID [****************XYYY]: XXXXXXXXYYY
    AWS Secret Access Key [****************ZOOO]:ZZZZZZZOOO
    Default region name [us-east-1]:us-east-1
    Default output format [None]: json
    

    TL;DR To get started right away you can download the project from github vagrant-aws-puppetserver, otherwise follow the guide below.

    Create Vagrantfile

    The below Vagrantfile utilizes a yaml file (servers.yaml) to provide the data, it allows you to control data using the yaml file and not have to modify the Vagrantfile code – separating code and data.

    // Vagrantfile

    # -*- mode: ruby -*-
    # vi: set ft=ruby :
    
    # Specify minimum Vagrant version and Vagrant API version
    Vagrant.require_version ">= 1.6.0"
    VAGRANTFILE_API_VERSION = "2"
    
    # Require YAML module
    require 'yaml'
    
    # Read YAML file with box details
    servers = YAML.load_file('servers.yaml')
    
    Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
    
        # Iterate through entries in YAML file
        servers.each do |server|
        
            config.vm.define server['name'] do |srv|
    
                srv.vm.box = server['box']
                srv.vm.box_url = server['box_url']
    
                srv.vm.provider :aws do |aws, override|
    
                    ### Dont do these here, better to use awscli with a profile
                    #aws.access_key_id = "YOUR KEY"
                    #aws.secret_access_key = "YOUR SECRET KEY"
                    #aws.session_token = "SESSION TOKEN"
    
                    aws.region = server["aws_region"] if server["aws_region"]
                    aws.keypair_name = server["aws_keypair_name"] if server["aws_keypair_name"]
                    aws.subnet_id = server["aws_subnet_id"] if server["aws_subnet_id"]
                    aws.associate_public_ip = server["aws_associate_public_ip"] if server["aws_associate_public_ip"]
                    aws.security_groups = server["aws_security_groups"] if server["aws_security_groups"]
                    aws.iam_instance_profile_name = server['aws_iam_role'] if server['aws_iam_role']
                    aws.ami = server["aws_ami"] if server["aws_ami"]
                    aws.instance_type = server["aws_instance_type"] if server["aws_instance_type"]
                    aws.tags = server["aws_tags"] if server["aws_tags"]
                    aws.user_data = server["aws_user_data"] if server["aws_user_data"]
    
                    override.ssh.username = server["aws_ssh_username"] if server["aws_ssh_username"]
                    override.ssh.private_key_path = server["aws_ssh_private_key_path"] if server["aws_ssh_private_key_path"]           
    
                    config.vm.synced_folder ".", "/vagrant", type: "rsync"
    
                    config.vm.provision :shell, path: server["provision"] if server["provision"]
    
                end  
            end
        end
    end
    

    Create servers.yaml

    This file contains the information that will be used by the Vagrantfile, this includes
    AWS region: Which region will this EC2 server run
    AWS keypair: Key used to connect to your launched EC2 instance
    AWS subnet id: Where will this EC2 instance sit in the AWS network
    AWS associate public ip: Do you need a public IP? true or false
    AWS security group: What AWS security group should be associated, should allow Puppetserver needed ports and whatever else you need (ssh, etc)
    AWS ami: Which AMI will you be using I am using a CentOS7
    AWS instance type: Puppetserver needs enough CPU/RAM, during my testing m3.xlarge was appropriate
    AWS SSH username: The EC2 instance user (depends on which AMI you choose), the CentOS AMI expects ec2-user
    AWS SSH private key path: The local path to the SSH key pair
    AWS User Data: I am adding user data which will execute a bash script that allows Vagrant to interact with the launched EC2 instance
    AWS Tags: This is not required for Vagrant and AWS/EC2, but in my provision script I am using the AWS Name Tag to be the system’s hostname, the other 2 tags are there for demonstration purposes
    provision: This is a provisioning script that will be run on the EC2 instance – this is the script that install the Puppet Enterprise server
    AWS IAM Role: You don’t need to add a role when working with Vagrant and AWS/EC2, but I am using a specific IAM role to allow the launched EC2 instance to be able to get information about its AWS Tags, so it is important that you provide it with a Role that allows DescribeTags, see below IAM policy:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "XXXXXXX",
                "Action": [
                    "ec2:DescribeTags"
                ],
                "Effect": "Allow",
                "Resource": "*"
            }
        ]
    }

    Now use your data in the servers.yaml file
    // servers.yaml

    ---
    - name: puppet4
      box: dummy
      box_url: https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
      aws_region: "us-east-1"
      aws_keypair_name: "john-key"
      aws_subnet_id: "subnet-0001" 
      aws_associate_public_ip: false
      aws_security_groups: ['sg-0001'] 
      aws_ami: "ami-0001"
      aws_instance_type: m3.xlarge
      aws_ssh_username: "ec2-user"
      aws_iam_role: "iam-able-to-describe-tags"
      aws_ssh_private_key_path: "/Users/john/.ssh/john-key.pem"
      aws_user_data: "#!/bin/bash\nsed -i -e 's/^Defaults.*requiretty/# Defaults requiretty/g' /etc/sudoers"
      aws_tags: 
        Name: 'puppet4'
        tier: 'stage'
        application: 'puppetserver'
      update: false
      provision: install_puppetserver.sh
    

    At this point you can spin up EC2 instances using the above Vagrantfile and servers.yaml file. If you add provision: install_puppetserver.sh to the servers.yaml file as I did and add the below script you will have a Puppet Enterprise server ready to go.

    // install_puppetserver.sh

    #!/bin/bash -xe
    
    # Ensure pip is installed
    if ! which pip; then
    	curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
    	python get-pip.py
    fi
    
    # Upgrade pip 
    /bin/pip install --upgrade pip
    
    # Upgrade awscli tools (They are installed in the AMI)
    /bin/pip install --upgrade awscli
    
    # Get hostname from AWS Name Tag (requires the EC2 instance to have an IAM role that allows DescribeTags)
    AWS_INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
    AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | grep region | cut -d\" -f4)
    AWSHOSTNAME=$(aws ec2 describe-tags --region ${AWS_REGION} --filters "Name=resource-id,Values=${AWS_INSTANCE_ID}" --query "Tags[?Key=='Name'].Value[] | [0]" | cut -d\" -f2)
    
    # Set hostname (use the AWS Name Tag)
    hostnamectl set-hostname ${AWSHOSTNAME}.cpg.org
    
    # Update system and install wget, git
    yum update -y
    yum install wget git -y
    
    # Set puppet.cpg.org as hostname in hosts file
    echo "$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) puppet puppet.cpg.org" >> /etc/hosts
    
    # Download puppet 
    wget "https://pm.puppetlabs.com/cgi-bin/download.cgi?dist=el&rel=7&arch=x86_64&ver=2016.4.0" -O puppet.2016.4.0.tar.gz
    tar -xvzf puppet.2016.4.0.tar.gz
    cd puppet-enterprise*
    
    # Create pe.conf file
    touch pe.conf
    echo '{' >> pe.conf
    echo '"console_admin_password": "puppet"' >> pe.conf
    echo '"puppet_enterprise::puppet_master_host": "%{::trusted.certname}"' >> pe.conf
    echo '}'  >> pe.conf
    
    echo "Install Puppetserver"
    ./puppet-enterprise-installer -c pe.conf
    
    echo "Adding * to autosign.conf"
    cat >> /etc/puppetlabs/puppet/autosign.conf <<'AUTOSIGN'
    *
    AUTOSIGN
    
    # Run puppet agent
    /usr/local/bin/puppet agent -t