Technologist

Tech stuff about Cloud, DevOps, SysAdmin, Virtualization, SAN, Hardware, Scripting, Automation and Development

Browsing Posts in Vagrant

In this guide I go over how to use Vagrant with AWS and in the process have an automated way to install Puppet Enterprise.
I am separating data and code by having a generic Vagrantfile with the code and have a servers.yaml file with all the data that will change from user to user.
For installing the Puppet Enterprise server I am including the automated provisioning script I am using with Vagrant and using AWS Tags to set the hostname of the launched server.

Pre-requisites:

  • Vagrant
  • vagrant-aws plug-in:
  • $ vagrant plugin install vagrant-aws
    
  • AWS pre-requisites:
  • While you can add your AWS credentials to the Vagrantfile, it is not recommended. A better way is to have the AWS CLI tools installed and configured

    $ aws configure
    AWS Access Key ID [****************XYYY]: XXXXXXXXYYY
    AWS Secret Access Key [****************ZOOO]:ZZZZZZZOOO
    Default region name [us-east-1]:us-east-1
    Default output format [None]: json
    

    TL;DR To get started right away you can download the project from github vagrant-aws-puppetserver, otherwise follow the guide below.

    Create Vagrantfile

    The below Vagrantfile utilizes a yaml file (servers.yaml) to provide the data, it allows you to control data using the yaml file and not have to modify the Vagrantfile code – separating code and data.

    // Vagrantfile

    # -*- mode: ruby -*-
    # vi: set ft=ruby :
    
    # Specify minimum Vagrant version and Vagrant API version
    Vagrant.require_version ">= 1.6.0"
    VAGRANTFILE_API_VERSION = "2"
    
    # Require YAML module
    require 'yaml'
    
    # Read YAML file with box details
    servers = YAML.load_file('servers.yaml')
    
    Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
    
        # Iterate through entries in YAML file
        servers.each do |server|
        
            config.vm.define server['name'] do |srv|
    
                srv.vm.box = server['box']
                srv.vm.box_url = server['box_url']
    
                srv.vm.provider :aws do |aws, override|
    
                    ### Dont do these here, better to use awscli with a profile
                    #aws.access_key_id = "YOUR KEY"
                    #aws.secret_access_key = "YOUR SECRET KEY"
                    #aws.session_token = "SESSION TOKEN"
    
                    aws.region = server["aws_region"] if server["aws_region"]
                    aws.keypair_name = server["aws_keypair_name"] if server["aws_keypair_name"]
                    aws.subnet_id = server["aws_subnet_id"] if server["aws_subnet_id"]
                    aws.associate_public_ip = server["aws_associate_public_ip"] if server["aws_associate_public_ip"]
                    aws.security_groups = server["aws_security_groups"] if server["aws_security_groups"]
                    aws.iam_instance_profile_name = server['aws_iam_role'] if server['aws_iam_role']
                    aws.ami = server["aws_ami"] if server["aws_ami"]
                    aws.instance_type = server["aws_instance_type"] if server["aws_instance_type"]
                    aws.tags = server["aws_tags"] if server["aws_tags"]
                    aws.user_data = server["aws_user_data"] if server["aws_user_data"]
    
                    override.ssh.username = server["aws_ssh_username"] if server["aws_ssh_username"]
                    override.ssh.private_key_path = server["aws_ssh_private_key_path"] if server["aws_ssh_private_key_path"]           
    
                    config.vm.synced_folder ".", "/vagrant", type: "rsync"
    
                    config.vm.provision :shell, path: server["provision"] if server["provision"]
    
                end  
            end
        end
    end
    

    Create servers.yaml

    This file contains the information that will be used by the Vagrantfile, this includes
    AWS region: Which region will this EC2 server run
    AWS keypair: Key used to connect to your launched EC2 instance
    AWS subnet id: Where will this EC2 instance sit in the AWS network
    AWS associate public ip: Do you need a public IP? true or false
    AWS security group: What AWS security group should be associated, should allow Puppetserver needed ports and whatever else you need (ssh, etc)
    AWS ami: Which AMI will you be using I am using a CentOS7
    AWS instance type: Puppetserver needs enough CPU/RAM, during my testing m3.xlarge was appropriate
    AWS SSH username: The EC2 instance user (depends on which AMI you choose), the CentOS AMI expects ec2-user
    AWS SSH private key path: The local path to the SSH key pair
    AWS User Data: I am adding user data which will execute a bash script that allows Vagrant to interact with the launched EC2 instance
    AWS Tags: This is not required for Vagrant and AWS/EC2, but in my provision script I am using the AWS Name Tag to be the system’s hostname, the other 2 tags are there for demonstration purposes
    provision: This is a provisioning script that will be run on the EC2 instance – this is the script that install the Puppet Enterprise server
    AWS IAM Role: You don’t need to add a role when working with Vagrant and AWS/EC2, but I am using a specific IAM role to allow the launched EC2 instance to be able to get information about its AWS Tags, so it is important that you provide it with a Role that allows DescribeTags, see below IAM policy:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "XXXXXXX",
                "Action": [
                    "ec2:DescribeTags"
                ],
                "Effect": "Allow",
                "Resource": "*"
            }
        ]
    }

    Now use your data in the servers.yaml file
    // servers.yaml

    ---
    - name: puppet4
      box: dummy
      box_url: https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
      aws_region: "us-east-1"
      aws_keypair_name: "john-key"
      aws_subnet_id: "subnet-0001" 
      aws_associate_public_ip: false
      aws_security_groups: ['sg-0001'] 
      aws_ami: "ami-0001"
      aws_instance_type: m3.xlarge
      aws_ssh_username: "ec2-user"
      aws_iam_role: "iam-able-to-describe-tags"
      aws_ssh_private_key_path: "/Users/john/.ssh/john-key.pem"
      aws_user_data: "#!/bin/bash\nsed -i -e 's/^Defaults.*requiretty/# Defaults requiretty/g' /etc/sudoers"
      aws_tags: 
        Name: 'puppet4'
        tier: 'stage'
        application: 'puppetserver'
      update: false
      provision: install_puppetserver.sh
    

    At this point you can spin up EC2 instances using the above Vagrantfile and servers.yaml file. If you add provision: install_puppetserver.sh to the servers.yaml file as I did and add the below script you will have a Puppet Enterprise server ready to go.

    // install_puppetserver.sh

    #!/bin/bash -xe
    
    # Ensure pip is installed
    if ! which pip; then
    	curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
    	python get-pip.py
    fi
    
    # Upgrade pip 
    /bin/pip install --upgrade pip
    
    # Upgrade awscli tools (They are installed in the AMI)
    /bin/pip install --upgrade awscli
    
    # Get hostname from AWS Name Tag (requires the EC2 instance to have an IAM role that allows DescribeTags)
    AWS_INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
    AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | grep region | cut -d\" -f4)
    AWSHOSTNAME=$(aws ec2 describe-tags --region ${AWS_REGION} --filters "Name=resource-id,Values=${AWS_INSTANCE_ID}" --query "Tags[?Key=='Name'].Value[] | [0]" | cut -d\" -f2)
    
    # Set hostname (use the AWS Name Tag)
    hostnamectl set-hostname ${AWSHOSTNAME}.cpg.org
    
    # Update system and install wget, git
    yum update -y
    yum install wget git -y
    
    # Set puppet.cpg.org as hostname in hosts file
    echo "$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) puppet puppet.cpg.org" >> /etc/hosts
    
    # Download puppet 
    wget "https://pm.puppetlabs.com/cgi-bin/download.cgi?dist=el&rel=7&arch=x86_64&ver=2016.4.0" -O puppet.2016.4.0.tar.gz
    tar -xvzf puppet.2016.4.0.tar.gz
    cd puppet-enterprise*
    
    # Create pe.conf file
    touch pe.conf
    echo '{' >> pe.conf
    echo '"console_admin_password": "puppet"' >> pe.conf
    echo '"puppet_enterprise::puppet_master_host": "%{::trusted.certname}"' >> pe.conf
    echo '}'  >> pe.conf
    
    echo "Install Puppetserver"
    ./puppet-enterprise-installer -c pe.conf
    
    echo "Adding * to autosign.conf"
    cat >> /etc/puppetlabs/puppet/autosign.conf <<'AUTOSIGN'
    *
    AUTOSIGN
    
    # Run puppet agent
    /usr/local/bin/puppet agent -t
    

In a previous post I wrote about how to Create a custom Vagrant Box from scratch.

In this post I will walk through the usage of Packer to automate the creation of a CentOS 7 image that can be used in Vagrant or even vSphere.

Packer is a cool Hashicorp tool that helps you automate the creation of machine images.

You can download the project/git repo at https://github.com/parcejohn/packer-centos7

From their website:
“Packer is easy to use and automates the creation of any type of machine image. It embraces modern configuration management by encouraging you to use automated scripts to install and configure the software within your Packer-made images. Packer brings machine images into the modern age, unlocking untapped potential and opening new opportunities.”

I use Packer to continue the journey to Infrastructure As Code, where even my golden images/templates are automated and source controlled (on Git).

Requirements:

* Packer // I installed on Mac using $ brew install packer
* vagrant // I installed on Mac using $ brew install vagrant
* vmware fusion // I installed using $ brew cask install vmware-fusion
* CentOS 7 ISO file

Packer uses a JSON template file to orchestrate the image creation and there are different stages in the process, I will concentrate on the below three, but you should familiarize with all of them (https://www.packer.io/docs/basics/terminology.html)

Builders: Packer component to create a machine image for a single platform, in this case I will be using the VMware builder.
Provisioners: Packer component that installs and configures software within a running machine prior to that machine being turned into a static image. Example provisioners include shell scripts, Chef, Puppet, etc. I will be using the shell provisioner.
Post-Processors: Packer component that takes the result of the builder or another post-processor and process that to create a new artifact. Examples of post-processors are compress to compress artifacts, upload to upload artifacts, etc. I will be creating a Vagrant box as the artifact, and also demonstrate how to upload the image to vSphere.

It’s time to walk through the process of creating a CentOS 7 image using Packer.

1) Create directory structure
The http folder will host a kickstart file, Packer will use its built in web server to serve this kickstart file
The scripts folder will host the provisioning scripts that will define the machine

centos/
├── http
└── scripts

2) Populate with the following configuration files and scripts (the contents of these files are shown in the next steps)

centos/
├── centos-7.1-x64-vmware.json
├── http
│   └── ks.cfg
├── scripts
│   ├── base.sh
│   ├── cleanup.sh
│   ├── hgfs.sh
│   ├── vmware.sh
│   └── zerodisk.sh
├── template.json
└── vagrant_rsa_key

3) Create a SSH key pair, you will need this to login to the server to complete install and configuration

The Public Key will be injected to the ‘vagrant’ user as part of the kickstart, it will then be used in the Packer JSON template to allow Packer to login to the machine
The Private Key will stay on your system to allow Packer and Vagrant to login to the created machine.

$ ssh-keygen -t rsa -b 4096 -C "vagrant" -N '' -q -f ./vagrant_rsa_key

4) Create a Packer JSON Template file (template.json)

{
  "variables": {
    "vm_name": "centos-7.1-vmware",
    "iso_url": "{{env `ISO_URL`}}",
    "iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284"
  },
  "builders": [
    {
      "headless": true,
      "type": "vmware-iso",
      "boot_command": [
        " text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg"
      ],
      "boot_wait": "10s",
      "disk_size": 8192,
      "guest_os_type": "centos-64",
      "http_directory": "http",
      "iso_url": "{{user `iso_url`}}",
      "iso_checksum_type": "sha256",
      "iso_checksum": "{{user `iso_sha256`}}",
      "ssh_username": "vagrant",
      "ssh_private_key_file": "vagrant_rsa",
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
      "shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
      "tools_upload_flavor": "linux",
      "tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",
      "vmx_data": {
        "memsize": "1024",
        "numvcpus": "1",
        "cpuid.coresPerSocket": "1"
      }
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
      "override": {
        "vmware-iso": {
          "scripts": [
            "scripts/base.sh",
            "scripts/vmware.sh",
            "scripts/hgfs.sh",
            "scripts/cleanup.sh",
            "scripts/zerodisk.sh"
          ]
        }
      }
    }
  ],
  "post-processors": [
    {
      "type": "vagrant",
      "override": {
        "vmware": {
          "output": "centos-7.1-x64-vmware.box"
        }
      }
    }
  ]
}

Packer Components/Sections:

Variables:
This is just to centralize variables that will be used by other components (e.g. Builder, provisioner, etc)
I can also set a variable to use Environment variables or command line provided values:
User provided values:
“iso_url”: “{{user `iso_url`}}”,

Environment variable values:
“iso_url”: “{{env `ISO_URL`}}”,

Builders:
Here is an array of builders and the needed parameters for each builder, in this case the only builder is of type vmware-iso.
Parameters and some comments about them.

      "headless": true, // Means not to open the VMware Fusion Console
      "type": "vmware-iso", // VMware type builder
      "boot_command": [
        " text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg"  // Send these key combination to the VMWare Fusion Console
      ],
      "boot_wait": "10s",
      "disk_size": 8192,
      "guest_os_type": "centos-64",
      "http_directory": "http",  // Directory where the kickstart file is placed
      "iso_url": "{{user `iso_url`}}", // Location of ISO image, defined at runtime, more on that later
      "iso_checksum_type": "sha256", 
      "iso_checksum": "{{user `iso_sha256`}}",
      "ssh_username": "vagrant",  // Packer will log in to resulting machine for provisioning, this user must exist (user created from the Kickstart) 
      "ssh_private_key_file": "vagrant_rsa", // Packer will log in using ssh key created earlier
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
      "shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
      "tools_upload_flavor": "linux",
      "tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso", 
      "vmx_data": {
        "memsize": "1024",
        "numvcpus": "1",
        "cpuid.coresPerSocket": "1"

Provisioners:
This section uses the shell provisioner and runs all of those scripts listed, which are located in the scripts folder.

      "type": "shell",
      "execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
      "override": {
        "vmware-iso": {
          "scripts": [
            "scripts/base.sh",
            "scripts/vmware.sh",
            "scripts/hgfs.sh",
            "scripts/cleanup.sh",
            "scripts/zerodisk.sh"

Post-processor:
This section states that I want a vagrant box and I am giving it the name.

      "type": "vagrant",
      "override": {
        "vmware": {
          "output": "centos-7.1-x64-vmware.box"

5) Kickstart file (should be http/ks.cfg based on the template.json file)
This is a minimal install of CentOS 7
Note: The private key (XXXX) given to the vagrant user, that is the key that we created in a previous step

install
cdrom
lang en_US.UTF-8
keyboard us
network --onboot yes --device eth0 --bootproto dhcp --noipv6
rootpw  --plaintext vagrant
firewall --enabled --service=ssh
authconfig --enableshadow --passalgo=sha512
selinux --disabled
timezone --utc America/New_York
bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet"

text
skipx
zerombr

clearpart --all --initlabel
autopart

auth  --useshadow  --enablemd5
firstboot --disabled
reboot

%packages --nobase --ignoremissing 
@core
bzip2
kernel-devel
kernel-headers
-ipw2100-firmware
-ipw2200-firmware
-ivtv-firmware
%end

%post
# Install SUDO
/usr/bin/yum -y install sudo

# Create vagrant user
/usr/sbin/useradd vagrant
/bin/mkdir /home/vagrant/.ssh
/bin/chmod 700 /home/vagrant/.ssh
cat >           /home/vagrant/.ssh/authorized_keys <<'VAGRANT_RSA'
ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX vagrant
VAGRANT_RSA

/bin/chmod 600 /home/vagrant/.ssh/authorized_keys
/bin/chown -R vagrant /home/vagrant/.ssh

# Add vagrant user to SUDO
echo "vagrant        ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers.d/vagrant
echo "Defaults:vagrant !requiretty"                 >> /etc/sudoers.d/vagrant
chmod 0440 /etc/sudoers.d/vagrant
%end

6 ) Provisioning Scrips
Packer calls these scripts (from the JSON template) on the created machine before making it a template/image.

base.sh // Install basic packages

#!/usr/bin/env bash
set -x

sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers
yum -y install gcc make gcc-c++ kernel-devel-`uname -r` perl

vmware.sh // Install VMTools, I separated this from base as I may want to call different scripts on a VirtualBox/etc builder.

#!/usr/bin/env bash
set -x

yum install -y fuse-libs open-vm-tools

hgfs.sh // For vagrant to work properly with folders

#!/usr/bin/env bash
set -x

VMWARE_ISO=/tmp/vmware_tools_linux.iso
VMWARE_MNTDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_mnt_XXXXXX)
VMWARE_TMPDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_XXXXXX)

# Extract tools
mount -o loop $VMWARE_ISO $VMWARE_MNTDIR
tar zxf $VMWARE_MNTDIR/VMwareTools*.tar.gz -C $VMWARE_TMPDIR
umount $VMWARE_MNTDIR

# Install tools
$VMWARE_TMPDIR/vmware-tools-distrib/vmware-install.pl -d

# Clean up
rm -f $VMWARE_ISO
rm -rf $VMWARE_MNTDIR
rm -rf $VMWARE_TMPDIR

cleanup.sh // Clean up before converting to image

#!/usr/bin/env bash
set -x

yum -y erase gtk2 libX11 hicolor-icon-theme avahi freetype bitstream-vera-fonts
rpm --rebuilddb
yum -y clean all

zerodisk.sh // This is so that the resulting box is as small as possible

#!/usr/bin/env bash
set -x

dd if=/dev/zero of=/EMPTY bs=1M
rm -f /EMPTY

7) Run Packer against the template.json file created earlier and watch it build the machine image

$ packer build -var ‘iso_url=/Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso’ -only=vmware-iso template.json
vmware-iso output will be in this color.

==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: file:///Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso
==> vmware-iso: Creating virtual machine disk
==> vmware-iso: Building and writing VMX file
==> vmware-iso: Starting HTTP server on port 8101
==> vmware-iso: Starting virtual machine…
vmware-iso: The VM will be run headless, without a GUI. If you want to
vmware-iso: view the screen of the VM, connect via VNC without a password to
vmware-iso: 127.0.0.1:5989
==> vmware-iso: Waiting 10s for boot…
==> vmware-iso: Connecting to VM via VNC
==> vmware-iso: Typing the boot command over VNC…
==> vmware-iso: Waiting for SSH to become available…
==> vmware-iso: Connected to SSH!
==> vmware-iso: Uploading the ‘linux’ VMware Tools
==> vmware-iso: Provisioning with shell script: scripts/base.sh
vmware-iso: + sed -i ‘s/^.*requiretty/#Defaults requiretty/’ /etc/sudoers



==> vmware-iso: Provisioning with shell script: scripts/zerodisk.sh
vmware-iso: + dd if=/dev/zero of=/EMPTY bs=1M
vmware-iso: dd: error writing ‘/EMPTY’: No space left on device
vmware-iso: 5488+0 records in
vmware-iso: 5487+0 records out
vmware-iso: 5754265600 bytes (5.8 GB) copied, 5.27091 s, 1.1 GB/s
vmware-iso: + rm -f /EMPTY
==> vmware-iso: Gracefully halting virtual machine…
vmware-iso: Waiting for VMware to clean up after itself…
==> vmware-iso: Deleting unnecessary VMware files…
vmware-iso: Deleting: output-vmware-iso/564de113-2fc5-2010-8e5d-63fec92f12f7.vmem
vmware-iso: Deleting: output-vmware-iso/packer-vmware-iso.plist
vmware-iso: Deleting: output-vmware-iso/vmware.log
==> vmware-iso: Cleaning VMX prior to finishing up…
vmware-iso: Unmounting floppy from VMX…
vmware-iso: Detaching ISO from CD-ROM device…
vmware-iso: Disabling VNC server…
==> vmware-iso: Compacting the disk image
==> vmware-iso: Running post-processor: vagrant
==> vmware-iso (vagrant): Creating Vagrant box for ‘vmware’ provider
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s001.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s002.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s003.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.nvram
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmsd
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmxf
vmware-iso (vagrant): Compressing: Vagrantfile
vmware-iso (vagrant): Compressing: disk-s001.vmdk
vmware-iso (vagrant): Compressing: disk-s002.vmdk
vmware-iso (vagrant): Compressing: disk-s003.vmdk
vmware-iso (vagrant): Compressing: disk.vmdk
vmware-iso (vagrant): Compressing: metadata.json
vmware-iso (vagrant): Compressing: packer-vmware-iso.nvram
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmsd
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmx
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmxf
Build ‘vmware-iso’ finished.

==> Builds finished. The artifacts of successful builds are:
–> vmware-iso: ‘vmware’ provider box: centos-7.1-x64-vmware.box

8) Add box to Vagrant

$ vagrant box add –name centos-7.1-010-x64-vmware.box centos-7.1-x64-vmware.box
==> box: Adding box ‘centos-7.1-010-x64-vmware.box’ (v0) for provider:
box: Downloading: file:///Users/john/packer/centos/centos-7.1-x64-vmware.box
==> box: Successfully added box ‘centos-7.1-010-x64-vmware.box’ (v0) for ‘vmware_desktop’!

The above is all you need to have your Infrastructure be Code that can be versioned and automatically create your images and use them on Vagrant.
The next item will show you how to use Packer to move the same image to your vSphere environment.

9) Sending Image to vSphere
To send the image created by Packer to vSphere you will need to add another post-provisioning entry to the JSON array:

    {
      "type": "vsphere",
      "host": "{{user `vm_host`}}",
      "username": "{{user `vm_user`}}",
      "password": "{{user `vm_pass`}}",
      "datacenter": "{{user `vm_dc`}}",
      "cluster": "{{user `vm_cluster`}}",
      "resource_pool": " ",
      "datastore": "{{user `vm_datastore`}}",
      "vm_folder": "{{user `vm_folder`}}",
      "vm_name": "{{user `vm_name`}}", 
      "vm_network": "{{user `vm_network`}}",
      "insecure" : "true"
    }

As you can see I am using variables to make this portable, so that means you have to add variables to the variables section as well, below is the new template to do both vagrant and vSphere image provisioning.

{
  "variables": {
    "vm_name": "centos-7.1-vmware",
    "iso_url": "{{env `ISO_URL`}}",
    "iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284",
    "vm_host": "{{ user `vm_host` }}",
    "vm_user": "{{ user `vm_user` }}",
    "vm_pass": "{{ env `vm_pass` }}",
    "vm_dc":   "{{ user `vm_dc` }}",
    "vm_cluster": "{{user `vm_cluster`}}",
    "vm_datastore": "{{user `vm_datastore`}}",
    "vm_folder": "{{user `vm_folder`}}",
    "vm_name": "{{user `vm_name`}}", 
    "vm_network": "{{user `vm_network`}}"
  },
  "builders": [
    {
      "headless": true,
      "type": "vmware-iso",
      "boot_command": [
        " text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg"
      ],
      "boot_wait": "10s",
      "disk_size": 8192,
      "guest_os_type": "centos-64",
      "http_directory": "http",
      "iso_url": "{{user `iso_url`}}",
      "iso_checksum_type": "sha256",
      "iso_checksum": "{{user `iso_sha256`}}",
      "ssh_username": "vagrant",
      "ssh_private_key_file": "vagrant_rsa",
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
      "shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
      "tools_upload_flavor": "linux",
      "tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",
      "vmx_data": {
        "memsize": "1024",
        "numvcpus": "1",
        "cpuid.coresPerSocket": "1"
      }
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
      "override": {
        "vmware-iso": {
          "scripts": [
            "scripts/base.sh",
            "scripts/vmware.sh",
            "scripts/hgfs.sh",
            "scripts/cleanup.sh",
            "scripts/zerodisk.sh"
          ]
        }
      }
    }
  ],
  "post-processors": [
    {
      "type": "vagrant",
      "override": {
        "vmware": {
          "output": "centos-7.1-x64-vmware.box"
        }
      }
    },
    {
      "type": "vsphere",
      "host": "{{user `vm_host`}}",
      "username": "{{user `vm_user`}}",
      "password": "{{user `vm_pass`}}",
      "datacenter": "{{user `vm_dc`}}",
      "cluster": "{{user `vm_cluster`}}",
      "resource_pool": " ",
      "datastore": "{{user `vm_datastore`}}",
      "vm_folder": "{{user `vm_folder`}}",
      "vm_name": "{{user `vm_name`}}", 
      "vm_network": "{{user `vm_network`}}",
      "insecure" : "true"
    }
  ]
}

10) Re-Run Packer against the template.json file and watch it build the machine image an

$ packer build \
> -var ‘iso_url=/Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso’ \
> -var ‘vm_host=vc.example.com’ \
> -var ‘vm_user=john@example.com’ \
> -var ‘vm_pass=XXXXXXX’ \
> -var ‘vm_dc=vDC’ \
> -var ‘vm_cluster=Folder/Cluster’ \
> -var ‘vm_datastore=store’ \
> -var ‘vm_folder=Images’ \
> -var ‘vm_name=centos71’ \
> -var ‘vm_network=dvs-net1’ \
> -only=vmware-iso template.json

vmware-iso output will be in this color.

==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: file:///Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso
==> vmware-iso: Creating virtual machine disk


==> vmware-iso: Running post-processor: vagrant
==> vmware-iso (vagrant): Creating Vagrant box for ‘vmware’ provider
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s001.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s002.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s003.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.nvram
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmsd
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmxf
vmware-iso (vagrant): Compressing: Vagrantfile
vmware-iso (vagrant): Compressing: disk-s001.vmdk
vmware-iso (vagrant): Compressing: disk-s002.vmdk
vmware-iso (vagrant): Compressing: disk-s003.vmdk
vmware-iso (vagrant): Compressing: disk.vmdk
vmware-iso (vagrant): Compressing: metadata.json
vmware-iso (vagrant): Compressing: packer-vmware-iso.nvram
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmsd
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmx
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmxf
==> vmware-iso: Running post-processor: vsphere
vmware-iso (vsphere): Uploading output-vmware-iso/packer-vmware-iso.vmx to vSphere
vmware-iso (vsphere): Opening VMX source: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vsphere): Opening VI target: vi://john%40example.com@vc.exmaple.com:443/vDC/host/Cluster/Resources/
vmware-iso (vsphere): Deploying to VI: vi://john%40example.com@vc.example.com:443/vDC/host/Cluster/Resources/
Transfer Completed
vmware-iso (vsphere): Completed successfully
vmware-iso (vsphere):
Build ‘vmware-iso’ finished.

==> Builds finished. The artifacts of successful builds are:
–> vmware-iso: ‘vmware’ provider box: centos-7.1-x64-vmware.box
–> vmware-iso:

Now you should have the same image running on both Vagrant and vSphere.

In a previous post I showed how to create a vagrant box from scratch and add it to Vagrant: Creating a custom Vagrant Box

Traditionally you will place the Vagrant Box in a web server, and then use it in a Vagrantfile as follows:

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "vagrant_rhel6_vanilla"
  config.vm.box_url = "http://vagrant.example.com/vagrant/vagrant_rhel6_vanilla.box"
...

In this post I will show you how to add versioning to your vagrant boxes, so that whenever the base box is changed/updated/patched your end-users/developers will be automatically notified and will be able to get the latest vagrant box with a simple ‘vagrant box update’ command.

1) Create the updated Vagrant Box
(e.g. vagrant_rhel6_vanilla_010.box for version 0.1.0) – you can use the procedure: Creating a custom Vagrant Box

2) Get a checksum of the box – You will need it later for the JSON Box catalog

$ shasum vagrant_rhel6_vanilla_010.box
b42efdb1e789976a45ffc26a321f00a42932f792  vagrant_rhel6_vanilla_010.box

OR

$ openssl sha1  vagrant_rhel6_vanilla_010.box
SHA1(vagrant_rhel6_vanilla_010.box)= b42efdb1e789976a45ffc26a321f00a42932f792

3) Upload the Box to the Vagrant Box web server repository:

I recommend to use the following web directory structure:
../vagrant/boxes serves the actual boxes/VMs
../vagrant serves the Box metadata JSON file (I explain what this is below)

vagrant/
|-- boxes
|   |-- vagrant_rhel6_vanilla_010.box
`-- vagrant_rhel6_vanilla.json

Also, ensure the web server returns json as the Content-Type.
For Apache you need to check/add that the following configuration is present:
AddType application/json .json

To verify if your webserver is returning the correct Content-Type:

$ curl -sI  http://vagrant.example.com/vagrant/vagrant_rhel6_vanilla.json | grep Content-Type
Content-Type: application/json

4) Create a Box catalog to be served by the web server:

The box catalog is a JSON metadata file allows you to server multiple versions of a box and enable update status notification.
Fields:
name: Name of the Vagrant Box, this is what you will put under config.vm.box on the Vagrantfile
Versions: This is an array/list/collection where you will put the different versions you are hosting/serving
url: This is the actual location of the box, this is what you will put under config.vm.box_url on the Vagrantfile

$ cat vagrant_rhel6_vanilla.json

{
  "name": "vagrant_rhel6_vanilla",
  "description": "This box contains RHEL 64-bit.",
  "versions": [
    {
      "version": "0.1.0",
      "providers": [
        {
          "name": "vmware_desktop",
          "url": "http://vagrant.example.com/vagrant/boxes/vagrant_rhel6_vanilla_010.box",
          "checksum_type": "sha1",
          "checksum": "b42efdb1e789976a45ffc26a321f00a42932f792"
        }
      ]
    }
  ]
}

5) Modify the Vagrantfile to use the json box catalog file, instead of the actual box name

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "vagrant_rhel6_vanilla"
  config.vm.box_url = "http://vagrant.example.com/vagrant/vagrant_rhel6_vanilla.json"
...

The first time you move from traditional box to versioned box, you need to delete the traditional box that is cached on your system.
This is only needed if you have a traditional box cached.

$ vagrant box list
vagrant_rhel6_vanilla      (vmware_desktop, 0)  <-- Traditional Box

$ vagrant box remove vagrant_rhel6_vanilla
Removing box 'vagrant_rhel6_vanilla' (v0) with provider 'vmware_desktop'...

6) Vagrant up and see Vagrant download the versioned Box

$ vagrant up
Bringing machine 'default' up with 'vmware_fusion' provider...
==> default: Box 'vagrant_rhel6_vanilla' could not be found. Attempting to find and install...
    default: Box Provider: vmware_desktop, vmware_fusion, vmware_workstation
    default: Box Version: >= 0
==> default: Loading metadata for box 'http://vagrant.example.com/vagrant/vagrant_rhel6_vanilla.json'
    default: URL: http://vagrant.example.com/vagrant/vagrant_rhel6_vanilla.json
==> default: Adding box 'vagrant_rhel6_vanilla' (v0.1.0) for provider: vmware_desktop
    default: Downloading: http://vagrant.example.com/vagrant/boxes/vagrant_rhel6_vanilla_010.box
    default: Calculating and comparing box checksum...
==> default: Successfully added box 'vagrant_rhel6_vanilla' (v0.1.0) for 'vmware_desktop'!
==> default: Cloning VMware VM: 'vagrant_rhel6_vanilla'. This can take some time...
==> default: Checking if box 'vagrant_rhel6_vanilla' is up to date...
==> default: Verifying vmnet devices are healthy...
...

7) Verify cached Vagrant Box is the correct version

$ vagrant box list
vagrant_rhel6_vanilla      (vmware_desktop, 0.1.0)

At this point you have a Vagrant environment that supports versions and update status notifications, let’s create a new version of the Box and see it in action.

8) The Vagrant administrator creates an updated box
(e.g. vagrant_rhel6_vanilla_010.box for version 0.1.1) – you can use the procedure: Creating a custom Vagrant Box

9) The Vagrant administrator places the Vagrant Box in the web server

vagrant/
|-- boxes
|   |-- vagrant_rhel6_vanilla_010.box
|   `-- vagrant_rhel6_vanilla_011.box  <-- New Vagrant box
`-- vagrant_rhel6_vanilla.json

10) The Vagrant administrator adds the new Vagrant Box to the Box manifest Catalog

$ cat vagrant_rhel6_vanilla.json

{
  "name": "vagrant_rhel6_vanilla",
  "description": "This box contains RHEL 64-bit.",
  "versions": [
    {
      "version": "0.1.0",
      "providers": [
        {
          "name": "vmware_desktop",
          "url": "http://vagrant.example.com/vagrant/boxes/vagrant_rhel6_vanilla_010.box",
          "checksum_type": "sha1",
          "checksum": "b42efdb1e789976a45ffc26a321f00a42932f792"
        }
      ]
    },
    {
      "version": "0.1.1",
      "providers": [
        {
          "name": "vmware_desktop",
          "url": "http://vagrant.example.com/vagrant/boxes/vagrant_rhel6_vanilla_011.box",
          "checksum_type": "sha1",
          "checksum": "8ca8207afced5ca4ba387547fe9ef2b79a7d2b5b"
        }
      ]
    }
  ]
}

That’s it! from a Vagrant administrator perspective

11) End user/developer experience: User gets notified on the next Vagrant up and can easily update:

$ vagrant up
Bringing machine 'default' up with 'vmware_fusion' provider...
==> default: Checking if box 'vagrant_rhel6_vanilla' is up to date...
==> default: A newer version of the box 'vagrant_rhel6_vanilla' is available! You currently
==> default: have version '0.1.0'. The latest is version '0.1.1'. Run
==> default: `vagrant box update` to update.
==> default: Verifying vmnet devices are healthy...
...

Everytime there is an update the end user gets notified and he/she can upgrade by simply:
vagrant box update

$ vagrant box update
==> default: Checking for updates to 'vagrant_rhel6_vanilla'
    default: Latest installed version: 0.1.0
    default: Version constraints:
    default: Provider: vmware_desktop
==> default: Updating 'vagrant_rhel6_vanilla' with provider 'vmware_desktop' from version
==> default: '0.1.0' to '0.1.1'...
==> default: Loading metadata for box 'http://vagrant.example.com/vagrant/vagrant_rhel6_vanilla.json'
==> default: Adding box 'vagrant_rhel6_vanilla' (v0.1.1) for provider: vmware_desktop
    default: Downloading: http://vagrant.example.com/vagrant/boxes/vagrant_rhel6_vanilla_011.box
    default: Calculating and comparing box checksum...
==> default: Successfully added box 'vagrant_rhel6_vanilla' (v0.1.1) for 'vmware_desktop'!

To verify, check cached Vagrant Boxes:

$ vagrant box list
vagrant_rhel6_vanilla      (vmware_desktop, 0.1.0)
vagrant_rhel6_vanilla      (vmware_desktop, 0.1.1) <-- New Vagrant Box will be used

To delete older cached Vagrant Box no longer needed:

$ vagrant box remove vagrant_rhel6_vanilla --box-version '0.1.0'
Removing box 'vagrant_rhel6_vanilla' (v0.1.0) with provider 'vmware_desktop'...

This improves the user/developer experience because they just need to do ‘vagrant box update‘ anytime there is an updated box from the Vagrant administrator.

You can use the plethora of Vagrant Boxes available online, but there are times that you want/need to create your own Vagrant Box, based on specific requirements, with specific packages and most importantly with a specific ssh key pair.

In this guide I will walk through the process of creating a Vagrant BOX from scratch. I will be using a Mac with VMware Fusion and my Vagrant Box will be based on RHEL/CentOS.

1)Create a kickstart configuration file that you will use to automate the installation.
You can do it manually but I prefer/recommend using a kickstart file because you are software-defining the resulting machine and you can source control it. I placed the kickstart on a web server, in this case in my own Mac system so that it is reachable by the VM during installation.

Pre-requisites:
root password:
You can use the password ‘vagrant‘ as is on the kickstart, or you can provide the SHA512 hash of the root password you want. You can do this in several ways from a Linux terminal

mkpasswd -m sha-512
OR
python -c 'import crypt; print crypt.crypt("password", "$6$somesalt$")'

//You then place this hash in the kickstart below, for example:
// rootpw  --iscrypted $6$AcXXXXXX$ZfE9iCSvnezqs3/t47dcPRxozcWACkdITx38T.oTMESLzVv8yjSm1pVE4JC5FmdGojpg8StRhfi.peduXXXXX

SSH key pair:
You need to create a key pair that you will use for Vagrant. The way it works is that the generated public key will be added to the authorized keys allowed to ssh as the user ‘vagrant’ in the custom Vagrant BOX while the vagrant software on the host(Mac) will use the private key to connect from the host(Mac) to the Vagrant BOX(vm)
You can create the ssh key pair as follows:

ssh-keygen -t rsa -C "Vagrant" -f 'vagrant_rsa'

// You then place the contents of vagrant_rsa.pub on the kickstart below, replacing the XXXX.....'s

Once you have the password and SSH keys, you can use the below kickstart (replacing the password and SSH public key accordingly)

# ks.cfg

install
cdrom
text
lang en_US.UTF-8
keyboard us
network --onboot yes --device eth0 --bootproto dhcp --noipv6
rootpw  vagrant
firewall --disabled
authconfig --enableshadow --passalgo=sha512
selinux --disabled
timezone --utc America/New_York
bootloader --location=mbr --driveorder=sda --append="crashkernel=auto"
zerombr
clearpart --initlabel --drives=sda --all
part /boot --fstype=ext4 --size=500
part pv.100 --grow --size=1
volgroup vg0 --pesize=4096 pv.100
logvol swap --name=lv_swap --vgname=vg0 --size=2048
logvol / --fstype=ext4 --name=lv_root --vgname=vg0 --grow --size=1024
poweroff

## Install needed software (this is for ESX5/Fusion5 compatibility)
repo  --name=vmware-tools-collection   --baseurl=http://packages.vmware.com/tools/esx/5.0u3/rhel6/x86_64

%packages --nobase
@core
vmware-tools-hgfs
vmware-tools-esx-nox
openssh-clients
%end

%post
############################### BEGIN post-kickstart actions

# See: https://www.kernel.org/pub/linux/utils/kernel/hotplug/udev/udev.html
/bin/rm -f /etc/udev/rules.d/70-persistent-net.rules
/bin/ln -s /dev/null /etc/udev/rules.d/70-persistent-net.rules
/bin/rm -rf /dev/.udev/
/bin/rm /lib/udev/rules.d/75-persistent-net-generator.rules

# Clean hardcoded MAC address
/bin/sed -i -e 's/^HWADDR=.*$//' -e 's/^UUID=.*$//' /etc/sysconfig/network-scripts/ifcfg-eth0

# Make boot verbose
/bin/sed -i -e 's/^splash/#splash/' -e 's/^hidden/#hidden/' -e 's/rhgb//' -e 's/quiet//' /boot/grub/grub.conf

# Remove screensaver/blank screen 
/bin/sed -i -r s/"(^[\t]*kernel.*)"/"\1 consoleblank=0"/ /boot/grub/grub.conf 

### BEGIN: Vagrant specific pre-requisites ###
# Enable vmhgfs
/bin/echo "modprobe vmhgfs" > /etc/sysconfig/modules/vmhgfs.modules
/bin/chmod +x /etc/sysconfig/modules/vmhgfs.modules

# Create vagrant user
/usr/sbin/useradd vagrant
/bin/mkdir /home/vagrant/.ssh
/bin/chmod 700 /home/vagrant/.ssh
cat >           /home/vagrant/.ssh/authorized_keys <<'VAGRANT_RSA'
ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX vagrant
VAGRANT_RSA

/bin/chmod 600 /home/vagrant/.ssh/authorized_keys
/bin/chown -R vagrant /home/vagrant/.ssh

## Allow vagrant user to sudo
/bin/sed -i -e 's/Defaults[[:space:]]*requiretty/###Allow vagrant user to sudo\nvagrant ALL=(ALL:ALL) NOPASSWD:ALL\nDefaults requiretty\nDefaults:vagrant !requiretty/' /etc/sudoers 
### END: Vagrant specific pre-requisites ###

# Clean yum stuff to make resulting machine leaner
yum clean all

############################### END post-kickstart actions

%end

2) Creating the custom VM

Add new VM
vm_create_new

Continue without Disc

vm_continuewithout_cd

Create a custom virtual machine
vm_create_custom

Choose RHEL6_64
vm_os_choice

Click on Customize settings
vm_customize_settings

 

Select 1CPU and 1GB of RAM (The VM should be as minimal as possible, you can increase CPU and RAM from Vagrant)
cpu_ram

 

Disable unnecessary stuff (print sharing, bluetooth, sound card)
disable_bluetooth

disable_printsharing

disable_sound

 

3) Start installation

Boot VM and add the RHEL/CentOS ISO disk (Choose Disc or Disc Image...)
connect_cd

Point to the kickstart file served by the web server
rhel_bootscreen2box

Installation will proceed and the VM will be powered off (based on the kickstart 'poweroff' option)

4) Once VM is off Disable CD drive
disable_cd

5) Quit VMware Fusion (this is to clear the lock files so that you have a pristine VM)

 

6) Difragment and Shrink the VMDK

john@mac:~$ /Applications/VMware\ Fusion.app/Contents/Library/vmware-vdiskmanager -d ~/Documents/Virtual\ Machines.localized/vagrant_rhel6_64.vmwarevm/Virtual\ Disk.vmdk
Defragment: 100% done.

john@mac:~$ /Applications/VMware\ Fusion.app/Contents/Library/vmware-vdiskmanager -k ~/Documents/Virtual\ Machines.localized/vagrant_rhel6_64.vmwarevm/Virtual\ Disk.vmdk
Shrink: 100% done.
Shrink completed successfully.

 

7) On the folder where the virtual machine was created, create the following metadata.json file:

john@mac:~/Documents/Virtual Machines.localized/vagrant_rhel6_64.vmwarevm $ printf "{\n\t\"provider\":\"vmware_desktop\"\n}\n" > metadata.json
john@mac:~/Documents/Virtual Machines.localized/vagrant_rhel6_64.vmwarevm $ cat metadata.json
{
"provider":"vmware_desktop"
}

8) Create the Vagrant box container

john@mac:~/Documents/Virtual Machines.localized/vagrant_rhel6_64.vmwarevm $ tar cvzf vagrant-rhel6.box ./*
a ./Virtual Disk-s001.vmdk
a ./Virtual Disk-s002.vmdk
a ./Virtual Disk-s003.vmdk
a ./Virtual Disk-s004.vmdk
a ./Virtual Disk-s005.vmdk
a ./Virtual Disk-s006.vmdk
a ./Virtual Disk-s007.vmdk
a ./Virtual Disk-s008.vmdk
a ./Virtual Disk-s009.vmdk
a ./Virtual Disk-s010.vmdk
a ./Virtual Disk-s011.vmdk
a ./Virtual Disk.vmdk
a ./metadata.json
a ./vagrant_rhel6_64.nvram
a ./vagrant_rhel6_64.plist
a ./vagrant_rhel6_64.vmsd
a ./vagrant_rhel6_64.vmx
a ./vagrant_rhel6_64.vmxf
a ./vmware.log

9) Add the Box to Vagrant

john@mac:~$ vagrant box add -name vagrant-rhel6 ~/Documents/workspace/vagrant/boxes/vagrant-rhel6.box
==> box: Adding box 'vagrant-rhel6' (v0) for provider:
box: Downloading: file:///Users/john/Documents/workspace/vagrant/boxes/vagrant-rhel6.box
==> box: Successfully added box 'vagrant-rhel6' (v0) for 'vmware_desktop'!

Note: You can also add this Vagrant Box to a webserver so that it can be shared with others, in my case I uploaded the Box to a web server:

john@mac:~$ scp ~/Documents/workspace/vagrant/boxes/vagrant-rhel6.box john@webserver:/var/www/html/vagrant/

Update: You can create a Vagrant Box Catalog that will allow versioning of the Boxes and allows the end-users/developers to get automatic update notifications when you have an updated Vagrant box: Versioning and Update Notifications for your Vagrant boxes