In a previous post I wrote about how to Create a custom Vagrant Box from scratch.
In this post I will walk through the usage of Packer to automate the creation of a CentOS 7 image that can be used in Vagrant or even vSphere.
Packer is a cool Hashicorp tool that helps you automate the creation of machine images.
You can download the project/git repo at https://github.com/parcejohn/packer-centos7
From their website:
“Packer is easy to use and automates the creation of any type of machine image. It embraces modern configuration management by encouraging you to use automated scripts to install and configure the software within your Packer-made images. Packer brings machine images into the modern age, unlocking untapped potential and opening new opportunities.”
I use Packer to continue the journey to Infrastructure As Code, where even my golden images/templates are automated and source controlled (on Git).
Requirements:
* Packer // I installed on Mac using $ brew install packer
* vagrant // I installed on Mac using $ brew install vagrant
* vmware fusion // I installed using $ brew cask install vmware-fusion
* CentOS 7 ISO file
Packer uses a JSON template file to orchestrate the image creation and there are different stages in the process, I will concentrate on the below three, but you should familiarize with all of them (https://www.packer.io/docs/basics/terminology.html)
Builders: Packer component to create a machine image for a single platform, in this case I will be using the VMware builder.
Provisioners: Packer component that installs and configures software within a running machine prior to that machine being turned into a static image. Example provisioners include shell scripts, Chef, Puppet, etc. I will be using the shell provisioner.
Post-Processors: Packer component that takes the result of the builder or another post-processor and process that to create a new artifact. Examples of post-processors are compress to compress artifacts, upload to upload artifacts, etc. I will be creating a Vagrant box as the artifact, and also demonstrate how to upload the image to vSphere.
It’s time to walk through the process of creating a CentOS 7 image using Packer.
1) Create directory structure
The http folder will host a kickstart file, Packer will use its built in web server to serve this kickstart file
The scripts folder will host the provisioning scripts that will define the machine
centos/
├── http
└── scripts
2) Populate with the following configuration files and scripts (the contents of these files are shown in the next steps)
centos/
├── centos-7.1-x64-vmware.json
├── http
│ └── ks.cfg
├── scripts
│ ├── base.sh
│ ├── cleanup.sh
│ ├── hgfs.sh
│ ├── vmware.sh
│ └── zerodisk.sh
├── template.json
└── vagrant_rsa_key
3) Create a SSH key pair, you will need this to login to the server to complete install and configuration
The Public Key will be injected to the ‘vagrant’ user as part of the kickstart, it will then be used in the Packer JSON template to allow Packer to login to the machine
The Private Key will stay on your system to allow Packer and Vagrant to login to the created machine.
$ ssh-keygen -t rsa -b 4096 -C "vagrant" -N '' -q -f ./vagrant_rsa_key
4) Create a Packer JSON Template file (template.json)
{
"variables": {
"vm_name": "centos-7.1-vmware",
"iso_url": "{{env `ISO_URL`}}",
"iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284"
},
"builders": [
{
"headless": true,
"type": "vmware-iso",
"boot_command": [
" text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg"
],
"boot_wait": "10s",
"disk_size": 8192,
"guest_os_type": "centos-64",
"http_directory": "http",
"iso_url": "{{user `iso_url`}}",
"iso_checksum_type": "sha256",
"iso_checksum": "{{user `iso_sha256`}}",
"ssh_username": "vagrant",
"ssh_private_key_file": "vagrant_rsa",
"ssh_port": 22,
"ssh_wait_timeout": "10000s",
"shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
"tools_upload_flavor": "linux",
"tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",
"vmx_data": {
"memsize": "1024",
"numvcpus": "1",
"cpuid.coresPerSocket": "1"
}
}
],
"provisioners": [
{
"type": "shell",
"execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
"override": {
"vmware-iso": {
"scripts": [
"scripts/base.sh",
"scripts/vmware.sh",
"scripts/hgfs.sh",
"scripts/cleanup.sh",
"scripts/zerodisk.sh"
]
}
}
}
],
"post-processors": [
{
"type": "vagrant",
"override": {
"vmware": {
"output": "centos-7.1-x64-vmware.box"
}
}
}
]
}
Packer Components/Sections:
Variables:
This is just to centralize variables that will be used by other components (e.g. Builder, provisioner, etc)
I can also set a variable to use Environment variables or command line provided values:
User provided values:
“iso_url”: “{{user `iso_url`}}”,
Environment variable values:
“iso_url”: “{{env `ISO_URL`}}”,
Builders:
Here is an array of builders and the needed parameters for each builder, in this case the only builder is of type vmware-iso.
Parameters and some comments about them.
"headless": true, // Means not to open the VMware Fusion Console
"type": "vmware-iso", // VMware type builder
"boot_command": [
" text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg" // Send these key combination to the VMWare Fusion Console
],
"boot_wait": "10s",
"disk_size": 8192,
"guest_os_type": "centos-64",
"http_directory": "http", // Directory where the kickstart file is placed
"iso_url": "{{user `iso_url`}}", // Location of ISO image, defined at runtime, more on that later
"iso_checksum_type": "sha256",
"iso_checksum": "{{user `iso_sha256`}}",
"ssh_username": "vagrant", // Packer will log in to resulting machine for provisioning, this user must exist (user created from the Kickstart)
"ssh_private_key_file": "vagrant_rsa", // Packer will log in using ssh key created earlier
"ssh_port": 22,
"ssh_wait_timeout": "10000s",
"shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
"tools_upload_flavor": "linux",
"tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",
"vmx_data": {
"memsize": "1024",
"numvcpus": "1",
"cpuid.coresPerSocket": "1"
Provisioners:
This section uses the shell provisioner and runs all of those scripts listed, which are located in the scripts folder.
"type": "shell",
"execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
"override": {
"vmware-iso": {
"scripts": [
"scripts/base.sh",
"scripts/vmware.sh",
"scripts/hgfs.sh",
"scripts/cleanup.sh",
"scripts/zerodisk.sh"
Post-processor:
This section states that I want a vagrant box and I am giving it the name.
"type": "vagrant",
"override": {
"vmware": {
"output": "centos-7.1-x64-vmware.box"
5) Kickstart file (should be http/ks.cfg based on the template.json file)
This is a minimal install of CentOS 7
Note: The private key (XXXX) given to the vagrant user, that is the key that we created in a previous step
install cdrom lang en_US.UTF-8 keyboard us network --onboot yes --device eth0 --bootproto dhcp --noipv6 rootpw --plaintext vagrant firewall --enabled --service=ssh authconfig --enableshadow --passalgo=sha512 selinux --disabled timezone --utc America/New_York bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet" text skipx zerombr clearpart --all --initlabel autopart auth --useshadow --enablemd5 firstboot --disabled reboot %packages --nobase --ignoremissing @core bzip2 kernel-devel kernel-headers -ipw2100-firmware -ipw2200-firmware -ivtv-firmware %end %post # Install SUDO /usr/bin/yum -y install sudo # Create vagrant user /usr/sbin/useradd vagrant /bin/mkdir /home/vagrant/.ssh /bin/chmod 700 /home/vagrant/.ssh cat > /home/vagrant/.ssh/authorized_keys <<'VAGRANT_RSA' ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX vagrant VAGRANT_RSA /bin/chmod 600 /home/vagrant/.ssh/authorized_keys /bin/chown -R vagrant /home/vagrant/.ssh # Add vagrant user to SUDO echo "vagrant ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/vagrant echo "Defaults:vagrant !requiretty" >> /etc/sudoers.d/vagrant chmod 0440 /etc/sudoers.d/vagrant %end
6 ) Provisioning Scrips
Packer calls these scripts (from the JSON template) on the created machine before making it a template/image.
base.sh // Install basic packages
#!/usr/bin/env bash set -x sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers yum -y install gcc make gcc-c++ kernel-devel-`uname -r` perl
vmware.sh // Install VMTools, I separated this from base as I may want to call different scripts on a VirtualBox/etc builder.
#!/usr/bin/env bash set -x yum install -y fuse-libs open-vm-tools
hgfs.sh // For vagrant to work properly with folders
#!/usr/bin/env bash set -x VMWARE_ISO=/tmp/vmware_tools_linux.iso VMWARE_MNTDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_mnt_XXXXXX) VMWARE_TMPDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_XXXXXX) # Extract tools mount -o loop $VMWARE_ISO $VMWARE_MNTDIR tar zxf $VMWARE_MNTDIR/VMwareTools*.tar.gz -C $VMWARE_TMPDIR umount $VMWARE_MNTDIR # Install tools $VMWARE_TMPDIR/vmware-tools-distrib/vmware-install.pl -d # Clean up rm -f $VMWARE_ISO rm -rf $VMWARE_MNTDIR rm -rf $VMWARE_TMPDIR
cleanup.sh // Clean up before converting to image
#!/usr/bin/env bash set -x yum -y erase gtk2 libX11 hicolor-icon-theme avahi freetype bitstream-vera-fonts rpm --rebuilddb yum -y clean all
zerodisk.sh // This is so that the resulting box is as small as possible
#!/usr/bin/env bash set -x dd if=/dev/zero of=/EMPTY bs=1M rm -f /EMPTY
7) Run Packer against the template.json file created earlier and watch it build the machine image
$ packer build -var ‘iso_url=/Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso’ -only=vmware-iso template.json
vmware-iso output will be in this color.==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: file:///Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso
==> vmware-iso: Creating virtual machine disk
==> vmware-iso: Building and writing VMX file
==> vmware-iso: Starting HTTP server on port 8101
==> vmware-iso: Starting virtual machine…
vmware-iso: The VM will be run headless, without a GUI. If you want to
vmware-iso: view the screen of the VM, connect via VNC without a password to
vmware-iso: 127.0.0.1:5989
==> vmware-iso: Waiting 10s for boot…
==> vmware-iso: Connecting to VM via VNC
==> vmware-iso: Typing the boot command over VNC…
==> vmware-iso: Waiting for SSH to become available…
==> vmware-iso: Connected to SSH!
==> vmware-iso: Uploading the ‘linux’ VMware Tools
==> vmware-iso: Provisioning with shell script: scripts/base.sh
vmware-iso: + sed -i ‘s/^.*requiretty/#Defaults requiretty/’ /etc/sudoers
…
…
…
==> vmware-iso: Provisioning with shell script: scripts/zerodisk.sh
vmware-iso: + dd if=/dev/zero of=/EMPTY bs=1M
vmware-iso: dd: error writing ‘/EMPTY’: No space left on device
vmware-iso: 5488+0 records in
vmware-iso: 5487+0 records out
vmware-iso: 5754265600 bytes (5.8 GB) copied, 5.27091 s, 1.1 GB/s
vmware-iso: + rm -f /EMPTY
==> vmware-iso: Gracefully halting virtual machine…
vmware-iso: Waiting for VMware to clean up after itself…
==> vmware-iso: Deleting unnecessary VMware files…
vmware-iso: Deleting: output-vmware-iso/564de113-2fc5-2010-8e5d-63fec92f12f7.vmem
vmware-iso: Deleting: output-vmware-iso/packer-vmware-iso.plist
vmware-iso: Deleting: output-vmware-iso/vmware.log
==> vmware-iso: Cleaning VMX prior to finishing up…
vmware-iso: Unmounting floppy from VMX…
vmware-iso: Detaching ISO from CD-ROM device…
vmware-iso: Disabling VNC server…
==> vmware-iso: Compacting the disk image
==> vmware-iso: Running post-processor: vagrant
==> vmware-iso (vagrant): Creating Vagrant box for ‘vmware’ provider
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s001.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s002.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s003.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.nvram
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmsd
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmxf
vmware-iso (vagrant): Compressing: Vagrantfile
vmware-iso (vagrant): Compressing: disk-s001.vmdk
vmware-iso (vagrant): Compressing: disk-s002.vmdk
vmware-iso (vagrant): Compressing: disk-s003.vmdk
vmware-iso (vagrant): Compressing: disk.vmdk
vmware-iso (vagrant): Compressing: metadata.json
vmware-iso (vagrant): Compressing: packer-vmware-iso.nvram
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmsd
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmx
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmxf
Build ‘vmware-iso’ finished.==> Builds finished. The artifacts of successful builds are:
–> vmware-iso: ‘vmware’ provider box: centos-7.1-x64-vmware.box
8) Add box to Vagrant
$ vagrant box add –name centos-7.1-010-x64-vmware.box centos-7.1-x64-vmware.box
==> box: Adding box ‘centos-7.1-010-x64-vmware.box’ (v0) for provider:
box: Downloading: file:///Users/john/packer/centos/centos-7.1-x64-vmware.box
==> box: Successfully added box ‘centos-7.1-010-x64-vmware.box’ (v0) for ‘vmware_desktop’!
The above is all you need to have your Infrastructure be Code that can be versioned and automatically create your images and use them on Vagrant.
The next item will show you how to use Packer to move the same image to your vSphere environment.
9) Sending Image to vSphere
To send the image created by Packer to vSphere you will need to add another post-provisioning entry to the JSON array:
{
"type": "vsphere",
"host": "{{user `vm_host`}}",
"username": "{{user `vm_user`}}",
"password": "{{user `vm_pass`}}",
"datacenter": "{{user `vm_dc`}}",
"cluster": "{{user `vm_cluster`}}",
"resource_pool": " ",
"datastore": "{{user `vm_datastore`}}",
"vm_folder": "{{user `vm_folder`}}",
"vm_name": "{{user `vm_name`}}",
"vm_network": "{{user `vm_network`}}",
"insecure" : "true"
}
As you can see I am using variables to make this portable, so that means you have to add variables to the variables section as well, below is the new template to do both vagrant and vSphere image provisioning.
{
"variables": {
"vm_name": "centos-7.1-vmware",
"iso_url": "{{env `ISO_URL`}}",
"iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284",
"vm_host": "{{ user `vm_host` }}",
"vm_user": "{{ user `vm_user` }}",
"vm_pass": "{{ env `vm_pass` }}",
"vm_dc": "{{ user `vm_dc` }}",
"vm_cluster": "{{user `vm_cluster`}}",
"vm_datastore": "{{user `vm_datastore`}}",
"vm_folder": "{{user `vm_folder`}}",
"vm_name": "{{user `vm_name`}}",
"vm_network": "{{user `vm_network`}}"
},
"builders": [
{
"headless": true,
"type": "vmware-iso",
"boot_command": [
" text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg"
],
"boot_wait": "10s",
"disk_size": 8192,
"guest_os_type": "centos-64",
"http_directory": "http",
"iso_url": "{{user `iso_url`}}",
"iso_checksum_type": "sha256",
"iso_checksum": "{{user `iso_sha256`}}",
"ssh_username": "vagrant",
"ssh_private_key_file": "vagrant_rsa",
"ssh_port": 22,
"ssh_wait_timeout": "10000s",
"shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
"tools_upload_flavor": "linux",
"tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",
"vmx_data": {
"memsize": "1024",
"numvcpus": "1",
"cpuid.coresPerSocket": "1"
}
}
],
"provisioners": [
{
"type": "shell",
"execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
"override": {
"vmware-iso": {
"scripts": [
"scripts/base.sh",
"scripts/vmware.sh",
"scripts/hgfs.sh",
"scripts/cleanup.sh",
"scripts/zerodisk.sh"
]
}
}
}
],
"post-processors": [
{
"type": "vagrant",
"override": {
"vmware": {
"output": "centos-7.1-x64-vmware.box"
}
}
},
{
"type": "vsphere",
"host": "{{user `vm_host`}}",
"username": "{{user `vm_user`}}",
"password": "{{user `vm_pass`}}",
"datacenter": "{{user `vm_dc`}}",
"cluster": "{{user `vm_cluster`}}",
"resource_pool": " ",
"datastore": "{{user `vm_datastore`}}",
"vm_folder": "{{user `vm_folder`}}",
"vm_name": "{{user `vm_name`}}",
"vm_network": "{{user `vm_network`}}",
"insecure" : "true"
}
]
}
10) Re-Run Packer against the template.json file and watch it build the machine image an
$ packer build \
> -var ‘iso_url=/Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso’ \
> -var ‘vm_host=vc.example.com’ \
> -var ‘vm_user=john@example.com’ \
> -var ‘vm_pass=XXXXXXX’ \
> -var ‘vm_dc=vDC’ \
> -var ‘vm_cluster=Folder/Cluster’ \
> -var ‘vm_datastore=store’ \
> -var ‘vm_folder=Images’ \
> -var ‘vm_name=centos71’ \
> -var ‘vm_network=dvs-net1’ \
> -only=vmware-iso template.json
vmware-iso output will be in this color.==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: file:///Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso
==> vmware-iso: Creating virtual machine disk
…
…
==> vmware-iso: Running post-processor: vagrant
==> vmware-iso (vagrant): Creating Vagrant box for ‘vmware’ provider
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s001.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s002.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s003.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.nvram
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmsd
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmxf
vmware-iso (vagrant): Compressing: Vagrantfile
vmware-iso (vagrant): Compressing: disk-s001.vmdk
vmware-iso (vagrant): Compressing: disk-s002.vmdk
vmware-iso (vagrant): Compressing: disk-s003.vmdk
vmware-iso (vagrant): Compressing: disk.vmdk
vmware-iso (vagrant): Compressing: metadata.json
vmware-iso (vagrant): Compressing: packer-vmware-iso.nvram
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmsd
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmx
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmxf
==> vmware-iso: Running post-processor: vsphere
vmware-iso (vsphere): Uploading output-vmware-iso/packer-vmware-iso.vmx to vSphere
vmware-iso (vsphere): Opening VMX source: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vsphere): Opening VI target: vi://john%40example.com@vc.exmaple.com:443/vDC/host/Cluster/Resources/
vmware-iso (vsphere): Deploying to VI: vi://john%40example.com@vc.example.com:443/vDC/host/Cluster/Resources/
Transfer Completed
vmware-iso (vsphere): Completed successfully
vmware-iso (vsphere):
Build ‘vmware-iso’ finished.==> Builds finished. The artifacts of successful builds are:
–> vmware-iso: ‘vmware’ provider box: centos-7.1-x64-vmware.box
–> vmware-iso:
Now you should have the same image running on both Vagrant and vSphere.
