Technologist

Tech stuff about Cloud, DevOps, SysAdmin, Virtualization, SAN, Hardware, Scripting, Automation and Development

Browsing Posts in Puppet

In this guide I go over how to use Vagrant with AWS and in the process have an automated way to install Puppet Enterprise.
I am separating data and code by having a generic Vagrantfile with the code and have a servers.yaml file with all the data that will change from user to user.
For installing the Puppet Enterprise server I am including the automated provisioning script I am using with Vagrant and using AWS Tags to set the hostname of the launched server.

Pre-requisites:

  • Vagrant
  • vagrant-aws plug-in:
  • AWS pre-requisites:
  • While you can add your AWS credentials to the Vagrantfile, it is not recommended. A better way is to have the AWS CLI tools installed and configured

    TL;DR To get started right away you can download the project from github vagrant-aws-puppetserver, otherwise follow the guide below.

    Create Vagrantfile

    The below Vagrantfile utilizes a yaml file (servers.yaml) to provide the data, it allows you to control data using the yaml file and not have to modify the Vagrantfile code – separating code and data.

    // Vagrantfile

    Create servers.yaml

    This file contains the information that will be used by the Vagrantfile, this includes
    AWS region: Which region will this EC2 server run
    AWS keypair: Key used to connect to your launched EC2 instance
    AWS subnet id: Where will this EC2 instance sit in the AWS network
    AWS associate public ip: Do you need a public IP? true or false
    AWS security group: What AWS security group should be associated, should allow Puppetserver needed ports and whatever else you need (ssh, etc)
    AWS ami: Which AMI will you be using I am using a CentOS7
    AWS instance type: Puppetserver needs enough CPU/RAM, during my testing m3.xlarge was appropriate
    AWS SSH username: The EC2 instance user (depends on which AMI you choose), the CentOS AMI expects ec2-user
    AWS SSH private key path: The local path to the SSH key pair
    AWS User Data: I am adding user data which will execute a bash script that allows Vagrant to interact with the launched EC2 instance
    AWS Tags: This is not required for Vagrant and AWS/EC2, but in my provision script I am using the AWS Name Tag to be the system’s hostname, the other 2 tags are there for demonstration purposes
    provision: This is a provisioning script that will be run on the EC2 instance – this is the script that install the Puppet Enterprise server
    AWS IAM Role: You don’t need to add a role when working with Vagrant and AWS/EC2, but I am using a specific IAM role to allow the launched EC2 instance to be able to get information about its AWS Tags, so it is important that you provide it with a Role that allows DescribeTags, see below IAM policy:

    Now use your data in the servers.yaml file
    // servers.yaml

    At this point you can spin up EC2 instances using the above Vagrantfile and servers.yaml file. If you add provision: install_puppetserver.sh to the servers.yaml file as I did and add the below script you will have a Puppet Enterprise server ready to go.

    // install_puppetserver.sh

In this post I am showing how to use generate a MySQL 5 password-hash that can be used to create MySQL GRANTS using a hash instead of a password.

To use a password-hash to create GRANTs:

A good use case is the Puppet puppetlabs-mysql module to automate the MySQL environment, You can automate/define USER and GRANT creation by using the code below, but notice that it requires a password-hash instead of a password:

OR:

They recommend using mysql_password() for creating such a hash. But that means you need to have a MySQL server available.
In this post I am writing about getting those hashes using Python, I wrote a program/script to get the password-hash programatically.

The Python program/script can be found at:
https://github.com/parcejohn/mysql_password_hash

Usage

# Using Command line arguments – User provided password (e.g. ‘secret’)

# Using Command line arguments – Random password with length=20 (default length=12)

# interactive mode (no arguments)

You can use the plethora of Vagrant Boxes available online, but there are times that you want/need to create your own Vagrant Box, based on specific requirements, with specific packages and most importantly with a specific ssh key pair.

In this guide I will walk through the process of creating a Vagrant BOX from scratch. I will be using a Mac with VMware Fusion and my Vagrant Box will be based on RHEL/CentOS.

1)Create a kickstart configuration file that you will use to automate the installation.
You can do it manually but I prefer/recommend using a kickstart file because you are software-defining the resulting machine and you can source control it. I placed the kickstart on a web server, in this case in my own Mac system so that it is reachable by the VM during installation.

Pre-requisites:
root password:
You can use the password ‘vagrant‘ as is on the kickstart, or you can provide the SHA512 hash of the root password you want. You can do this in several ways from a Linux terminal

SSH key pair:
You need to create a key pair that you will use for Vagrant. The way it works is that the generated public key will be added to the authorized keys allowed to ssh as the user ‘vagrant’ in the custom Vagrant BOX while the vagrant software on the host(Mac) will use the private key to connect from the host(Mac) to the Vagrant BOX(vm)
You can create the ssh key pair as follows:

Once you have the password and SSH keys, you can use the below kickstart (replacing the password and SSH public key accordingly)

2) Creating the custom VM

Add new VM
vm_create_new

Continue without Disc

vm_continuewithout_cd

Create a custom virtual machine
vm_create_custom

Choose RHEL6_64
vm_os_choice

Click on Customize settings
vm_customize_settings

 

Select 1CPU and 1GB of RAM (The VM should be as minimal as possible, you can increase CPU and RAM from Vagrant)
cpu_ram

 

Disable unnecessary stuff (print sharing, bluetooth, sound card)
disable_bluetooth

disable_printsharing

disable_sound

 

3) Start installation

Boot VM and add the RHEL/CentOS ISO disk (Choose Disc or Disc Image…)
connect_cd

Point to the kickstart file served by the web server
rhel_bootscreen2box

Installation will proceed and the VM will be powered off (based on the kickstart ‘poweroff’ option)

4) Once VM is off Disable CD drive
disable_cd

5) Quit VMware Fusion (this is to clear the lock files so that you have a pristine VM)

 

6) Difragment and Shrink the VMDK

 

7) On the folder where the virtual machine was created, create the following metadata.json file:

8) Create the Vagrant box container

9) Add the Box to Vagrant

Note: You can also add this Vagrant Box to a webserver so that it can be shared with others, in my case I uploaded the Box to a web server:

Update: You can create a Vagrant Box Catalog that will allow versioning of the Boxes and allows the end-users/developers to get automatic update notifications when you have an updated Vagrant box: Versioning and Update Notifications for your Vagrant boxes