John | Technologist

Browsing Posts published by John

Installing Puppet Enterprise on AWS EC2 using Vagrant

February 6, 2017 John No comments

In this guide I go over how to use Vagrant with AWS and in the process have an automated way to install Puppet Enterprise.
I am separating data and code by having a generic Vagrantfile with the code and have a servers.yaml file with all the data that will change from user to user.
For installing the Puppet Enterprise server I am including the automated provisioning script I am using with Vagrant and using AWS Tags to set the hostname of the launched server.

Pre-requisites:

Vagrant
vagrant-aws plug-in:

$ vagrant plugin install vagrant-aws

1	$ vagrant plugin install vagrant-aws

AWS pre-requisites:

While you can add your AWS credentials to the Vagrantfile, it is not recommended. A better way is to have the AWS CLI tools installed and configured

$ aws configure
AWS Access Key ID [****************XYYY]: XXXXXXXXYYY
AWS Secret Access Key [****************ZOOO]:ZZZZZZZOOO
Default region name [us-east-1]:us-east-1
Default output format [None]: json

$ aws configure

AWS Access Key ID [****************XYYY]: XXXXXXXXYYY

AWS Secret Access Key [****************ZOOO]:ZZZZZZZOOO

Default region name [us-east-1]:us-east-1

Default output format [None]: json

TL;DR To get started right away you can download the project from github vagrant-aws-puppetserver, otherwise follow the guide below.

Create Vagrantfile

The below Vagrantfile utilizes a yaml file (servers.yaml) to provide the data, it allows you to control data using the yaml file and not have to modify the Vagrantfile code – separating code and data.

// Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Specify minimum Vagrant version and Vagrant API version
Vagrant.require_version ">= 1.6.0"
VAGRANTFILE_API_VERSION = "2"

# Require YAML module
require 'yaml'

# Read YAML file with box details
servers = YAML.load_file('servers.yaml')

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

    # Iterate through entries in YAML file
    servers.each do |server|
    
        config.vm.define server['name'] do |srv|

            srv.vm.box = server['box']
            srv.vm.box_url = server['box_url']

            srv.vm.provider :aws do |aws, override|

                ### Dont do these here, better to use awscli with a profile
                #aws.access_key_id = "YOUR KEY"
                #aws.secret_access_key = "YOUR SECRET KEY"
                #aws.session_token = "SESSION TOKEN"

                aws.region = server["aws_region"] if server["aws_region"]
                aws.keypair_name = server["aws_keypair_name"] if server["aws_keypair_name"]
                aws.subnet_id = server["aws_subnet_id"] if server["aws_subnet_id"]
                aws.associate_public_ip = server["aws_associate_public_ip"] if server["aws_associate_public_ip"]
                aws.security_groups = server["aws_security_groups"] if server["aws_security_groups"]
                aws.iam_instance_profile_name = server['aws_iam_role'] if server['aws_iam_role']
                aws.ami = server["aws_ami"] if server["aws_ami"]
                aws.instance_type = server["aws_instance_type"] if server["aws_instance_type"]
                aws.tags = server["aws_tags"] if server["aws_tags"]
                aws.user_data = server["aws_user_data"] if server["aws_user_data"]

                override.ssh.username = server["aws_ssh_username"] if server["aws_ssh_username"]
                override.ssh.private_key_path = server["aws_ssh_private_key_path"] if server["aws_ssh_private_key_path"]           

                config.vm.synced_folder ".", "/vagrant", type: "rsync"

                config.vm.provision :shell, path: server["provision"] if server["provision"]

            end  
        end
    end
end

# -*- mode: ruby -*-

# vi: set ft=ruby :

# Specify minimum Vagrant version and Vagrant API version

Vagrant.require_version ">= 1.6.0"

VAGRANTFILE_API_VERSION = "2"

# Require YAML module

require 'yaml'

# Read YAML file with box details

servers = YAML.load_file('servers.yaml')

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

# Iterate through entries in YAML file

servers.each do |server|

config.vm.define server['name'] do |srv|

srv.vm.box = server['box']

srv.vm.box_url = server['box_url']

srv.vm.provider :aws do |aws, override|

### Dont do these here, better to use awscli with a profile

#aws.access_key_id = "YOUR KEY"

#aws.secret_access_key = "YOUR SECRET KEY"

#aws.session_token = "SESSION TOKEN"

aws.region = server["aws_region"] if server["aws_region"]

aws.keypair_name = server["aws_keypair_name"] if server["aws_keypair_name"]

aws.subnet_id = server["aws_subnet_id"] if server["aws_subnet_id"]

aws.associate_public_ip = server["aws_associate_public_ip"] if server["aws_associate_public_ip"]

aws.security_groups = server["aws_security_groups"] if server["aws_security_groups"]

aws.iam_instance_profile_name = server['aws_iam_role'] if server['aws_iam_role']

aws.ami = server["aws_ami"] if server["aws_ami"]

aws.instance_type = server["aws_instance_type"] if server["aws_instance_type"]

aws.tags = server["aws_tags"] if server["aws_tags"]

aws.user_data = server["aws_user_data"] if server["aws_user_data"]

override.ssh.username = server["aws_ssh_username"] if server["aws_ssh_username"]

override.ssh.private_key_path = server["aws_ssh_private_key_path"] if server["aws_ssh_private_key_path"]

config.vm.synced_folder ".", "/vagrant", type: "rsync"

config.vm.provision :shell, path: server["provision"] if server["provision"]

end

Create servers.yaml

This file contains the information that will be used by the Vagrantfile, this includes
AWS region: Which region will this EC2 server run
AWS keypair: Key used to connect to your launched EC2 instance
AWS subnet id: Where will this EC2 instance sit in the AWS network
AWS associate public ip: Do you need a public IP? true or false
AWS security group: What AWS security group should be associated, should allow Puppetserver needed ports and whatever else you need (ssh, etc)
AWS ami: Which AMI will you be using I am using a CentOS7
AWS instance type: Puppetserver needs enough CPU/RAM, during my testing m3.xlarge was appropriate
AWS SSH username: The EC2 instance user (depends on which AMI you choose), the CentOS AMI expects ec2-user
AWS SSH private key path: The local path to the SSH key pair
AWS User Data: I am adding user data which will execute a bash script that allows Vagrant to interact with the launched EC2 instance
AWS Tags: This is not required for Vagrant and AWS/EC2, but in my provision script I am using the AWS Name Tag to be the system’s hostname, the other 2 tags are there for demonstration purposes
provision: This is a provisioning script that will be run on the EC2 instance – this is the script that install the Puppet Enterprise server
AWS IAM Role: You don’t need to add a role when working with Vagrant and AWS/EC2, but I am using a specific IAM role to allow the launched EC2 instance to be able to get information about its AWS Tags, so it is important that you provide it with a Role that allows DescribeTags, see below IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "XXXXXXX",
            "Action": [
                "ec2:DescribeTags"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "XXXXXXX",

"Action": [

"ec2:DescribeTags"

"Effect": "Allow",

"Resource": "*"

}

]

}

Now use your data in the servers.yaml file
// servers.yaml

---
- name: puppet4
  box: dummy
  box_url: https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
  aws_region: "us-east-1"
  aws_keypair_name: "john-key"
  aws_subnet_id: "subnet-0001" 
  aws_associate_public_ip: false
  aws_security_groups: ['sg-0001'] 
  aws_ami: "ami-0001"
  aws_instance_type: m3.xlarge
  aws_ssh_username: "ec2-user"
  aws_iam_role: "iam-able-to-describe-tags"
  aws_ssh_private_key_path: "/Users/john/.ssh/john-key.pem"
  aws_user_data: "#!/bin/bash\nsed -i -e 's/^Defaults.*requiretty/# Defaults requiretty/g' /etc/sudoers"
  aws_tags: 
    Name: 'puppet4'
    tier: 'stage'
    application: 'puppetserver'
  update: false
  provision: install_puppetserver.sh

---

- name: puppet4

box: dummy

box_url: https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box

aws_region: "us-east-1"

aws_keypair_name: "john-key"

aws_subnet_id: "subnet-0001"

aws_associate_public_ip: false

aws_security_groups: ['sg-0001']

aws_ami: "ami-0001"

aws_instance_type: m3.xlarge

aws_ssh_username: "ec2-user"

aws_iam_role: "iam-able-to-describe-tags"

aws_ssh_private_key_path: "/Users/john/.ssh/john-key.pem"

aws_user_data: "#!/bin/bash\nsed -i -e 's/^Defaults.*requiretty/# Defaults requiretty/g' /etc/sudoers"

aws_tags:

Name: 'puppet4'

tier: 'stage'

application: 'puppetserver'

update: false

provision: install_puppetserver.sh

At this point you can spin up EC2 instances using the above Vagrantfile and servers.yaml file. If you add provision: install_puppetserver.sh to the servers.yaml file as I did and add the below script you will have a Puppet Enterprise server ready to go.

// install_puppetserver.sh

#!/bin/bash -xe

# Ensure pip is installed
if ! which pip; then
	curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
	python get-pip.py
fi

# Upgrade pip 
/bin/pip install --upgrade pip

# Upgrade awscli tools (They are installed in the AMI)
/bin/pip install --upgrade awscli

# Get hostname from AWS Name Tag (requires the EC2 instance to have an IAM role that allows DescribeTags)
AWS_INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | grep region | cut -d\" -f4)
AWSHOSTNAME=$(aws ec2 describe-tags --region ${AWS_REGION} --filters "Name=resource-id,Values=${AWS_INSTANCE_ID}" --query "Tags[?Key=='Name'].Value[] | [0]" | cut -d\" -f2)

# Set hostname (use the AWS Name Tag)
hostnamectl set-hostname ${AWSHOSTNAME}.cpg.org

# Update system and install wget, git
yum update -y
yum install wget git -y

# Set puppet.cpg.org as hostname in hosts file
echo "$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) puppet puppet.cpg.org" >> /etc/hosts

# Download puppet 
wget "https://pm.puppetlabs.com/cgi-bin/download.cgi?dist=el&rel=7&arch=x86_64&ver=2016.4.0" -O puppet.2016.4.0.tar.gz
tar -xvzf puppet.2016.4.0.tar.gz
cd puppet-enterprise*

# Create pe.conf file
touch pe.conf
echo '{' >> pe.conf
echo '"console_admin_password": "puppet"' >> pe.conf
echo '"puppet_enterprise::puppet_master_host": "%{::trusted.certname}"' >> pe.conf
echo '}'  >> pe.conf

echo "Install Puppetserver"
./puppet-enterprise-installer -c pe.conf

echo "Adding * to autosign.conf"
cat >> /etc/puppetlabs/puppet/autosign.conf <<'AUTOSIGN'
*
AUTOSIGN

# Run puppet agent
/usr/local/bin/puppet agent -t

#!/bin/bash -xe

# Ensure pip is installed

if ! which pip; then

curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"

python get-pip.py

# Upgrade pip

/bin/pip install --upgrade pip

# Upgrade awscli tools (They are installed in the AMI)

/bin/pip install --upgrade awscli

# Get hostname from AWS Name Tag (requires the EC2 instance to have an IAM role that allows DescribeTags)

AWS_INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | grep region | cut -d\" -f4)

AWSHOSTNAME=$(aws ec2 describe-tags --region ${AWS_REGION} --filters "Name=resource-id,Values=${AWS_INSTANCE_ID}" --query "Tags[?Key=='Name'].Value[] | [0]" | cut -d\" -f2)

# Set hostname (use the AWS Name Tag)

hostnamectl set-hostname ${AWSHOSTNAME}.cpg.org

# Update system and install wget, git

yum update -y

yum install wget git -y

# Set puppet.cpg.org as hostname in hosts file

echo "$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4) puppet puppet.cpg.org" >> /etc/hosts

# Download puppet

wget "https://pm.puppetlabs.com/cgi-bin/download.cgi?dist=el&rel=7&arch=x86_64&ver=2016.4.0" -O puppet.2016.4.0.tar.gz

tar -xvzf puppet.2016.4.0.tar.gz

cd puppet-enterprise*

# Create pe.conf file

touch pe.conf

echo '{' >> pe.conf

echo '"console_admin_password": "puppet"' >> pe.conf

echo '"puppet_enterprise::puppet_master_host": "%{::trusted.certname}"' >> pe.conf

echo '}' >> pe.conf

echo "Install Puppetserver"

./puppet-enterprise-installer -c pe.conf

echo "Adding * to autosign.conf"

cat >> /etc/puppetlabs/puppet/autosign.conf <<'AUTOSIGN'

AUTOSIGN

# Run puppet agent

/usr/local/bin/puppet agent -t

aws, Cloud, DevOps, Puppet, Scripting, Vagrant aws, devops, puppet, puppetserver, vagrant

Docker Registry with Authentication and SSL/TLS

September 25, 2016 John 2 comments

In this post I will go over the installation of Docker Registry with Basic Authentication and over SSL/TLS (self-signed cert for demonstration purposes). I will also do this on Docker for Mac, which has some interesting things to note.

Let me start by saying that you can use the Docker Hub which is a freely available Hosted registry, but there may be instances where you want to host your own registry to control where your images are being stored, you can also opt to use the commercial Docker Trusted Registry, but I will go over the free open-source Registry.

Create project structure
Password file to support Basic Auth
Certificates to support TLS
Location to store the images
Create Docker Registry container
Using the Registry
Registry API

Create project structure

mac$ mkdir -p registry/{volumes/{auth,certs,data},docker-registry}
mac$ cd registry
mac$ tree
.
└── registry
    ├── docker-registry // WhereI will store the docker-compose.yml file
    └── volumes 
        ├── auth  // Volume for Basic Auth password file 
        ├── certs // Volume for TLS certs
        └── data  // Volumes for images storage

mac$ mkdir -p registry/{volumes/{auth,certs,data},docker-registry}

mac$ cd registry

mac$ tree

└── registry

├── docker-registry // WhereI will store the docker-compose.yml file

└── volumes

├── auth // Volume for Basic Auth password file

├── certs // Volume for TLS certs

└── data // Volumes for images storage

Password file to support Basic Auth

Let’s create a username and password

mac$ htpasswd -Bbn <USER> <PASSWORD> > volumes/auth/htpasswd

mac$ cat volumes/auth/htpasswd
user1:$2y$05$xOuwA8oaflSZ5wpiJto2/OBOIMTSpZrH69wsaQOsnaKt07Vzu1sBW

mac$ htpasswd -Bbn <USER> <PASSWORD> > volumes/auth/htpasswd

mac$ cat volumes/auth/htpasswd

user1:$2y$05$xOuwA8oaflSZ5wpiJto2/OBOIMTSpZrH69wsaQOsnaKt07Vzu1sBW

Certificates to support TLS

Create a root CA key, which will sign the actual certificates

mac$ openssl genrsa -out volumes/certs/rootCA.key 2048

1	mac$ openssl genrsa -out volumes/certs/rootCA.key 2048

Create root CA certificate which will need to be installed on the systems that will use the Registry

mac$ openssl req -x509 -new -nodes -key volumes/certs/rootCA.key -days 365 -out volumes/certs/rootCA.crt

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:NY
Locality Name (eg, city) []:NY
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Technologist
Organizational Unit Name (eg, section) []:Tech
Common Name (e.g. server FQDN or YOUR name) []:Technologist CA
Email Address []:johnATtechnologist

mac$ openssl req -x509 -new -nodes -key volumes/certs/rootCA.key -days 365 -out volumes/certs/rootCA.crt

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]:US

State or Province Name (full name) [Some-State]:NY

Locality Name (eg, city) []:NY

Organization Name (eg, company) [Internet Widgits Pty Ltd]:Technologist

Organizational Unit Name (eg, section) []:Tech

Common Name (e.g. server FQDN or YOUR name) []:Technologist CA

Email Address []:johnATtechnologist

Now that you have a CA, you can sign certificates.

Create a key for the Registry system

mac$ openssl genrsa -out volumes/certs/host.key 2048

1	mac$ openssl genrsa -out volumes/certs/host.key 2048

Create a Certificate Signing Request (CSR) for the CA to sign

mac$ openssl req -new -key volumes/certs/host.key -out volumes/certs/host.csr

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:NY
Locality Name (eg, city) []:NY
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Technologist
Organizational Unit Name (eg, section) []:Tech
Common Name (e.g. server FQDN or YOUR name) []:registry.example.com
Email Address []:johnATtechnologist

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

mac$ openssl req -new -key volumes/certs/host.key -out volumes/certs/host.csr

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]:US

State or Province Name (full name) [Some-State]:NY

Locality Name (eg, city) []:NY

Organization Name (eg, company) [Internet Widgits Pty Ltd]:Technologist

Organizational Unit Name (eg, section) []:Tech

Common Name (e.g. server FQDN or YOUR name) []:registry.example.com

Email Address []:johnATtechnologist

Please enter the following 'extra' attributes

to be sent with your certificate request

A challenge password []:

An optional company name []:

Finally have the root CA sign the CSR

mac$ openssl x509 -req -in volumes/certs/host.csr -CA volumes/certs/rootCA.crt -CAkey volumes/certs/rootCA.key -CAcreateserial -out volumes/certs/host.crt -days 365

Signature ok
subject=/C=US/ST=NY/L=NY/O=Technologist/OU=Tech/CN=registry.example.com/emailAddress=johnATtechnologist
Getting CA Private Key

mac$ openssl x509 -req -in volumes/certs/host.csr -CA volumes/certs/rootCA.crt -CAkey volumes/certs/rootCA.key -CAcreateserial -out volumes/certs/host.crt -days 365

Signature ok

subject=/C=US/ST=NY/L=NY/O=Technologist/OU=Tech/CN=registry.example.com/emailAddress=johnATtechnologist

Getting CA Private Key

Location to store the images

The tree structure should look like below, having the data folder be the image repository

mac $ tree
.
├── docker-registry
│   └── docker-compose.yml
└── volumes
    ├── auth
    │   └── htpasswd
    ├── certs
    │   ├── host.crt
    │   ├── host.csr
    │   ├── host.key
    │   ├── rootCA.crt
    │   ├── rootCA.key
    │   └── rootCA.srl
    └── data

mac $ tree

├── docker-registry

│ └── docker-compose.yml

└── volumes

├── auth

│ └── htpasswd

├── certs

│ ├── host.crt

│ ├── host.csr

│ ├── host.key

│ ├── rootCA.crt

│ ├── rootCA.key

│ └── rootCA.srl

└── data

Create Docker Registry container

Run your container using docker command

mac$ docker run -d -p 5000:5000 --restart=always --name registry \
  -v /Users/john/docker/blog/registry/volumes/auth:/auth \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
  -v /Users/john/docker/blog/registry/volumes/certs:/certs \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/host.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/host.key \
  registry:2

mac$ docker run -d -p 5000:5000 --restart=always --name registry \

-v /Users/john/docker/blog/registry/volumes/auth:/auth \

-e "REGISTRY_AUTH=htpasswd" \

-e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \

-e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \

-v /Users/john/docker/blog/registry/volumes/certs:/certs \

-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/host.crt \

-e REGISTRY_HTTP_TLS_KEY=/certs/host.key \

registry:2

Or even better, use docker-compose:

mac$ cd docker-registry
mac$ cat docker-compose.yml

1 2	mac$ cd docker-registry mac$ cat docker-compose.yml

registry:
  restart: always
  image: registry:2
  ports:
    - 5000:5000
  environment:
    REGISTRY_HTTP_TLS_CERTIFICATE: /certs/host.crt
    REGISTRY_HTTP_TLS_KEY: /certs/host.key
    REGISTRY_AUTH: htpasswd
    REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
    REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
  volumes:
    - /Users/john/docker/blog/registry/volumes/data:/var/lib/registry
    - /Users/john/docker/blog/registry/volumes/certs:/certs
    - /Users/john/docker/blog/registry/volumes/auth:/auth

registry:

restart: always

image: registry:2

ports:

- 5000:5000

environment:

REGISTRY_HTTP_TLS_CERTIFICATE: /certs/host.crt

REGISTRY_HTTP_TLS_KEY: /certs/host.key

REGISTRY_AUTH: htpasswd

REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd

REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm

volumes:

- /Users/john/docker/blog/registry/volumes/data:/var/lib/registry

- /Users/john/docker/blog/registry/volumes/certs:/certs

- /Users/john/docker/blog/registry/volumes/auth:/auth

Start up container based on docker-compose.yml

mac$ docker-compose up -d
Creating dockerregistry_registry_1

1 2	mac$ docker-compose up -d Creating dockerregistry_registry_1

See it running

mac$ docker-compose ps
          Name                         Command               State           Ports
-------------------------------------------------------------------------------------------
dockerregistry_registry_1   /entrypoint.sh /etc/docker ...   Up      0.0.0.0:5000->5000/tcp

mac$ docker-compose ps

Name Command State Ports

-------------------------------------------------------------------------------------------

dockerregistry_registry_1 /entrypoint.sh /etc/docker ... Up 0.0.0.0:5000->5000/tcp

At this point the Docker Registry is running

Using the Registry

Download an image from Docker Hub (or you can test with your own images built from Dockerfile)

mac$ docker pull ubuntu

1	mac$ docker pull ubuntu

Tag the image to point to your new Registry

mac$ docker tag ubuntu registry.example.com:5000/john/ubuntu

1	mac$ docker tag ubuntu registry.example.com:5000/john/ubuntu

Push image to your Registry (see failure due to ‘unknow CA authority’)

mac$ docker push registry.example.com:5000/john/ubuntu
The push refers to a repository [registry.example.com:5000/john/ubuntu]
Get https://registry.example.com:5000/v1/_ping: x509: certificate signed by unknown authority

mac$ docker push registry.example.com:5000/john/ubuntu

The push refers to a repository [registry.example.com:5000/john/ubuntu]

Get https://registry.example.com:5000/v1/_ping: x509: certificate signed by unknown authority

At this point, you need to add the root CA cert to your trusted certificates.
On the machine that will pull or push to the registry, you will need to install the rootCA.crt CA certificate created earlier:

docker-engine$ mkdir /etc/docker/certs.d/registry.example.com:5000/ 
docker-engine$ cp rootCA.crt /etc/docker/certs.d/registry.example.com:5000/rootCA.crt

1 2	docker-engine$ mkdir /etc/docker/certs.d/registry.example.com:5000/ docker-engine$ cp rootCA.crt /etc/docker/certs.d/registry.example.com:5000/rootCA.crt

On the Mac, you are probably using Docker for Mac, which actually runs a small hypervisor xhyve that virtualizes the docker engine. That means we need to place the rootCA.crt CA certificate inside the xhyve VM running the Docker Engine. There is no simple way of doing it and I am using the following method (which must be done everytime you restart the Docker service).

mac$ docker run -v /Users/john/docker/blog/registry/volumes/certs:/macdata -v /etc/docker:/hostdata ubuntu bash -c "mkdir -p /hostdata/certs.d/registry.example.com:5000/;cat /macdata/rootCA.crt > /hostdata/certs.d/registry.example.com:5000/ca.crt"

1	mac$ docker run -v /Users/john/docker/blog/registry/volumes/certs:/macdata -v /etc/docker:/hostdata ubuntu bash -c "mkdir -p /hostdata/certs.d/registry.example.com:5000/;cat /macdata/rootCA.crt > /hostdata/certs.d/registry.example.com:5000/ca.crt"

Now let’s try to Push the image to the Registry again

linux$ docker push registry.example.com:5000/john/ubuntu
The push refers to a repository [registry.example.com:5000/john/ubuntu]
4f9d527ff23f: Preparing
cc8df16ebbaf: Preparing
c3c1fd75be70: Preparing
5c42a1311e7e: Preparing
c8305edc6321: Preparing
unauthorized: authentication required

linux$ docker push registry.example.com:5000/john/ubuntu

The push refers to a repository [registry.example.com:5000/john/ubuntu]

4f9d527ff23f: Preparing

cc8df16ebbaf: Preparing

c3c1fd75be70: Preparing

5c42a1311e7e: Preparing

c8305edc6321: Preparing

unauthorized: authentication required

Good, we are getting somewhere, TLS is working, and it now asks for Authentication
Authenticate to the Docker Registry

linux$ docker login registry.example.com:5000
Username (testuser): user1
Password:
Login Succeeded

linux$ docker login registry.example.com:5000

Username (testuser): user1

Password:

Now let’s successfully push our image

linux$ docker push registry.example.com:5000/john/ubuntu
The push refers to a repository [registry.example.com:5000/john/ubuntu]
4f9d527ff23f: Pushed
cc8df16ebbaf: Pushed
c3c1fd75be70: Pushed
5c42a1311e7e: Pushed
c8305edc6321: Pushed
latest: digest: sha256:35dca5fc91c4890f787a266a057ad76b98ff3af43c75aa7e10962d06f06fd6e5 size: 1357

linux$ docker push registry.example.com:5000/john/ubuntu

The push refers to a repository [registry.example.com:5000/john/ubuntu]

4f9d527ff23f: Pushed

cc8df16ebbaf: Pushed

c3c1fd75be70: Pushed

5c42a1311e7e: Pushed

c8305edc6321: Pushed

latest: digest: sha256:35dca5fc91c4890f787a266a057ad76b98ff3af43c75aa7e10962d06f06fd6e5 size: 1357

At this point you can share your Docker image, or incorporate into a development or integration workflow by pulling the image
From any Docker engine system:

linux$ docker login registry.example.com:5000
Username: user1
Password:
Login Succeeded

linux$ docker login registry.example.com:5000

Username: user1

Password:

linux$ docker pull registry.example.com:5000/john/ubuntu
Using default tag: latest
latest: Pulling from john/ubuntu
Digest: sha256:35dca5fc91c4890f787a266a057ad76b98ff3af43c75aa7e10962d06f06fd6e5
Status: Downloaded newer image for registry.example.com:5000/john/ubuntu:latest

linux$ docker pull registry.example.com:5000/john/ubuntu

Using default tag: latest

latest: Pulling from john/ubuntu

Digest: sha256:35dca5fc91c4890f787a266a057ad76b98ff3af43c75aa7e10962d06f06fd6e5

Status: Downloaded newer image for registry.example.com:5000/john/ubuntu:latest

Registry API

You can use the Docker Registry API https://docs.docker.com/registry/spec/api/ to interact with your Registry.

For example to see a catalog of your images:

$ curl --user user1:password1 --cacert rootCA.crt -X GET https://registry.example.com:5000/v2/_catalog

{"repositories":["john/ubuntu"]}

$ curl --user user1:password1 --cacert rootCA.crt -X GET https://registry.example.com:5000/v2/_catalog

{"repositories":["john/ubuntu"]}

DevOps containers, docker, registry, tls

AWS EC2 automation using AWS CLI and User-Data

August 22, 2016 John No comments

In this post I will go over the installation and usage of the AWS CLI to deploy EC2 machines, also combined with AWS user-data to automate actions/scripts that will run on the EC2 machines at install time.

Install the AWS CLI
You can install the AWS CLI in many ways (e.g. zip file, brew, PIP, etc) for details you can follow http://docs.aws.amazon.com/cli/latest/userguide/installing.html
I will install via the Python package manager (PIP) inside a virtual environment

Create virtual environment

$ virtualenv env
New python executable in env/bin/python2.7
Also creating executable in env/bin/python
Installing setuptools, pip…done.

Activate virtual environment

$ source env/bin/activate

Install the awscli

$ pip install awscli
Collecting awscli
Downloading awscli-1.10.53-py2.py3-none-any.whl (970kB)
100% |################################| 970kB 582kB/s
Collecting botocore==1.4.43 (from awscli)
Downloading botocore-1.4.43-py2.py3-none-any.whl (2.5MB)
100% |################################| 2.5MB 206kB/s
Collecting s3transfer<0.2.0,>=0.1.0 (from awscli)
Downloading s3transfer-0.1.1-py2.py3-none-any.whl (49kB)
100% |################################| 49kB 1.4MB/s
Collecting rsa<=3.5.0,>=3.1.2 (from awscli)
Downloading rsa-3.4.2-py2.py3-none-any.whl (46kB)
100% |################################| 49kB 6.2MB/s
Collecting colorama<=0.3.7,>=0.2.5 (from awscli)
Downloading colorama-0.3.7-py2.py3-none-any.whl
Collecting docutils>=0.10 (from awscli)
Downloading docutils-0.12.tar.gz (1.6MB)
100% |################################| 1.6MB 312kB/s
Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.4.43->awscli)
Downloading jmespath-0.9.0-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.4.43->awscli)
Downloading python_dateutil-2.5.3-py2.py3-none-any.whl (201kB)
100% |################################| 204kB 1.9MB/s
Collecting futures<4.0.0,>=2.2.0 (from s3transfer<0.2.0,>=0.1.0->awscli)
Downloading futures-3.0.5-py2-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)
Downloading pyasn1-0.1.9-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.4.43->awscli)
Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six, pyasn1, futures, python-dateutil, jmespath, docutils, colorama, rsa, s3transfer, botocore, awscli
Running setup.py install for docutils
changing mode of build/scripts-2.7/rst2html.py from 644 to 755
changing mode of build/scripts-2.7/rst2s5.py from 644 to 755
…
Successfully installed awscli-1.10.53 botocore-1.4.43 colorama-0.3.7 docutils-0.12 futures-3.0.5 jmespath-0.9.0 pyasn1-0.1.9 python-dateutil-2.5.3 rsa-3.4.2 s3transfer-0.1.1 six-1.10.0

Verify installation (and review help documentation)

$ aws help

Configure the AWS CLI to use AWS credentials
You will need to provide an AWS Access Key ID and AWS secreate Access Key, these will map to a AWS user and the privileges this user have. The user can (probably should have) been created from the AWS console.

$ aws configure
AWS Access Key ID [****************XYYY]: XXXXXXXXYYY
AWS Secret Access Key [****************ZOOO]:ZZZZZZZOOO
Default region name [us-east-1]:us-east-1
Default output format [None]: json

The above will create a default configuration and set of credentials under ~/.aws/{config,credentials}

Launching an EC2 Instance from awscli

To connect to the EC2 instance you will need at least a SSH keypair and allow access to SSH into the EC2 instance

Create key pair

$ aws ec2 create-key-pair –key-name john-keypair –query ‘KeyMaterial’ –output text > ~/.ssh/john-keypair.pem

$ chmod 400 ~/.ssh/john-keypair.pem

Verify key existence

$ aws ec2 describe-key-pairs –key-name john-keypair
KEYPAIRS 03:eb:3a:d3:13:ba:d3:e3:03:13:b3:f1:43:83:cc:03:ec:d8:4b:43 john-keypair

Create a Security group that allows ingress SSH (tcp 22)

$ aws ec2 create-security-group –group-name allow_tcp_22 –description “Allow SSH”

sg-b1740cd5

// TAG it, I cannot stress enough how important it is to tag every resource with at least a Name

$ aws ec2 create-tags –resources sg-b1740cd5 –tags Key=Name,Value=allow_tcp_22

Add rules, for example: allow inbound (ingress) tcp port 22 to all (0.0.0.0/0)

$ aws ec2 authorize-security-group-ingress –group-id sg-b1740cd5 –protocol tcp –port 22 –cidr 0.0.0.0/0

Select the AWS AMI (aka template) you will use to clone and create your EC2 instance.

See your images (For example I have two Encrypted Ubuntu images) These are using output=text in ~/.aws/config, just to show you what that looks like as oposed to JSON.

$ aws ec2 describe-images –owners self
IMAGES x86_64 2016-07-30T13:16:39.000Z xen ami-a64403c2 /Encrypted Ubuntu Linux 14.04 (HVM) machine Encrypted Red Hat Enterprise Linux (HVM) False /dev/sda1 ebs simple available hvm
BLOCKDEVICEMAPPINGS /dev/sda1
EBS True True snap-bc158f41 10 gp2
IMAGES x86_64 2016-07-30T13:22:03.000Z xen ami-a74403c3 /Encrypted Ubuntu Linux 16.04 (HVM) machine Encrypted Red Hat Enterprise Linux (HVM) False /dev/sda1 ebs simple available hvm
BLOCKDEVICEMAPPINGS /dev/sda1
EBS True True snap-900aa0a1 10 gp2

Or see available images from Redhat

$ aws ec2 describe-images –owners 309956199498 –filters “Name=architecture,Values=x86_64” | grep RHEL-7.1
IMAGES x86_64 2015-02-26T16:27:33.000Z Provided by Red Hat, Inc. xen ami-a540a5e1 309956199498/RHEL-7.1_HVM_GA-20150225-x86_64-1-Hourly2-GP2 machine RHEL-7.1_HVM_GA-20150225-x86_64-1-Hourly2-GP2 309956199498 True /dev/sda1 ebs available hvm
IMAGES x86_64 2015-08-04T17:22:47.000Z Provided by Red Hat, Inc. xen ami-c1996685 309956199498/RHEL-7.1_HVM-20150803-x86_64-1-Hourly2-GP2 machine RHEL-7.1_HVM-20150803-x86_64-1-Hourly2-GP2309956199498 True /dev/sda1 ebs simple available hvm

Or even the images from Amazon, which are FREE – this is the one I will use

$ aws ec2 describe-images –owners amazon –filters “Name=root-device-type,Values=ebs” “Name=architecture,Values=x86_64” | grep ‘Amazon Linux AMI 2016’
…
IMAGES x86_64 2016-06-22T08:08:12.000Z Amazon Linux AMI 2016.03.3 x86_64 HVM GP2 xen ami-31490d51 amazon/amzn-ami-hvm-2016.03.3.x86_64-gp2 amazon machine amzn-ami-hvm-2016.03.3.x86_64-gp2 137112412989 True /dev/xvda ebs simple available hvm
…

Launch/Run the instance based on the chosen AMI (e.g. ami-31490d51)

$ aws ec2 run-instances –image-id ami-31490d51 –count 1 –instance-type m3.medium –key-name john-keypair –security-group-ids sg-b1740cd5

{
    "OwnerId": "XXXX",
    "ReservationId": "r-XXc3852a",
    "Groups": [],
    "Instances": [
        {
            "Monitoring": {
                "State": "disabled"
            },
            "PublicDnsName": "",
            "RootDeviceType": "ebs",
            "State": {
                "Code": 0,
                "Name": "pending"
            },
            "EbsOptimized": false,
            "LaunchTime": "2016-07-30T23:44:30.000Z",
            "PrivateIpAddress": "172.31.13.82",
            "ProductCodes": [],
            "VpcId": "vpc-d491d9b1",
            "StateTransitionReason": "",
            "InstanceId": "i-e1aeaf54",
...
...

{

"OwnerId": "XXXX",

"ReservationId": "r-XXc3852a",

"Groups": [],

"Instances": [

{

"Monitoring": {

"State": "disabled"

"PublicDnsName": "",

"RootDeviceType": "ebs",

"State": {

"Code": 0,

"Name": "pending"

"EbsOptimized": false,

"LaunchTime": "2016-07-30T23:44:30.000Z",

"PrivateIpAddress": "172.31.13.82",

"ProductCodes": [],

"VpcId": "vpc-d491d9b1",

"StateTransitionReason": "",

"InstanceId": "i-e1aeaf54",

...

TAG IT

$ aws ec2 create-tags –resources i-e1aeaf54 –tags Key=Name,Value=JohnInstance1

Check its status

$ aws ec2 describe-instance-status –instance-ids i-e1aeaf54

Check its details

$ aws ec2 describe-instances –instance-ids i-e1aeaf54

Retrive from its details specifc information using jq (jq is a lightweight and flexible command-line JSON processor)

$ aws ec2 describe-instances –instance-ids i-e1aeaf54 | jq ‘.Reservations[].Instances[].PublicDnsName’

“ec2-54-183-59-200.us-west-1.compute.amazonaws.com”

Connect to the EC2 instance using your SSH key-pair

ssh -i ~/.ssh/john-keypair.pem ec2-user@ec2-54-183-59-200.us-west-1.compute.amazonaws.com

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2016.03-release-notes/
10 package(s) needed for security, out of 22 available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-172-31-13-82 ~]$

ssh -i ~/.ssh/john-keypair.pem ec2-user@ec2-54-183-59-200.us-west-1.compute.amazonaws.com

__| __|_ )

_| ( / Amazon Linux AMI

___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2016.03-release-notes/

10 package(s) needed for security, out of 22 available

Run "sudo yum update" to apply all updates.

[ec2-user@ip-172-31-13-82 ~]$

Destroy EC2 instance

$ aws ec2 terminate-instances –instance-ids i-e1aeaf54

{
    "TerminatingInstances": [
        {
            "InstanceId": "i-e1aeaf54",
            "CurrentState": {
                "Code": 32,
                "Name": "shutting-down"
            },
            "PreviousState": {
                "Code": 16,
                "Name": "running"
            }
        }
    ]
}

{

"TerminatingInstances": [

{

"InstanceId": "i-e1aeaf54",

"CurrentState": {

"Code": 32,

"Name": "shutting-down"

"PreviousState": {

"Code": 16,

"Name": "running"

}

]

}

Create EC2 instance with user data script
Now let’s create an EC2 instance that will be ready to serve a website, we will do that by adding user data content that installs a webserver and adds some content.

Notes:
Amazon EC2 limits the size of user-data to 16KB (This limit applies to the data in raw form, not base64-encoded form.)
You can download a larger script and run it, from S3 for example.
You can run any language that supports the shabang(#!) (e.g. Bash, Python, Ruby, Perl)

//user_data.sh

#!/bin/bash
set -e -x
yum install httpd -y
chkconfig httpd on
service httpd start
INSTANCEID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
echo "Hello from ${INSTANCEID}" > /var/www/html/index.html

#!/bin/bash

set -e -x

yum install httpd -y

chkconfig httpd on

service httpd start

INSTANCEID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

echo "Hello from ${INSTANCEID}" > /var/www/html/index.html

Launch/run instance with user-data:

$ aws ec2 run-instances –image-id ami-31490d51 –count 1 –instance-type m3.medium –key-name john-keypair –security-group-ids sg-b1740cd5 –user-data file://user_data.sh

{
    "OwnerId": "XXXX",
    "ReservationId": "r-7ed096cc",
    "Groups": [],
    "Instances": [
        {
            "Monitoring": {
                "State": "disabled"
            },
            "PublicDnsName": "",
            "RootDeviceType": "ebs",
            "State": {
                "Code": 0,
                "Name": "pending"
            },
            "EbsOptimized": false,
            "LaunchTime": "2016-07-30T22:34:29.000Z",
            "PrivateIpAddress": "172.31.6.82",
            "ProductCodes": [],
            "VpcId": "vpc-d491d9b1",
            "StateTransitionReason": "",
            "InstanceId": "i-6b6a6ade",
...

{

"OwnerId": "XXXX",

"ReservationId": "r-7ed096cc",

"Groups": [],

"Instances": [

{

"Monitoring": {

"State": "disabled"

"PublicDnsName": "",

"RootDeviceType": "ebs",

"State": {

"Code": 0,

"Name": "pending"

"EbsOptimized": false,

"LaunchTime": "2016-07-30T22:34:29.000Z",

"PrivateIpAddress": "172.31.6.82",

"ProductCodes": [],

"VpcId": "vpc-d491d9b1",

"StateTransitionReason": "",

"InstanceId": "i-6b6a6ade",

...

TAG it

$ aws ec2 create-tags –resources i-e1aeaf54 –tags Key=Name,Value=JohnInstance

Get its public DNS name

$ aws ec2 describe-instances –instance-ids i-e974745c | jq ‘.Reservations[].Instances[].PublicDnsName’

“ec2-52-53-165-15.us-west-1.compute.amazonaws.com”

Connect to it

$ ssh -i ~/.ssh/john-keypair.pem ec2-user@ec2-52-53-165-15.us-west-1.compute.amazonaws.com

Verify that httpd is running and the right content is displayed

[ec2-user@ip-172-31-9-131 ~]$ curl http://localhost

Hello from i-e974745c

Troubleshooting: Check the /var/log/cloud-init-output.log file

[ec2-user@ip-172-31-9-131 ~]$ sudo tail /var/log/cloud-init-output.log

  apr-util-ldap.x86_64 0:1.4.1-4.17.amzn1 httpd-tools.x86_64 0:2.2.31-1.8.amzn1

Complete!
+ chkconfig httpd on
+ service httpd start
Starting httpd: [  OK  ]
++ curl -s http://169.254.169.254/latest/meta-data/instance-id
+ INSTANCEID=i-e974745c
+ echo 'Hello from i-e974745c'
Cloud-init v. 0.7.6 finished at Thu, 18 Aug 2016 19:51:46 +0000. Datasource DataSourceEc2.  Up 63.66 seconds

apr-util-ldap.x86_64 0:1.4.1-4.17.amzn1 httpd-tools.x86_64 0:2.2.31-1.8.amzn1

Complete!

+ chkconfig httpd on

+ service httpd start

Starting httpd: [ OK ]

++ curl -s http://169.254.169.254/latest/meta-data/instance-id

+ INSTANCEID=i-e974745c

+ echo 'Hello from i-e974745c'

Cloud-init v. 0.7.6 finished at Thu, 18 Aug 2016 19:51:46 +0000. Datasource DataSourceEc2. Up 63.66 seconds

Extra:
To allow access to it from the public you will need to add a security group that allows ports 80 and 443.
You can create one as we did for the SSH access, or you can see if you already have a group.

Look at the descriptions of you security groups (this is why it is important to add good descriptions)
“allow SSH access”
“Allow SSH”
“Allow Web tcp 80 and 443 from 0.0.0.0/0”
…

Looks like “Allow Web tcp 80 and 443 from 0.0.0.0/0” will do the trick, let’s verify

$ aws ec2 describe-security-groups | jq ‘.SecurityGroups[2]’

{
  "IpPermissionsEgress": [
    {
      "IpProtocol": "-1",
      "IpRanges": [
        {
          "CidrIp": "0.0.0.0/0"
        }
      ],
      "UserIdGroupPairs": [],
      "PrefixListIds": []
    }
  ],
  "Description": "Allow Web tcp 80 and 443 from 0.0.0.0/0",
  "Tags": [
    {
      "Value": "allow_tcp_web",
      "Key": "Name"
    }
  ],
  "IpPermissions": [
    {
      "PrefixListIds": [],
      "FromPort": 80,
      "IpRanges": [
        {
          "CidrIp": "0.0.0.0/0"
        }
      ],
      "ToPort": 80,
      "IpProtocol": "tcp",
      "UserIdGroupPairs": []
    },
    {
      "PrefixListIds": [],
      "FromPort": 443,
      "IpRanges": [
        {
          "CidrIp": "0.0.0.0/0"
        }
      ],
      "ToPort": 443,
      "IpProtocol": "tcp",
      "UserIdGroupPairs": []
    }
  ],
  "GroupName": "allow_tcp_web",
  "VpcId": "vpc-XXX",
  "OwnerId": "XXXXXXX",
  "GroupId": "sg-a3672ac7"
}

{

"IpPermissionsEgress": [

{

"IpProtocol": "-1",

"IpRanges": [

{

"CidrIp": "0.0.0.0/0"

}

"UserIdGroupPairs": [],

"PrefixListIds": []

}

"Description": "Allow Web tcp 80 and 443 from 0.0.0.0/0",

"Tags": [

{

"Value": "allow_tcp_web",

"Key": "Name"

}

"IpPermissions": [

{

"PrefixListIds": [],

"FromPort": 80,

"IpRanges": [

{

"CidrIp": "0.0.0.0/0"

}

"ToPort": 80,

"IpProtocol": "tcp",

"UserIdGroupPairs": []

{

"PrefixListIds": [],

"FromPort": 443,

"IpRanges": [

{

"CidrIp": "0.0.0.0/0"

}

"ToPort": 443,

"IpProtocol": "tcp",

"UserIdGroupPairs": []

}

"GroupName": "allow_tcp_web",

"VpcId": "vpc-XXX",

"OwnerId": "XXXXXXX",

"GroupId": "sg-a3672ac7"

}

Let’s add this security group (sg-a3672ac7) to our EC2 instance (i-e974745c)
// This does not append, you have to specify all groups, in this case I am adding the SSH and WEB security groups to the instance

$ aws ec2 modify-instance-attribute –instance-id i-e974745c –groups sg-b1740cd5 sg-a3672ac7

Verify that the instance has both security groups

$ aws ec2 describe-instances –instance-ids i-e974745c | jq ‘.Reservations[].Instances[].SecurityGroups[]’

{
  "GroupName": "allow_tcp_web",
  "GroupId": "sg-a3672ac7"
}
{
  "GroupName": "allow_tcp_22",
  "GroupId": "sg-b1740cd5"
}

{

"GroupName": "allow_tcp_web",

"GroupId": "sg-a3672ac7"

}

{

"GroupName": "allow_tcp_22",

"GroupId": "sg-b1740cd5"

}

You can now verify that the content is publicly available

aws, Cloud, DevOps automation, aws, awscli, cloud, ec2, user-data

Infrastructure As Code: Create Linux (RHEL/CentOS) base Images Using Packer

January 25, 2016 John 3 comments

In a previous post I wrote about how to Create a custom Vagrant Box from scratch.

In this post I will walk through the usage of Packer to automate the creation of a CentOS 7 image that can be used in Vagrant or even vSphere.

Packer is a cool Hashicorp tool that helps you automate the creation of machine images.

You can download the project/git repo at https://github.com/parcejohn/packer-centos7

From their website:
“Packer is easy to use and automates the creation of any type of machine image. It embraces modern configuration management by encouraging you to use automated scripts to install and configure the software within your Packer-made images. Packer brings machine images into the modern age, unlocking untapped potential and opening new opportunities.”

I use Packer to continue the journey to Infrastructure As Code, where even my golden images/templates are automated and source controlled (on Git).

Requirements:

* Packer // I installed on Mac using $ brew install packer
* vagrant // I installed on Mac using $ brew install vagrant
* vmware fusion // I installed using $ brew cask install vmware-fusion
* CentOS 7 ISO file

Packer uses a JSON template file to orchestrate the image creation and there are different stages in the process, I will concentrate on the below three, but you should familiarize with all of them (https://www.packer.io/docs/basics/terminology.html)

Builders: Packer component to create a machine image for a single platform, in this case I will be using the VMware builder.
Provisioners: Packer component that installs and configures software within a running machine prior to that machine being turned into a static image. Example provisioners include shell scripts, Chef, Puppet, etc. I will be using the shell provisioner.
Post-Processors: Packer component that takes the result of the builder or another post-processor and process that to create a new artifact. Examples of post-processors are compress to compress artifacts, upload to upload artifacts, etc. I will be creating a Vagrant box as the artifact, and also demonstrate how to upload the image to vSphere.

It’s time to walk through the process of creating a CentOS 7 image using Packer.

1) Create directory structure
The http folder will host a kickstart file, Packer will use its built in web server to serve this kickstart file
The scripts folder will host the provisioning scripts that will define the machine

centos/
├── http
└── scripts

2) Populate with the following configuration files and scripts (the contents of these files are shown in the next steps)

centos/
├── centos-7.1-x64-vmware.json
├── http
│   └── ks.cfg
├── scripts
│   ├── base.sh
│   ├── cleanup.sh
│   ├── hgfs.sh
│   ├── vmware.sh
│   └── zerodisk.sh
├── template.json
└── vagrant_rsa_key

3) Create a SSH key pair, you will need this to login to the server to complete install and configuration

The Public Key will be injected to the ‘vagrant’ user as part of the kickstart, it will then be used in the Packer JSON template to allow Packer to login to the machine
The Private Key will stay on your system to allow Packer and Vagrant to login to the created machine.

$ ssh-keygen -t rsa -b 4096 -C "vagrant" -N '' -q -f ./vagrant_rsa_key

1	$ ssh-keygen -t rsa -b 4096 -C "vagrant" -N '' -q -f ./vagrant_rsa_key

4) Create a Packer JSON Template file (template.json)

{
  "variables": {
    "vm_name": "centos-7.1-vmware",
    "iso_url": "{{env `ISO_URL`}}",
    "iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284"
  },
  "builders": [
    {
      "headless": true,
      "type": "vmware-iso",
      "boot_command": [
        "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"
      ],
      "boot_wait": "10s",
      "disk_size": 8192,
      "guest_os_type": "centos-64",
      "http_directory": "http",
      "iso_url": "{{user `iso_url`}}",
      "iso_checksum_type": "sha256",
      "iso_checksum": "{{user `iso_sha256`}}",
      "ssh_username": "vagrant",
      "ssh_private_key_file": "vagrant_rsa",
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
      "shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
      "tools_upload_flavor": "linux",
      "tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",
      "vmx_data": {
        "memsize": "1024",
        "numvcpus": "1",
        "cpuid.coresPerSocket": "1"
      }
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
      "override": {
        "vmware-iso": {
          "scripts": [
            "scripts/base.sh",
            "scripts/vmware.sh",
            "scripts/hgfs.sh",
            "scripts/cleanup.sh",
            "scripts/zerodisk.sh"
          ]
        }
      }
    }
  ],
  "post-processors": [
    {
      "type": "vagrant",
      "override": {
        "vmware": {
          "output": "centos-7.1-x64-vmware.box"
        }
      }
    }
  ]
}

{

"variables": {

"vm_name": "centos-7.1-vmware",

"iso_url": "{{env `ISO_URL`}}",

"iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284"

"builders": [

{

"headless": true,

"type": "vmware-iso",

"boot_command": [

"<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"

"boot_wait": "10s",

"disk_size": 8192,

"guest_os_type": "centos-64",

"http_directory": "http",

"iso_url": "{{user `iso_url`}}",

"iso_checksum_type": "sha256",

"iso_checksum": "{{user `iso_sha256`}}",

"ssh_username": "vagrant",

"ssh_private_key_file": "vagrant_rsa",

"ssh_port": 22,

"ssh_wait_timeout": "10000s",

"shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",

"tools_upload_flavor": "linux",

"tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",

"vmx_data": {

"memsize": "1024",

"numvcpus": "1",

"cpuid.coresPerSocket": "1"

}

"provisioners": [

{

"type": "shell",

"execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",

"override": {

"vmware-iso": {

"scripts": [

"scripts/base.sh",

"scripts/vmware.sh",

"scripts/hgfs.sh",

"scripts/cleanup.sh",

"scripts/zerodisk.sh"

]

}

"post-processors": [

{

"type": "vagrant",

"override": {

"vmware": {

"output": "centos-7.1-x64-vmware.box"

}

]

}

Packer Components/Sections:

Variables:
This is just to centralize variables that will be used by other components (e.g. Builder, provisioner, etc)
I can also set a variable to use Environment variables or command line provided values:
User provided values:
“iso_url”: “{{user iso_url}}”,

Environment variable values:
“iso_url”: “{{env ISO_URL}}”,

Builders:
Here is an array of builders and the needed parameters for each builder, in this case the only builder is of type vmware-iso.
Parameters and some comments about them.

      "headless": true, // Means not to open the VMware Fusion Console
      "type": "vmware-iso", // VMware type builder
      "boot_command": [
        "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"  // Send these key combination to the VMWare Fusion Console
      ],
      "boot_wait": "10s",
      "disk_size": 8192,
      "guest_os_type": "centos-64",
      "http_directory": "http",  // Directory where the kickstart file is placed
      "iso_url": "{{user `iso_url`}}", // Location of ISO image, defined at runtime, more on that later
      "iso_checksum_type": "sha256", 
      "iso_checksum": "{{user `iso_sha256`}}",
      "ssh_username": "vagrant",  // Packer will log in to resulting machine for provisioning, this user must exist (user created from the Kickstart) 
      "ssh_private_key_file": "vagrant_rsa", // Packer will log in using ssh key created earlier
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
      "shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
      "tools_upload_flavor": "linux",
      "tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso", 
      "vmx_data": {
        "memsize": "1024",
        "numvcpus": "1",
        "cpuid.coresPerSocket": "1"

"headless": true, // Means not to open the VMware Fusion Console

"type": "vmware-iso", // VMware type builder

"boot_command": [

"<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>" // Send these key combination to the VMWare Fusion Console

"boot_wait": "10s",

"disk_size": 8192,

"guest_os_type": "centos-64",

"http_directory": "http", // Directory where the kickstart file is placed

"iso_url": "{{user `iso_url`}}", // Location of ISO image, defined at runtime, more on that later

"iso_checksum_type": "sha256",

"iso_checksum": "{{user `iso_sha256`}}",

"ssh_username": "vagrant", // Packer will log in to resulting machine for provisioning, this user must exist (user created from the Kickstart)

"ssh_private_key_file": "vagrant_rsa", // Packer will log in using ssh key created earlier

"ssh_port": 22,

"ssh_wait_timeout": "10000s",

"shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",

"tools_upload_flavor": "linux",

"tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",

"vmx_data": {

"memsize": "1024",

"numvcpus": "1",

"cpuid.coresPerSocket": "1"

Provisioners:
This section uses the shell provisioner and runs all of those scripts listed, which are located in the scripts folder.

      "type": "shell",
      "execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
      "override": {
        "vmware-iso": {
          "scripts": [
            "scripts/base.sh",
            "scripts/vmware.sh",
            "scripts/hgfs.sh",
            "scripts/cleanup.sh",
            "scripts/zerodisk.sh"

"type": "shell",

"execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",

"override": {

"vmware-iso": {

"scripts": [

"scripts/base.sh",

"scripts/vmware.sh",

"scripts/hgfs.sh",

"scripts/cleanup.sh",

"scripts/zerodisk.sh"

Post-processor:
This section states that I want a vagrant box and I am giving it the name.

      "type": "vagrant",
      "override": {
        "vmware": {
          "output": "centos-7.1-x64-vmware.box"

"type": "vagrant",

"override": {

"vmware": {

"output": "centos-7.1-x64-vmware.box"

5) Kickstart file (should be http/ks.cfg based on the template.json file)
This is a minimal install of CentOS 7
Note: The private key (XXXX) given to the vagrant user, that is the key that we created in a previous step

install
cdrom
lang en_US.UTF-8
keyboard us
network --onboot yes --device eth0 --bootproto dhcp --noipv6
rootpw  --plaintext vagrant
firewall --enabled --service=ssh
authconfig --enableshadow --passalgo=sha512
selinux --disabled
timezone --utc America/New_York
bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet"

text
skipx
zerombr

clearpart --all --initlabel
autopart

auth  --useshadow  --enablemd5
firstboot --disabled
reboot

%packages --nobase --ignoremissing 
@core
bzip2
kernel-devel
kernel-headers
-ipw2100-firmware
-ipw2200-firmware
-ivtv-firmware
%end

%post
# Install SUDO
/usr/bin/yum -y install sudo

# Create vagrant user
/usr/sbin/useradd vagrant
/bin/mkdir /home/vagrant/.ssh
/bin/chmod 700 /home/vagrant/.ssh
cat >           /home/vagrant/.ssh/authorized_keys <<'VAGRANT_RSA'
ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX vagrant
VAGRANT_RSA

/bin/chmod 600 /home/vagrant/.ssh/authorized_keys
/bin/chown -R vagrant /home/vagrant/.ssh

# Add vagrant user to SUDO
echo "vagrant        ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers.d/vagrant
echo "Defaults:vagrant !requiretty"                 >> /etc/sudoers.d/vagrant
chmod 0440 /etc/sudoers.d/vagrant
%end

install

cdrom

lang en_US.UTF-8

keyboard us

network --onboot yes --device eth0 --bootproto dhcp --noipv6

rootpw --plaintext vagrant

firewall --enabled --service=ssh

authconfig --enableshadow --passalgo=sha512

selinux --disabled

timezone --utc America/New_York

bootloader --location=mbr --driveorder=sda --append="crashkernel=auto rhgb quiet"

text

skipx

zerombr

clearpart --all --initlabel

autopart

auth --useshadow --enablemd5

firstboot --disabled

reboot

%packages --nobase --ignoremissing

@core

bzip2

kernel-devel

kernel-headers

-ipw2100-firmware

-ipw2200-firmware

-ivtv-firmware

%end

%post

# Install SUDO

/usr/bin/yum -y install sudo

# Create vagrant user

/usr/sbin/useradd vagrant

/bin/mkdir /home/vagrant/.ssh

/bin/chmod 700 /home/vagrant/.ssh

cat > /home/vagrant/.ssh/authorized_keys <<'VAGRANT_RSA'

ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX vagrant

VAGRANT_RSA

/bin/chmod 600 /home/vagrant/.ssh/authorized_keys

/bin/chown -R vagrant /home/vagrant/.ssh

# Add vagrant user to SUDO

echo "vagrant ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/vagrant

echo "Defaults:vagrant !requiretty" >> /etc/sudoers.d/vagrant

chmod 0440 /etc/sudoers.d/vagrant

%end

6 ) Provisioning Scrips
Packer calls these scripts (from the JSON template) on the created machine before making it a template/image.

base.sh // Install basic packages

#!/usr/bin/env bash
set -x

sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers
yum -y install gcc make gcc-c++ kernel-devel-`uname -r` perl

#!/usr/bin/env bash

set -x

sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers

yum -y install gcc make gcc-c++ kernel-devel-`uname -r` perl

vmware.sh // Install VMTools, I separated this from base as I may want to call different scripts on a VirtualBox/etc builder.

#!/usr/bin/env bash
set -x

yum install -y fuse-libs open-vm-tools

#!/usr/bin/env bash

set -x

yum install -y fuse-libs open-vm-tools

hgfs.sh // For vagrant to work properly with folders

#!/usr/bin/env bash
set -x

VMWARE_ISO=/tmp/vmware_tools_linux.iso
VMWARE_MNTDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_mnt_XXXXXX)
VMWARE_TMPDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_XXXXXX)

# Extract tools
mount -o loop $VMWARE_ISO $VMWARE_MNTDIR
tar zxf $VMWARE_MNTDIR/VMwareTools*.tar.gz -C $VMWARE_TMPDIR
umount $VMWARE_MNTDIR

# Install tools
$VMWARE_TMPDIR/vmware-tools-distrib/vmware-install.pl -d

# Clean up
rm -f $VMWARE_ISO
rm -rf $VMWARE_MNTDIR
rm -rf $VMWARE_TMPDIR

#!/usr/bin/env bash

set -x

VMWARE_ISO=/tmp/vmware_tools_linux.iso

VMWARE_MNTDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_mnt_XXXXXX)

VMWARE_TMPDIR=$(mktemp --tmpdir=/tmp -q -d -t vmware_XXXXXX)

# Extract tools

mount -o loop $VMWARE_ISO $VMWARE_MNTDIR

tar zxf $VMWARE_MNTDIR/VMwareTools*.tar.gz -C $VMWARE_TMPDIR

umount $VMWARE_MNTDIR

# Install tools

$VMWARE_TMPDIR/vmware-tools-distrib/vmware-install.pl -d

# Clean up

rm -f $VMWARE_ISO

rm -rf $VMWARE_MNTDIR

rm -rf $VMWARE_TMPDIR

cleanup.sh // Clean up before converting to image

#!/usr/bin/env bash
set -x

yum -y erase gtk2 libX11 hicolor-icon-theme avahi freetype bitstream-vera-fonts
rpm --rebuilddb
yum -y clean all

#!/usr/bin/env bash

set -x

yum -y erase gtk2 libX11 hicolor-icon-theme avahi freetype bitstream-vera-fonts

rpm --rebuilddb

yum -y clean all

zerodisk.sh // This is so that the resulting box is as small as possible

#!/usr/bin/env bash
set -x

dd if=/dev/zero of=/EMPTY bs=1M
rm -f /EMPTY

#!/usr/bin/env bash

set -x

dd if=/dev/zero of=/EMPTY bs=1M

rm -f /EMPTY

7) Run Packer against the template.json file created earlier and watch it build the machine image

$ packer build -var ‘iso_url=/Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso’ -only=vmware-iso template.json
vmware-iso output will be in this color.

==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: file:///Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso
==> vmware-iso: Creating virtual machine disk
==> vmware-iso: Building and writing VMX file
==> vmware-iso: Starting HTTP server on port 8101
==> vmware-iso: Starting virtual machine…
vmware-iso: The VM will be run headless, without a GUI. If you want to
vmware-iso: view the screen of the VM, connect via VNC without a password to
vmware-iso: 127.0.0.1:5989
==> vmware-iso: Waiting 10s for boot…
==> vmware-iso: Connecting to VM via VNC
==> vmware-iso: Typing the boot command over VNC…
==> vmware-iso: Waiting for SSH to become available…
==> vmware-iso: Connected to SSH!
==> vmware-iso: Uploading the ‘linux’ VMware Tools
==> vmware-iso: Provisioning with shell script: scripts/base.sh
vmware-iso: + sed -i ‘s/^.*requiretty/#Defaults requiretty/’ /etc/sudoers
…
…
…
==> vmware-iso: Provisioning with shell script: scripts/zerodisk.sh
vmware-iso: + dd if=/dev/zero of=/EMPTY bs=1M
vmware-iso: dd: error writing ‘/EMPTY’: No space left on device
vmware-iso: 5488+0 records in
vmware-iso: 5487+0 records out
vmware-iso: 5754265600 bytes (5.8 GB) copied, 5.27091 s, 1.1 GB/s
vmware-iso: + rm -f /EMPTY
==> vmware-iso: Gracefully halting virtual machine…
vmware-iso: Waiting for VMware to clean up after itself…
==> vmware-iso: Deleting unnecessary VMware files…
vmware-iso: Deleting: output-vmware-iso/564de113-2fc5-2010-8e5d-63fec92f12f7.vmem
vmware-iso: Deleting: output-vmware-iso/packer-vmware-iso.plist
vmware-iso: Deleting: output-vmware-iso/vmware.log
==> vmware-iso: Cleaning VMX prior to finishing up…
vmware-iso: Unmounting floppy from VMX…
vmware-iso: Detaching ISO from CD-ROM device…
vmware-iso: Disabling VNC server…
==> vmware-iso: Compacting the disk image
==> vmware-iso: Running post-processor: vagrant
==> vmware-iso (vagrant): Creating Vagrant box for ‘vmware’ provider
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s001.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s002.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s003.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.nvram
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmsd
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmxf
vmware-iso (vagrant): Compressing: Vagrantfile
vmware-iso (vagrant): Compressing: disk-s001.vmdk
vmware-iso (vagrant): Compressing: disk-s002.vmdk
vmware-iso (vagrant): Compressing: disk-s003.vmdk
vmware-iso (vagrant): Compressing: disk.vmdk
vmware-iso (vagrant): Compressing: metadata.json
vmware-iso (vagrant): Compressing: packer-vmware-iso.nvram
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmsd
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmx
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmxf
Build ‘vmware-iso’ finished.

==> Builds finished. The artifacts of successful builds are:
–> vmware-iso: ‘vmware’ provider box: centos-7.1-x64-vmware.box

8) Add box to Vagrant

$ vagrant box add –name centos-7.1-010-x64-vmware.box centos-7.1-x64-vmware.box
==> box: Adding box ‘centos-7.1-010-x64-vmware.box’ (v0) for provider:
box: Downloading: file:///Users/john/packer/centos/centos-7.1-x64-vmware.box
==> box: Successfully added box ‘centos-7.1-010-x64-vmware.box’ (v0) for ‘vmware_desktop’!

The above is all you need to have your Infrastructure be Code that can be versioned and automatically create your images and use them on Vagrant.
The next item will show you how to use Packer to move the same image to your vSphere environment.

9) Sending Image to vSphere
To send the image created by Packer to vSphere you will need to add another post-provisioning entry to the JSON array:

    {
      "type": "vsphere",
      "host": "{{user `vm_host`}}",
      "username": "{{user `vm_user`}}",
      "password": "{{user `vm_pass`}}",
      "datacenter": "{{user `vm_dc`}}",
      "cluster": "{{user `vm_cluster`}}",
      "resource_pool": " ",
      "datastore": "{{user `vm_datastore`}}",
      "vm_folder": "{{user `vm_folder`}}",
      "vm_name": "{{user `vm_name`}}", 
      "vm_network": "{{user `vm_network`}}",
      "insecure" : "true"
    }

{

"type": "vsphere",

"host": "{{user `vm_host`}}",

"username": "{{user `vm_user`}}",

"password": "{{user `vm_pass`}}",

"datacenter": "{{user `vm_dc`}}",

"cluster": "{{user `vm_cluster`}}",

"resource_pool": " ",

"datastore": "{{user `vm_datastore`}}",

"vm_folder": "{{user `vm_folder`}}",

"vm_name": "{{user `vm_name`}}",

"vm_network": "{{user `vm_network`}}",

"insecure" : "true"

}

As you can see I am using variables to make this portable, so that means you have to add variables to the variables section as well, below is the new template to do both vagrant and vSphere image provisioning.

{
  "variables": {
    "vm_name": "centos-7.1-vmware",
    "iso_url": "{{env `ISO_URL`}}",
    "iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284",
    "vm_host": "{{ user `vm_host` }}",
    "vm_user": "{{ user `vm_user` }}",
    "vm_pass": "{{ env `vm_pass` }}",
    "vm_dc":   "{{ user `vm_dc` }}",
    "vm_cluster": "{{user `vm_cluster`}}",
    "vm_datastore": "{{user `vm_datastore`}}",
    "vm_folder": "{{user `vm_folder`}}",
    "vm_name": "{{user `vm_name`}}", 
    "vm_network": "{{user `vm_network`}}"
  },
  "builders": [
    {
      "headless": true,
      "type": "vmware-iso",
      "boot_command": [
        "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"
      ],
      "boot_wait": "10s",
      "disk_size": 8192,
      "guest_os_type": "centos-64",
      "http_directory": "http",
      "iso_url": "{{user `iso_url`}}",
      "iso_checksum_type": "sha256",
      "iso_checksum": "{{user `iso_sha256`}}",
      "ssh_username": "vagrant",
      "ssh_private_key_file": "vagrant_rsa",
      "ssh_port": 22,
      "ssh_wait_timeout": "10000s",
      "shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",
      "tools_upload_flavor": "linux",
      "tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",
      "vmx_data": {
        "memsize": "1024",
        "numvcpus": "1",
        "cpuid.coresPerSocket": "1"
      }
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",
      "override": {
        "vmware-iso": {
          "scripts": [
            "scripts/base.sh",
            "scripts/vmware.sh",
            "scripts/hgfs.sh",
            "scripts/cleanup.sh",
            "scripts/zerodisk.sh"
          ]
        }
      }
    }
  ],
  "post-processors": [
    {
      "type": "vagrant",
      "override": {
        "vmware": {
          "output": "centos-7.1-x64-vmware.box"
        }
      }
    },
    {
      "type": "vsphere",
      "host": "{{user `vm_host`}}",
      "username": "{{user `vm_user`}}",
      "password": "{{user `vm_pass`}}",
      "datacenter": "{{user `vm_dc`}}",
      "cluster": "{{user `vm_cluster`}}",
      "resource_pool": " ",
      "datastore": "{{user `vm_datastore`}}",
      "vm_folder": "{{user `vm_folder`}}",
      "vm_name": "{{user `vm_name`}}", 
      "vm_network": "{{user `vm_network`}}",
      "insecure" : "true"
    }
  ]
}

{

"variables": {

"vm_name": "centos-7.1-vmware",

"iso_url": "{{env `ISO_URL`}}",

"iso_sha256": "f90e4d28fa377669b2db16cbcb451fcb9a89d2460e3645993e30e137ac37d284",

"vm_host": "{{ user `vm_host` }}",

"vm_user": "{{ user `vm_user` }}",

"vm_pass": "{{ env `vm_pass` }}",

"vm_dc": "{{ user `vm_dc` }}",

"vm_cluster": "{{user `vm_cluster`}}",

"vm_datastore": "{{user `vm_datastore`}}",

"vm_folder": "{{user `vm_folder`}}",

"vm_name": "{{user `vm_name`}}",

"vm_network": "{{user `vm_network`}}"

"builders": [

{

"headless": true,

"type": "vmware-iso",

"boot_command": [

"<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"

"boot_wait": "10s",

"disk_size": 8192,

"guest_os_type": "centos-64",

"http_directory": "http",

"iso_url": "{{user `iso_url`}}",

"iso_checksum_type": "sha256",

"iso_checksum": "{{user `iso_sha256`}}",

"ssh_username": "vagrant",

"ssh_private_key_file": "vagrant_rsa",

"ssh_port": 22,

"ssh_wait_timeout": "10000s",

"shutdown_command": "echo '/sbin/halt -h -p' > /tmp/shutdown.sh; echo 'vagrant'|sudo -S sh '/tmp/shutdown.sh'",

"tools_upload_flavor": "linux",

"tools_upload_path": "/tmp/vmware_tools_{{.Flavor}}.iso",

"vmx_data": {

"memsize": "1024",

"numvcpus": "1",

"cpuid.coresPerSocket": "1"

}

"provisioners": [

{

"type": "shell",

"execute_command": "echo 'vagrant'|sudo -S sh '{{.Path}}'",

"override": {

"vmware-iso": {

"scripts": [

"scripts/base.sh",

"scripts/vmware.sh",

"scripts/hgfs.sh",

"scripts/cleanup.sh",

"scripts/zerodisk.sh"

]

}

"post-processors": [

{

"type": "vagrant",

"override": {

"vmware": {

"output": "centos-7.1-x64-vmware.box"

}

{

"type": "vsphere",

"host": "{{user `vm_host`}}",

"username": "{{user `vm_user`}}",

"password": "{{user `vm_pass`}}",

"datacenter": "{{user `vm_dc`}}",

"cluster": "{{user `vm_cluster`}}",

"resource_pool": " ",

"datastore": "{{user `vm_datastore`}}",

"vm_folder": "{{user `vm_folder`}}",

"vm_name": "{{user `vm_name`}}",

"vm_network": "{{user `vm_network`}}",

"insecure" : "true"

}

]

}

10) Re-Run Packer against the template.json file and watch it build the machine image an

$ packer build \
> -var ‘iso_url=/Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso’ \
> -var ‘vm_host=vc.example.com’ \
> -var ‘vm_user=john@example.com’ \
> -var ‘vm_pass=XXXXXXX’ \
> -var ‘vm_dc=vDC’ \
> -var ‘vm_cluster=Folder/Cluster’ \
> -var ‘vm_datastore=store’ \
> -var ‘vm_folder=Images’ \
> -var ‘vm_name=centos71’ \
> -var ‘vm_network=dvs-net1’ \
> -only=vmware-iso template.json
vmware-iso output will be in this color.

==> vmware-iso: Downloading or copying ISO
vmware-iso: Downloading or copying: file:///Users/john/iso/CentOS-7-x86_64-Minimal-1511.iso
==> vmware-iso: Creating virtual machine disk
…
…
==> vmware-iso: Running post-processor: vagrant
==> vmware-iso (vagrant): Creating Vagrant box for ‘vmware’ provider
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s001.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s002.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk-s003.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/disk.vmdk
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.nvram
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmsd
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vagrant): Copying: output-vmware-iso/packer-vmware-iso.vmxf
vmware-iso (vagrant): Compressing: Vagrantfile
vmware-iso (vagrant): Compressing: disk-s001.vmdk
vmware-iso (vagrant): Compressing: disk-s002.vmdk
vmware-iso (vagrant): Compressing: disk-s003.vmdk
vmware-iso (vagrant): Compressing: disk.vmdk
vmware-iso (vagrant): Compressing: metadata.json
vmware-iso (vagrant): Compressing: packer-vmware-iso.nvram
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmsd
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmx
vmware-iso (vagrant): Compressing: packer-vmware-iso.vmxf
==> vmware-iso: Running post-processor: vsphere
vmware-iso (vsphere): Uploading output-vmware-iso/packer-vmware-iso.vmx to vSphere
vmware-iso (vsphere): Opening VMX source: output-vmware-iso/packer-vmware-iso.vmx
vmware-iso (vsphere): Opening VI target: vi://john%40example.com@vc.exmaple.com:443/vDC/host/Cluster/Resources/
vmware-iso (vsphere): Deploying to VI: vi://john%40example.com@vc.example.com:443/vDC/host/Cluster/Resources/
Transfer Completed
vmware-iso (vsphere): Completed successfully
vmware-iso (vsphere):
Build ‘vmware-iso’ finished.

==> Builds finished. The artifacts of successful builds are:
–> vmware-iso: ‘vmware’ provider box: centos-7.1-x64-vmware.box
–> vmware-iso:

Now you should have the same image running on both Vagrant and vSphere.

DevOps, Packer, Vagrant centos, devops, infrastructure, linux, packer, rhel, vagrant, vsphere

Troubleshooting Secure Connectivity using SSL/TLS Certificates

August 24, 2015 John 1 comment

Have you had issues with an application complaining about SSL/TLS connectivity to a server?

If the server is using a valid third party signed certificate you should not see issues connecting from a web browser, but maybe you will issues connecting from the command line or from an application.

The way SSL/TLS work in short, is that the client(web browser, command line shell, application, etc) connects to the server and the server responds with a certificate.
This server certificate is signed by a Certificate Authority (CA) and the client checks if the CA is in its trusted Certificate Authorities local database to be able to trust the server certificate.

Web browsers have their own trusted Certifcate Authority databases or use the Operating Systems’ and that is why your browser does rarely complain if the server certificate is signed by a valid CA.

Flow:
Client —-> Server
Server — SSL/TLS certificate –> Client
Client checks server certificate against its CA database to trust or not

Now let’s test from the Linux command line:
// The items in RED are the errors we are getting
// The items in BOLD is the information we need, the certificate and the issuer

We are missing the Issuer (CA) certificate:
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3

…which is what we need to be able to trust the server certificate:
s:/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.example.com

[vagrant@vagrant ~]$ openssl s_client -connect site.example.com:443
CONNECTED(00000003)
depth=0 C = US, ST = California, L = City, O = Example Corporation, OU = CIT, CN = *.example.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = California, L = City, O = Example Corporation, OU = CIT, CN = *.example.com
verify error:num=27:certificate not trusted
verify return:1
depth=0 C = US, ST = California, L = City, O = Example Corporation, OU = CIT, CN = *.example.com
verify error:num=21:unable to verify the first certificate
verify return:1
—
Certificate chain
0 s:/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.example.com
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
Server certificate
—–BEGIN CERTIFICATE—–
XXXFTjCCBDagAwIBAgIQGtNpqqohGc6UzXNUYvmQMTANBgkqhkiG9w0BAQUFADCB
tTELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2Ug
YXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMm
VmVyaVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwHhcNMTUwNjI1
MDAwMDAwWhcNMTYwOTIzMjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgT
CkNhbGlmb3JuaWExFzAVBgNVBAcUDlJlZHdvb2QgU2hvcmVzMRswGQYDVQQKFBJP
cmFjbGUgQ29ycG9yYXRpb24xDDAKBgNVBAsUA0NJVDEgMB4GA1UEAxQXKi5vcmFj
bGVvdXRzb3VyY2luZy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQDTDH4+V9VK1ri9uh5zFpV6+VzpIYQOEHmd4Rg68uS3lzMV/xD+HBVEkenarpVP
uXSQnlP79hguSWfOpX1LpNjHeiy68pWMLMUuXmZiXS8BI0ctyGYb7LntAsGe6Y0N
YTBybpxS8szLPXQ6u+wa+6dfHxswEY6CUc+4wFyFgAVqWaKOKoJwaIUd//BoaGMv
E2Jj4ygnesolWWdqTWURsFS3X08P0nNKbXMCQoMvSp3f3rS+p+q/xVXqfJuAo7wZ
yTEBkTxWI2wDnjyWw3hrkfuxrdrs0YhmddHtchNTt3if3Q1LerDOaR3Hhc+H9fVB
cZYgsuNmSABBqQ30T0B8ZSz1AgMBAAGjggGDMIIBfzA5BgNVHREEMjAwghcqLm9y
YWNsZW91dHNvdXJjaW5nLmNvbYIVb3JhY2xlb3V0c291cmNpbmcuY29tMAkGA1Ud
EwQCMAAwDgYDVR0PAQH/BAQDAgWgMCsGA1UdHwQkMCIwIKAeoByGGmh0dHA6Ly9z
ZC5zeW1jYi5jb20vc2QuY3JsMGEGA1UdIARaMFgwVgYGZ4EMAQICMEwwIwYIKwYB
BQUHAgEWF2h0dHBzOi8vZC5zeW1jYi5jb20vY3BzMCUGCCsGAQUFBwICMBkMF2h0
dHBzOi8vZC5zeW1jYi5jb20vcnBhMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjAfBgNVHSMEGDAWgBQNRFwWU0TBgn4dIKsl9AFj2L55pTBXBggrBgEFBQcB
AQRLMEkwHwYIKwYBBQUHMAGGE2h0dHA6Ly9zZC5zeW1jZC5jb20wJgYIKwYBBQUH
MAKGGmh0dHA6Ly9zZC5zeW1jYi5jb20vc2QuY3J0MA0GCSqGSIb3DQEBBQUAA4IB
AQBrQknzr4wntvKOGc8YK6RfqaXS+r4XH34DlEPp2kFleNRaiJpo2DTVXr1a+WXQ
omrKl9j4Irbf0Jq4vI0XqYw5Ei8WsvqCMoFXqnCwY2MazYVHRF/IYvCWfjIcySGu
tR/RS+bvpN1+ATt8vNnW7imdca3LjkM0Fw2GVjmaX2CeINpCRkuQ/d28ob46o9D9
VF+/kjc2tVwfabQQCNykRlhYLJDmhKcVI5+swD10+JjddbSAsXpIQFy3FI77idvC
rngbwc0G36Rj3DvcoypUbXoFzsnisy58oieFWRzHPe9DVYh0o/uJj40NJDOLuyEA
EIV237/q67gIYEn6LM8CLXXX
—–END CERTIFICATE—–
subject=/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.example.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
No client certificate CA names sent
—
SSL handshake has read 1547 bytes and written 591 bytes
—
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : AES128-SHA
Session-ID: 6A6F797F1318EC872CBD4DD899493DB949E08F78A050580F70F9621ABB0ACE0C
Session-ID-ctx:
Master-Key: 4E12620FCE8592D4D272E58D9DC9C313D4A6D83B2956880279F58526526B9AB0AF8A9660AEB40F03FF46CDDFCB970A74
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1439626705
Timeout : 300 (sec)
Verify return code: 21 (unable to verify the first certificate)
—
DONE

# Get CA Issuer Certificate(s)

Now that we know we are missing the Issuer certificate we need to get it from the Certificate Authority website. In this specific case we need:
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3

Later you will have the exact same issue with the ‘VeriSign Class 3 Secure Server CA – G3’ certificate because it will be asking for its Issuer certificate ‘VeriSign Class 3 Public Primary Certification Authority – G5’, so you will need to get that Certificate as well.

I searched online for both and found them at:
https://www.tbs-certificates.co.uk/FAQ/en/565.html
https://www.tbs-certificates.co.uk/FAQ/en/599.html

Take everything from —–BEGIN CERTIFICATE—– to —–END CERTIFICATE—– and put them in files (e.g. verisign_class_3_secure_server_ca_g3.pem) and (e.g. verisign_class_3_secure_server_ca_g5.pem)

Having both files will ensure you have the entire Trust Chain:
*.example.com CERT trusted by ‘VeriSign Class 3 Secure Server CA – G3’ CERT trusted by ‘VeriSign Class 3 Public Primary Certification Authority – G5’

$ cat /tmp/ca_trust/verisign_class_3_secure_server_ca_g3.pem

1	$ cat /tmp/ca_trust/verisign_class_3_secure_server_ca_g3.pem

—–BEGIN CERTIFICATE—–
MIIF7DCCBNSgAwIBAgIQbsx6pacDIAm4zrz06VLUkTANBgkqhkiG9w0BAQUFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzUwHhcNMTAwMjA4MDAwMDAwWhcNMjAwMjA3MjM1OTU5WjCBtTEL
MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQg
aHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMmVmVy
aVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCxh4QfwgxF9byrJZenraI+nLr2wTm4i8rCrFbG
5btljkRPTc5v7QlK1K9OEJxoiy6Ve4mbE8riNDTB81vzSXtig0iBdNGIeGwCU/m8
f0MmV1gzgzszChew0E6RJK2GfWQS3HRKNKEdCuqWHQsV/KNLO85jiND4LQyUhhDK
tpo9yus3nABINYYpUHjoRWPNGUFP9ZXse5jUxHGzUL4os4+guVOc9cosI6n9FAbo
GLSa6Dxugf3kzTU2s1HTaewSulZub5tXxYsU5w7HnO1KVGrJTcW/EbGuHGeBy0RV
M5l/JJs/U0V/hhrzPPptf4H1uErT9YU3HLWm0AnkGHs4TvoPAgMBAAGjggHfMIIB
2zA0BggrBgEFBQcBAQQoMCYwJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLnZlcmlz
aWduLmNvbTASBgNVHRMBAf8ECDAGAQH/AgEAMHAGA1UdIARpMGcwZQYLYIZIAYb4
RQEHFwMwVjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL2Nw
czAqBggrBgEFBQcCAjAeGhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMDQG
A1UdHwQtMCswKaAnoCWGI2h0dHA6Ly9jcmwudmVyaXNpZ24uY29tL3BjYTMtZzUu
Y3JsMA4GA1UdDwEB/wQEAwIBBjBtBggrBgEFBQcBDARhMF+hXaBbMFkwVzBVFglp
bWFnZS9naWYwITAfMAcGBSsOAwIaBBSP5dMahqyNjmvDz4Bq1EgYLHsZLjAlFiNo
dHRwOi8vbG9nby52ZXJpc2lnbi5jb20vdnNsb2dvLmdpZjAoBgNVHREEITAfpB0w
GzEZMBcGA1UEAxMQVmVyaVNpZ25NUEtJLTItNjAdBgNVHQ4EFgQUDURcFlNEwYJ+
HSCrJfQBY9i+eaUwHwYDVR0jBBgwFoAUf9Nlp8Ld7LvwMAnzQzn6Aq8zMTMwDQYJ
KoZIhvcNAQEFBQADggEBAAyDJO/dwwzZWJz+NrbrioBL0aP3nfPMU++CnqOh5pfB
WJ11bOAdG0z60cEtBcDqbrIicFXZIDNAMwfCZYP6j0M3m+oOmmxw7vacgDvZN/R6
bezQGH1JSsqZxxkoor7YdyT3hSaGbYcFQEFn0Sc67dxIHSLNCwuLvPSxe/20majp
dirhGi2HbnTTiN0eIsbfFrYrghQKlFzyUOyvzv9iNw2tZdMGQVPtAhTItVgooazg
W+yzf5VK+wPIrSbb5mZ4EkrZn0L74ZjmQoObj49nJOhhGbXdzbULJgWOw27EyHW4
Rs/iGAZeqa6ogZpHFt4MKGwlJ7net4RYxh84HqTEy2Y=
—–END CERTIFICATE—–

$ cat /tmp/ca_certs/verisign_class_3_secure_server_ca_g5.pem

1	$ cat /tmp/ca_certs/verisign_class_3_secure_server_ca_g5.pem

—–BEGIN CERTIFICATE—–
MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzUwHhcNMDYxMTA4MDAwMDAwWhcNMzYwNzE2MjM1OTU5WjCByjEL
MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2ln
biwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJp
U2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9y
aXR5IC0gRzUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1
nmAMqudLO07cfLw8RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbex
t0uz/o9+B1fs70PbZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIz
SdhDY2pSS9KP6HBRTdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQG
BO+QueQA5N06tRn/Arr0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+
rCpSx4/VBEnkjWNHiDxpg8v+R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/
NIeWiu5T6CUVAgMBAAGjgbIwga8wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8E
BAMCAQYwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1hZ2UvZ2lmMCEwHzAH
BgUrDgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xvZ28udmVy
aXNpZ24uY29tL3ZzbG9nby5naWYwHQYDVR0OBBYEFH/TZafC3ey78DAJ80M5+gKv
MzEzMA0GCSqGSIb3DQEBBQUAA4IBAQCTJEowX2LP2BqYLz3q3JktvXf2pXkiOOzE
p6B4Eq1iDkVwZMXnl2YtmAl+X6/WzChl8gGqCBpH3vn5fJJaCGkgDdk+bW48DW7Y
5gaRQBi5+MHt39tBquCWIMnNZBU4gcmU7qKEKQsTb47bDN0lAtukixlE0kF6BWlK
WE9gyn6CagsCqiUXObXbf+eEZSqVir2G3l6BFoMtEMze/aiCKm0oHw0LxOXnGiYZ
4fQRbxC1lfznQgUy286dUV4otp6F01vvpX1FQHKOtw5rDgb7MzVIcbidJ4vEZV8N
hnacRHr2lVz2XTIIM6RUthg/aFzyQkqFOFSDX9HoLPKsEdao7WNq
—–END CERTIFICATE—–

# Check CA Issuer Certificates Details

$ openssl x509 -in verisign_class_3_secure_server_ca_g3.pem -text

1	$ openssl x509 -in verisign_class_3_secure_server_ca_g3.pem -text

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
6e:cc:7a:a5:a7:03:20:09:b8:ce:bc:f4:e9:52:d4:91
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, O=VeriSign, Inc., OU=VeriSign Trust Network, OU=(c) 2006 VeriSign, Inc. – For authorized use only, CN=VeriSign Class 3 Public Primary Certification Authority – G5
Validity
Not Before: Feb 8 00:00:00 2010 GMT
Not After : Feb 7 23:59:59 2020 GMT
Subject: C=US, O=VeriSign, Inc., OU=VeriSign Trust Network, OU=Terms of use at https://www.verisign.com/rpa (c)10, CN=VeriSign Class 3 Secure Server CA – G3
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (2048 bit)
Modulus (2048 bit):
00:b1:87:84:1f:c2:0c:45:f5:bc:ab:25:97:a7:ad:
a2:3e:9c:ba:f6:c1:39:b8:8b:ca:c2:ac:56:c6:e5:
bb:65:8e:44:4f:4d:ce:6f:ed:09:4a:d4:af:4e:10:
9c:68:8b:2e:95:7b:89:9b:13:ca:e2:34:34:c1:f3:
5b:f3:49:7b:62:83:48:81:74:d1:88:78:6c:02:53:
f9:bc:7f:43:26:57:58:33:83:3b:33:0a:17:b0:d0:
4e:91:24:ad:86:7d:64:12:dc:74:4a:34:a1:1d:0a:
ea:96:1d:0b:15:fc:a3:4b:3b:ce:63:88:d0:f8:2d:
0c:94:86:10:ca:b6:9a:3d:ca:eb:37:9c:00:48:35:
86:29:50:78:e8:45:63:cd:19:41:4f:f5:95:ec:7b:
98:d4:c4:71:b3:50:be:28:b3:8f:a0:b9:53:9c:f5:
ca:2c:23:a9:fd:14:06:e8:18:b4:9a:e8:3c:6e:81:
fd:e4:cd:35:36:b3:51:d3:69:ec:12:ba:56:6e:6f:
9b:57:c5:8b:14:e7:0e:c7:9c:ed:4a:54:6a:c9:4d:
c5:bf:11:b1:ae:1c:67:81:cb:44:55:33:99:7f:24:
9b:3f:53:45:7f:86:1a:f3:3c:fa:6d:7f:81:f5:b8:
4a:d3:f5:85:37:1c:b5:a6:d0:09:e4:18:7b:38:4e:
fa:0f
Exponent: 65537 (0x10001)
X509v3 extensions:
Authority Information Access:
OCSP – URI:http://ocsp.verisign.com

X509v3 Basic Constraints: critical
CA:TRUE, pathlen:0
X509v3 Certificate Policies:
Policy: 2.16.840.1.113733.1.7.23.3
CPS: https://www.verisign.com/cps
User Notice:
Explicit Text: https://www.verisign.com/rpa

X509v3 CRL Distribution Points:
URI:http://crl.verisign.com/pca3-g5.crl

X509v3 Key Usage: critical
Certificate Sign, CRL Sign
1.3.6.1.5.5.7.1.12:
0_.].[0Y0W0U..image/gif0!0.0…+…………..k…j.H.,{..0%.#http://logo.verisign.com/vslogo.gif
X509v3 Subject Alternative Name:
DirName:/CN=VeriSignMPKI-2-6
X509v3 Subject Key Identifier:
0D:44:5C:16:53:44:C1:82:7E:1D:20:AB:25:F4:01:63:D8:BE:79:A5
X509v3 Authority Key Identifier:
keyid:7F:D3:65:A7:C2:DD:EC:BB:F0:30:09:F3:43:39:FA:02:AF:33:31:33

Signature Algorithm: sha1WithRSAEncryption
0c:83:24:ef:dd:c3:0c:d9:58:9c:fe:36:b6:eb:8a:80:4b:d1:
a3:f7:9d:f3:cc:53:ef:82:9e:a3:a1:e6:97:c1:58:9d:75:6c:
e0:1d:1b:4c:fa:d1:c1:2d:05:c0:ea:6e:b2:22:70:55:d9:20:
33:40:33:07:c2:65:83:fa:8f:43:37:9b:ea:0e:9a:6c:70:ee:
f6:9c:80:3b:d9:37:f4:7a:6d:ec:d0:18:7d:49:4a:ca:99:c7:
19:28:a2:be:d8:77:24:f7:85:26:86:6d:87:05:40:41:67:d1:
27:3a:ed:dc:48:1d:22:cd:0b:0b:8b:bc:f4:b1:7b:fd:b4:99:
a8:e9:76:2a:e1:1a:2d:87:6e:74:d3:88:dd:1e:22:c6:df:16:
b6:2b:82:14:0a:94:5c:f2:50:ec:af:ce:ff:62:37:0d:ad:65:
d3:06:41:53:ed:02:14:c8:b5:58:28:a1:ac:e0:5b:ec:b3:7f:
95:4a:fb:03:c8:ad:26:db:e6:66:78:12:4a:d9:9f:42:fb:e1:
98:e6:42:83:9b:8f:8f:67:24:e8:61:19:b5:dd:cd:b5:0b:26:
05:8e:c3:6e:c4:c8:75:b8:46:cf:e2:18:06:5e:a9:ae:a8:81:
9a:47:16:de:0c:28:6c:25:27:b9:de:b7:84:58:c6:1f:38:1e:
a4:c4:cb:66
—–BEGIN CERTIFICATE—–
MIIF7DCCBNSgAwIBAgIQbsx6pacDIAm4zrz06VLUkTANBgkqhkiG9w0BAQUFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzUwHhcNMTAwMjA4MDAwMDAwWhcNMjAwMjA3MjM1OTU5WjCBtTEL
MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2UgYXQg
aHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMmVmVy
aVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCxh4QfwgxF9byrJZenraI+nLr2wTm4i8rCrFbG
5btljkRPTc5v7QlK1K9OEJxoiy6Ve4mbE8riNDTB81vzSXtig0iBdNGIeGwCU/m8
f0MmV1gzgzszChew0E6RJK2GfWQS3HRKNKEdCuqWHQsV/KNLO85jiND4LQyUhhDK
tpo9yus3nABINYYpUHjoRWPNGUFP9ZXse5jUxHGzUL4os4+guVOc9cosI6n9FAbo
GLSa6Dxugf3kzTU2s1HTaewSulZub5tXxYsU5w7HnO1KVGrJTcW/EbGuHGeBy0RV
M5l/JJs/U0V/hhrzPPptf4H1uErT9YU3HLWm0AnkGHs4TvoPAgMBAAGjggHfMIIB
2zA0BggrBgEFBQcBAQQoMCYwJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLnZlcmlz
aWduLmNvbTASBgNVHRMBAf8ECDAGAQH/AgEAMHAGA1UdIARpMGcwZQYLYIZIAYb4
RQEHFwMwVjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL2Nw
czAqBggrBgEFBQcCAjAeGhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20vcnBhMDQG
A1UdHwQtMCswKaAnoCWGI2h0dHA6Ly9jcmwudmVyaXNpZ24uY29tL3BjYTMtZzUu
Y3JsMA4GA1UdDwEB/wQEAwIBBjBtBggrBgEFBQcBDARhMF+hXaBbMFkwVzBVFglp
bWFnZS9naWYwITAfMAcGBSsOAwIaBBSP5dMahqyNjmvDz4Bq1EgYLHsZLjAlFiNo
dHRwOi8vbG9nby52ZXJpc2lnbi5jb20vdnNsb2dvLmdpZjAoBgNVHREEITAfpB0w
GzEZMBcGA1UEAxMQVmVyaVNpZ25NUEtJLTItNjAdBgNVHQ4EFgQUDURcFlNEwYJ+
HSCrJfQBY9i+eaUwHwYDVR0jBBgwFoAUf9Nlp8Ld7LvwMAnzQzn6Aq8zMTMwDQYJ
KoZIhvcNAQEFBQADggEBAAyDJO/dwwzZWJz+NrbrioBL0aP3nfPMU++CnqOh5pfB
WJ11bOAdG0z60cEtBcDqbrIicFXZIDNAMwfCZYP6j0M3m+oOmmxw7vacgDvZN/R6
bezQGH1JSsqZxxkoor7YdyT3hSaGbYcFQEFn0Sc67dxIHSLNCwuLvPSxe/20majp
dirhGi2HbnTTiN0eIsbfFrYrghQKlFzyUOyvzv9iNw2tZdMGQVPtAhTItVgooazg
W+yzf5VK+wPIrSbb5mZ4EkrZn0L74ZjmQoObj49nJOhhGbXdzbULJgWOw27EyHW4
Rs/iGAZeqa6ogZpHFt4MKGwlJ7net4RYxh84HqTEy2Y=
—–END CERTIFICATE—–

$ openssl x509 -in verisign_class_3_secure_server_ca_g5.pem -text

1	$ openssl x509 -in verisign_class_3_secure_server_ca_g5.pem -text

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
18:da:d1:9e:26:7d:e8:bb:4a:21:58:cd:cc:6b:3b:4a
Signature Algorithm: sha1WithRSAEncryption
Issuer: C=US, O=VeriSign, Inc., OU=VeriSign Trust Network, OU=(c) 2006 VeriSign, Inc. – For authorized use only, CN=VeriSign Class 3 Public Primary Certification Authority – G5
Validity
Not Before: Nov 8 00:00:00 2006 GMT
Not After : Jul 16 23:59:59 2036 GMT
Subject: C=US, O=VeriSign, Inc., OU=VeriSign Trust Network, OU=(c) 2006 VeriSign, Inc. – For authorized use only, CN=VeriSign Class 3 Public Primary Certification Authority – G5
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:af:24:08:08:29:7a:35:9e:60:0c:aa:e7:4b:3b:
4e:dc:7c:bc:3c:45:1c:bb:2b:e0:fe:29:02:f9:57:
08:a3:64:85:15:27:f5:f1:ad:c8:31:89:5d:22:e8:
2a:aa:a6:42:b3:8f:f8:b9:55:b7:b1:b7:4b:b3:fe:
8f:7e:07:57:ec:ef:43:db:66:62:15:61:cf:60:0d:
a4:d8:de:f8:e0:c3:62:08:3d:54:13:eb:49:ca:59:
54:85:26:e5:2b:8f:1b:9f:eb:f5:a1:91:c2:33:49:
d8:43:63:6a:52:4b:d2:8f:e8:70:51:4d:d1:89:69:
7b:c7:70:f6:b3:dc:12:74:db:7b:5d:4b:56:d3:96:
bf:15:77:a1:b0:f4:a2:25:f2:af:1c:92:67:18:e5:
f4:06:04:ef:90:b9:e4:00:e4:dd:3a:b5:19:ff:02:
ba:f4:3c:ee:e0:8b:eb:37:8b:ec:f4:d7:ac:f2:f6:
f0:3d:af:dd:75:91:33:19:1d:1c:40:cb:74:24:19:
21:93:d9:14:fe:ac:2a:52:c7:8f:d5:04:49:e4:8d:
63:47:88:3c:69:83:cb:fe:47:bd:2b:7e:4f:c5:95:
ae:0e:9d:d4:d1:43:c0:67:73:e3:14:08:7e:e5:3f:
9f:73:b8:33:0a:cf:5d:3f:34:87:96:8a:ee:53:e8:
25:15
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
1.3.6.1.5.5.7.1.12:
0_.].[0Y0W0U..image/gif0!0.0…+…………..k…j.H.,{..0%.#http://logo.verisign.com/vslogo.gif
X509v3 Subject Key Identifier:
7F:D3:65:A7:C2:DD:EC:BB:F0:30:09:F3:43:39:FA:02:AF:33:31:33
Signature Algorithm: sha1WithRSAEncryption
93:24:4a:30:5f:62:cf:d8:1a:98:2f:3d:ea:dc:99:2d:bd:77:
f6:a5:79:22:38:ec:c4:a7:a0:78:12:ad:62:0e:45:70:64:c5:
e7:97:66:2d:98:09:7e:5f:af:d6:cc:28:65:f2:01:aa:08:1a:
47:de:f9:f9:7c:92:5a:08:69:20:0d:d9:3e:6d:6e:3c:0d:6e:
d8:e6:06:91:40:18:b9:f8:c1:ed:df:db:41:aa:e0:96:20:c9:
cd:64:15:38:81:c9:94:ee:a2:84:29:0b:13:6f:8e:db:0c:dd:
25:02:db:a4:8b:19:44:d2:41:7a:05:69:4a:58:4f:60:ca:7e:
82:6a:0b:02:aa:25:17:39:b5:db:7f:e7:84:65:2a:95:8a:bd:
86:de:5e:81:16:83:2d:10:cc:de:fd:a8:82:2a:6d:28:1f:0d:
0b:c4:e5:e7:1a:26:19:e1:f4:11:6f:10:b5:95:fc:e7:42:05:
32:db:ce:9d:51:5e:28:b6:9e:85:d3:5b:ef:a5:7d:45:40:72:
8e:b7:0e:6b:0e:06:fb:33:35:48:71:b8:9d:27:8b:c4:65:5f:
0d:86:76:9c:44:7a:f6:95:5c:f6:5d:32:08:33:a4:54:b6:18:
3f:68:5c:f2:42:4a:85:38:54:83:5f:d1:e8:2c:f2:ac:11:d6:
a8:ed:63:6a
—–BEGIN CERTIFICATE—–
MIIE0zCCA7ugAwIBAgIQGNrRniZ96LtKIVjNzGs7SjANBgkqhkiG9w0BAQUFADCB
yjELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJp
U2lnbiwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxW
ZXJpU2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0
aG9yaXR5IC0gRzUwHhcNMDYxMTA4MDAwMDAwWhcNMzYwNzE2MjM1OTU5WjCByjEL
MAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQLExZW
ZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTowOAYDVQQLEzEoYykgMjAwNiBWZXJpU2ln
biwgSW5jLiAtIEZvciBhdXRob3JpemVkIHVzZSBvbmx5MUUwQwYDVQQDEzxWZXJp
U2lnbiBDbGFzcyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9y
aXR5IC0gRzUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvJAgIKXo1
nmAMqudLO07cfLw8RRy7K+D+KQL5VwijZIUVJ/XxrcgxiV0i6CqqpkKzj/i5Vbex
t0uz/o9+B1fs70PbZmIVYc9gDaTY3vjgw2IIPVQT60nKWVSFJuUrjxuf6/WhkcIz
SdhDY2pSS9KP6HBRTdGJaXvHcPaz3BJ023tdS1bTlr8Vd6Gw9KIl8q8ckmcY5fQG
BO+QueQA5N06tRn/Arr0PO7gi+s3i+z016zy9vA9r911kTMZHRxAy3QkGSGT2RT+
rCpSx4/VBEnkjWNHiDxpg8v+R70rfk/Fla4OndTRQ8Bnc+MUCH7lP59zuDMKz10/
NIeWiu5T6CUVAgMBAAGjgbIwga8wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8E
BAMCAQYwbQYIKwYBBQUHAQwEYTBfoV2gWzBZMFcwVRYJaW1hZ2UvZ2lmMCEwHzAH
BgUrDgMCGgQUj+XTGoasjY5rw8+AatRIGCx7GS4wJRYjaHR0cDovL2xvZ28udmVy
aXNpZ24uY29tL3ZzbG9nby5naWYwHQYDVR0OBBYEFH/TZafC3ey78DAJ80M5+gKv
MzEzMA0GCSqGSIb3DQEBBQUAA4IBAQCTJEowX2LP2BqYLz3q3JktvXf2pXkiOOzE
p6B4Eq1iDkVwZMXnl2YtmAl+X6/WzChl8gGqCBpH3vn5fJJaCGkgDdk+bW48DW7Y
5gaRQBi5+MHt39tBquCWIMnNZBU4gcmU7qKEKQsTb47bDN0lAtukixlE0kF6BWlK
WE9gyn6CagsCqiUXObXbf+eEZSqVir2G3l6BFoMtEMze/aiCKm0oHw0LxOXnGiYZ
4fQRbxC1lfznQgUy286dUV4otp6F01vvpX1FQHKOtw5rDgb7MzVIcbidJ4vEZV8N
hnacRHr2lVz2XTIIM6RUthg/aFzyQkqFOFSDX9HoLPKsEdao7WNq
—–END CERTIFICATE—–

# Trust the CA Issuer Certificates

There are several ways to do it, see below for some and their pros/cons.

1) Put the CA Issuer certificates in pem format in a folder, and use them directly from the application

This is good for a quick test, but I don’t recommended it as standard operating procedure because you have to manage each set of certs and have the applications point to hardcoded paths.

// Create dir to hold certs

mkdir /tmp/ca_certs

1	mkdir /tmp/ca_certs

// Put the CA issuer cert in the directory

mv verisign_class_3_secure_server_ca_g3.pem /tmp/ca_certs
mv verisign_class_3_secure_server_ca_g5.pem /tmp/ca_certs

1 2	mv verisign_class_3_secure_server_ca_g3.pem /tmp/ca_certs mv verisign_class_3_secure_server_ca_g5.pem /tmp/ca_certs

// Create symbolic links needed

$ c_rehash /tmp/ca_certs/
Doing /tmp/ca_certs/
verisign_class_3_secure_server_ca_g3.pem => 46117fcc.0
verisign_class_3_secure_server_ca_g5.pem => b204d74a.0

$ c_rehash /tmp/ca_certs/

Doing /tmp/ca_certs/

verisign_class_3_secure_server_ca_g3.pem => 46117fcc.0

verisign_class_3_secure_server_ca_g5.pem => b204d74a.0

// Verify successful connectivity by passing the CApath

$ openssl s_client -CApath /tmp/ca_trust/ -connect site.example.com:443

1	$ openssl s_client -CApath /tmp/ca_trust/ -connect site.example.com:443

CONNECTED(00000003)
depth=2 C = US, O = “VeriSign, Inc.”, OU = VeriSign Trust Network, OU = “(c) 2006 VeriSign, Inc. – For authorized use only”, CN = VeriSign Class 3 Public Primary Certification Authority – G5
verify return:1
depth=1 C = US, O = “VeriSign, Inc.”, OU = VeriSign Trust Network, OU = Terms of use at https://www.verisign.com/rpa (c)10, CN = VeriSign Class 3 Secure Server CA – G3
verify return:1
depth=0 C = US, ST = California, L = City, O = Example Corporation, OU = CIT, CN = *.Example.com
verify return:1
—
Certificate chain
0 s:/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.Example.com
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
Server certificate
—–BEGIN CERTIFICATE—–
MIIFTjCCBDagAwIBAgIQGtNpqqohGc6UzXNUYvmQMTANBgkqhkiG9w0BAQUFADCB
tTELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2Ug
YXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMm
VmVyaVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwHhcNMTUwNjI1
MDAwMDAwWhcNMTYwOTIzMjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgT
CkNhbGlmb3JuaWExFzAVBgNVBAcUDlJlZHdvb2QgU2hvcmVzMRswGQYDVQQKFBJP
cmFjbGUgQ29ycG9yYXRpb24xDDAKBgNVBAsUA0NJVDEgMB4GA1UEAxQXKi5vcmFj
bGVvdXRzb3VyY2luZy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQDTDH4+V9VK1ri9uh5zFpV6+VzpIYQOEHmd4Rg68uS3lzMV/xD+HBVEkenarpVP
uXSQnlP79hguSWfOpX1LpNjHeiy68pWMLMUuXmZiXS8BI0ctyGYb7LntAsGe6Y0N
YTBybpxS8szLPXQ6u+wa+6dfHxswEY6CUc+4wFyFgAVqWaKOKoJwaIUd//BoaGMv
E2Jj4ygnesolWWdqTWURsFS3X08P0nNKbXMCQoMvSp3f3rS+p+q/xVXqfJuAo7wZ
yTEBkTxWI2wDnjyWw3hrkfuxrdrs0YhmddHtchNTt3if3Q1LerDOaR3Hhc+H9fVB
cZYgsuNmSABBqQ30T0B8ZSz1AgMBAAGjggGDMIIBfzA5BgNVHREEMjAwghcqLm9y
YWNsZW91dHNvdXJjaW5nLmNvbYIVb3JhY2xlb3V0c291cmNpbmcuY29tMAkGA1Ud
EwQCMAAwDgYDVR0PAQH/BAQDAgWgMCsGA1UdHwQkMCIwIKAeoByGGmh0dHA6Ly9z
ZC5zeW1jYi5jb20vc2QuY3JsMGEGA1UdIARaMFgwVgYGZ4EMAQICMEwwIwYIKwYB
BQUHAgEWF2h0dHBzOi8vZC5zeW1jYi5jb20vY3BzMCUGCCsGAQUFBwICMBkMF2h0
dHBzOi8vZC5zeW1jYi5jb20vcnBhMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjAfBgNVHSMEGDAWgBQNRFwWU0TBgn4dIKsl9AFj2L55pTBXBggrBgEFBQcB
AQRLMEkwHwYIKwYBBQUHMAGGE2h0dHA6Ly9zZC5zeW1jZC5jb20wJgYIKwYBBQUH
MAKGGmh0dHA6Ly9zZC5zeW1jYi5jb20vc2QuY3J0MA0GCSqGSIb3DQEBBQUAA4IB
AQBrQknzr4wntvKOGc8YK6RfqaXS+r4XH34DlEPp2kFleNRaiJpo2DTVXr1a+WXQ
omrKl9j4Irbf0Jq4vI0XqYw5Ei8WsvqCMoFXqnCwY2MazYVHRF/IYvCWfjIcySGu
tR/RS+bvpN1+ATt8vNnW7imdca3LjkM0Fw2GVjmaX2CeINpCRkuQ/d28ob46o9D9
VF+/kjc2tVwfabQQCNykRlhYLJDmhKcVI5+swD10+JjddbSAsXpIQFy3FI77idvC
rngbwc0G36Rj3DvcoypUbXoFzsnisy58oieFWRzHPe9DVYh0o/uJj40NJDOLuyEA
EIV237/q67gIYEn6LM8CLah9
—–END CERTIFICATE—–
subject=/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.Example.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
No client certificate CA names sent
—
SSL handshake has read 1547 bytes and written 591 bytes
—
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : AES128-SHA
Session-ID: 6A6F797F1318E8462CBD4DD899493DB949E08F78A0505C4E70F9621ABB0AA280
Session-ID-ctx:
Master-Key: 50D309E782DF245FA36BD1028197459FE1A56B976D947D2994BBB75A0754D18AD44A90F29AC818DB28562F0E9E2E5442
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1439631686
Timeout : 300 (sec)
Verify return code: 0 (ok)
—

2) Append the CA Issuer Certificates to the OS certificate bundle (/etc/pki/tls/certs/ca-bundle.crt)

This is not recommended as standard operating procedure because patches/updates will overwrite the certificate bundle file (/etc/pki/tls/certs/ca-bundle.crt) and the certificates you had added wont be trusted anymore

$ sudo bash -c  'cat /tmp/ca_certs/verisign_class_3_secure_server_ca_g{3,5}* >> /etc/pki/tls/certs/ca-bundle.crt'

1	$ sudo bash -c 'cat /tmp/ca_certs/verisign_class_3_secure_server_ca_g{3,5}* >> /etc/pki/tls/certs/ca-bundle.crt'

// Verify successful connectivity

$ openssl s_client -connect site.example.com:443

1	$ openssl s_client -connect site.example.com:443

CONNECTED(00000003)
depth=2 C = US, O = “VeriSign, Inc.”, OU = VeriSign Trust Network, OU = “(c) 2006 VeriSign, Inc. – For authorized use only”, CN = VeriSign Class 3 Public Primary Certification Authority – G5
verify return:1
depth=1 C = US, O = “VeriSign, Inc.”, OU = VeriSign Trust Network, OU = Terms of use at https://www.verisign.com/rpa (c)10, CN = VeriSign Class 3 Secure Server CA – G3
verify return:1
depth=0 C = US, ST = California, L = City, O = Example Corporation, OU = CIT, CN = *.Example.com
verify return:1
—
Certificate chain
0 s:/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.Example.com
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
Server certificate
—–BEGIN CERTIFICATE—–
MIIFTjCCBDagAwIBAgIQGtNpqqohGc6UzXNUYvmQMTANBgkqhkiG9w0BAQUFADCB
tTELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2Ug
YXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMm
VmVyaVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwHhcNMTUwNjI1
MDAwMDAwWhcNMTYwOTIzMjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgT
CkNhbGlmb3JuaWExFzAVBgNVBAcUDlJlZHdvb2QgU2hvcmVzMRswGQYDVQQKFBJP
cmFjbGUgQ29ycG9yYXRpb24xDDAKBgNVBAsUA0NJVDEgMB4GA1UEAxQXKi5vcmFj
bGVvdXRzb3VyY2luZy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQDTDH4+V9VK1ri9uh5zFpV6+VzpIYQOEHmd4Rg68uS3lzMV/xD+HBVEkenarpVP
uXSQnlP79hguSWfOpX1LpNjHeiy68pWMLMUuXmZiXS8BI0ctyGYb7LntAsGe6Y0N
YTBybpxS8szLPXQ6u+wa+6dfHxswEY6CUc+4wFyFgAVqWaKOKoJwaIUd//BoaGMv
E2Jj4ygnesolWWdqTWURsFS3X08P0nNKbXMCQoMvSp3f3rS+p+q/xVXqfJuAo7wZ
yTEBkTxWI2wDnjyWw3hrkfuxrdrs0YhmddHtchNTt3if3Q1LerDOaR3Hhc+H9fVB
cZYgsuNmSABBqQ30T0B8ZSz1AgMBAAGjggGDMIIBfzA5BgNVHREEMjAwghcqLm9y
YWNsZW91dHNvdXJjaW5nLmNvbYIVb3JhY2xlb3V0c291cmNpbmcuY29tMAkGA1Ud
EwQCMAAwDgYDVR0PAQH/BAQDAgWgMCsGA1UdHwQkMCIwIKAeoByGGmh0dHA6Ly9z
ZC5zeW1jYi5jb20vc2QuY3JsMGEGA1UdIARaMFgwVgYGZ4EMAQICMEwwIwYIKwYB
BQUHAgEWF2h0dHBzOi8vZC5zeW1jYi5jb20vY3BzMCUGCCsGAQUFBwICMBkMF2h0
dHBzOi8vZC5zeW1jYi5jb20vcnBhMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjAfBgNVHSMEGDAWgBQNRFwWU0TBgn4dIKsl9AFj2L55pTBXBggrBgEFBQcB
AQRLMEkwHwYIKwYBBQUHMAGGE2h0dHA6Ly9zZC5zeW1jZC5jb20wJgYIKwYBBQUH
MAKGGmh0dHA6Ly9zZC5zeW1jYi5jb20vc2QuY3J0MA0GCSqGSIb3DQEBBQUAA4IB
AQBrQknzr4wntvKOGc8YK6RfqaXS+r4XH34DlEPp2kFleNRaiJpo2DTVXr1a+WXQ
omrKl9j4Irbf0Jq4vI0XqYw5Ei8WsvqCMoFXqnCwY2MazYVHRF/IYvCWfjIcySGu
tR/RS+bvpN1+ATt8vNnW7imdca3LjkM0Fw2GVjmaX2CeINpCRkuQ/d28ob46o9D9
VF+/kjc2tVwfabQQCNykRlhYLJDmhKcVI5+swD10+JjddbSAsXpIQFy3FI77idvC
rngbwc0G36Rj3DvcoypUbXoFzsnisy58oieFWRzHPe9DVYh0o/uJj40NJDOLuyEA
EIV237/q67gIYEn6LM8CLah9
—–END CERTIFICATE—–
subject=/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.Example.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
No client certificate CA names sent
—
SSL handshake has read 1547 bytes and written 591 bytes
—
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : AES128-SHA
Session-ID: 6A6F797F1318E8462CBD4DD899493DB949E08F78A0505C4E70F9621ABB0AA280
Session-ID-ctx:
Master-Key: 50D309E782DF245FA36BD1028197459FE1A56B976D947D2994BBB75A0754D18AD44A90F29AC818DB28562F0E9E2E5442
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1439631686
Timeout : 300 (sec)
Verify return code: 0 (ok)
—

3) Use the SharedSystemCertificates (*** Recommended ***)
This is the recommended way of adding trusted CA certificates to your RHEL/CentOS systems.
The idea is to make NSS, GnuTLS, OpenSSL and Java share a default source for retrieving system certificate anchors and black list information, details @ https://fedoraproject.org/wiki/Features/SharedSystemCertificates

// Enable the CA trust capability

/usr/bin/update-ca-trust enable

1	/usr/bin/update-ca-trust enable

// Copy/Move the Issuer CA certificates to /etc/pki/ca-trust/source/anchors/

$ sudo cp /tmp/ca_certs/verisign_class_3_secure_server_ca_g{3,5}* /etc/pki/ca-trust/source/anchors/

1	$ sudo cp /tmp/ca_certs/verisign_class_3_secure_server_ca_g{3,5}* /etc/pki/ca-trust/source/anchors/

// Update the CA trust

/usr/bin/update-ca-trust extract

1	/usr/bin/update-ca-trust extract

// Verify successful connectivity

$ openssl s_client -connect site.example.com:443

1	$ openssl s_client -connect site.example.com:443

CONNECTED(00000003)
depth=2 C = US, O = “VeriSign, Inc.”, OU = VeriSign Trust Network, OU = “(c) 2006 VeriSign, Inc. – For authorized use only”, CN = VeriSign Class 3 Public Primary Certification Authority – G5
verify return:1
depth=1 C = US, O = “VeriSign, Inc.”, OU = VeriSign Trust Network, OU = Terms of use at https://www.verisign.com/rpa (c)10, CN = VeriSign Class 3 Secure Server CA – G3
verify return:1
depth=0 C = US, ST = California, L = City, O = Example Corporation, OU = CIT, CN = *.Example.com
verify return:1
—
Certificate chain
0 s:/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.Example.com
i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
Server certificate
—–BEGIN CERTIFICATE—–
MIIFTjCCBDagAwIBAgIQGtNpqqohGc6UzXNUYvmQMTANBgkqhkiG9w0BAQUFADCB
tTELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMR8wHQYDVQQL
ExZWZXJpU2lnbiBUcnVzdCBOZXR3b3JrMTswOQYDVQQLEzJUZXJtcyBvZiB1c2Ug
YXQgaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYSAoYykxMDEvMC0GA1UEAxMm
VmVyaVNpZ24gQ2xhc3MgMyBTZWN1cmUgU2VydmVyIENBIC0gRzMwHhcNMTUwNjI1
MDAwMDAwWhcNMTYwOTIzMjM1OTU5WjCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgT
CkNhbGlmb3JuaWExFzAVBgNVBAcUDlJlZHdvb2QgU2hvcmVzMRswGQYDVQQKFBJP
cmFjbGUgQ29ycG9yYXRpb24xDDAKBgNVBAsUA0NJVDEgMB4GA1UEAxQXKi5vcmFj
bGVvdXRzb3VyY2luZy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB
AQDTDH4+V9VK1ri9uh5zFpV6+VzpIYQOEHmd4Rg68uS3lzMV/xD+HBVEkenarpVP
uXSQnlP79hguSWfOpX1LpNjHeiy68pWMLMUuXmZiXS8BI0ctyGYb7LntAsGe6Y0N
YTBybpxS8szLPXQ6u+wa+6dfHxswEY6CUc+4wFyFgAVqWaKOKoJwaIUd//BoaGMv
E2Jj4ygnesolWWdqTWURsFS3X08P0nNKbXMCQoMvSp3f3rS+p+q/xVXqfJuAo7wZ
yTEBkTxWI2wDnjyWw3hrkfuxrdrs0YhmddHtchNTt3if3Q1LerDOaR3Hhc+H9fVB
cZYgsuNmSABBqQ30T0B8ZSz1AgMBAAGjggGDMIIBfzA5BgNVHREEMjAwghcqLm9y
YWNsZW91dHNvdXJjaW5nLmNvbYIVb3JhY2xlb3V0c291cmNpbmcuY29tMAkGA1Ud
EwQCMAAwDgYDVR0PAQH/BAQDAgWgMCsGA1UdHwQkMCIwIKAeoByGGmh0dHA6Ly9z
ZC5zeW1jYi5jb20vc2QuY3JsMGEGA1UdIARaMFgwVgYGZ4EMAQICMEwwIwYIKwYB
BQUHAgEWF2h0dHBzOi8vZC5zeW1jYi5jb20vY3BzMCUGCCsGAQUFBwICMBkMF2h0
dHBzOi8vZC5zeW1jYi5jb20vcnBhMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjAfBgNVHSMEGDAWgBQNRFwWU0TBgn4dIKsl9AFj2L55pTBXBggrBgEFBQcB
AQRLMEkwHwYIKwYBBQUHMAGGE2h0dHA6Ly9zZC5zeW1jZC5jb20wJgYIKwYBBQUH
MAKGGmh0dHA6Ly9zZC5zeW1jYi5jb20vc2QuY3J0MA0GCSqGSIb3DQEBBQUAA4IB
AQBrQknzr4wntvKOGc8YK6RfqaXS+r4XH34DlEPp2kFleNRaiJpo2DTVXr1a+WXQ
omrKl9j4Irbf0Jq4vI0XqYw5Ei8WsvqCMoFXqnCwY2MazYVHRF/IYvCWfjIcySGu
tR/RS+bvpN1+ATt8vNnW7imdca3LjkM0Fw2GVjmaX2CeINpCRkuQ/d28ob46o9D9
VF+/kjc2tVwfabQQCNykRlhYLJDmhKcVI5+swD10+JjddbSAsXpIQFy3FI77idvC
rngbwc0G36Rj3DvcoypUbXoFzsnisy58oieFWRzHPe9DVYh0o/uJj40NJDOLuyEA
EIV237/q67gIYEn6LM8CLah9
—–END CERTIFICATE—–
subject=/C=US/ST=California/L=City/O=Example Corporation/OU=CIT/CN=*.Example.com
issuer=/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA – G3
—
No client certificate CA names sent
—
SSL handshake has read 1547 bytes and written 591 bytes
—
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : AES128-SHA
Session-ID: 6A6F797F1318E8462CBD4DD899493DB949E08F78A0505C4E70F9621ABB0AA280
Session-ID-ctx:
Master-Key: 50D309E782DF245FA36BD1028197459FE1A56B976D947D2994BBB75A0754D18AD44A90F29AC818DB28562F0E9E2E5442
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1439641898
Timeout : 300 (sec)
Verify return code: 0 (ok)
—

Linux, Mac, Security none

Using Python Virtual Environments with Sublime Text

August 21, 2015 John No comments

When developing in Python I enjoy using Sublime Text, especially its ability to run/build right from Sublime by simply pressing COMMAND+B. Meaning that I dont have to go out of the editor to the terminal to run the Python program.

Sample Python Program:

#!/usr/bin/env python

print "Hello World"

#!/usr/bin/env python

print "Hello World"

In sublime, I can program, run the program and see the result, increasing my productivity:

But what if I want to run a Python Virtual Environment and its benefits (see developing-in-python-using-python-virtual-environments).
Once the virtual environment is activated your terminal will use the Python Virtual Environment just fine, but Sublime Text will not, it will continue to use the system-wide Python environment by default.

Example: Having a Python module (e.g. IPy) installed in your virtual environment, but not system-wide, and having the Python virtual environment active:

# Python code:

#!/usr/bin/env python

import IPy

print "Hello World"

#!/usr/bin/env python

import IPy

print "Hello World"

# Running from terminal – (OK):

$ python hello.py
Hello World

1 2	$ python hello.py Hello World

# Running from Sublime – (NOT OK):

To make Sublime Text use the virtual environment you need to create a Sublime project and edit its properties to use the Virtual Env.

# Create Sublime Project
Go to Project -> Save Project As…
// I named mine blog.sublime-project

# Edit project properties by editing the file in sublime to be as follows:
Note:
Remember the name: label as this is how you will use the build with… option
The shell_cmd: label is the command that will run when you build. It is far from perfect as you have to hardcode the Python program in the configuration, but at least works.

{
	"folders":
	[
		{
			"path": "."
		}
	],
	"virtualenv": "env",
	"build_systems": [
	  {
	  "name": "PYTHON_VENV",
	  "shell_cmd": "env/bin/python hello.py"
	  }
 	] 
}

{

"folders":

[

{

"path": "."

}

"virtualenv": "env",

"build_systems": [

{

"name": "PYTHON_VENV",

"shell_cmd": "env/bin/python hello.py"

}

]

}

# Prepare Sublime to use your new build system:
Tools -> Build Systems -> PYTHON_VENV

# Now you can now continue to code and use COMMAND+B to build/run your programs without leaving your Sublime Text editor:

Development, DevOps, Python python, sublime, virtualenv

Developing in Python using Python Virtual Environments

August 10, 2015 John No comments

When developing in Python you most likely will need to install Python modules that will provide some type of functionality, the process is very simple:

Install the Python package manager:

sudo yum install python-pip -y

1	sudo yum install python-pip -y

Install modules, for example ‘yaml’, this is without virtual environments, this is basically installing a Python module system-wide:

sudo pip install pyaml

1	sudo pip install pyaml

The above will install Python modules for the system, but to do so you need ‘root’ privileges (e.g. sudo), and there is a possibility that developers don’t have ‘root’ privileges.
Another important scenario to consider is that installing/updating/modifying system-wide Python modules could affect other existing Python projects in the same system.

A way to overcome the system-wide python dependencies is to work with your own virtual environment, which allows you to install Python modules in a Python virtual environment and without the need for ‘root’ privileges.

What this accomplishes is to have your Python projects have their specific dependencies met without affecting anyone else.

To develop using Python virtual environments, the system should have the virtualenv Python package first:

[vagrant@vagrant-box ~]$  sudo yum install python-virtualenv

1	[vagrant@vagrant-box ~]$ sudo yum install python-virtualenv

Now as a regular user (no root or sudo needed):
# Create (or cd into) your Python project folder

$ mkdir my_python_project
$ cd my_python_project

1 2	$ mkdir my_python_project $ cd my_python_project

# Create Python Virtual Environment, you can name it what you want, I chose to name it ‘env’

[vagrant@vagrant-box my_python_project]$ virtualenv env
New python executable in env/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.

[vagrant@vagrant-box my_python_project]$ virtualenv env

New python executable in env/bin/python

Installing Setuptools..............................................................................................................................................................................................................................done.

Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.

# Activating the virtual environment
To use the virtual environment (as opposed to the system-wide Python environment) you need to activate it first. After it is activated you will see it in the left your command prompt.

# System-wide Python

[vagrant@vagrant-box my_python_project]$ which python
/usr/local/bin/python

1 2	[vagrant@vagrant-box my_python_project]$ which python /usr/local/bin/python

# Activate Virtual Env

[vagrant@vagrant-box my_python_project]$ source env/bin/activate

1	[vagrant@vagrant-box my_python_project]$ source env/bin/activate

# Ready to use Python Virtual Env

(env)[vagrant@vagrant-box my_python_project]$ which python
/vagrant/my_python_project/env/bin/python

1 2	(env)[vagrant@vagrant-box my_python_project]$ which python /vagrant/my_python_project/env/bin/python

# Installing modules in the virtual environment

[vagrant@vagrant-box my_python_project]$ pip install pyvmomi
Downloading/unpacking pyvmomi
  Downloading pyvmomi-5.5.0.2014.1.1.tar.gz (198kB): 198kB downloaded
  Running setup.py egg_info for package pyvmomi
Downloading/unpacking requests>=2.3.0 (from pyvmomi)
  Downloading requests-2.5.1.tar.gz (443kB): 443kB downloaded
  Running setup.py egg_info for package requests
Downloading/unpacking six>=1.7.3 (from pyvmomi)
  Downloading six-1.9.0.tar.gz
  Running setup.py egg_info for package six
    no previously-included directories found matching 'documentation/_build'
Installing collected packages: pyvmomi, requests, six
  Running setup.py install for pyvmomi
  Running setup.py install for requests
  Running setup.py install for six
    no previously-included directories found matching 'documentation/_build'
Successfully installed pyvmomi requests six
Cleaning up...

[vagrant@vagrant-box my_python_project]$ pip install pyvmomi

Downloading/unpacking pyvmomi

Downloading pyvmomi-5.5.0.2014.1.1.tar.gz (198kB): 198kB downloaded

Running setup.py egg_info for package pyvmomi

Downloading/unpacking requests>=2.3.0 (from pyvmomi)

Downloading requests-2.5.1.tar.gz (443kB): 443kB downloaded

Running setup.py egg_info for package requests

Downloading/unpacking six>=1.7.3 (from pyvmomi)

Downloading six-1.9.0.tar.gz

Running setup.py egg_info for package six

no previously-included directories found matching 'documentation/_build'

Installing collected packages: pyvmomi, requests, six

Running setup.py install for pyvmomi

Running setup.py install for requests

Running setup.py install for six

no previously-included directories found matching 'documentation/_build'

Successfully installed pyvmomi requests six

Cleaning up...

# Uninstalling modules in the virtual environment

[vagrant@vagrant-box my_python_project]$pip uninstall pyvmomi
Uninstalling pyvmomi:
  /vagrant/my_python_project/env/LICENSE.txt
  /vagrant/my_python_project/env/MANIFEST.in
  /vagrant/my_python_project/env/NOTICE.txt
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/__init__.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/__init__.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/connect.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/connect.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Cache.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Cache.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/CoreTypes.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/CoreTypes.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Differ.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Differ.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/DynamicTypeManagerHelper.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/DynamicTypeManagerHelper.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Iso8601.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Iso8601.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ManagedMethodExecutorHelper.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ManagedMethodExecutorHelper.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ServerObjects.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ServerObjects.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/SoapAdapter.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/SoapAdapter.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/StubAdapterAccessorImpl.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/StubAdapterAccessorImpl.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Version.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Version.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/VmomiSupport.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/VmomiSupport.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/__init__.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/__init__.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/pyVmomiSettings.py
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/pyVmomiSettings.pyc
  /vagrant/my_python_project/env/lib/python2.6/site-packages/pyvmomi-5.5.0.2014.1.1-py2.6.egg-info
  /vagrant/my_python_project/env/setup.cfg
  /vagrant/my_python_project/env/setup.py
  /vagrant/my_python_project/env/tox.ini
Proceed (y/n)? y
  Successfully uninstalled pyvmomi

[vagrant@vagrant-box my_python_project]$pip uninstall pyvmomi

Uninstalling pyvmomi:

/vagrant/my_python_project/env/LICENSE.txt

/vagrant/my_python_project/env/MANIFEST.in

/vagrant/my_python_project/env/NOTICE.txt

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/__init__.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/__init__.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/connect.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVim/connect.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Cache.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Cache.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/CoreTypes.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/CoreTypes.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Differ.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Differ.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/DynamicTypeManagerHelper.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/DynamicTypeManagerHelper.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Iso8601.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Iso8601.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ManagedMethodExecutorHelper.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ManagedMethodExecutorHelper.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ServerObjects.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/ServerObjects.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/SoapAdapter.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/SoapAdapter.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/StubAdapterAccessorImpl.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/StubAdapterAccessorImpl.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Version.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/Version.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/VmomiSupport.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/VmomiSupport.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/__init__.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/__init__.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/pyVmomiSettings.py

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyVmomi/pyVmomiSettings.pyc

/vagrant/my_python_project/env/lib/python2.6/site-packages/pyvmomi-5.5.0.2014.1.1-py2.6.egg-info

/vagrant/my_python_project/env/setup.cfg

/vagrant/my_python_project/env/setup.py

/vagrant/my_python_project/env/tox.ini

Proceed (y/n)? y

Successfully uninstalled pyvmomi

# Install multiple python modules from a requirements file
If your project has multiple Python modules required, it is better to create a requirements.txt file with the list of Python modules.

[vagrant@vagrant-box my_python_project]$ cat requirements.txt
argparse
pyaml

[vagrant@vagrant-box my_python_project]$ cat requirements.txt

argparse

pyaml

Then you can install all the listed Python modules in your virtual environment as follows:

[vagrant@vagrant-box my_python_project]$ pip install -r requirements.txt

1	[vagrant@vagrant-box my_python_project]$ pip install -r requirements.txt

# Deactivate virtual environment (go back to using system-wide Python)

[vagrant@vagrant-box my_python_project]$ deactivate

1	[vagrant@vagrant-box my_python_project]$ deactivate

Development, DevOps, Python python, virtualenv

Showing Differences Between two Git Commits

August 3, 2015 John No comments

From time to time I get questions about how to check differences between two Git commits or two branches, I will answer those questions in this post.

How to check differences between two branches:

Example:
you have a dev branch and a master branch. You develop in the dev branch and would like to know if you were to merge the dev changes to master, what will those changes be:
Please note the order of the branch this states differences that will be added to master from dev.

$ git diff master dev

1	$ git diff master dev

How to check differences between two commits:

Example:
When you do git log you will see your different commits over time, you need to check the differences between two commits.

commit eab54d0ec21a1d7e351fee4c67139ada740e7e6b
Author: John
Date: Wed Jul 8 22:18:05 2015 -0400

Add feature 2

commit b722f1650b9fb33e0990beec027c097526c61478
Author: John
Date: Wed Jul 8 22:31:18 2015 -0400

Add feature 1

// Remember the order the first argument is the point in time, and the second item is what is added to the first argument’s commit

$ git diff b722f1650b9fb33e0990beec027c097526c61478  eab54d0ec21a1d7e351fee4c67139ada740e7e6b
diff --git a/test/script.sh b/test/script.sh
index 33f9e16..cc40434 100755
--- a//test/script.sh
+++ b//test/script.sh
@@ -14,7 +14,7 @@ fi

...
-blah1
+blah2
...

$ git diff b722f1650b9fb33e0990beec027c097526c61478 eab54d0ec21a1d7e351fee4c67139ada740e7e6b

diff --git a/test/script.sh b/test/script.sh

index 33f9e16..cc40434 100755

--- a//test/script.sh

+++ b//test/script.sh

@@ -14,7 +14,7 @@ fi

...

-blah1

+blah2

...

How to check differences between the HEAD (current state/latest revision) and a previous commit:

$ git diff HEAD b722f1650b9fb33e0990beec027c097526c61478 
diff --git a/test/script.sh b/test/script.sh
index 33f9e16..cc40434 100755
--- a//test/script.sh
+++ b//test/script.sh
@@ -14,7 +14,7 @@ fi

...
-blah1
+blah2
...

$ git diff HEAD b722f1650b9fb33e0990beec027c097526c61478

diff --git a/test/script.sh b/test/script.sh

index 33f9e16..cc40434 100755

--- a//test/script.sh

+++ b//test/script.sh

@@ -14,7 +14,7 @@ fi

...

-blah1

+blah2

...

Or you can specify the last commit or a number of commits:

// Last commit

$ git diff HEAD~1
diff --git a/test/script.sh b/test/script.sh
index 33f9e16..cc40434 100755
--- a//test/script.sh
+++ b//test/script.sh
@@ -14,7 +14,7 @@ fi

...
-blah1
+blah2
...

$ git diff HEAD~1

diff --git a/test/script.sh b/test/script.sh

index 33f9e16..cc40434 100755

--- a//test/script.sh

+++ b//test/script.sh

@@ -14,7 +14,7 @@ fi

...

-blah1

+blah2

...

//Last 2 commits

$ git diff HEAD~2
diff --git a/test/script2.pp b/test/script2.pp
index 8def87e..be66b71 100644
--- a/test/script2.pp
+++ b/test/script2.pp
@@ -4,11 +4,10 @@

 class test {
-  stage { "first": before => Stage["main"] }

-  class { "repos": stage => "first" }
+  class { "test": stage => "first" }

-  class repos {
+  class test {
...
diff --git a/test/script.sh b/test/script.sh
index 33f9e16..cc40434 100755
--- a//test/script.sh
+++ b//test/script.sh
@@ -14,7 +14,7 @@ fi

...
-blah1
+blah2
...

$ git diff HEAD~2

diff --git a/test/script2.pp b/test/script2.pp

index 8def87e..be66b71 100644

--- a/test/script2.pp

+++ b/test/script2.pp

@@ -4,11 +4,10 @@

class test {

- stage { "first": before => Stage["main"] }

- class { "repos": stage => "first" }

+ class { "test": stage => "first" }

- class repos {

+ class test {

...

diff --git a/test/script.sh b/test/script.sh

index 33f9e16..cc40434 100755

--- a//test/script.sh

+++ b//test/script.sh

@@ -14,7 +14,7 @@ fi

...

-blah1

+blah2

...

How to check what changed since yesterday or a specific date:

git diff @{yesterday}
diff --git a/.gitignore b/.gitignore
index 4327d2e..8fa1531 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,2 +1,3 @@
-.svn
-.project
+.com_puppetlabs_geppetto_pptp_target/
+.metadata/
+Geppetto.AutoFileSystemLinked/

git diff @{yesterday}

diff --git a/.gitignore b/.gitignore

index 4327d2e..8fa1531 100644

--- a/.gitignore

+++ b/.gitignore

@@ -1,2 +1,3 @@

-.svn

-.project

+.com_puppetlabs_geppetto_pptp_target/

+.metadata/

+Geppetto.AutoFileSystemLinked/

// OR a specific date

git diff @{2015-05-01}
diff --git a/.gitignore b/.gitignore
index 4327d2e..8fa1531 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,2 +1,3 @@
-.svn
-.project
+.com_puppetlabs_geppetto_pptp_target/
+.metadata/
+Geppetto.AutoFileSystemLinked/

git diff @{2015-05-01}

diff --git a/.gitignore b/.gitignore

index 4327d2e..8fa1531 100644

--- a/.gitignore

+++ b/.gitignore

@@ -1,2 +1,3 @@

-.svn

-.project

+.com_puppetlabs_geppetto_pptp_target/

+.metadata/

+Geppetto.AutoFileSystemLinked/

How to check all changes to a specific file:

$ git log --follow /path/to/file
commit eab54d0ec21a1d7e351fee4c67139ada740e7e6b
Author: John <john@example.org>
Date:   Wed Jul 8 22:18:05 2015 -0400

    Add feature 2

commit b722f1650b9fb33e0990beec027c097526c61478
Author: John <john@example.org>
Date:   Wed Jul 8 22:31:18 2015 -0400

    Add feature 1

$ git log --follow /path/to/file

commit eab54d0ec21a1d7e351fee4c67139ada740e7e6b

Author: John <john@example.org>

Date: Wed Jul 8 22:18:05 2015 -0400

Add feature 2

commit b722f1650b9fb33e0990beec027c097526c61478

Author: John <john@example.org>

Date: Wed Jul 8 22:31:18 2015 -0400

Add feature 1

How to check size differences between two trees

I found this one in stackoverflow and has been very useful:

Reference Source: http://stackoverflow.com/questions/10845051/git-show-total-file-size-difference-between-two-commits/10847242#10847242

“git cat-file -s will output the size in bytes of an object in git. git diff-tree can tell you the differences between one tree and another. Putting this together into a script called git-file-size-diff located somewhere on your PATH will give you the ability to call git file-size-diff . Putting this together we can try something like the following” – Stackoverflow

$ cat /usr/local/bin/git-file-size-diff
#!/bin/sh
. git sh-setup
args=$(git rev-parse --sq "$@")
eval "git diff-tree -r $args" | {
  total=0
  while read A B C D M P
  do
    case $M in
      M) bytes=$(( $(git cat-file -s $D) - $(git cat-file -s $C) )) ;;
      A) bytes=$(git cat-file -s $D) ;;
      D) bytes=-$(git cat-file -s $C) ;;
      *)
        echo >&2 warning: unhandled mode $M in \"$A $B $C $D $M $P\"
        continue
        ;;
    esac
    total=$(( $total + $bytes ))
    printf '%d\t%s\n' $bytes "$P"
  done
  echo total $total Bytes
  echo total $(($total/1000)) KBytes
}

$ cat /usr/local/bin/git-file-size-diff

#!/bin/sh

. git sh-setup

args=$(git rev-parse --sq "$@")

eval "git diff-tree -r $args" | {

total=0

while read A B C D M P

case $M in

M) bytes=$(( $(git cat-file -s $D) - $(git cat-file -s $C) )) ;;

A) bytes=$(git cat-file -s $D) ;;

D) bytes=-$(git cat-file -s $C) ;;

echo >&2 warning: unhandled mode $M in \"$A $B $C $D $M $P\"

continue

;;

esac

total=$(( $total + $bytes ))

printf '%d\t%s\n' $bytes "$P"

done

echo total $total Bytes

echo total $(($total/1000)) KBytes

}

// Using it to check size differences between two branches:

$ git file-size-diff master dev
66	.gitignore
239	modules/test/Modulefile
171	modules/test/README
167	modules/test/files/file.txt
1425	modules/test/manifests/init.pp
total 2068 Bytes
total 2 KBytes

$ git file-size-diff master dev

66 .gitignore

239 modules/test/Modulefile

171 modules/test/README

167 modules/test/files/file.txt

1425 modules/test/manifests/init.pp

total 2068 Bytes

total 2 KBytes

// Using it to check size differences between the local master branch and the remote master branch, this is useful to know how much data will be downloaded when doing a ‘git pull’:

$ git file-size-diff master origin/master
633	modules/test/manifests/config.pp
326	modules/test/manifests/service.pp
60	modules/test/manifests/params.pp
573	modules/test/manifests/install.pp
13	modules/test/manifests/init.pp
86	modules/mod/manifests/params.pp
total 1691 Bytes
total 1 KBytes

$ git file-size-diff master origin/master

633 modules/test/manifests/config.pp

326 modules/test/manifests/service.pp

60 modules/test/manifests/params.pp

573 modules/test/manifests/install.pp

13 modules/test/manifests/init.pp

86 modules/mod/manifests/params.pp

total 1691 Bytes

total 1 KBytes

Development, DevOps development, diff, git, linux

Rename Multiple Files in Linux Using Regex and perform Arithmetic

July 26, 2015 John No comments

While working on my video library I noticed that I had to rename a bunch of files to have correct file names that include Show name and Episode name and number.

I have about 275 files named from 000-274, but I needed to rename them to specify the season and episode they correspond to.

For example, files from 125 – 150 are from season 5 of said show, and that is what I will use for this post.

An example of a file in the 125 – 150 range:
‘SHOW – 135 – Some Episode _www.unneededStuff.mp4’

As you can see the filename has the following structure (which I will modify later in this post)

(ShowName) – (filename #) – (EpisodeName)_(Extra unneeded stuff).(extension)

I will be using the Linux/Mac rename command to perform the renaming of these files (125-150).
The end result should be that these files are renamed to have numbers from 501- 526 (for Season 5), so I will use arithmetic manipulate the numbers to be what I want.
I will also clean up the name a little bit so that it does not include the ‘extra unneeded stuff’.
The name of a file should be of form:
‘SHOW – 511 – Some Episode.mp4’

First make sure you work with the specific files you want to modify and nothing else:

$ ls SHOW\ -\ 1{2{5-9},3,4,50}*

1	$ ls SHOW\ -\ 1{2{5-9},3,4,50}*

// The above will show only files between 125 – 150 will be touched/modified.
// Using Bash brace expansion will help create the list between 125 – 150
// the brace expansion will use anything that has numbers 125-129, and any 13*, 14*, 150

Use the rename command with a RegEx expression that will modify the name the way you want.
I have the following regex expression:

's/(^.+)\s-\s(\d\d\d)(.+)_www.+(\..+$)/$1." - ".($2+376).$3.$4/e'

1	's/(^.+)\s-\s(\d\d\d)(.+)_www.+(\..+$)/$1." - ".($2+376).$3.$4/e'

Explanation:
// (^.+)\s-\s is the Show name plus ‘ – ‘, I am grouping just the name of the show to use it later
// (\d\d\d) is the incorrect episode number used in the filename
// (.+)_www.+ is the Episode name plus ‘unneeded stuff’, I am grouping just the episode name to use it later
// (\..+$) is the file extension, I am grouping it to use it later

// Now what was found with the above Regex will be substituted as follows:
$1 will have the Show name
“ – ” I am concatenating literal “ – “ after the show name
($2+376) I am adding the episode number + 376, which will give me the correct episode number. For example for 135, now it will be 511 (which is Season 5 episode 11)
$3 The Episode name captured from the previous regex
$4 The file extension captured from the previous regex

Run rename with -n (dry run) using the regex and file list created above to verify it will give you what you want/expect.

$ rename -n 's/(^.+)\s-\s(\d\d\d)(.+)_www.+(\..+$)/$1." - ".($2+376).$3.$4/e' SHOW\ -\ 1{2{5..9},3,4,50}*

1	$ rename -n 's/(^.+)\s-\s(\d\d\d)(.+)_www.+(\..+$)/$1." - ".($2+376).$3.$4/e' SHOW\ -\ 1{2{5..9},3,4,50}*

Output:
…
‘SHOW – 135 – Some Episode_www.unneededStuff.mp4’ would be renamed to ‘SHOW – 511 – Some Episode.mp4’
…

Now run it again without -n:

$ rename  's/(^.+)\s-\s(\d\d\d)(.+)_www.+(\..+$)/$1." - ".($2+376).$3.$4/e' SHOW\ -\ 1{2{5..9},3,4,50}*

1	$ rename 's/(^.+)\s-\s(\d\d\d)(.+)_www.+(\..+$)/$1." - ".($2+376).$3.$4/e' SHOW\ -\ 1{2{5..9},3,4,50}*

Linux, Mac, Scripting none

Generate MySQL Password Hash using Python

June 26, 2015 John 1 comment

In this post I am showing how to use generate a MySQL 5 password-hash that can be used to create MySQL GRANTS using a hash instead of a password.

To use a password-hash to create GRANTs:

GRANT ALL ON *.* to user@% identified by PASSWORD '<PASS-HASH>';

1	GRANT ALL ON . to user@% identified by PASSWORD '<PASS-HASH>';

A good use case is the Puppet puppetlabs-mysql module to automate the MySQL environment, You can automate/define USER and GRANT creation by using the code below, but notice that it requires a password-hash instead of a password:

users => {
  'someuser@localhost' => {
    ensure                   => 'present',
    max_connections_per_hour => '0',
    max_queries_per_hour     => '0',
    max_updates_per_hour     => '0',
    max_user_connections     => '0',
    password_hash            => '*F3A2A51A9B0F2BE2468926B4132313728C250DBF',
  },
}

users => {

'someuser@localhost' => {

ensure => 'present',

max_connections_per_hour => '0',

max_queries_per_hour => '0',

max_updates_per_hour => '0',

max_user_connections => '0',

password_hash => '*F3A2A51A9B0F2BE2468926B4132313728C250DBF',

}

OR:

mysql_user { 'root@127.0.0.1':
  ensure                   => 'present',
  max_connections_per_hour => '0',
  max_queries_per_hour     => '0',
  max_updates_per_hour     => '0',
  max_user_connections     => '0',
  password_hash            => '*F3A2A51A9B0F2BE2468926B4132313728C250DBF',
}

mysql_user { 'root@127.0.0.1':

ensure => 'present',

max_connections_per_hour => '0',

max_queries_per_hour => '0',

max_updates_per_hour => '0',

max_user_connections => '0',

password_hash => '*F3A2A51A9B0F2BE2468926B4132313728C250DBF',

}

They recommend using mysql_password() for creating such a hash. But that means you need to have a MySQL server available.
In this post I am writing about getting those hashes using Python, I wrote a program/script to get the password-hash programatically.

The Python program/script can be found at:
https://github.com/parcejohn/mysql_password_hash

Usage

$ ./mysql_password_hash -h
usage: mysql_password_hash [-h] [-p PASSWORD | -r] [-l PASSWORD_LENGTH]

MySQL Password Hash Generator

optional arguments:
  -h, --help            show this help message and exit
  -p PASSWORD, --password PASSWORD
                        Enter a password
  -r, --generate_random
                        Generate a random password
  -l PASSWORD_LENGTH, --password_length PASSWORD

$ ./mysql_password_hash -h

usage: mysql_password_hash [-h] [-p PASSWORD | -r] [-l PASSWORD_LENGTH]

MySQL Password Hash Generator

optional arguments:

-h, --help show this help message and exit

-p PASSWORD, --password PASSWORD

Enter a password

-r, --generate_random

Generate a random password

-l PASSWORD_LENGTH, --password_length PASSWORD

# Using Command line arguments – User provided password (e.g. ‘secret’)

$ mysql_password_hash -p secret
PASSWORD: secret
HASH: *14e65567abdb5135d0cfd9a70b3032c179a49ee7

$ mysql_password_hash -p secret

PASSWORD: secret

HASH: *14e65567abdb5135d0cfd9a70b3032c179a49ee7

# Using Command line arguments – Random password with length=20 (default length=12)

$ mysql_password_hash -r -l 20
PASSWORD: gnlrn96^g18jcblmssa6
HASH: *e3cbe60709e8abe2082c92cc5e72a762d5f18e22

$ mysql_password_hash -r -l 20

PASSWORD: gnlrn96^g18jcblmssa6

HASH: *e3cbe60709e8abe2082c92cc5e72a762d5f18e22

# interactive mode (no arguments)

mysql_password_hash

1	mysql_password_hash

Development, DevOps, Puppet, Python none

Older Entries

Technologist

Browsing Posts published by John

Installing Puppet Enterprise on AWS EC2 using Vagrant

Pre-requisites:

Create Vagrantfile

Create servers.yaml

Docker Registry with Authentication and SSL/TLS

Create project structure

Password file to support Basic Auth

Certificates to support TLS

Location to store the images

Create Docker Registry container

Using the Registry

Registry API

AWS EC2 automation using AWS CLI and User-Data

Infrastructure As Code: Create Linux (RHEL/CentOS) base Images Using Packer

Troubleshooting Secure Connectivity using SSL/TLS Certificates

# Get CA Issuer Certificate(s)

# Check CA Issuer Certificates Details

# Trust the CA Issuer Certificates

Using Python Virtual Environments with Sublime Text

Developing in Python using Python Virtual Environments

Showing Differences Between two Git Commits

Rename Multiple Files in Linux Using Regex and perform Arithmetic

Generate MySQL Password Hash using Python

Recent Posts

Browse by Tags

Categories

Meta

Related Links