The following guide describes the neccessary steps to install and configure a pair of cisco nexus 1000v switches to be used in a vSphere cluster.
These will connect to Cisco Nexus 5020 Upstream Switches.
In this guide the hardware used consists of:
Hardware:
3x HPProliant DL380 G6 with 2 4-port NICs.
2x Cisco 5200Nexus Switches
Software:
vSphere 4 Update 1 Enterprise Plus (needed to use Cisco nexus1000v)
vCenter installed as a virtual machine – 192.168.10.10 (on VLAN 10)
Cisco Nexus 1000v 4.0.4.SV1.3b –
Primary 192.168.101.10 domain id 100 (on VLAN 101)
I am assuming you have already installed and configured vCenter and the ESX cluster.
Cisco recommends that you use 3 separate VLANs for Nexus traffic, I am using the following VLANs:
100 – Control – Control connectivity between Nexus 1000V VSM and VEMs (Non Routable)
101 – Management – ssh/telnet/scp to the cisco Nexux 1000v int mgmt0 (Routable)
102 – Packet – Internal connectivity between Nexus 1000v (Non Routable)
And I will also use VLAN 10 and 20 for VM traffic (10 for Production, 20 for Development)
1) Install vSphere (I assume you have done this step)
2) Configure Cisco Nexus 5020 Upstream Switchports
You need to configure the ports on the upstream switches in order to pass VLAN information to the ESX hosts’ uplink NICs
On the Nexus5020s, run the following:
// These commands give a description to the port and allow trunking of VLANs.
// The allowed VLANs are listed
// spanning-tree port type edge trunk is the recommended spanning-tree typeinterface Ethernet1/1/10
description “ESX1-eth0”
switchport mode trunk
switchport trunk allowed vlan 10-20,100-102
spanning-tree port type edge trunk
3) Service Console VLAN !!!
When I installed the ESX server, I used the native VLAN, but after you change the switch port from switchport mode access to switchport mode trunk, the ESX server needs to be configured to send specific VLAN traffic to the Service Console.
My Service Console IP is 192.168.10.11 on VLAN 10, so you will need to console to the ESX host and enter the following:
[root@esx1]# esxcfg-vswitch -v 10 -p “Service Console” vSwitch0
4) Add Port Groups for the Control,Packet and Management VLANs.
I add these Port Groups to VMware Network Virtual Switch vSwitch0 on all the ESX hosts. Make sure to select the right VLANs for your environment.
5) Now that you have configured the Control,Packet and Management Port Groups with their respective VLANs, you can install the Cisco Nexus 1000v.
I chose to install the Virtual Appliace (OVA) file downloaded from Cisco. The installation is very simple, make sure to select to Manually Configure Nexus 1000v and to Map the VLANs to Control, Packet and Management. The rest is just like installing a regular virtual appliance.
6) Power on and open a console window to the Nexus1000v VM(appliance) you just installed. A setup script will start running and will ask you a few questions.
admin password
domain ID // This is used to identify the VSM and VEM. If you want to have 2 Nexus 1000v for high availability, both Nexus 1000v will use the same domain ID. I chose 100
High Availability mode // If you plan to use 2 Nexus 1000v for high availability, then for the first installation select primary, otherwise standalone
Network Information // Things like IP, netmask, gateway Disable Telnet! Enable SSH!
The other stuff we will configure later (Not from the Setup script)
7) Register vCenter Nexus 1000v Plug-in
Once you have the Nexus 1000v basics configured, you should be able to access it. Try to SSH to it (Hopefully you enabled SSH).
Open a browser and point it to the Nexus 1000v management IP address (in this case 192.168.101.10) and you will get a webpage like the following
- Download the cisco_nexus_1000v_extension.xml
- Open vSphere client and connect to the vCenter.
- Go to Plug-ins > Manage Plug-ins
- Right-click under Available Plug-ins and select New Plu-ins, Browse to the cisco_nexus_1000v_extension.xml
- Click Register Plug-in (disregard security warning about new SSL cert)
You do NOT need to Download and Install the Plug-in, just Register it.
Now we can start the “advanced” configuration of the Nexus 1000v
8 ) Configure SVS domain ID on VSM
n1kv(config)# svs-domain
n1kv(config-svs-domain)# domain id 100
n1kv(config-svs-domain)# exit
9) Configure Control and Packet VLANs
n1kv(config)# svs-domain
n1kv(config-svs-domain)# control vlan 100
n1kv(config-svs-domain)# packet vlan 102
n1kv(config-svs-domain)# svs mode L2
n1kv(config-svs-domain)# exit
10) Connect Nexus 1000v to vCenter
In this step we are defining the SVS connection which is the link between the VSM and vCenter.
n1kv(config)# svs connection vcenter
n1kv(config-svs-conn)# protocol vmware-vim
n1kv(config-svs-conn)# vmware dvs datacenter-name myDatacenter
n1kv(config-svs-conn)# remote ip address 192.168.10.10
n1kv(config-svs-conn)# connect
n1kv(config-svs-conn)# exit
n1kv(config)# exit
n1kv# copy run start
//Verify the SVS connection
n1kv# show svs connections vcenter
connection vcenter:
ip address: 192.168.10.10
remote port: 80
protocol: vmware-vim https
certificate: default
datacenter name: myDatacenter
DVS uuid: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
config status: Enabled
operational status: Connected
sync status: Complete
version: VMware vCenter Server 4.0.0 build-258672
12) Create the VLANs on the VSM
n1kv# conf t
n1kv(config)# vlan 100
n1kv(config-vlan)# name Control
n1kv(config-vlan)# exit
n1kv(config)# vlan 102
n1kv(config-vlan)# name Packet
n1kv(config-vlan)# exit
n1kv(config)# vlan 101
n1kv(config-vlan)# name Management
n1kv(config-vlan)# exit
n1kv(config)# vlan 10
n1kv(config-vlan)# name Production
n1kv(config-vlan)# exit
n1kv(config)# vlan 20
n1kv(config-vlan)# name Development
n1kv(config-vlan)# exit
// Verify VLANs
n1kv(config)# show vlan VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 1 default active 10 Production active 20 Development active 100 Control active 101 Management active 102 Packet active VLAN Type ---- ----- 1 enet 10 enet 20 enet 100 enet 101 enet 102 enet
13) Create Uplink Port-Profile
The Cisco Nexus 1000v acts like the VMware DVS. Before you can add hosts to the Nexus1000v you will need to create uplink port-profiles; which will allow VEMs to connect with the VSM.
n1kv(config)# port-profile system-uplink
n1kv(config-port-prof)# switchport mode trunk
n1kv(config-port-prof)# switchport trunk allowed vlan 10,20,100-102
n1kv(config-port-prof)# no shutdown
n1kv(config-port-prof)# system vlan 100,102
n1kv(config-port-prof)# vmware port-group dv-system-uplink
n1kv(config-port-prof)# capability uplink
n1kv(config-port-prof)# state enabled
// Verify Uplink Port-Profile
n1kv(config-port-prof)# show port-profile name system-uplink
port-profile system-uplink
description:
type: ethernet
status: enabled
capability l3control: no
pinning control-vlan: -
pinning packet-vlan: -
system vlans: 100,102
port-group: dv-system-uplink
max ports: -
inherit:
config attributes:
switchport mode trunk
switchport trunk allowed vlan 10-20,100-102
no shutdown
evaluated config attributes:
switchport mode trunk
switchport trunk allowed vlan 10-20,100-102
no shutdown
assigned interfaces:
14) It is now time to install the VEM on the ESX hosts.
The preferred way to do this is using VUM(VMware Update Manager). If you have VUM in the system the installation will be very simple.
Simply go to Home->Inventory->Networking
Right Click on the Nexus Switch and add host
// Verify that the task is successfull
// Also take a look at the VSM console
n1kv# 2011 Jan 14 14:43:03 n1kv %PLATFORM-2-MOD_PWRUP: Module 3 powered up (Serial number ) n1kv# show module Mod Ports Module-Type Model Status --- ----- -------------------------------- ------------------ ------------ 1 0 Virtual Supervisor Module Nexus1000V active * 3 248 Virtual Ethernet Module NA ok Mod Sw Hw --- --------------- ------ 1 4.0(4)SV1(3b) 0.0 3 4.0(4)SV1(3b) 1.20 Mod MAC-Address(es) Serial-Num --- -------------------------------------- ---------- 1 xx-xx-xx-xx-xx-xx to xx-xx-xx-xx-xx-xx NA 3 xx-xx-xx-xx-xx-xx to xx-xx-xx-xx-xx-xx NA Mod Server-IP Server-UUID Server-Name --- --------------- ------------------------------------ -------------------- 1 192.168.101.10 NA NA 3 192.168.11.82 XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX esx1 * this terminal session
// Do the same for all the other ESX Hosts
15) Create the Port-Profile(s) (VMware Port-Groups)
Port-Profile configure interfaces on the VEM.
From the VMware point of view a port-profile is represented as a port-group.
// The Port-Profile below will be the VLAN 10 PortGroup on vCenter
n1kv# conf t
n1kv(config)# port-profile VLAN_10
n1kv(config-port-prof)# vmware port-group
n1kv(config-port-prof)# switchport mode access
n1kv(config-port-prof)# switchport access vlan 10
n1kv(config-port-prof)# vmware max-ports 200 // By default it has only 32 ports, I want 200 available
n1kv(config-port-prof)# no shutdown
n1kv(config-port-prof)# state enabled
n1kv(config-port-prof)# exit
16) Select the PortGroup you want your VM to connect to
17) Verify Port Profile/Port Groups from the VSM console
n1kv# show port-profile usage
-------------------------------------------------------------------------------
Port Profile Port Adapter Owner
-------------------------------------------------------------------------------
VLAN_10 Veth1 Net Adapter 1 jeos_10
VLAN_20 Veth2 Net Adapter 1 jeos_20
system-uplink Eth3/5 vmnic4 esx1.example.com
Eth3/6 vmnic5 esx1.example.com
Eth3/9 vmnic8 esx1.example.com
Eth3/10 vmnic9 esx1.example.com
Eth4/5 vmnic4 esx2.example.com
Eth4/6 vmnic5 esx2.example.com
Eth4/9 vmnic8 esx2.example.com
Eth4/10 vmnic9 esx2.example.com
At this point you are ready to use the Cisco 1000v, but if you plan to run this in a production environment, it is strongly recommended you run the VSM in High Availability mode.
Follow this post to learn how to install and configure VSM High Availability:
cisco-nexus-1000v-vsm-high-availability
Comments
Leave a comment Trackback