How to enable SmartFabric for VxRail Step 1

I decided to document the steps I used to deploy SmartFabric Services on VxRail.  These are partly notes for myself and Customer Solution Centre engineers that will likely need to showcase this capability very soon.  The demand for this solution is very high and the customers I have met are impressed by what is now possible (wait until you see the roadmap!). 

Note: This is not a guide for end user customers because a lot of what I write about is handled thru our automated deployment appliance; VxRail. A note of thanks to Allan Scott from New York CSC that helped with the first SFS deployment and documentation.

Step 1: Enable SmartFabric Services on the ToR Switch

  • Cabling the ToR Switches
  • Installing/Upgrading OS10 on Dell EMC Switch
  • Enable the VxRail Personality on OS10
  • Ready for Part2 – Deploying VxRail with Smart Fabric Services

Next Blog: Step 2: Deploy VxRail Cluster incl. ToR with VxRail Manager

Step 3: Deploy the SmartFabric OMNI plugin in VMware vSphere

Step 4: Virtualization engineer controls Day 2 Ops for the Full Stack


Getting Started :


How to enable SmartFabric Services on the ToRs

SmartFabric is supported on the 4100 series from Dell EMC. Current models are 10G – S4112 F/T, S4128 F/T or S4148 F/T  (25G coming soon). Sales can order these switches to be delivered from factory with OS10 and licenses already applied.

If you need the latest version of OS10 – get it here: force10networks.com , request login thru support page and download 10.4.1.x. Put OS10 .bin file and licence .xml file in a USB drive – insert USB drive into switch.

Cabling the TORs

Cable up ports 29 and 30 – 100GB cables for ISL (VLT).

Cable up ports 25 and 26 – 100GB cables for Uplink.

Cable laptop into port 1 on switch.

Cable new VxRail appliances in any other port starting at port 2.

Installing or Upgrading OS10 on switch:

This is an optional step. The switches can be ordered and configured in factory, and so should arrive ready to begin at Step 2.

Connect the laptop to the serial port on one of the switches and start putty. Putty settings are 115200, 8, stop, none, none. I used a USB serial port so my COM port was COM3.

Powerup the serial connected switch – and break into ONIE mode by hitting ESC during bootup.

Choose “onie-discovery-stop” from the menu.

At prompt type: fdisk -l

USB Thumbdrive should be /dev/sdb1

mkdir /mnt/usb

mount -t vfat /dev/sdb1 /mnt/usb

Install OS10:

cd /mnt/usb

onie-nos-install /mnt/usb/XXXXXXXXXX.bin

Check IOS10 Version & Install License:

show version

show license status (skip next step if already installed)

license install usb://xxxxxxxx-NOSEnterprise-License.xml

Configure mgmt interface if required:

conf

int mgmt 1/1/1

no ip address dhcp

ip address 10.204.86.250/24

no shut

exit

management route 10.204.86.0/24 managementethernet

exit

Repeat these steps for the second ToR switch.

Optional step. Configure 40GB uplinks:

My showcase lab is using 40GB uplinks rather than 100GB so I needed to change the profile of the uplinks before applying the VxRail SFS personality. You can skip this step if you are using 100GB links.

OS10(config)# switch-port-profile 1/1 profile-2

Warning: Switch port profile will be applied only after a save and reload. All management port configurations will be retained but all other configurations will be wiped out after the reload.

OS10(config)# exit

OS10# write memory

OS10# reload

Enable the VxRail personality:

The SFS personality script is included in OS10. Once applied to each ToR switch, the switches will reboot with SmartFabric Mode enabled and you are now ready to perform a VxRail deployment from the VxRail Manager.

system bash

sudo sfs_enable_vxrail_personality.py -d 20 -a-m 2002

‘-d 20’ is a unique Domain ID that you assign to each cluster

‘-m 2002’ is a non-routed vlan used to do the initial build, local to the ToR switches only (Internal management network)

‘-a’ indicates that the port-channel on the upstream switches is configured with LACP

VxRail personality profile script options :


 
Domain -d <id> Required numeric value unique to data center (1 to 254) applied to ToR switch configuration settings Default: 1
Uplink -u <port,port> Override default 100Gb uplink ports Default: ports 25& 26
ISL -I <port,port> Override default 100Gb ISL ports Default: ports 29 & 30
Uplink tagging -t Whether external management VLAN is tagged or untagged when passed through uplinks. Default: untagged
Uplink LACP -a Whether LACP is active on uplink port channel (dynamic) or not (static). Default: static
Uplink breakout -b <2X50GE, 4X25GE, 4X10GE> Breakout 100Gb uplinks. Used to support connectivity to upstream switches without 100Gb ports
Management VLAN -m <VLAN> VxRail Cluster Build Network VLAN.
Default: 1

Validate Personality:

system bash

sudo sfs_validate_vxrail_personality.py

Links to useful guides that helped us document this build:

VxRail Fabric Automation SmartFabric Services User Guide

Dell EMC OpenManage Network Integration for VMware vCenter

How to Install Dell Networking FTOS on Dell Open Networking (ON) Switches


Please follow and like us:

SmartFabrics is Smart for VxRail HCI

My guide to enable SmartFabric on Dell EMC switches is here.

Catch out my interview with Barry Coombs from ComputerWorld UK on SmartFabric… does everyone hate their own voice or just me?

Is networking in HCI complex?

Back in October I asked the question, is HCI networking easy? I stand by my assertion that it is already pretty simple once you understand the converged design for HCI does not require separate physical fabrics. Once you setup your required VLANS and appropriate MTU and multicast for IPv6 you are 90% of the way done.  So if it’s so easy already, then why am I so excited about Smart Fabrics for VxRail appliances? Start up a conversation with Ned the network engineer about the automated deployment and simplified life-cycle management capabilities of SmartFabrics and you will get back a blank dull stare. “That’s just a python script” Ned the Network Admin will say, “Take your fancy sales patter down to the Virtualization guys, we don’t want your kind round here!”

Ned the NetAdmin says “You’ll never take my switches!”

It’s not easy to impress a NetAdmin

Ned has a point I suppose. The Network Admins job is to move packets reliably from one part of the network to the other, monitor the network for any problems, and to design and build future networks.  The last thing Ned needs is dealing with end users complaining about network problems caused by poorly designed applications (its never the networks fault!). This is actually the main selling point FOR the use of Smart Fabrics. Let me explain why.

Before you understand why SmartFabrics for VxRail, you have to first understand the reason for choosing VxRail appliances in the first place.  I have written a few Blogs on this already here and here . In short it’s an engineered solution for vSAN that comes from Dell EMC fully validated and tested and automatically deployed and updated throughout its life-cycle.

VxRail appliances don’t require a Storage expert or Server guru and it even saves the Virtualization admin from having to spend countless hours reading design and deploy documentation. After deployment is done, day 2 simplified operations begins. As a result maintenance and updates and upgrades are made easy through a single bundle file that covers the entire stack (not just the software on top of somebody else’ hypervisor – and includes the hardware too.

Can SmartFabrics simplify HCI?

The last part of the HCI architecture that needed simplification was the network, so Dell EMC has had its sites on Ned’s cubicle for a while now. If we are going to provide a fully automated deployment experience for our customers, it only makes sense to include the Top of Rack switches that are being used by VxRail appliances.  After all why would the network admin want to be responsible for those HCI host ports anyways? If something goes wrong in the vSAN stack, Ned doesn’t want to be dragged in to a War Room to defend last weekend’s network changes. Ned knows the changes the network team made at the weekend were to the core only.

Now with Smart Fabrics for VxRail, Ned can still own the core, and leave the HCI network problems to the Virtualization team.  SmartFabric will fully configure a redundant TOR fabric for VxRail, and continue to maintain the network for the life-cycle of the HCI solution. When it’s time to patch the HCI network, SmartFabric will provide a bundle file, and perform a non-disruptive rolling upgrade to the network TORs leaving Ned free to watch old episodes of Futurama. If the HCI team needs to expand their existing VxRail cluster by adding a new node, then Smart Fabric will fully automate the changes to the TOR switch, no need for Ned to ever get involved.

Is BYO Networking still an Option?

One of the advantages for VxRail customers has been the fact that it is BYON (Bring Your Own Networking). This means that Dell EMC does not force you to take a switch from their portfolio into your datacenter. For some customers, this would be non-negotiable.  They may have standardized on a specific brand and prefer to stay that way, no matter what they run at the Storage or Virtualization layer.  VxRail networking is compatible with any modern low latent switch and the introduction of VxRail SmartFabrics does NOT mean that the BYON option is no longer a choice. Hopefully the automation that comes with SmartFabric for VxRail will entice some customers to converge the entire HCI stack and give Ned some peace of mind.

Here is a great Blog on vSAN and Network Switch choices from @LostSignal on this Blog: https://blogs.vmware.com/virtualblocks/2019/03/21/designing-vsan-networks-2019-update/

Update! This week I will be meeting Hasan Mansur at the Limerick Customer Solution Center who writes a great Dell EMC networking blog at https://hasanmansur.com/ . Hasan has written two great articles there about SmartFabric Services. Please check it out Part 1 here and Part 2 here.

Please follow and like us:

Is HCI networking easy?

Even though hyper-converged solutions have been  one of the hottest trends in the Datacenter since virtualization, you will still meet traditional architects that are seeing this technology for the first time.  Many times the customer will come to the conversation with just the virtualization lead,  sometimes they will bring the Storage or Compute team, but often they will forget to tell the Networking team any of their plans (no wonder the network engineer can be so grumpy).  This can prove problematic for a networking team that is not familiar with a few of the basic HCI requirements. Continue reading “Is HCI networking easy?”

Please follow and like us: