Tag: networking

SmartFabric Services for VxRail latest Updates and Install Guide

Its been a little over a year since we launched SmartFabric Services for VxRail. This was initially a neat little solution for single rack deployments. We were the first in the industry to automate the entire HCI deployment. The vision was always to go beyond the Top of Rack and automate Leaf and Spine architectures as networking is still the challenge for HCI solutions. Now we have.

SmartFabric for VxRail New Features

  • Zero-Touch automated deployment of Leaf/Spine
  • Enhanced GUI for Leaf and Spine Personality Management
  • Single Rack or Multi Rack VxRail Cluster deployments in a single site
  • One or more VxRail Clusters connected to a single fabric
  • Ability to connect non VxRail devices to the fabric
  • Fabric expansion automation
  • Lifecycle management of Leaf/Spine from vSphere OMNI plugin
  • Switch replacement automation

I have been keeping a close eye on the release of this software as I wanted to get it setup to demo to customers at the Customer Solution Center. I also wanted to quickly get a out a new Blog series to replace the original one that was so popular with our Partners and the PreSales community.

So before we get started, lets review the latest supported versions of the new SmartFabric Services for Vxrail solution.

Supported Software versions

Link to InfoHub where the latest detail on supported versions and more!

Lets get started. Follow these Steps.

Getting SmartFabric for VxRail is now even easier than before and I am going to document the 4 Steps I followed to get it up and running in my lab.

Step 1 – Enable SmartFabric Services on OS10 Leaf switch

Note: This is not a guide for end user customers because a lot of what I write about is handled thru our automated deployment appliance by Partners or Services teams. So please proceed with caution.

Check that the OS10 version is EXACTLY 10.5.0.5

Login to the Leaf and Spine switches and check that they are running the correct supported version of OS10 for VxRail installs. If the version is not running OS10 version 10.5.0.5 (not older or newer) then please upgrade or downgrade. You can use this blog i wrote earlier to perform an upgrade or downgrade from the switch CLI.

Configure Leaf switch OOB Management.

Put an ip address on both of the Leaf switches before you enable the SmartFabric personality in the following step.

OS10# configure terminal
OS10(config)# interface mgmt 1/1/1
OS10 (conf-if-ma-1/1/1)# no ip address dhcp
OS10(conf-if-ma-1/1/1)# ip address 192.168.105.235/24
OS10 (conf-if-ma-1/1/1)# no shutdown
OS10(conf-if-ma-1/1/1)# exit
OS10(config)# management route 192.168.0.0/16 192.168.105.254
OS10(config)# end
OS10# write memory

Enable SmartFabric Services for VxRail on the Leaf and Spine.

OS10(config)# smartfabric l3fabric enable role LEAF vlti ethernet 1/1/14-1/1/15

We can also enable the Spine switches now. These are not required yet for the VxRail deployment so we will set them up later.

OS10(config)# smartfabric l3fabric enable role SPINE

When the SmartFabric Services for VxRail personality is applied, the switches will reload.

Verify SFS Cluster & verify the Master

OS10# show smartfabric personality

Personality :L3 Fabric
Role :LEAF
ICL :ethernet1/1/14, ethernet1/1/15

OS10# show smartfabric cluster

CLUSTER DOMAIN ID : 100
VIP : fde2:53ba:e9a0:cccc:0:5eff:fe00:1100
ROLE : MASTER
SERVICE-TAG : D21WNK2
MASTER-IPV4 : 192.168.105.235
PREFERRED-MASTER : true

Connect to SmartFabric GUI

Now we can connect to either Leaf OOB Management ip and access the GUI. If we know the Master IP – then use this URL to access GUI:

https://MASTER_IP_ADDRESS

The next blog will explain the new SmartFabric GUI and simplified VxRail deployment process. Stay Tuned.

How to Upgrade SmartFabric OS10 via cli

Download the latest SmartFabric OS10 operating system.

Before we upgrade SmartFabric OS10 operating system on our switches we need to get the latest compatible OS10 version. The latest version of SmartFabric OS10 Enterprise Edition software is located on the Force10 Networks portal here. If you plan to upgrade SmartFabric OS10 with VxRail then you should consult the guide here. This matrix tracks the OS10 Switch Operating system version, OMNI plugin version and VxRail software that align.

Follow this guide to manually upgrade #DellNetworking OS10 operating system for the latest SmartFabric features. Click To Tweet

Show version

OS10# show version
Dell EMC Networking OS10-Enterprise
Copyright (c) 1999-2019 by Dell Inc. All Rights Reserved.
OS Version: 10.4.3.4
Build Version: 10.4.3.4.213
Build Time: 2019-06-10T09:54:17-0700
System Type: S4112F-ON
Architecture: x86_64
Up Time: 00:39:25

show switch-operating-mode

OS10# show switch-operating-mode

Switch-Operating-Mode : Full Switch Mode

Note that I was rebuilding a new SmartFabric VxRail cluster so I did not want to retain the existing switch configuration. I wanted to demonstrate installing VxRail and SmartFabric using the automated deployment on the latest hardware. By using the sfs_disable.py script – I am destroying the configuration of the existing switch.

Sudo sfs_disable.py

System bash
Sudo sfs_disable.py

image download via scp

OS10# image download scp://root:Password01@192.168.105.129/root/PKGS_OS10-Enterprise-10.5.0.5.661stretch-installer-x86_64.bin

Use ‘show image status‘ for updates

image install

OS10# image install image://PKGS_OS10-Enterprise-10.5.0.5.661stretch-installer-x86_64.bin

Image Upgrade State: install

Installation State: install

State Detail: In progress: Installing
Task Start: 2020-03-20T18:43:30Z
Task End: 0000-00-00T00:00:00Z

boot system standby

OS10# boot system standby

OS10# reload

show boot

OS10# show boot

Current system image information:

Type Boot Type Active Standby Next-Boot

Node-id 1 Flash Boot [A] 10.5.0.5 [B] 10.5.0.2 [A] active

write memory

OS10# write memory

show version

OS10# show version
Dell EMC Networking OS10 Enterprise
Copyright (c) 1999-2020 by Dell Inc. All Rights Reserved.
OS Version: 10.5.0.5
Build Version: 10.5.0.5.661
Build Time: 2020-02-15T00:45:32+0000
System Type: S4112F-ON
Architecture: x86_64
Up Time: 2 days 05:23:04

Next Step; Enable SmartFabric Personality

The next step once the switch is upgraded to 10.5.0.5 would be to enable the SmartFabric for VxRail personality. Follow this guide here.

How to enable SmartFabric for VxRail Step 1

I decided to document the steps I used to enable SmartFabric Services on VxRail.  These are partly notes for myself and Customer Solution Centre engineers that will likely need to showcase this capability very soon.  The demand for this solution is very high and the customers I have met are impressed by what is now possible (wait until you see the roadmap!). 

Note: This is not a guide for end user customers because a lot of what I write about is handled thru our automated deployment appliance; VxRail. A note of thanks to Allan Scott from New York CSC that helped with the first SFS deployment and documentation.

Step 1: Enable SmartFabric Services on the ToR Switch

  • Cabling the ToR Switches
  • Installing/Upgrading OS10 on Dell EMC Switch
  • Enable the VxRail Personality on OS10
  • Ready for Part2 – Deploying VxRail with Smart Fabric Services

Next Blog: Step 2: Deploy VxRail Cluster incl. ToR with VxRail Manager

Step 3: Deploy the SmartFabric OMNI plugin in VMware vSphere

Step 4: Virtualization engineer controls Day 2 Ops for the Full Stack


Getting Started :


How to enable SmartFabric Services on the ToRs

SmartFabric is supported on the 4100 series from Dell EMC. Current models are 10G – S4112 F/T, S4128 F/T or S4148 F/T  (25G coming soon). Sales can order these switches to be delivered from factory with OS10 and licenses already applied.

If you need the latest version of OS10 – get it here: force10networks.com , request login thru support page and download 10.4.1.x. Put OS10 .bin file and licence .xml file in a USB drive – insert USB drive into switch.

Cabling the TORs

First cable up ports 29 and 30 – 100GB cables for ISL (VLT).

Next cable up ports 25 and 26 – 100GB cables for Uplink.

Plug laptop into port 1 on switch.

Connect new VxRail appliances in any other port starting at port 2.

Installing or Upgrading OS10 on switch:

This is an optional step. The switches can be ordered and configured in factory, and so should arrive ready to begin at Step 2.

Connect the laptop to the serial port on one of the switches and start putty. Putty settings are 115200, 8, stop, none, none. I used a USB serial port so my COM port was COM3.

Powerup the serial connected switch – and break into ONIE mode by hitting ESC during bootup.

Choose “onie-discovery-stop” from the menu.

At prompt type: fdisk -l

USB Thumbdrive should be /dev/sdb1

mkdir /mnt/usb

mount -t vfat /dev/sdb1 /mnt/usb

Install OS10:

cd /mnt/usb

onie-nos-install /mnt/usb/XXXXXXXXXX.bin

Check IOS10 Version & Install License:

show version

show license status (skip next step if already installed)

license install usb://xxxxxxxx-NOSEnterprise-License.xml

Configure mgmt interface if required:

conf

int mgmt 1/1/1

no ip address dhcp

ip address 10.204.86.250/24

no shut

exit

management route 10.204.86.0/24 managementethernet

exit

Repeat these steps for the second ToR switch.

Optional step. Configure 40GB uplinks:

My showcase lab is using 40GB uplinks rather than 100GB so I needed to change the profile of the uplinks before applying the VxRail SFS personality. You can skip this step if you are using 100GB links.

OS10(config)# switch-port-profile 1/1 profile-2

Warning: Switch port profile will be applied only after a save and reload. All management port configurations will be retained but all other configurations will be wiped out after the reload.

OS10(config)# exit

OS10# write memory

OS10# reload

Enable the VxRail personality:

The SFS personality script is included in OS10. Once applied to each ToR switch, the switches will reboot with SmartFabric Mode enabled and you are now ready to perform a VxRail deployment from the VxRail Manager.

system bash

sudo sfs_enable_vxrail_personality.py -d 20 -a-m 2002

‘-d 20’ is a unique Domain ID that you assign to each cluster

‘-m 2002’ is a non-routed vlan used to do the initial build, local to the ToR switches only (Internal management network)

‘-a’ indicates that the port-channel on the upstream switches is configured with LACP

VxRail personality profile script options :


  
Domain -d <id> Required numeric value unique to data center (1 to 254) applied to ToR switch configuration settings Default: 1
Uplink -u <port,port> Override default 100Gb uplink ports Default: ports 25& 26
ISL -I <port,port> Override default 100Gb ISL ports Default: ports 29 & 30
Uplink tagging -t Whether external management VLAN is tagged or untagged when passed through uplinks. Default: untagged
Uplink LACP -a Whether LACP is active on uplink port channel (dynamic) or not (static). Default: static
Uplink breakout -b <2X50GE, 4X25GE, 4X10GE> Breakout 100Gb uplinks. Used to support connectivity to upstream switches without 100Gb ports
Management VLAN -m <VLAN> VxRail Cluster Build Network VLAN.
Default: 1

Validate Personality:

system bash

sudo sfs_validate_vxrail_personality.py

Links to useful guides that helped us document this build:

VxRail Fabric Automation SmartFabric Services User Guide

Dell EMC OpenManage Network Integration for VMware vCenter

How to Install Dell Networking FTOS on Dell Open Networking (ON) Switches


SmartFabric is Smart for VxRail HCI

My guide to enable SmartFabric on Dell EMC switches is here.

No time to read? Listen to an interview with Barry Coombs from ComputerWorld UK on SmartFabric.

Now with SmartFabric for @VxRail, Ned can still own the core, and leave the #HCI network problems to the Virtualization team. Click To Tweet

Is networking in HCI complex?

Back in October I asked the question, is HCI networking easy? I stand by my assertion that it is already pretty simple once you understand the converged design for HCI does not require separate physical fabrics. Once you setup your required VLANS and appropriate MTU and multicast for IPv6 you are 90% of the way done.  So if it’s so easy already, then why am I so excited about SmartFabric for VxRail appliances? Start up a conversation with Ned the network engineer about the automated deployment and simplified life-cycle management capabilities of SmartFabrics and you will get back a blank dull stare. “That’s just a python script” Ned the Network Admin will say, “Take your fancy sales patter down to the Virtualization guys, we don’t want your kind round here!”

Ned the NetAdmin says “You’ll never take my switches!”

It’s not easy to impress a NetAdmin

Ned has a point I suppose. The Network Admins job is to move packets reliably from one part of the network to the other, monitor the network for any problems, and to design and build future networks.  The last thing Ned needs is dealing with end users complaining about network problems caused by poorly designed applications (its never the networks fault!). This is actually the main selling point FOR the use of Smart Fabrics. Let me explain why.

Before you understand why SmartFabric for VxRail, you have to first understand the reason for choosing VxRail appliances in the first place.  I have written a few Blogs on this already here and here . In short it’s an engineered solution for vSAN that comes from Dell EMC fully validated and tested and automatically deployed and updated throughout its life-cycle.

VxRail appliances don’t require a Storage expert or Server guru and it even saves the Virtualization admin from having to spend countless hours reading design and deploy documentation. After deployment is done, day 2 simplified operations begins. As a result maintenance and updates and upgrades are made easy through a single bundle file that covers the entire stack (not just the software on top of somebody else’ hypervisor – and includes the hardware too.

Can SmartFabric simplify HCI?

The last part of the HCI architecture that needed simplification was the network, so Dell EMC has had its sites on Ned’s cubicle for a while now. If we are going to provide a fully automated deployment experience for our customers, it only makes sense to include the Top of Rack switches that are being used by VxRail appliances.  After all why would the network admin want to be responsible for those HCI host ports anyways? If something goes wrong in the vSAN stack, Ned doesn’t want to be dragged in to a War Room to defend last weekend’s network changes. Ned knows the changes the network team made at the weekend were to the core only.

Now with Smart Fabric for VxRail, Ned can still own the core, and leave the HCI network problems to the Virtualization team.  SmartFabric will fully configure a redundant TOR fabric for VxRail, and continue to maintain the network for the life-cycle of the HCI solution. When it’s time to patch the HCI network, SmartFabric will provide a bundle file, and perform a non-disruptive rolling upgrade to the network TORs leaving Ned free to watch old episodes of Futurama. If the HCI team needs to expand their existing VxRail cluster by adding a new node, then SmartFabric will fully automate the changes to the TOR switch, no need for Ned to ever get involved.

Is BYO Networking still an Option?

One of the advantages for VxRail customers has been the fact that it is BYON (Bring Your Own Networking). This means that Dell EMC does not force you to take a switch from their portfolio into your datacenter. For some customers, this would be non-negotiable.  They may have standardized on a specific brand and prefer to stay that way, no matter what they run at the Storage or Virtualization layer.  VxRail networking is compatible with any modern low latent switch and the introduction of VxRail SmartFabric does NOT mean that the BYON option is no longer a choice. Hopefully the automation that comes with SmartFabric for VxRail will entice some customers to converge the entire HCI stack and give Ned some peace of mind.

Here is a great Blog on vSAN and Network Switch choices from @LostSignal on this Blog: https://blogs.vmware.com/virtualblocks/2019/03/21/designing-vsan-networks-2019-update/

Update! This week I will be meeting Hasan Mansur at the Limerick Customer Solution Center who writes a great Dell EMC networking blog at https://hasanmansur.com/ . Hasan has written two great articles there about SmartFabric Services. Please check it out Part 1 here and Part 2 here.

Is HCI networking easy?

Even though hyper-converged solutions have been  one of the hottest trends in the Datacenter since virtualization, you will still meet traditional architects that are seeing this technology for the first time.  Many times the customer will come to the conversation with just the virtualization lead,  sometimes they will bring the Storage or Compute team, but often they will forget to tell the Networking team any of their plans (no wonder the network engineer can be so grumpy).  This can prove problematic for a networking team that is not familiar with a few of the basic HCI requirements. Continue reading “Is HCI networking easy?”

error

Enjoy this blog? Please spread the word :)