Using the new GUI for SmartFabric VxRail Deployment

In the last SmartFabric blog I covered how to enable SmartFabric on your Dell OS10 switch and to declare the personality. Building out a new SmartFabric VxRail cluster from a greenfield install requires two Top of Rack switches with the Leaf personality. Once this task is done, simply rebooting the ToRs will give you access to the GUI where everything required to setup the new fabric is achieved. This could not be simpler.

Login to SmartFabric VxRail Master

In the last step of the previous blog we setup the ICL ports between the Leafs and triggered an election by rebooting them. Now we need to logon to the Master switch of the new Leaf pair. If you hit the wrong one, you will be presented with the view below and know you are on the secondary. Just click the link on the ip address to reach the Master.

We can logon to the SmartFabric VxRail master by the default username/password of admin/admin. Here we are presented with the dashboard. There are two required steps to perform here.

Complete Fabric Setup

We can already see that the two Leaf switches have begun the initial fabric creation. The new topology shows the ICL connections. The first required step before we deploy SmartFabric VxRail is to Update Default Fabric Switch Names and Descriptions.

The second required step is to configure Uplinks. You have choice to setup Layer 2 or Layer 3 uplinks here. This can be changed later, but required for the VxRail deployment so you can reach DNS and NTP services normally. Add the Management network on the last step to reach these services outside of the VxRail rack.

Optional Settings

You are ready to deploy VxRail at this stage unless you require a few optional steps. You might need to make a few changes if your setup falls outside the default port speeds for example.

Configure Breakouts

You can change the uplinks speed from the default port speed (100GB). There are several options available depending on the switch you choose and the port you end up using. You might also need to adjust the Jump port speed if you plan to use a 1GB laptop nic in a 25GB switch.

Configure Jump Port

The jump port is used to deploy VxRail by using a laptop to reach the default client control vlan where VxRail Manager is setup initially. The Jump port can now be selected on any port. This is much better than the last version that always selected port 0 as the default jump port.

Deploy VxRail on SmartFabric

You can now deploy your first VxRail cluster into the SmartFabric enabled fabric. There will be one new step in the VxRail Manager deployment GUI. Once you confirm that you want to allow VxRail Manager to control the switch configuration, the rest of the setup is totally automated!

SmartFabric Services for VxRail latest Updates and Install Guide

Its been a little over a year since we launched SmartFabric Services for VxRail. This was initially a neat little solution for single rack deployments. We were the first in the industry to automate the entire HCI deployment. The vision was always to go beyond the Top of Rack and automate Leaf and Spine architectures as networking is still the challenge for HCI solutions. Now we have.

SmartFabric for VxRail New Features

  • Zero-Touch automated deployment of Leaf/Spine
  • Enhanced GUI for Leaf and Spine Personality Management
  • Single Rack or Multi Rack VxRail Cluster deployments in a single site
  • One or more VxRail Clusters connected to a single fabric
  • Ability to connect non VxRail devices to the fabric
  • Fabric expansion automation
  • Lifecycle management of Leaf/Spine from vSphere OMNI plugin
  • Switch replacement automation

I have been keeping a close eye on the release of this software as I wanted to get it setup to demo to customers at the Customer Solution Center. I also wanted to quickly get a out a new Blog series to replace the original one that was so popular with our Partners and the PreSales community.

So before we get started, lets review the latest supported versions of the new SmartFabric Services for Vxrail solution.

Supported Software versions

Link to InfoHub where the latest detail on supported versions and more!

Lets get started. Follow these Steps.

Getting SmartFabric for VxRail is now even easier than before and I am going to document the 4 Steps I followed to get it up and running in my lab.

Step 1 – Enable SmartFabric Services on OS10 Leaf switch

Note: This is not a guide for end user customers because a lot of what I write about is handled thru our automated deployment appliance by Partners or Services teams. So please proceed with caution.

Check that the OS10 version is EXACTLY 10.5.0.5

Login to the Leaf and Spine switches and check that they are running the correct supported version of OS10 for VxRail installs. If the version is not running OS10 version 10.5.0.5 (not older or newer) then please upgrade or downgrade. You can use this blog i wrote earlier to perform an upgrade or downgrade from the switch CLI.

Configure Leaf switch OOB Management.

Put an ip address on both of the Leaf switches before you enable the SmartFabric personality in the following step.

OS10# configure terminal
OS10(config)# interface mgmt 1/1/1
OS10 (conf-if-ma-1/1/1)# no ip address dhcp
OS10(conf-if-ma-1/1/1)# ip address 192.168.105.235/24
OS10 (conf-if-ma-1/1/1)# no shutdown
OS10(conf-if-ma-1/1/1)# exit
OS10(config)# management route 192.168.0.0/16 192.168.105.254
OS10(config)# end
OS10# write memory

Enable SmartFabric Services for VxRail on the Leaf and Spine.

OS10(config)# smartfabric l3fabric enable role LEAF vlti ethernet 1/1/14-1/1/15

We can also enable the Spine switches now. These are not required yet for the VxRail deployment so we will set them up later.

OS10(config)# smartfabric l3fabric enable role SPINE

When the SmartFabric Services for VxRail personality is applied, the switches will reload.

Verify SFS Cluster & verify the Master

OS10# show smartfabric personality

Personality :L3 Fabric
Role :LEAF
ICL :ethernet1/1/14, ethernet1/1/15


OS10# show smartfabric cluster

CLUSTER DOMAIN ID : 100
VIP : fde2:53ba:e9a0:cccc:0:5eff:fe00:1100
ROLE : MASTER
SERVICE-TAG : D21WNK2
MASTER-IPV4 : 192.168.105.235
PREFERRED-MASTER : true

Connect to SmartFabric GUI

Now we can connect to either Leaf OOB Management ip and access the GUI. If we know the Master IP – then use this URL to access GUI:

https://MASTER_IP_ADDRESS

The next blog will explain the new SmartFabric GUI and simplified VxRail deployment process. Stay Tuned.

How to Upgrade SmartFabric OS10 via cli

Download the latest SmartFabric OS10 operating system.

Before we upgrade SmartFabric OS10 operating system on our switches we need to get the latest compatible OS10 version. The latest version of SmartFabric OS10 Enterprise Edition software is located on the Force10 Networks portal here. If you plan to upgrade SmartFabric OS10 with VxRail then you should consult the guide here. This matrix tracks the OS10 Switch Operating system version, OMNI plugin version and VxRail software that align.

Follow this guide to manually upgrade #DellNetworking OS10 operating system for the latest SmartFabric features. Click To Tweet

Show version

OS10# show version
Dell EMC Networking OS10-Enterprise
Copyright (c) 1999-2019 by Dell Inc. All Rights Reserved.
OS Version: 10.4.3.4
Build Version: 10.4.3.4.213
Build Time: 2019-06-10T09:54:17-0700
System Type: S4112F-ON
Architecture: x86_64
Up Time: 00:39:25

show switch-operating-mode

OS10# show switch-operating-mode

Switch-Operating-Mode : Full Switch Mode

Note that I was rebuilding a new SmartFabric VxRail cluster so I did not want to retain the existing switch configuration. I wanted to demonstrate installing VxRail and SmartFabric using the automated deployment on the latest hardware. By using the sfs_disable.py script – I am destroying the configuration of the existing switch.

Sudo sfs_disable.py

System bash
Sudo sfs_disable.py

image download via scp

OS10# image download scp://root:Password01@192.168.105.129/root/PKGS_OS10-Enterprise-10.5.0.5.661stretch-installer-x86_64.bin

Use ‘show image status‘ for updates

image install

OS10# image install image://PKGS_OS10-Enterprise-10.5.0.5.661stretch-installer-x86_64.bin

Image Upgrade State: install

Installation State: install

State Detail: In progress: Installing
Task Start: 2020-03-20T18:43:30Z
Task End: 0000-00-00T00:00:00Z

boot system standby

OS10# boot system standby

OS10# reload

show boot

OS10# show boot

Current system image information:

Type Boot Type Active Standby Next-Boot

Node-id 1 Flash Boot [A] 10.5.0.5 [B] 10.5.0.2 [A] active

write memory

OS10# write memory

show version

OS10# show version
Dell EMC Networking OS10 Enterprise
Copyright (c) 1999-2020 by Dell Inc. All Rights Reserved.
OS Version: 10.5.0.5
Build Version: 10.5.0.5.661
Build Time: 2020-02-15T00:45:32+0000
System Type: S4112F-ON
Architecture: x86_64
Up Time: 2 days 05:23:04

Next Step; Enable SmartFabric Personality

The next step once the switch is upgraded to 10.5.0.5 would be to enable the SmartFabric for VxRail personality. Follow this guide here.

VxRail SmartFabric networking for Day 2 Ops

Step 1: Enable SmartFabric Services on the ToR Switch

Step 2: Deploy VxRail Cluster incl. ToR with VxRail Manager

Step 3: Deploy the SmartFabric OMNI plugin in VMware vSphere

Step 4: Virtualization engineer controls Day 2 Ops for the Full Stack

Network Admins calm down and look here.

The network admins are not usually in favor of anything that replaces their day to day jobs. If you can show them the finished solution, you may be able to sway them in favour. Make sure to give them a demo of the OMNI plugin running inside vSphere using VxRail SmartFabric enabled switches.

Click into the Omni dashboard.

The OMNI dashboard displays the current information on the VxRail SmartFabric enabled Switches. Here the Virtualization engineer has visibility to the OS10 version and the VLANS currently setup. That’s it? Not sure what NetAdmin Ned was expecting but there is no Network-Chaos-Now button!

Lets give Ned a use-case for SmartFabrics

You can describe a scenario to Ned that is common. It’s time to add a new host to the existing cluster. Normally you would ask Ned to prepare the host network ports. He would label them add all the required VLANS and to make sure the settings match with the existing host ports.

New Hosts detected with VxRail SmartFabric

With SmartFabrics no need to bother Ned

Adding hosts to HCI clusters are now done solely by the Virtualization Admin without any need to ask the NetAdmin to prepare host ports. It is all taken care of automatically by the VxRail SmartFabric services and the OMNI plugin.

Choose the discovered host.

Choose the discovered VxRail host

Enter vCenter credentials.

vCenter credentials

Add the Ip address information.

configure ip address

Run validation.

Validate the VxRail SmartFabric hosts

Validation must pass first.

validation can catch human errors

Once passed, host is added automatically.

new hosts added automatically to VxRail SmartFabric cluster

Watch quick Ned! Ports are configured.

hosts connected to VxRail SmartFabric are automatically configured

Hands free with VxRail SmartFabric services.

vSphere automation for VxRail SmartFabric

How to deploy the SmartFabric OMNI plugin in VMware vSphere Step 3

Step 1: Enable SmartFabric Services on the ToR Switch

Step 2: Deploy VxRail Cluster incl. ToR with VxRail Manager

Step 3: Deploy the SmartFabric OMNI plugin in VMware vSphere

EXTRA STEP Added Feb. 2020. See Below in yellow.

Next Blog : Step 4: Virtualization engineer controls Day 2 Ops for the Full Stack

Installing the SmartFabric OMNI plugin

At this point we have already run the first two steps to get SmartFabric installed, our Switches have SmartFabric services enabled and the VxRail Manager Gui deployment has automatically configured the switch and disabled the CLI. Now we will add the Open Manage Network Integration or SmartFabric OMNI plugin to vSphere, giving the virtualization engineer full visibility and control of the dedicated HCI switch fabric.

Download the OMNI plugin VM

We begin by deploying the OMNI VM ova, which is available from VMware Solution Exchange here. It seems to bounce you now to the Dell Support site where you can get the latest version here. As of June 2019 the latest version is 1.1.18.

Deploy the OVA

These steps are straightforward. Isn’t great that we can do the OVA deployment thru the HTML5 interface? Be sure to match up the source and destination networks correctly at Step 7 in the process. Later on we will configure an ipv4 address on vCenter network and enable ipv6 on the MGMT network.

Power on the VM and connect to a console on the OMNI vm. You need to establish a password here for the admin user. The default username and default password is setup as admin / admin.

Configure the OMNI appliance

Since this is a first run deployment, we are going to select option “0. Full Setup” at the Menu. As you can see you can re-run and do other admin tasks later if required from this interface.

Before we active the first connection, we will setup the Profile name “vCenter Server Network” and assign a valid ipv4 address for the OMNI VM here.

The second connection is for ipv6 discovery on the VxRail Management network, no ipv4 address needed here, set to disabled.

Set IPv6 Config to “Link-Local”.

EXTRA STEP Added Feb. 2020.

Navigate to Routing and select Edit. Select Add.

For “Destination/Prefix” enter “fde1:53ba:e9a0:cccc::/64”.

Leave “Next Hop” empty, Leave”Metric” default of 0. Click OK

Be sure to activate both networks on the last screen.

You will be prompted at the CLI again for NTP, SSL Cert, the appliance ip address, FQDN for VCSA, VC Username and Password, and finally the OMNI ip for REST and REST username and password.

Log in and out of vSphere and the OMNI plugin will appear.

How to deploy SmartFabric for VxRail Step 2

Step 1: Enable SmartFabric Services on the ToR Switch

Step 2: Deploy SmartFabric VxRail Cluster incl. ToR with VxRail Manager

      • Connect VxRail nodes to deploy SmartFabric enabled Switches
      • Power on first 4 VxRail nodes
      • Deploy SmartFabric VxRail thru the VxRail Manager GUI
      • Sit Back and Relax while VxRail Manager fully automates Bring Up of the entire Hyperconverged Stack (Compute, Storage AND the Network!)

Next Blog: Step 3: Deploy the OMNI plugin in VMware vSphere

Step 4: Virtualization engineer controls Day 2 Ops for the Full Stack

Beginning the VxRail GUI deployment

Connect the VxRail nodes to deploy SmartFabric enabled switches. Power on the nodes. Connect the laptop to the first port (port 1 only!) in the SFS switch. This is the “jump port” and will allow access to the default ip address [https://192.168.10.200] of the VxRail Manager VM. The VxRail Manager VM is already deployed in the appliances in factory.

VxRail Manager GUI install
The Vxrail Manager GUI interface used to deploy a new cluster.

VxRail will deploy SmartFabric now

Since VxRail version 4.7.100 there is now the ability for the GUI driven VxRail Manager install to detect Dell EMC SmartFabric enabled switches. This automates configuration of the switches during the full stack deployment.

deploy SmartFabric
Here we can see that VxRail Manager has detected a Smart Fabric is available to configure thru VLAN 3939 the discovery VLAN for nodes and switches.

Setup the REST user to deploy SmartFabric

When the installer selects to deploy SmartFabric switches the next step will be to configure the Rest User account on the SFS enabled switch. If its a first run, a new password will be created. If you are setting up a second VxRail cluster into an existing SmartFabric, then the current password will be requested. The Rest User account will be documented in the Pre-Install Checklist by the installer.

Setting REST User password
The REST user account needs a password set.

New option to choose 2-Node cluster

The ability to deploy 2-node clusters is now an option with VxRail 4.7 versions. The GUI install gives you a choice for 2-Node here. For all other VxRail deployments the default option is selected.

New 2 node VxRail option during GUI install
You need at least 3 or 4 nodes to start the Smart Fabric node setup.

The initial cluster deployment needs to be 3 or 4 nodes. Normally the node with lowest serial number wins the VxRail Manager election process. This is signified by the small “house” icon. If you don’t see all the powered on nodes, try a reboot of the missing node and wait. It should be detected by ipv6 multicast using the loudmouth service on the discovery VLAN 3939.

VxRail node discovery
VxRail node discovery

Automate the build with JSON

Remember we use the Pre-Install checklist is used to capture all the required information to make a VxRail deployment in a customer Datacenter go perfectly. It can also be used to generate a JSON file to help eliminate human error during the GUI setup.

using JSON file
Use the JSON file – its way quicker!

Here we have used the JSON file to populate all the fields in the GUI install of VxRail. You only need to verify that everything is correct and provide the passwords.

New external MGMT VLAN option

There is a new option during the VxRail 4.7 version of the install that asks for a Management VLAN ID. The default is 0 to allow for Native VLAN to be used, or you can specify a different public management VLAN here. It should be different to the internal management VLAN used to separate the cluster build. The Internal Management VLAN was the -m option set in Step 1. This is very easy with SmartFabric now, as the VxRail Manager will make the necessary port configuration changes on the SFS enabled switches.

Now you can specify a external Management VLAN during GUI install rather than Native VLAN.

VxRail Manager automates the Switch configuration

There is a new task at the end of the GUI install and just before validation. SmartFabric services will now configure all the required VLANS that were specified, including Management, vMotion, vSAN and VM traffic.

SmartFabric switch automation
SmartFabric is smart! See it does the Network Admin tasks now!
switch fully configured.
Switch fully configured.

Validation eliminates human error

Trust but verify. Even with the SmartFabric automation, VxRail Manager still performs the important task of validating the install. Validation must pass before the actual deployment begins. This will capture anything amiss such as missing DNS entries, IP address conflicts , missing NTP. Basically anything that could prevent the build from completing successfully.

Validation
Trust but Verify = Validate!

Once validation has passed there is an opportunity to download a copy of the JSON file (useful if Step-By-Step was selected).

validation passed
Validation successful.

It is now time to kick-off the build. This is a fully automated process. There is zero value in manually setting up VSAN. By automating the process Dell EMC can be certain that every customer install of VxRail clusters are fully aligned to best practice. No snowflakes!

fully automated vSAN deployment
Go get a coffee – VxRail Manager will take care of the rest!
deploy SmartFabric
Hooray! The VxRail vSAN cluster is complete. Time to setup the OMNI plugin for day 2 operations of the SmartFabric.

SmartFabric is Smart for VxRail HCI

My guide to enable SmartFabric on Dell EMC switches is here.

No time to read? Listen to an interview with Barry Coombs from ComputerWorld UK on SmartFabric.

Now with SmartFabric for @VxRail, Ned can still own the core, and leave the #HCI network problems to the Virtualization team. Click To Tweet

Is networking in HCI complex?

Back in October I asked the question, is HCI networking easy? I stand by my assertion that it is already pretty simple once you understand the converged design for HCI does not require separate physical fabrics. Once you setup your required VLANS and appropriate MTU and multicast for IPv6 you are 90% of the way done.  So if it’s so easy already, then why am I so excited about SmartFabric for VxRail appliances? Start up a conversation with Ned the network engineer about the automated deployment and simplified life-cycle management capabilities of SmartFabrics and you will get back a blank dull stare. “That’s just a python script” Ned the Network Admin will say, “Take your fancy sales patter down to the Virtualization guys, we don’t want your kind round here!”

Ned the NetAdmin says “You’ll never take my switches!”

It’s not easy to impress a NetAdmin

Ned has a point I suppose. The Network Admins job is to move packets reliably from one part of the network to the other, monitor the network for any problems, and to design and build future networks.  The last thing Ned needs is dealing with end users complaining about network problems caused by poorly designed applications (its never the networks fault!). This is actually the main selling point FOR the use of Smart Fabrics. Let me explain why.

Before you understand why SmartFabric for VxRail, you have to first understand the reason for choosing VxRail appliances in the first place.  I have written a few Blogs on this already here and here . In short it’s an engineered solution for vSAN that comes from Dell EMC fully validated and tested and automatically deployed and updated throughout its life-cycle.

VxRail appliances don’t require a Storage expert or Server guru and it even saves the Virtualization admin from having to spend countless hours reading design and deploy documentation. After deployment is done, day 2 simplified operations begins. As a result maintenance and updates and upgrades are made easy through a single bundle file that covers the entire stack (not just the software on top of somebody else’ hypervisor – and includes the hardware too.

Can SmartFabric simplify HCI?

The last part of the HCI architecture that needed simplification was the network, so Dell EMC has had its sites on Ned’s cubicle for a while now. If we are going to provide a fully automated deployment experience for our customers, it only makes sense to include the Top of Rack switches that are being used by VxRail appliances.  After all why would the network admin want to be responsible for those HCI host ports anyways? If something goes wrong in the vSAN stack, Ned doesn’t want to be dragged in to a War Room to defend last weekend’s network changes. Ned knows the changes the network team made at the weekend were to the core only.

Now with Smart Fabric for VxRail, Ned can still own the core, and leave the HCI network problems to the Virtualization team.  SmartFabric will fully configure a redundant TOR fabric for VxRail, and continue to maintain the network for the life-cycle of the HCI solution. When it’s time to patch the HCI network, SmartFabric will provide a bundle file, and perform a non-disruptive rolling upgrade to the network TORs leaving Ned free to watch old episodes of Futurama. If the HCI team needs to expand their existing VxRail cluster by adding a new node, then SmartFabric will fully automate the changes to the TOR switch, no need for Ned to ever get involved.

Is BYO Networking still an Option?

One of the advantages for VxRail customers has been the fact that it is BYON (Bring Your Own Networking). This means that Dell EMC does not force you to take a switch from their portfolio into your datacenter. For some customers, this would be non-negotiable.  They may have standardized on a specific brand and prefer to stay that way, no matter what they run at the Storage or Virtualization layer.  VxRail networking is compatible with any modern low latent switch and the introduction of VxRail SmartFabric does NOT mean that the BYON option is no longer a choice. Hopefully the automation that comes with SmartFabric for VxRail will entice some customers to converge the entire HCI stack and give Ned some peace of mind.

Here is a great Blog on vSAN and Network Switch choices from @LostSignal on this Blog: https://blogs.vmware.com/virtualblocks/2019/03/21/designing-vsan-networks-2019-update/

Update! This week I will be meeting Hasan Mansur at the Limerick Customer Solution Center who writes a great Dell EMC networking blog at https://hasanmansur.com/ . Hasan has written two great articles there about SmartFabric Services. Please check it out Part 1 here and Part 2 here.

error

Enjoy this blog? Please spread the word :)