InterWeb.org.uk

Interweb Blog Hyper-V, App-V, Dell Equallogic, Forefront TMG Security, DPM 2010, SCVMM

21Apr/1113

Dell EqualLogic PS Series iSCSI SAN in a Hyper-V Environment

Following on from my Hyper-V Post I thought I would carry on the theme of High Availability and clustered services.  Dell's EqualLogic PS Series iSCSI SAN  optimise's resources by automating volume and network load balancing.  A PS Series Group consists of one or more PS Series Array.  Group Members are connected to an IP network and managed as a single system.

iSCSI Protocol.

iSCSI¬†or internet small computer systems interface to give it its full title, is a block level storage protocol that can be used to create a storage network over Ethernet.¬† iSCSI¬†uses ethernet as a transport medium for data from servers to storage devices or SAN'S¬†(Storage Area Networks) such as EqualLogic.¬† iSCSI¬†ÔĽŅencapsulates SCSI commands into TCP/IP and sends these commands over standard Ethernet, because of this iSCSI can be sent over LAN's and WAN's and these Arrays do not have to be located in the same room or even the same¬†Galaxy as your servers.

iSCSI Initiators.

iSCSI initiator's are the clients that connect to the iSCSI targets.

  • Software initiators, can use an existing¬†network card and stack¬†on the same network as¬†the iSCSI target¬†to discover and connect.¬†¬†As this is done in the operating system, there is a certain¬†overhead on processing this.
  • Hardware initiators¬†use dedicated hardware to implement iSCSI¬†and removes the iSCSI¬†and TCP processing overhead away from the operating system and¬†over to¬†the dedicated hardware.¬† HBA's¬†or Host Bus Adapters are typically 10GB NIC's with a TCP Offload Engine¬†(TOE) and a SCSI bus adapter.¬† TCP Offload NICS¬†are a half way house between software initiator's¬†and full HBA's,¬† the processing of TCP/IP is offloaded on to the TOE cards processor while the host Operating System still handles the iSCSI processing.

iSCSI Targets.

  • Hardware iSCSI¬†targets usually reside on an array¬†of some description, such as a EqualLogic.¬† You can have many iSCSI¬†targets on an iSCSI¬†server or array.¬† Software targets are available in many operating systems such as Linux, BSD, and Windows with third-party¬†software such as StarWinds¬†iSCSI SAN software.
  • LUN or Logical Unit Number is¬†in essence¬†a numbered disk created on your iSCSI¬†SAN with management software and emulates a SCSI disk drive (The Target).¬† An initiator hardware or software can now negotiate a connection with this target, now you have an iSCSI¬†connection that¬†sends SCSI commands encapsulated within TCP/IP providing a connection to¬†a LUN via a target, emulating a SCSI disk drive.¬† iSCSI¬†Luns appear as physical¬†raw disks, the system to which the iSCSI target is attached treats the LUN as a local disk and thus manages the filesystem itself.

To create a highly available virtual infrastructure with Microsoft Hyper-V Server you will need to implement an SAN of some description whether Fibre Channel or iSCSI hardware based or software, this decision will be completely personal to your organisation and budget available.  I have read blogs outlining implementations of a HA Hyper-V Cluster using StarWinds iSCSI SAN software, personally I would always go for a hardware SAN implementation.

We currently have a Dell EqualLogic PS4000 and PS6000 in a Group with a 3TB LUN for use as a Cluster Shared Volume for our Hyper-V Cluster. I cover CSV's in my Hyper-V post.  A PS series group is an iSCSI SAN consisting of one or more PS series storage arrays connected to an IP network and managed as a single system.

EqualLogic Hardware Setup.

Providing availability and performance are the key goals of a SAN and implementing the correct hardware setup is essential.  As a minimum one network interface and one active control module connected to a switch is needed to provide connectivity to the EqualLogic array, this is a basic requirement and provides no redundancy.

to provide best availability and performance the following should be taken into consideration;

Redundant Power  Supply Modules.

Two Power Supply Modules providing power redundancy in the event of failure.

Redundant Network Connections.

Control Modules are made up of more than one Network interface depending on the type of controller Dell PS4000 Type8 (3 interfaces) and PS6000 Type 7 (4 interfaces), this provides network interface redundancy on the active control module as well as increased performance.  The EqualLogic intelligently load balances traffic through the active network controller and its interfaces.

PS4000 Rear View - Redundant Control Modules - Redundant Power Supply Modules

PS4000 Rear View

 

Two Control Modules to provide Control Module failover.

Two Control Modules to provide control module failover, one active control module and one Secondary control module (Active-Passive), if the active control module fails then the secondary control module becomes active and network communication carries on with no interruption to services.  This is also the case when performing Firmware upgrades, the control modules are upgraded and failover with no interruption to service.

Redundant Network Switches to provide Switch failover.

Distribute your interface connections across more than one networked switch, these switches should be gigabit.  The switches will be connected with either high-capacity stacking cables connected directly into the backplane of the switch or with an ISL (inter-switch link) with enough capacity to carry any intra-array traffic. Currently we have 4x gigabit ports in a Link Aggregation Group or LAG in use as an ISL, I will re-evaluate this as and when network load dictates.

Equallogic Redundant Network Configuration

EqualLogic Redundant Network Configuration-

 

Redundant Network Paths - between Client and Target.

Creating a Multipathed I/O (MPIO) connection between the iSCSI Client and the iSCSI target ensures that there is no single point of failure between your client and array.  EqualLogic provide some good Multipathing tools in its Host Integration Toolkit (HKIT) for achieving this.  Multipathing requires more than one interface at the client providing interface redundancy, connecting to each networked switch in the network setup, in the event of a switch failure the multipathed client will still have an active connection remaining switch.

Redundant Network paths MPIO

Redundant Network paths MPIO

 

Note that if you have the management network enabled 1 link from each control module will be to a seperate physical switch or VLAN that can communicate with your management network.

EqualLogic Software Setup.

In a Multi-Subnet SAN group, each member must have one interface on the same subnet as the Group IP address and the Group Management network if you have chosen to enable this.  Enabling the Management network allows you to manage the EqualLogic Array or group from your management network using SANHQ or the Java based Group Manager from the array.

Host Integration Tool KIT (HITKIT)

Now that the EqualLogic is set to go you can use the Dell Host Integration Tool Kit or HIT KIT to set up the EqualLogic.  The latest version of the HITKIT can be downloaded from EqualLogic Support Site; https://www.equallogic.com/support/ - you will need to register to obtain a login if you don't already have one.

Host Integration Tool Kit

Host Integration Tool Kit

 

Remote Setup Wizard (RSW)

The EqualLogic Host integration Tool Kit for Microsoft allows for fast provisioning of Windows systems on a PS Series SAN.  The Remote Setup Wizard or RSW in the HITKIT discovers any uninitialized PS Series Arrays and takes you through th setup of the Array.  It goes without saying that the computer you are running the HITKIT and RSW on need to be one the same subnet as the PS Series Array.  Setting up an EqualLogic Array is fast and easy and after the setup is done managing and monitoring your array or group is even easier with the tools Dell have provided.

Remote Setup Wizard

Remote Setup Wizard

 

Multipath IO (MPIO)

Multi-path I/O MPIO for short provides fault tollerance and performance benefits, creating more than one path for network traffic from your client (server) to your Equallogic ensures that if connectivity is interrupted on interface A then traffic will continue to be routed through interface B and will carry on with no interruption to service.

HIT KIT MPIO

HIT KIT MPIO

 

Once installed a new tab is added to the iSCSI Initiator Properties control panel, (Dell EqualLogic MPIO).  As you can see from the screenshots below multiple sessions are connected to the EqualLogic target IP, creating a multipath I/O connection to the EqualLogic Volume from the Client.

iSCSIcplMPIO

iSCSIcpl MPIO

MPIOSessions

MPIOSessions

 

With Dell EqualLogic and the Host Integration tool kit for Microsoft, setting up MPIO is simply easy, the setup is clear and concise and saves a lot of time.  MPIO is a must for a highly available setup that requires maximum performance from your EqualLogic iSCSI array.

VSS Hardware Provider

EqualLogics VSS Hardware provider and Microsoft Data Protection Manager 2010 provide a robust backup solution, once again installing the EqualLogic VSS is easy to do and once installed, provides a more efficient way of backing up your VM's using DPM2010.  A snapshot is taken by the EqualLogic and DPM2010 reads the data for the backup from the EqualLogic array snapshot of the volume rather than from live Cluster Shared Volume.  EqualLogic's Hardware based VSS is far more efficient than using the Windows VSS writer and eliminates some of the complications of backing up from a single CSV.  I will do a blog purely on DPM2010 and EqualLogic at which point I will go into further detail on the EqualLogic hardware VSS writer.

Once you have initialized your array you can then manage it with SANHQ also available for download from the EqualLogic customer support site.

Dell SAN Head Quarters for Equallogic

Dell SAN Head Quarters for EqualLogic

 

SANHQ monitors all PS Series Arrays from one location, providing information on performance, Alarms and Health status and can be configured to send out E-Mail alerts based on this information.  SANHQ provides performance reporting this information is very important when analysing trends and performance.  SANHQ is very intuitive and provides and excellent view of what is happening with your EqualLoigc Array/Group.

From SANHQ the EqualLogic Group Manager can be launched where configuration tasks such as creating volumes, setting snapshot schedules and upgrading member firmware can be completed through a nice clear GUI.

grpmanager

EqualLogic Group Manager

 

In conclusion, Dell's EqualLogic PS Series iSCSI SAN has proven to be a reliable, robust and easy to use SAN.  Also with the Host Integration Tools provided integrating it with Microsoft Hyper-V Server 2008 R2 has been quick and easy.  There is a lot more I would like to go into about EqualLogic but there is far too much to go into in one post.  I suggest checking out EqualLogic http://www.equallogic.com/ and see for yourself that power that comes with this Enterprise level SAN.

My next post will be about configuring Dell PowerConnect 5400 Series non-stackable switches for use as a backend iSCSI network, best practices and an example config.  Also as of today there is a new firmware update for EqualLogic V5-0-5 I will document and post all steps involved in performing a firmware update on an EqualLogic member.

 

 

    Comments (13) Trackbacks (0)
    1. What a great implementation.

    2. Hi there would you mind letting me know which hosting company you’re utilizing? I’ve loaded your blog in 3 completely different browsers and I must say this blog loads a lot faster then most. Can you recommend a good web hosting provider at a fair price? Thanks, I appreciate it!

    3. Hi, great blog…
      I have implemented a 4 node Hyper-V R2 SP1 Cluster with CSV and PS4000XV. All was good until I started using DPM 2010 to backup my guest VMs and now I continually get lots of ‘5120 cluster errors “BAD NETWORK PATH” all IO will be queued’ when backups start.
      You seem to have a very similar setup to me… have you come across these cluster errors? I am using the Equallogic VSS provider and SAN firmware is 5.04 with HIT Kit 3.5.1. HELP!!! ūüôĀ

      • Hi, Thanks for the comment.

        I have never experienced this issue. First things I would check are that ‘Client For Microsoft Networks’ and ‘File and Printer Sharing for Microsoft Networks’ are enabled on all nic’s that are used for cluster traffic these need to be enabled to support SMB. If you use your iSCSI nic’s for cluster traffic then these items also need to be enabled on them, although I do not suggest using your iSCSI nic’s for cluster communication.

        If you are using Hyper-V Server Core then take a look at my nvspbind blog for how to check adapter bindings.

        If you have to use your iSCSI nic’s in your cluster network then you have no option but to enable ‘Client For Microsoft Networks’ and ‘File and Printer Sharing for Microsoft Networks’.

        Let me know how it goes.

        • Hi, quick update that this looks like an issue around the configuration of the Etherchannel (LAG) link between our two ISCSI switches. We are using Cisco 3560 switches and not powerconnect. Lots and lots of discards on these ports. As a test I turned off one of the switches and the 5120 errors disappeared. Looks like flowcontrol / jumbo frames on the etherchannel and am investigating further…

    4. Hi,

      great blog!

      We have implemented a similar Architecture using Dell m1000 and m610 Blades and an P6010. Everything works great until DPM starts tu backup. Some of the issues are already known and fixed by Microsoft with hot fixes after Server 2008 R2 SP1. But I have still one unanswered question: When backup up a Virtual Machine which has it’s VHDs on a CSV, it seems, that after doing the snapshot (of course, with the Equallogic Hardware VSS Provider), the whole VHD’s are read from the snapshot (according to the information of SANHQ), and then delta is transported to the DPM Server. From my point of view, this seems to be the only way, how it could work with CVS’s – isn’t it? Oder Vendors seem to be smarter: (http://www.hds.com/assets/pdf/hitachi-implementation-guide-protecting-hyper-v-csvs.pdf) – they argue, that they have a change list on the storage, so that only the delta has to be read from the snapshot. What is your opinion about that?
      Finaly we moved to the strategy of backing up the VMs from “inside”, but to be honest, it would be much more easy to maintain from “outside”.
      Thanks,
      Thomas

      • Yes you are right in thinking that when using the EqualLogic Hardware VSS provider a snapshot of the CSV is taken by the EqualLogic and the vhd’s are backed up from the snapshot of the CSV,when the backup is complete the snapshot is then deleted. This is done so that the load on the live CSV is reduced and that performance is not degraded, also the host that the Virtual Machine resides on is only in redirected IO mode while the snashop is taken which is normally a very short period of time.

        When DPM2010 performs an Express Full Backup of Hyper-V protected data the protection agent transfers a snapshot of all blocks that have changed since the previous express full backup, so although the EqualLogic Hardware VSS creates a snapshot of the CSV, DPM2010 only copies the changed data since the last Express Full Backup. DPM2010 does this by using its filter technology to identify the changed blocks instead of reading all of the data or using checksums, this is aimed at reducing the load on the host.

        This filter technology resides with DPM2010 and not with the EqualLogic, the hardware VSS writer just snapshots the CSV so that DPM2010 can read the data. DPM2010 then works out what data has changed and needs to be copied.

        Thanks for you comments, I hope this helps a little.

        Mark

    5. Great read! A very simliar setup that i have got.
      You mentioned that “I will do a blog purely on DPM2010 and EqualLogic at which point I will go into further detail on the EqualLogic hardware VSS writer”.
      I am particularly interested in this.
      I have a CSV from an EqualLogic PS4000 and DPM installed on a seperate physical server. I am struggling with how to use hardware VSS writers for DPM to backup the VM’s on the SAN.
      Any help you could offer (or the afore mentioned blog) would be great!

      Cheers

      • Hi, Thanks for the comment.

        I have been quite busy of late, but have started writing a blog about DPM 2010 and EqualLogic. If you need any help in the meantime, drop me an E-Mail to [email protected] and I will be happy to help as much as possible. I will be pluging on with the blog so should be up soon.

    6. I’m very interested in this as well! I have the same setup as the above poster and am just trying to grasp all the moving parts required to utilize the hardware vss writers. Looking forward to your post!

    7. Excellent post, really enjoyed reading it. Can’t wait for your DPM2010 and EqualLogic hardware VSS blog.

    8. Great article and comments. I can’t wait to read your article. I’ve been having problems with DPM 2010 or EqualLogic hardware VSS writer leaving the snapshot online after backing up. This then has caused issues with some of my VMs.

      ‘servername’ was reset because an unrecoverable error occurred on a virtual processor that caused a triple fault. If the problem persists, contact Product Support. (Virtual machine ID F377C839-2E02-4D92-8011-C4C1AD6010D8)

      After I change the snapshots to offline then the problem clears up.


    Leave a comment

    No trackbacks yet.

    Get Adobe Flash player