SAN-NAS Build

Operating System/Storage Platform: OmniOS/Napp-it
CPU: Xeon E3-1230v2 CPU
Motherboard: Supermicro X9SCM-F-O
Chassis: Rosewill RSV-L4411 Chassis
Drives: 8x Hitachi 2TB Disks, 4x Toshiba 3TB Disks, 1x SanDisk SDSSDP 64gb SSD for ZIL and Read Cache, 1x Crucial M464GB SSD for OS
RAM: 16GB UDIMM Kingston RAM
Add-in Cards: 2x IBM M1015 crossflashed/9211 IT mode, 2x Intel Dual Port Server NIC
Power Supply: Antec EarthWatts 650 Power Supply
Usage Profile: Storage, iSCSI Target, SMB Shares, MediaTomb

IMG_20131111_195707

 

 

IMG_20131111_195858

Pools:

I decided to go with mirrors with both of my pools. While the performance penalty of ZFS pools versus mirrors were negligible, they were noticeable. I decided to go with speed and the redundancy of mirrors.

Bonnie Benchmarks:

Comstar – iSCSI Config

Switch Config:
I am using a 16 port ZyXEL GS1510-16 smart switch configured with 5x VLANs with Jumbo Frames enabled.

napp-it config – Comstar:

Microsoft Hyper-V to VMware ESXi 5.1 Migration

I am going to migrate my server that’s running Server 2008 R2 w/ Hyper-V to ESXi 5.1 and pass through the HBA to oi-napp-it for storage in ESXi, and for SMB/NFS shares.
I have been playing with oi/napp-it for about 5 days now, tweaking, reading, tweaking, troubleshooting failed disks (4 of 8 1TB disks I had at the house were failing or were failed), and I genuinely like it!
I have previous Solaris experience, so Openindiana (http://openindiana.org/) is in concept fairly easy enough for me to use. I like the kernel based SMB/CIFS, it’s a hell of a lot faster than Samba. On the Desktop version of Openindiana, the desktop environment uses GNOME 2 which I love, I am not a fan of GNOME 3, and everything is right where I know it should be in the GUI.
The pkg command for managing application packages works rationally, and I picked up on it’s switches and usage very fast.
napp-it (http://www.napp-it.org/) is very easy to use thanks to the user interface being rationally laid out, it puts all the power of ZFS into a few mouse clicks with gea’s(creator of napp-it) HTML and PERL magic. (I don’t have an issue with CLIs, I have been using command line based operating systems daily since 1995)
I would like to recover the additional 3.63 TB of lost by the ServRAID M1015 w/ 8x 2TB Hitachi 7K3000 in RAID 10 @ 7.27 TB formatted, and use JBOD w/ RAID-Z2 which would get me 10.9 TB formatted.
I have been administering VMware servers for a living for the last 12 years, so of course, ESXi is old hat for me. ESXi 5.1 is a huge improvement over 5.0, which was riddled with bugs and issues that drove me nuts.
I had such a crappy time with ESXi 5.0 when it came out, I blew it away at home, and replaced it with Server 2008 R2 and Hyper-V because it just works.


Current Config:
  • Server 2008 R2 (2k8 box)
  • 16 GB RAM
  • Xeon E3 1230
  • Supermicro X9SCM-F
  • 2x Intel Dual Port NICs
  • Rosewill RSV-L4411
  • Crucial M4 64 GB Boot Drive – 8x 2TB Hitachi 7K3000 – 4x Western Digital RE3 500 GB
  • IBM ServRAID M1015 flashed w/ LSI SAS9211-8i in RAID 10 (Hitachi Drives formatted 7.27 TB) – 3Ware 9650SE in RAID 5(WD RE3 Drives formatted 1.4 TB)
  • The 4x WD drives in RAID 5 are blank w/ 1.4 TB free, everything else is sitting on the 7.27 TB RAID 10 array of Hitachi disks
  • Hyper-V Role – 1 2k8 r2 domain controller, 1 win7, 2x LAMP servers – VMs use 1 TB of data for VHDs that are thin provisioned
  • MS iSCSI Software Target consuming 1 TB data
  • Shares consuming 2 TB data

New Config:
  • ESXi 5.1
  • I will be re-configuring the ServRAID M1015 for JBOD w/ the 8 Hitachi 2 TB Disks and moving the 3Ware 9650SE and 4x WD 1.4 TB RAID 5 array to the oi-nappit box for migration purposes (the array will be reformatted)
  • 250 GB laptop spinner for backing up ESXi with ghettoVCB (http://communities.vmware.com/docs/DOC-8760)

Tools Being Used:
  • (NAS) – Physical oi-napp-it server with 3.9 TB pool, 1.4 TB pool (5.4 TB Total)
  • Regedit to export the Windows Shares registry keys from the 2008 R2 config for creating shares and setting security in the new oi/napp-it VM
  • Starwind Converter to convert the Hyper-V VHDs to VMware VMDKs
  • VMWare vCenter Converter to P2V the Physical Windows 2008 R2 machine incase things get real nasty(Hyper-V Role and iSCSI Target will be removed prior)
  • MS DNS to export the DNS Entries from Domain Controller for new VMWare VMs, the IP’s will need to be reentered upon recreation w/ ESXi
  • oi.iso on the NAS


Steps:
  1. Install Starwind Converter on 2k8 box
  2. Install VMware vCenter Converter on workstation
  3. Copy 2k8 box shares to NAS
  4. Export DNS Entries from MS DNS on Domain Controller
  5. Temporarily change DNS information on NAS, 2k8 server, and workstation to use router instead of Domain Controller
  6. Power off all Hyper-V VMs
  7. Power off server
  8. Power off NAS
  9. Move 9650SE controller and WD disks to NAS
  10. Power on 2k8 box
  11. Power on NAS
  12. Conigure 9650SE & WD disks on NAS, create pool.
  13. Copy contents of MS iSCSI target to NAS
  14. Start converting VHDs to VMDKs onto NAS
  15. Remove Hyper-V Role
  16. P2V 2k8 box as files to NAS
  17. Power off server
  18. Power on server and reconfigure ServRAID M1015 to JBOD w/ 8x 2TB Hitachi’s
  19. Install and configure ESXi 5.1x to 64 GB Crucial M4
  20. Upload oi.iso to ESXi, install, and configure oi w/ napp-it VM with same name and IP address as 2k8 box
  21. Create a pool in VM NAS for NFS share as Storage for ESXi, mount volumes in ESXi
  22. Configure Comstar for iSCSI Target
  23. Create new disksless Windows 2008 R2 server for the Domain Controller in ESXi, upload and attach converted VMDK to machine. Install VMware Tools and reboot.
  24. Assign IP address from exported DNS entries on Domain Controller, verify functionality.
  25. Delete 2k8 box from Active Directory
  26. Change physical NAS, virtual NAS, workstation, and ESXi network DNS settings to use migrated Domain Controller as the DNS server. Restart SMB service on physical NAS and the VM NAS.
  27. Join ESXi to Active Directory
  28. Join NAS VM to Active Directory
  29. Create ZFS Folders from exported 2k8 box file server registry keys
  30. Set permissions on new ZFS Folders from the exported 2k8r2 file server registry keys
  31. Copy shares from physical NAS to new VM NAS’ newly created ZFS Folders
  32. Re-create diskless Windows 7 machine, 2x diskless Linux machines, and upload and attach their respective VMDKs, install VMWare tools and reboot. Assign IP addresses to machines
  33. Connect machines to Comstar iSCSI Target

Finalization:
  1. Test!
  2. Test!
  3. Test!