Operating System/Storage Platform: OmniOS/Napp-it
CPU: Xeon E3-1230v2 CPU
Motherboard: Supermicro X9SCM-F-O
Chassis: Rosewill RSV-L4411 Chassis
Drives: 8x Hitachi 2TB Disks, 4x Toshiba 3TB Disks, 1x SanDisk SDSSDP 64gb SSD for ZIL and Read Cache, 1x Crucial M464GB SSD for OS
RAM: 16GB UDIMM Kingston RAM
Add-in Cards: 2x IBM M1015 crossflashed/9211 IT mode, 2x Intel Dual Port Server NIC
Power Supply: Antec EarthWatts 650 Power Supply
Usage Profile: Storage, iSCSI Target, SMB Shares, MediaTomb
I decided to go with mirrors with both of my pools. While the performance penalty of ZFS pools versus mirrors were negligible, they were noticeable. I decided to go with speed and the redundancy of mirrors.
Comstar – iSCSI Config
I am using a 16 port ZyXEL GS1510-16 smart switch configured with 5x VLANs with Jumbo Frames enabled.
napp-it config – Comstar:
After losing our host(it was free) we are finally back! More geekiness coming your way soon.
I am going to migrate my server that’s running Server 2008 R2 w/ Hyper-V to ESXi 5.1 and pass through the HBA to oi-napp-it for storage in ESXi, and for SMB/NFS shares.
I have been playing with oi/napp-it for about 5 days now, tweaking, reading, tweaking, troubleshooting failed disks (4 of 8 1TB disks I had at the house were failing or were failed), and I genuinely like it!
I have previous Solaris experience, so Openindiana (http://openindiana.org/) is in concept fairly easy enough for me to use. I like the kernel based SMB/CIFS, it’s a hell of a lot faster than Samba. On the Desktop version of Openindiana, the desktop environment uses GNOME 2 which I love, I am not a fan of GNOME 3, and everything is right where I know it should be in the GUI.
The pkg command for managing application packages works rationally, and I picked up on it’s switches and usage very fast.
napp-it (http://www.napp-it.org/) is very easy to use thanks to the user interface being rationally laid out, it puts all the power of ZFS into a few mouse clicks with gea’s(creator of napp-it) HTML and PERL magic. (I don’t have an issue with CLIs, I have been using command line based operating systems daily since 1995)
I would like to recover the additional 3.63 TB of lost by the ServRAID M1015 w/ 8x 2TB Hitachi 7K3000 in RAID 10 @ 7.27 TB formatted, and use JBOD w/ RAID-Z2 which would get me 10.9 TB formatted.
I have been administering VMware servers for a living for the last 12 years, so of course, ESXi is old hat for me. ESXi 5.1 is a huge improvement over 5.0, which was riddled with bugs and issues that drove me nuts.
I had such a crappy time with ESXi 5.0 when it came out, I blew it away at home, and replaced it with Server 2008 R2 and Hyper-V because it just works.
- Server 2008 R2 (2k8 box)
- 16 GB RAM
- Xeon E3 1230
- Supermicro X9SCM-F
- 2x Intel Dual Port NICs
- Rosewill RSV-L4411
- Crucial M4 64 GB Boot Drive – 8x 2TB Hitachi 7K3000 – 4x Western Digital RE3 500 GB
- IBM ServRAID M1015 flashed w/ LSI SAS9211-8i in RAID 10 (Hitachi Drives formatted 7.27 TB) – 3Ware 9650SE in RAID 5(WD RE3 Drives formatted 1.4 TB)
- The 4x WD drives in RAID 5 are blank w/ 1.4 TB free, everything else is sitting on the 7.27 TB RAID 10 array of Hitachi disks
- Hyper-V Role – 1 2k8 r2 domain controller, 1 win7, 2x LAMP servers – VMs use 1 TB of data for VHDs that are thin provisioned
- MS iSCSI Software Target consuming 1 TB data
- Shares consuming 2 TB data
- ESXi 5.1
- I will be re-configuring the ServRAID M1015 for JBOD w/ the 8 Hitachi 2 TB Disks and moving the 3Ware 9650SE and 4x WD 1.4 TB RAID 5 array to the oi-nappit box for migration purposes (the array will be reformatted)
- 250 GB laptop spinner for backing up ESXi with ghettoVCB (http://communities.vmware.com/docs/DOC-8760)
Tools Being Used:
- (NAS) – Physical oi-napp-it server with 3.9 TB pool, 1.4 TB pool (5.4 TB Total)
- Regedit to export the Windows Shares registry keys from the 2008 R2 config for creating shares and setting security in the new oi/napp-it VM
- Starwind Converter to convert the Hyper-V VHDs to VMware VMDKs
- VMWare vCenter Converter to P2V the Physical Windows 2008 R2 machine incase things get real nasty(Hyper-V Role and iSCSI Target will be removed prior)
- MS DNS to export the DNS Entries from Domain Controller for new VMWare VMs, the IP’s will need to be reentered upon recreation w/ ESXi
- oi.iso on the NAS
- Install Starwind Converter on 2k8 box
- Install VMware vCenter Converter on workstation
- Copy 2k8 box shares to NAS
- Export DNS Entries from MS DNS on Domain Controller
- Temporarily change DNS information on NAS, 2k8 server, and workstation to use router instead of Domain Controller
- Power off all Hyper-V VMs
- Power off server
- Power off NAS
- Move 9650SE controller and WD disks to NAS
- Power on 2k8 box
- Power on NAS
- Conigure 9650SE & WD disks on NAS, create pool.
- Copy contents of MS iSCSI target to NAS
- Start converting VHDs to VMDKs onto NAS
- Remove Hyper-V Role
- P2V 2k8 box as files to NAS
- Power off server
- Power on server and reconfigure ServRAID M1015 to JBOD w/ 8x 2TB Hitachi’s
- Install and configure ESXi 5.1x to 64 GB Crucial M4
- Upload oi.iso to ESXi, install, and configure oi w/ napp-it VM with same name and IP address as 2k8 box
- Create a pool in VM NAS for NFS share as Storage for ESXi, mount volumes in ESXi
- Configure Comstar for iSCSI Target
- Create new disksless Windows 2008 R2 server for the Domain Controller in ESXi, upload and attach converted VMDK to machine. Install VMware Tools and reboot.
- Assign IP address from exported DNS entries on Domain Controller, verify functionality.
- Delete 2k8 box from Active Directory
- Change physical NAS, virtual NAS, workstation, and ESXi network DNS settings to use migrated Domain Controller as the DNS server. Restart SMB service on physical NAS and the VM NAS.
- Join ESXi to Active Directory
- Join NAS VM to Active Directory
- Create ZFS Folders from exported 2k8 box file server registry keys
- Set permissions on new ZFS Folders from the exported 2k8r2 file server registry keys
- Copy shares from physical NAS to new VM NAS’ newly created ZFS Folders
- Re-create diskless Windows 7 machine, 2x diskless Linux machines, and upload and attach their respective VMDKs, install VMWare tools and reboot. Assign IP addresses to machines
- Connect machines to Comstar iSCSI Target