Friday, March 12, 2010

Virtualizing Windows Home Server On Hyper-V

The last post about virtualization was acronym galore. Double that, add a side of long-windedness, and here we go.

Primarily, some failing hardware prompted me to build the new server that I detailed in my previous post. Check that out for a list of the components I chose and why. I built it as a VMware ESXi 4.0 whitebox with an end-goal of virtualizing my various needs for a home server environment. VMware ESXi was initially my hypervisor of choice because it's the most popular and by most accounts it's the most advanced. It would also have been more helpful to learn for my job. Everything worked great, but when I started looking into SATA storage cards to support all of my hard drives currently residing in a native Windows Home Server install, things got touchy. Finding supported storage cards for ESXi can be both difficult and expensive. I later on decided to abort and go with Microsoft's free Hyper-V Server 2008 R2. This post deals with why that was needed in my particular situation, as well as some of my experiences installing and configuring Hyper-V Server.

As I said before, I have an existing native hardware home server. It's been almost 2 years ago to the day that I built it. I installed Windows Home Server (WHS) on it and all was well with the world. WHS makes for a perfect home server operating system in my eyes, even for the less than technical. It handles backups, file storage, and network sharing in a manner that's easy for everyone. For the casual user, you can buy a prebuilt, preinstalled box like the HP Media Smart series and just let the magic happen. But, what about the more technical and demanding user? WHS is designed to be headless and only accessed from the network, offering other services by installing 3rd party Add-Ins. It seems a waste that an always on and very capable machine can't (actually it can, but shouldn't) be used for running advanced services that aren't specifically written to be used as a WHS Add-In. Some common uses I had; SABnzbd for grabbing binaries from usenet, VirtualBox for running virtual machines (VM) and testing, Copssh for an SSH tunnel, any many others. You can do whatever you want with WHS, as I did, but these aren't supported use cases and could potentially cause problems if you aren't careful. since then it's been extremely reliable and useful.

Since I'm already running WHS in an unsupported fashion, and I'm needing all these services as well as the occasional testing environment, I figured, why not run WHS as a virtual machine(VM)? Then I could just have a separate VM that ran my unsupported services, as well as be able to rapidly deploy a test environment to really gimp up with no worries. This is why I built the new server and decided to virtualize WHS using ESXi.

OK, so best laid plans don't always work out just as you envision. I wanted the ability to yank out one of the Windows Home Server data pool drives and plug it into a PC to recover data if the need ever arose. This would require that the VM be able to write directly to the drive instead of writing inside virtual hard drive container files such as VMDK or VHD. After further research into ESXi in regards to my unique storage needs for WHS, I learned that you can't do what is called Raw Device Mapping (RDM) on local storage. The storage either has to be SAN based, or you have to use VMDirectPath if you want a VM to have full control and direct access to the hard drives. OK, fine. I'll just use VMDirectPath on my VT-d compatible motherboard to pass on the two storage cards directly to my Windows Home Server virtual machine. The only problem is that I just could not get VMDirectPath to work with this card, even though I had it working fine for a USB controller. I guess that is why they say most any PCI device SHOULD work, not that it will. I'm not going to go on record and say that there's no way to get it working, because there's tons of beeps and boops you can tweak, but it's definitely very obscure and hard to troubleshoot because information is almost non-existent.

My decision to call an audible and go with Hyper-V went something like this. "What's this? You say Hyper-V will allow me to map a local disk directly to a VM? Sweet! You also say that you can use Windows Server drivers for hardware support? Double sweet!" As a side note, Hyper-V also allows direct passthrough of USB devices to a VM. ESXi does not allow this.

I found the Supermicro AOC-SASLP-MV8 storage card, which is a PCI-e x4 card that is relatively cheap at around $100. It gives you 8 SATA ports to do with what you will. I bought two. Add these 16 SATA ports to the 8 existing on my motherboard and I have 24 total SATA ports. That matches up quite nicely to the maximum 23 SATA ports that I could possibly need on my new Norco 4020 case (20, hot swap, 2 internal bays, and 1 slim dvd drive). In addition to these, I had to pick up some SFF-8087 breakout cables to turn a single port on the storage controller into 4 SATA ports. I bought four 3Ware CBL-SFF8087OCF-10M for about $50 total on eBay. This will connect all the SATA ports to the backplane of the Norco 4020 and I'll be ready to pop a drive in whenever I'm ready.


The Supermicro AOC-SASLP-MV8 storage card.


Here's what the final build looks like, all wired up. Do note, I bought one meter length SFF-8087 cables. The 0.5 meter cables would have been more than sufficient. I would suggest those to help with wiring clutter.

Now it's time to finally install and configuring Hyper-V Server, but first, a list of requirements. For managing Hyper-V, you'll need either a Professional, Enterprise, or Ultimate version of Windows 7, or any version of Windows Vista. The management software for Windows 7 is here, and the two pieces you'll need for Vista are here and here. You'll also need the install image of Hyper-V Server 2008 R2 located here, and a very helpful script called HVRemote here. A small USB thumb drive will be required for getting the script onto the server. Or I guess a floppy would work if that's an option for you.

Installing is the easy part and I'll gloss over that quickly. The first step is to download, burn, and boot from the free Hyper-V Server 2008 R2 install image. You'll get a standard Microsoft wizard based install interface. Choose your language, accept the license, pick the install drive, etc. After a reboot, you'll be asked to create an Administrative password. After doing so, you should be greeted with two screens. One is a standard black windows command line, and the other is called SCONFIG.


Logged into the Hyper-V console, viewing SCONFIG and CMD.

SCONFIG helps you with some of the initial settings of the Hyper-V machine itself such at network, computer name, and user accounts. There's only a few steps you'll need to take here to get up and going, and it's actually a little easier if you'll be joining a domain. I'll be configuring mine for a workgroup.

Follow these steps to configure everything you should need in SCONFIG.
Step 1 - Select 1 and then W to choose Workgroup.
Step 2 - Select 2 and then type in the server name you want. I chose to be creative and name mine Hyper-V.
Step 3 - To manage the server, you'll need to create an account with the same name and password that you use to login to the PC on which you plan to manage the server. For instance, on my primary PC, that I'll be using to manage Hyper-V, I log in as Joe. On the Hyper-V server, I create a username Joe with my same password. If the username and password doesn't match, you won't be able to connect later on with the Microsoft Management Console (MMC) Snap-In. Be sure to pay attention to the case sensitivity. Also, if your username has a space, be sure to enclose your username in quotes. So, select option 3, type your username and then give it a password.
Step 4 - Select option 7 and E to enable remote desktop. Then select option 2 to allow any version of remote desktop to connect. This still requires a username and password, so it's still secure.
Step 5 - If you want your server to get an IP address via DHCP, you can skip this step. Otherwise, select 8 to get into network settings. Choose the number that correlates to your NIC and then give it an IP address with option 1 and a DNS server with option 2. At this point, you should actually be able to remote desktop to your server's IP address and finish configuration from there. It's up to you.

Now it's time to run the extremely handy HVRemote script created by John Howard, a senior program manager at Microsoft. This script will save you a ton of time while setting up all of the security policies needed to do remote management in a workgroup. Hyper-V is designed for a domain and Active Directory, and in that case all of the security policies would already be in place. Setting everything up for a workgroup can be done manually but wow it takes forever. Dive into the nitty gritty details if you like. Otherwise, download the script and put it on a USB thumb drive. Plug it into your server and pull up the CMD window.

Try to figure out which drive letter the thumb drive is and change to it. Mine was F:\ but yours is probably different. Once you're there, run the script by typing "cscript hvremote.wsf /add:Joe". Replacing Joe with your username of course. After a moment you'll be back at the command prompt and ready to move to the PC you intend on using for management.

You'll need to run a couple of commands using HVRemote on the management PC as well. First, open a CMD window with elevated privileges. This can be done by going to "Start > All Programs > Accessories". Right click on "Command Prompt" and choose "Run as administrator". Then find the directory where you have hvremote.swf and run these commands. "cscript hvremote.wsf /mmc:enable" and then run "cscript hvremote.wsf /anondcom:grant". Once you've done that, you should finally be done with all of the CMD windows.

Install the MMC Add-In software that I mentioned in the requirements for the OS you're using. Once the software is installed, you'll have to go to the Control Panel to turn the feature on (weird, right?). Go to "Control Panel > Programs and Features" and click on "Turn Windows features on or off". In this window, go to "Remote Server Administration Tools > Role Administration Tools" and check the "Hyper-V Tools" box and hit OK. Hyper-V Manager should now be listed under Administrative Tools on your Start Menu. You may have to enable the System Administrative Tools under the Start Menu properties if you don't see it, or just run it from the Control Panel.


Hyper-V Manager running.

Now that Hyper-V Manger is installed and running, it's time to really start playing. Click on "Connect to Server" and then enter your server's hostname and OK. If you have a problem connecting, you may have to create an entry in the HOSTS file (C:\Windows\System32\drivers\etc\) that points the server's hostname to it's IP address. I'm not exactly sure why this is an issue for some, but it didn't take me long to figure out that others were having the same problem. It's obviously some type of DNS resolution problem.

Hooray! We can now start making VM's willy nilly. Or can we? Depending on your situation, you may need to install some NIC or storage drivers or something. If you're network card isn't working, I'm sure you know this by now. Later on you may also discover that all of your storage devices aren't showing up. I had to manually install the drivers for the Supermicro AOC-SASLP-MV8 cards. It's a little unorthodox, but not really that hard. Take the x64 drivers (got mine from the driver CD) and copy them to a thumb drive. Basically you're looking for an inf file and it's associated files. Plug the thumb drive into the server and pull up the CMD window (either locally at the server or via RDP). Browse to the folder where the drivers are and install the driver by running "pnputil -i -a *.inf". This will install any inf drivers in the folder. After rebooting you should be good to go on the driver front.

Now, back to the Hyper-V Manager to create our Windows Home Server virtual machine. First, click on "Virtual Network Manager" on the right-hand side to set up, oddly enough, our virtual network. Choose "External" and then click "Add". I named mine the once again very creative "Home Network". Choose your NIC in the drop down and then click OK. Now at the top right of Hyper-V Manager click "New > Virtual Machine..." and you'll start the VM creation wizard. I named mine "WHS", gave it 4gb of RAM (overkill, but can be changed later), told it to connect to "Home Network", and then I created a 70GB hard virtual hard disk (VHD). You can create whatever size VHD you'd like, but 70GB seems to be right about the minimum for WHS. I want the WHS system drive to be as small as possible since it's virtual. Later, I'll add raw disks to the VM to add to the data pool.

For the actual install, you can go directly from a disc, or copy over an ISO file from which to install. I chose ISO since I figured it to be faster, but for some reason, I could never get an install to work from an ISO hosted on a network share. To get around this, I created a network share on the server so I could copy the ISO over to the server's local storage. Thanks goes to my friend and co-worker Darren (thanks Darren!) for telling me how to use Computer Management to connect to a remote PC. On the management PC, right click "Computer" and then "Manage". At the top left, right click "Computer Management (Local)" and then "Connect to another computer...". Yeah, it seems simple now but I didn't know you could do that. Enter the server's IP and click OK. Then, open up "System Tools > Shared Folders > Shares". Right click in the window on the right side and then "New Share...". You can then browse for or create a folder to share on the network. Select or create the folder you want, click OK, and then hit Next. Give the share a name and then Next. I chose the third option "Administrators have full access; other users have no access" and clicked Finish. Now you should be able to copy an ISO over to your server by going to "\\Hyper-V" (or whatever you named your server) and then into the share.


Connected to Hyper-V with Computer Management.


Browsing the share I created on Hyper-V for copying over ISO images.

Once you've got the disc in the drive, or the ISO is on local storage, choose the appropriate setting under "Installation Options" in the VM wizard. After you click finish, you should be back at the main Hyper-V Manager window. Select the VM you create at the top middle and then click the Start button near the bottom right. This obviously starts up the VM. Click Connect to open a console view of the VM. Now, just follow the standard Windows Home Server install process. Keep in mind, the install will be SLOOOOOOWWWWW. Microsoft hasn't optimized the install environment as well as VMware. After the OS installs though, everything is quite peppy. Especially after installing the Integration Services. Speaking of which, that's probably the first thing you should do after the install completes.

WHS should now be fully up and running. The last step is to connect all of the data drives to the WHS VM. If you plan on having WHS write to raw hard drives like I do, first you have to go into the CMD window on the server and offline the disk using Diskpart. Type in "diskpart" and hit enter. Now, type "list disk" to view all of the disks in the server. Decide which one it is that you want to present to WHS natively, and type "select disk 1" or whatever number yours is. Type "offline disk" and enter and the disk will now be offline and available to be presented as a raw disk.


All of my disks offlined except Disk 0 and 1 that I used for virtualization needs.

In Hyper-V Manager, select WHS and click on "Settings". On the left, click "SCSI Controller". Select "Hard Drive" and click Add. Choose "Physical hard disk:" and then select the actual drive you want to be associated directly to WHS. Click OK and that should be it. The drive will show up under WHS as a drive not added to the pool No rebooting required. Add this drive to the pool in WHS as you normally would and it will start using that drive just as if it were native. Wash, rinse, and repeat as necessary. WHS is now fully running and configured!


Connected to WHS through the Hyper-V console view. All of the disks are showing up as native except the virtual system disk.

One last thing I wanted to mention. For my build, I had a single 1TB drive on bay 1 of my server. This would house the Hyper-V system and all of the VMs, and I would back up the VMs to a second drive in bay 2. I initially planned on using the top 4 bays for VMs and VM backups, and the bottom 16 bays would be used for drives connecting directly to WHS. After evaluating various backup strategies for Hyper-V, none of them seem adequate to me. I believe host level backup on Hyper-V Server is still a work in progress unless you want to spend money on an enterprise solution of some sort. My solution was to enable software RAID1 on the C: drive. That way if the drive in bay 1 dies, everything is mirrored over to the drive in bay 2 and I haven't lost anything, and I don't even have to reinstall. I just yank out the dead drive, replace it and rebuild the mirror. Yes, I know it's not a true backup since it's off site, but it keeps the hard drive from being a single point of failure. As for the other 16 bays, WHS will take care of the duplication for all of that data.

Here's how to enable a software RAID1 volume in Hyper-V Server. For my example, "Disk 0" is the disk, and "Volume 0" is the partition where Hyper-V is installed. "Volume 1" is the 100MB volume that is created by default by Hyper-V (and Windows 7). "Disk 1" is the empty and formatted drive to which I want to mirror everything. You'll need to substitute the correct numbers for your specific setup.

First, go to the CMD window on the server and run "diskpart". Before we can mirror anything, we have to convert the disks to dynamic. Type "select disk 0" and then "convert dynamic". Do the same for "disk 1". Now, "select volume 0" and then "add disk=1". This starts the mirroring process. Do the same for "volume 1". It can take a few hours for it to build the mirror. During the build process, if you type "list volume" in diskpart, it will show "Mirror" as the type, and it will say "Rebuild" under the Status. After the build is complete, the Status will change to "Healthy".


Showing the disks and volumes after everything is configured how I like. Disk 0 is fully mirrored over to Disk 1, and the rest of the disks are offline so they can be tied directly to the WHS virtual machine.

It took us a while to get there but I'd say it wasn't all that difficult. As you can see in one of the screen shots above I've already created a few other VMs for various purposes and services. I now run WHS as it was intended, besides the fact that it's virtualized. There's less chance that I'll install something I'm not supposed to and mess everything up. I can easily roll out a VM for testing some new software and then just blow it away when I'm done. I have a linux VM set up for SSH tunneling. I even have a PowerUser VM created that is the only "user" in the household with write access to the shares on WHS (except for the Public share). That way I can point iTunes or something to the Music folder share without worrying about it totally messing up my organization structure or creating a ton of thumbs.db files. PowerUser also runs a TeamSpeak server, FTP server, SqueezeBox software, and a couple of other services that aren't supposed to be installed on WHS. Everything is much more flexible and functional. I've been very pleased.

56 comments:

  1. threeeyeMar 17, 2010 06:20 AM
    Hey Joe. Thanks for the write-up.

    I have been toying with the same idea for the same reasons for quite some time. I am still a bit worried that this 'unofficial' WHS configuration may end up losing a few TB of my data it protects so nicely but I'll probably do it with the next server hardware upgrade.

    Anyway, about that soft-raid "I don't even have to reinstall. I just yank out the dead drive, replace it and rebuild the mirror." What component actually controls this? The 'soft' part of the RAID must live somewhere so if the drive dies who manages the fail-over based on which data? Any idea?

    Thanks!
    ReplyDelete
  2. JoeMar 17, 2010 07:07 AM
    OK if you're not very familiar with virtualization or software RAID, this might be a tad hard to follow.

    Hard drive #1 holds the installation of Hyper-V, as well as all of the virtual hard drives (VHD) of the virtual machines (VM). The way I installed it, this all resides in the C partition on hard drive #1. For hard drive #2, I have a software mirror (RAID 1) of the C partition on hard drive #1. That means that anything that gets written to C on hard drive #1, it also gets written to hard drive #2. If hard drive #1 were to die, I would be able to boot from hard drive #2. I'd just have to replace hard drive #1 with a new drive, and then go into diskpart and tell it to rebuild the mirror. The same is true if hard drive #2 were to die. This basically keeps me from having to reinstall Hyper-V itself, as well as all of the VMs that it hosts because the VHD files would be saved.

    Even if I weren't doing any mirroring on drive #1 and #2, I wouldn't lose any of my important data on WHS just as long as I had duplication enabled on the folder shares like normal. The C partition for WHS is a VHD file on hard drive #1. The WHS data pool also has 50Gb in the VHD as well since WHS makes you have a 70Gb hard drive. So, if drive #1 died and I didn't have it mirrored, the only thing I'd lose is the C: drive of WHS and the 50Gb of data that's in the data pool. Once again though, the data in the data pool would be safe as long as I have duplication enabled in WHS because that data would be copied to another drive.

    My primary reason for doing the mirroring on drive #1 and #2 is just so I don't have to reinstall Hyper-V, WHS, or any other VMs I create. It's optional from a WHS perspective if duplication.
    ReplyDelete
  3. threeeyeMar 17, 2010 08:39 AM
    Hey Joe - yes thanks, I follow all that and am familiar with enterprise hardware RAID systems - just not with Hyper-V. I basically have the same motivation and objective as you describe incl. OS mirror. Given that this RAID is a Hyper-V software feature I wonder who is the controller if the boot drive goes down. A RAID card could detect a failed drive and boot off the mirror (the card would be the single point of failure of course...). Who detects this and boots off D if C is dead? Isn't there firmware involved?

    Also - were you able to migrate your existing WHS drives or did you have to start over when setting up the virtualized WHS?
    ReplyDelete
  4. Shadows of OrionMar 17, 2010 11:32 AM
    Whoa, I think this just solved my dilemma. Been needing to build a home server, but unwilling to devote an entire machine to WHS; mostly because WHS makes it harder to fully utilize the hardware for other purposes.

    I've been contemplating using *nix as a base, but I'm not nearly as familiar with it for it to be the primary.

    Plus having a way to run a MythTV or some other DVR software on the same hardware...
    ReplyDelete
  5. JoeMar 17, 2010 11:41 AM
    threeeye,

    The software RAID 1 isn't specific to Hyper-V. I think you can do it on pretty much anything since Windows 2000. Hyper-V just happens to support it too.

    As for who controls which drive to boot from, you have to go into the BIOS and tell it to boot from drive #2. Although, if the drive completely disappears, drive #2 is the boot device that's next in line anyway and it may boot automatically.

    I definitely was able to migrate my data, but it was a little bit of a pain. I just plugged in the drive with data on it and did NOT add it to the data pool. Then, copy the data from the drive to data pool and THEN add the drive. I did it this way because WHS will format the drive when you add it to the pool. Had to do that about 9 times, hah. All the data is 100% intact though. You could techincally do a server reinstall when going through the WHS install and it wouldn't mess with your data. I wanted to start completely from scratch though.
    ReplyDelete
  6. JoeMar 17, 2010 11:44 AM
    Shadows of Orion,

    Yeah sounds like you've got a plan! Keep in mind that any TV-tuner cards you plug into the server wouldn't be usable by the virtual machines. Sounds like you don't plan on using a tuner for MythTV though, so you should be good. If you DID need a tuner, ESXi may be an option instead of Hyper-V because it has VMDirectPath and can pass a PCI device to a VM. ESXi is a butt with local storage though.
    ReplyDelete
  7. Shadows of OrionMar 17, 2010 01:12 PM
    Ideally Hyper-V will get the ability eventually for PCI pass through. I can hope! There are the Silicon Dust tuners that would fix the lack of PCI pass through. There were rumors of a Cable Card version coming.

    Does the Hyper-V pass through restriction include USB devices too?
    ReplyDelete
  8. JoeMar 17, 2010 01:15 PM
    Yeah I'm hoping for a PCI passthrough with Hyper-V as well.

    For USB devices, you can pass it through no problem. Works much better than ESX.
    ReplyDelete
  9. threeeyeMar 18, 2010 08:01 AM
    Ok, sounds good. Maybe I give the reinstall a go when it's my turn to upgrade but your approach sounds safer. Thanks.
    ReplyDelete
  10. Mackenzie's DaddyMar 19, 2010 08:06 PM
    Thanks for the article Joe. It made me realize why I wasn't able to install WHS in a VM. I was loading my ISO image from a share. I copied the file on my local disk and voila!
    I was beginning to think I would try to fix my problem by installing ESXi also. I'm so glad I waited a bit before doing it because you saved me a lot of time and headaches I'm sure.
    Thanks again
    ReplyDelete
  11. AaronMar 20, 2010 04:52 PM
    I've been running my WHS as a VM on Hyper-V for some time now. What I can't figure out is the best way to maximize networking to the WHS. I have two gig nics in my Hyper-v box and I'd like to know the optimized method for best performance to the virtual WHS... advice?
    ReplyDelete
  12. AnonymousMar 21, 2010 10:07 AM
    Thanks for great article but i thought the main WHS drive should be as big as possible, (aka > 70 GB because of the 4Kb file for the location of the real datafile on the other drives) or did you just expanded that dynamically? If so what size are you having now?
    Did you have any performance issues while with transfer rates etc. after it was virtualized?
    ReplyDelete
  13. JoeMar 21, 2010 08:03 PM
    Mackenzie's Daddy,
    Glad it was helpful! I'm not sure why MS doesn't allow installing from an ISO on a share. Seems silly to me.

    Aaron,
    For improved performance, I would create two virtual networks, each one using one of your NICs. I would set WHS to one virtual network, and all of your other VMs to the other. This would ensure that WHS has a NIC all to itself. You may also look into NIC teaming, but from my brief research, it's not supported by Microsoft. Some hardware manufacturers claim to support it, but it seems to be spotty. It also doesn't look like something you would be able to do with Hyper-V Server, only full installs of Server 2008.

    Anonymous,
    My system drive is set at a maximum of 70GB. It's not dynamically expanding. Since the primary drive for a VM has to be a VHD, I chose to set the VHD for WHS as small as possible because I wanted the maximum amount of my data to be placed on a hard drive native to the VM instead of in a VHD. I understand what you're saying about the 4KB files (called tombstones). They're stored on Data (D):, which in my case is 50GB (for the VHD, WHS will partition 20GB of it as Sys C and the rest as Data D). If I'm doing the math right, 50GB is enough for 12.5 million 4KB files. I think I'll be OK for quite some time. I found noone else on the internet having the same problem. If the need ever actually did arise, it would be a simple matter to expand the drive.

    I did some limited benchmarking to make sure I had no major performance impact. So far, I've yet to see any discernible difference in performance. Network speeds are normal for me. In fact, if I'm copying files from one of my other VMs to a share on the virtualized WHS, it actually goes at disk speed instead of network speed. Pretty neat I thought.
    ReplyDelete
  14. LarryMar 21, 2010 10:38 PM
    This comment has been removed by the author.
    ReplyDelete
  15. MikeMar 21, 2010 11:10 PM
    You do not need to actually install to a VHD. If you want to avoid this, you can install directly on to a pass-through disk.
    ReplyDelete
  16. JoeMar 22, 2010 08:03 AM
    Mike,
    It looks like you're right. The startup disk CAN be a pass-through disk. The only limitation is that it has to be IDE, not SCSI. I'm not sure if I made some assumptions during my install, or if I read some incorrect information, but I didn't think you could do pass-through with IDE. Turns out you can, and it shouldn't be too hard to install the VM to an IDE pass-through disk. I still would have chosen to install the system partitions in a VHD though, just so I can make snapshots and possibly avoid a reinstall in the future. You can't make snapshots of a pass-through disk so my recovery options would have been more limited without installing to a VHD.

    Thanks for pointing that out Mike!
    ReplyDelete
  17. SamzMar 23, 2010 11:38 PM
    Great article Joe!
    I've gone thru and setup a new WHS installation as a guest in Hyper-V. Everything runs pretty well but now i just have to figure out how to 'migrate' my data from WHS which is on the other 5x1.5TB drives on the same server.

    I've been trying to work out how to perform a WHS reinstall rather than a new build. In a new install, WHS doesn't 'see' the raw SCSI drives, additional scsi driver. The ones extracted doesn't include a txtsetup.oem file so isn't recognized.

    So far I've managed to ...

    1. Create new VM for WHS
    2. Mount the required raw SCSI disks directly to that new VM.
    3. Extract drivers from Hyper-V Integration services.

    Any idea how to create this file? As i think this might be the missing link to allow a WHS to reinstall onto Hyper-V rather than having to rebuild it. Almost like a Physical to Virtual (P2V) of WHS!
    ReplyDelete
  18. JoeMar 24, 2010 07:30 AM
    Samz,

    When I migrated my data, I went with the brute force route. On my old server, I turned off duplication for one of the shares and made some empty space. After that, I told it to remove a hard drive and it moved all the data off, then I pulled it out and put it into the new server and let it add it to the pool. Then, I would pull out a drive, plug it into the new server, copy everything over from X:\DE\shares to the new data pool (using the network shares), and then add that drive to the pool. Then repeat the whole process for the next drive. It was a long and boring process, but I wanted to do it this way because I had previously written data to a drive in an unsupported fashion (by not using the network shares) and I wanted everything to be as pristine as possible in the new server.

    To answer your question more specifically though, I don't think it's possible for WHS to see the SCSI devices until after Integration Services is installed because the SCSI device isn't an emulated controller, it's a synthetic one. If you only had a couple of drives, you could probably could add them to the IDE controller and it should work. With 5 drives though, you have too many. With IDE, there's 2 controllers with 2 devices each, one of which is the DVD Drive. It would be nice if Hyper-V would let you add an additional IDE controller. It looks like you may have to take the long road like I did.
    ReplyDelete
  19. SamzMar 25, 2010 03:11 AM
    Hey Joe,
    Thanks for your quick response. Yeah i thought a manual data migration would be needed. But as always, I try to look for a 'smarter' approach. As i didn't realize the SCSI device was a synthetic one.

    I have both my Hyper-V and WHS are on the same physical server. I select different drives in BIOS to bootup the preferred OS currently.

    One note though, I'm running the ASUS P5BV-SAS motherboard which has an onboard LSI 1068 raid controller / 12 sata ports. I've managed to create a SCSI controller in Hyper-V and have successfully added all 5 of my drives as physically connected to the one SCSI controller on the virtual WHS. I've also tried created multiple SCSI controllers and have a drive added to each one. There's a hard coded 4 drive limit on SCSI controllers.

    With the virtual SCSI controller setup, i could just copy the data from the WHS data drives into my pooled drive from within Hyper-V WHS as it's visible. However my concern is more around any potential data corruption, so instead i'm taking the individual data copy approach like you have.
    ReplyDelete
  20. The KittyMar 27, 2010 04:15 PM
    Joe, great tutorial. I just built a new quad core server for mygrating my previous WHS.

    I have Win7HP so I created a Vista Ultimate x86 virtual machine. Took a long time to get from RTM to SP2, but it's all validated. I enabled the Management tools and all is well. But when I try to install the Hyper-V management snap-ins, I keep getting a message that the update is not supported. Do you know what might cause this? Thanks!
    ReplyDelete
  21. JoeMar 28, 2010 05:09 PM
    The Kitty,

    If I understand correctly, you only have Windows 7 Home Premium machine (not supported for Hyper-V Manager). So for management, you're trying to install the Hyper-V Manager on a Windows Vista virtual machine. I poked around the net for a bit and couldn't find anyone else who'd tried this, so I tried this myself. I got the same error you did. One would think that running Hyper-V Manager from a virtual machine isn't supported, but that's only an assumption. Sorry I can't give you a better answer.
    ReplyDelete
  22. KongMar 29, 2010 07:02 PM
    Hi Joe,

    Great guide! I followed it and was able to get my WHS up and running on top of Hyper-V. The only thing I've noticed is WHS is now a little sluggish when opening the console, RDP, overall UI. It was alot snappier when it was on it's own physical server.

    Hyper-V is running on a Core 2 Quad Q8400@3GHZ with 6GB RAM. All drives are Western Digital Black. The WHS VM is assigned 2 of those cores and 2GB of RAM.

    Any idea where I can start looking to troubleshoot the sluggishness?
    ReplyDelete
  23. JoeMar 29, 2010 07:24 PM
    Kong,

    Have you installed the Hyper-V Integration Services in the VM? This speeds things up by a great deal, especially network traffic and memory performance. If you have, I don't know what else it could be.

    I haven't personally done any performance tuning on my WHS install because I don't see any need for it. It seems to be lightening fast. I would expect to see a slight drop in performance if you compare it running on native hardware against running in a VM on the same hardware. The host OS takes up some resources and there's a little efficiency loss in all of the hardware translations that have to happen, so some performance loss is to be expected.

    My previous server was an AMD 3800+ X2 at 2.7ghz with 4gb of RAM and it was always quite snappy. Though, now that it's a VM on my new server, it's even faster even with only 2 cores and 2gb of RAM allocated to it. No tweaking whatsoever.
    ReplyDelete
  24. The KittyMar 30, 2010 09:29 AM
    Joe - I started over with Vista Ultimate in the VM, got it to SP1 level patches and then applied updates. Remote Management KB loaded fine, then the first Hyper-V KB pack. Now I can enter the Hyper-V manager. Maybe going from SP1 to SP2 kills the ability? The second Vista KB pack did not seem to load when run though :(

    I will try to load Hyper-V on the server hardware tonight. AMD Athlon II 630 2.8GHz quad core with 4 or 6 GB RAM. We'll see if the Vista Hyper-V manager in the VM talks nice to it. The users out there really appreciate your guide and patience with us.

    Now to try
    ReplyDelete
  25. KongMar 30, 2010 02:11 PM
    Hi Joe,

    The integration service was the first item that I installed, and then all the WHS updates from PP1 all the way too PP3.

    I'll try playing around with it some more, worse case I'll do a rebuild. Maybe one of the add in is causing the slow down.
    ReplyDelete
  26. AnonymousApr 1, 2010 05:55 PM
    Hi Joe - Great Write up, fantastic!

    How can we refuse a free server OS?

    So, I just entered the world of hyper V, and learned the hard way of locating multiple managment tool links, and trying to install the tools, that Vista Home is NOT supported. Sorry I did not see your post sooner.
    You mentioned tuner cards would not work in a virtual machine.

    Do you happen to know if my HDHOMERUN (soon to have the cable card version) will run fine in a virtual Vista Media Center? My virtual vista media center does see the extenders on my network. I will find some time to test over spring break, and post back my results.

    Also, any idea why, if I copy a VHD, and start it up as a new virtual machine, it give me a licensing error? No changes to any hardware have been made. I never had that issue on vmware.

    Thanks,

    Brian
    ReplyDelete
  27. JoeApr 1, 2010 08:44 PM
    Brian,

    I actually do know that the HDHomeRun will definitely work fine in a VM, but it would only be good for running extenders on it since you can't really watch video on it locally. It seems like that's not an issue for you though since you speak of using extenders. Sounds like a good use case.

    I'm not sure exactly what you mean about licensing issues when copying a VHD. You could be talking about a VHD from a native Win7 install, Virtual PC, or Virtual Server. However, to copy a VM within Hyper-V itself, I always just export it first, and then import it while checking the box to copy all files. I've never had a licensing issue doing it this way, although technically you're supposed to have a separate license for each virtualized Windows install.
    ReplyDelete
  28. AnonymousApr 4, 2010 10:37 AM
    Thanks for the write up, it is really useful.

    I just a have a few questions before configuring this setup.

    1) WHS is known to be quite high on disk I/O, I am just wondering if this causes any slowdown of the other VMs. If the VHDs are all located on the same (mirrored) disk does WHS not slow down their performance.

    I like your approach as it should prevent a system disk fail from requiring a reinstall. But if it is going to kill performance of all the other VMs I would rather install the WHS SYS disk on a seperate passthrough.

    2) You keep refering to the ease of USB passthrough. I can't find any mention of this being supported. Do you only mean passthrough for remote desktop sessions? If not please can you explain a bit more.

    Thanks
    ReplyDelete
  29. JoeApr 4, 2010 12:45 PM
    Anonymous,

    The high disk I/O in WHS is going to be primarily on the data drives when it starts balancing the data. In my case, for WHS, I only have a 70gb VHD created on the Hyper-V system drive, so there's not going to be a whole lot of balancing that can be done.

    For USB, I was talking about USB hard drives. It is possible to pass other USB devices to the VM through remote desktop like you suggested, but I have no experience with that.
    ReplyDelete
  30. Shadows of OrionApr 15, 2010 11:14 AM
    After poking around, I still haven't been able to figure out if Hyper-V requires any special chip features to pass thru raw disks (VT-d etc). You mentioned that ESXi does and that incompatibility sent you down the Hyper-V path.

    Ideally VT-d isn't required and some low end i3 is sufficient....
    ReplyDelete
  31. alexis.argyrisApr 23, 2010 11:19 AM
    After lots of adventures with the motherboard, drivers for three different raid controllers, diskpart, ten 2TB disks, etc, etc, I managed to duplicate your setup. Thanks a lot for the info. Greatly appreciated!
    ReplyDelete
  32. Martin RowanMay 26, 2010 01:50 PM
    Hi There. Great couple of posts. I wanted to take a look at Vail, so thought it time to move to a X64 capable platform for my always on PC in the loft currently running WHS.

    So I bought a Asus P6T, i7 920, 12GB RAM, etc. Installed Hyper-V 2008 R2. Installed WHS and another Win7 VM just fine. I then added a PCIE Intel Pro 1000 CT NIC to the system with a plan to dedicate this to WHS as you had. This is when I hit problems.

    From sConfig and HyperV manager is only shows the onboard RealTek NIC. I downloaded the Intel Windows 2008 x64 drivers (ProWinX64) the latest from the Intel site. I ran the setup program which installed the drivers, (since the problems I've also gone into the various directories and used the pnputil command to install drivers). Whatever I try no joy still doesn't appear. If I use the Intel prosetcl.exe command I can see the NIC, but it's shown as disabled and I can't see how to enable it.

    Any ideas, since you have a very similar config?
    ReplyDelete
  33. JoeMay 26, 2010 08:45 PM
    This comment has been removed by the author.
    ReplyDelete
  34. JoeMay 26, 2010 08:47 PM
    Martin,

    I had no problems with mine. I just installed the driver and it was working fine. Does the interface show up under ipconfig? If it's a matter of just being disabled, I think you can enable it via the command line with netsh.

    netsh interface set interface *interfacename* enabled

    Otherwise, DevCon is a cmd line utitily I've used for troubleshooting devices. Take a look at DevCon here.

    http://msdn.microsoft.com/en-us/library/ff544707.aspx

    Edit: Had to delete my previous comment because I typed the netsh command wrong.
    ReplyDelete
  35. Martin RowanMay 27, 2010 09:27 AM
    I managed to setup remote device manager and the NIC is showing there but with an exclamation mark indicating it's not working properly. Can you recall which driver you used and it's version, just in case the latest driver (15.2 IIRC) is causing the problem.

    I'll take another look when I get home tonight.
    ReplyDelete
  36. JoeMay 27, 2010 11:44 AM
    Martin,

    I think I have the same NIC. I picked up the $30 EXPI9301CT. I really don't recall having to install drivers for it manually, I only had to do that for my storage card. I'll check later on tonight on what driver version I'm using.
    ReplyDelete
  37. Martin RowanMay 27, 2010 01:44 PM
    Fixed it! Thanks.

    I went in ran the Intel Uninstaller for the driver I'd added and it started working. Yipee. It's now running with Driver ver: 11.0.5.22. I guess the newer driver needed to be avoided as it didn't work.
    ReplyDelete
  38. on3eighteenJun 17, 2010 01:52 PM
    Joe -- well done indeed. A quick question about your software RAID1 setup. My mobo supports hardware-level RAID. I actually have 2 of my 1TB SATA drives configured in RAID1. May I just keep this configuration or do I need to "un-RAID" them in the RAID setup utility & go the software RAID route that you took?

    Lastly, I am also thinking of installing the full blown Server 2008 R2 & installing Hyper-V as a role. Is it safe to assume that most, if not all, of the configuration setup for WHS will be the same? I imagine I'll take a hit on performance a bit across the board b/c I'm running a full blown OS vs. just a light-weight hypervisor.
    ReplyDelete
  39. JoeJun 17, 2010 08:14 PM
    on3eighteen,

    The hardware RAID is fine. My motherboard didn't have it so it wasn't an option. It doesn't really matter either way as almost all motherboard based RAID solutions are actually a hybrid hardware/software solution. It still uses the CPU to do all the work. There's not much benefit other than it being easier to set up that way.

    For the 2nd question, yep the WHS configuration will be the same. There's some slight overhead for the full version of Server 2008, but it's really small unless you're actively using the other services on it. When you enable the Hyper-V role, a lot of the low level stuff in the 'host' OS becomes virtualized. It's not much different than just having an extra VM running on the box.
    ReplyDelete
  40. on3eighteenJun 20, 2010 11:49 AM
    Joe -- I seem to be getting a very common error after Google'ing my error.

    "Cannot connect to the RPC service on computer '192.168.0.150'. Make sure your RPC service is running."

    I have read some threads about this issue creeping up if your Hyper-V server has a Broadcom NIC & I have read problems about security.

    I know I have not given you much to go by but have you seen this error before?
    ReplyDelete
  41. JoeJun 20, 2010 07:45 PM
    On3eighteen,

    I haven't experienced this problem myself, so I can offer little input. Right away it seems like permission issues from running the server in a Workgroup instead of a Domain. HVRemote set up all of my permissions perfectly, but I did a little searching and found a possible work around for you. It involves disabling the firewall, but if it's a weird driver error it may be your only option. First, though, I would try rerunning HVRemote on both the server and the client.

    http://blog.malevy.net/2009/11/home-server-part-2-installing-hyper-v.html
    ReplyDelete
  42. on3eighteenJun 21, 2010 07:54 AM
    I just read that the new version of WHS (codenamed "Vail") allows you to remove the system drive from the storage pool in order to speed up the OS. That should put some folk's fears about WHS being slow in a virtual hard disk b/c the writing/reading happens on separate, physical disks.
    ReplyDelete
  43. on3eighteenJun 22, 2010 04:11 AM
    Joe -- did you create a 70GB dynamically expanding VHD or a fixed size?
    ReplyDelete
  44. JoeJun 22, 2010 08:01 AM
    on3eighteen,

    I made it fixed, though it shouldn't really matter.
    ReplyDelete
  45. on3eighteenJun 23, 2010 06:43 AM
    Joe -- I am experimenting w/ WHS "Vail". A cool feature is that you can remove the system disk from the storage pool so you don't have to worry about losing data if some is stored on that disk. You can also gain some performance b/c the system disk is just dedicated to running the OS.

    One downside of Vail is that you can no longer access the storage drives (shares) from a computer not running the WHS software. This worries me a bit b/c I cannot just take that disk out & plug it into another computer & see all the files. Any idea if this is true and if that worries you too?

    Lastly, I know you put your Server 2008 on a RAID1 drive so you have a backup. Would it also make sense to use RAID0 for speed & just manually backup the OS from within WHS itself? And you could also copy the VHDs to a USB drive if you wanted. Thoughts?
    ReplyDelete
  46. JoeJun 23, 2010 07:10 AM
    on3eighteen,

    I've played around with Vail myself. Overall, I really like it, and I love the fact that you can remove the system disk from the pool as well. Though, I too am concerned about the disks being block based now. You are correct in that, currently, the disks can't be read by any other system. That's not to say that they couldn't write some software to allow you to do that, but it doesn't exist right now and it's not on their roadmap. The fact that you CAN do this with the current version is a primary reason I chose WHS. It's a warm and fuzzy blanket to me knowing that I can yank the drive out and plug it into anything and move the files if the need arose.

    Since this seems to be something that's going away with Vail, I've actually been researching alternative file systems such as ZFS and BTRFS. I really like the management pieces to Vail, but since one of the primary reasons I use WHS will be gone, it's no longer worth it to me to lose half my storage to duplication. Luckily, though, Vail supports iSCSI targets for storage disks. Right now, I'm playing around with putting Solaris in a VM and having it host my disks using ZFS RAIDZ (RAID5 basically). Then I'm using that to offer iSCSI targets to Vail for storage. This gives me RAID5 like storage where not as much space is wasted, but also the easy management that WHS offers. The only problem I've run into so far is that there are no Hyper-V enlightened SCSI drivers for Solaris (or even BSD that I can find). This limits you to the 4 IDE channels in a Hyper-V VM. BTRFS is very similar linux equivalent to ZFS, but it's still in heavy development and currently doesn't support RAID5 (though it shouldn't be long).

    I said all that to say, yeah, the block based system worries me too. If they were to offer an application that could be used to mount a Vail based disk and recover the files, though, that would probably alleviate my concerns altogether.

    Your last question, you could do RAID0 if you wanted and backup the OS to somewhere else. And even easier would be to copy the VHD's to a USB drive or something. But honestly, those are both manual processes, or they'd need to be scripted somehow. RAID1 is more than fast enough for me and the most hands-off solution, so that's why I went that route.
    ReplyDelete
  47. on3eighteenJun 23, 2010 09:33 AM
    Joe -- your knowledge amazes me. Just when I thought I knew a thing or two...

    I just installed Vail in an VM & I cannot access any of the shares from any of my computers on the network...even the WHS machine itself! I just found this article & it looks like WHS Vail cannot be installed on RAID disks. Well, that sucks. I am becoming less of a raving fan of Vail. The biggest thing I liked was the automated computer backup but that's not too hard to schedule w/ Windows 7 machines now. I don't care about the remote access stuff.

    One last thing I don't like about Vail is that now the shares are partitions on the machine. Before, w/ WHS v1, I mapped a network drive to the "shares" folder so that any other computer could map a drive to that share & "see" all the shared folders. Now, w/ Vail, I cannot map a drive that contains all the shares. Granted, I did read that writing directly to the disk & not using the file share was not good. I'm not sure if that screwed up disk balancing & duplication but I did it. I never had to recover from a backup so maybe I was breaking things behind the scenes & didn't know it.

    My main goal is to have 1 place w/ all my files (videos, movies, music, docs), have it on a drive(s) which are very fast, & have that stuff backed up. I don't think Vail will help me.
    ReplyDelete
  48. JoeJun 23, 2010 10:05 AM
    on3eighteen,

    Haha, thanks. I'm no expert, just an enthusiast.

    It may be that you don't have user accounts set up in Vail in order to see the shares. If that is the case, you could just enable the Guest account. And yeah, neither WHS v1 or Vail "support" RAID disks, although in most cases it will work fine as long as it's hardware RAID and transparent to the OS....or if it's virtualized as it is in my case on Hyper-V.

    The point you made about the way shares are handled, I actually do like that change. If you were mapping drives and writing data directly to d:\shares in WHS v1, you may have not experienced any problems yet, but it's highly probably that you will. I had this same problem with WHS when it first came out, I was doing the same thing as you and writing data directly to d:\shares instead of using UNC paths. Drive Extender kept trying to write data to a drive that was actually already full. It didn't know the drive was full because it only saw data written to the shares if it came from the network. With the new version of Drive Extender, it works below the NTFS level. So, any data written to the disks, either natively or over the network, will be picked up by Drive Extender fine. As an interim solution for WHS v1, instead of mapping a drive, I just made a shortcut on the desktop to \\server\ and that's been servicing me fine for years now.
    ReplyDelete
  49. Boarder2Jul 18, 2010 12:29 AM
    I've been trying all night to get my EXPI9301CT working with Hyper-V Server R2 to no avail.

    I've tried with both the Microsoft and Intel drivers and they both show up in Device Manager with the yellow exclamation point saying "This device is not working properly because Windows cannot load the drivers required for this device."

    This blog post is the only thing google searching got me even close to an answer. Any help would be greatly appreciated as I'm trying to do exactly the same thing here and want to give WHS a dedicated NIC.
    ReplyDelete
  50. AnonymousNov 30, 2010 08:36 PM
    Hey Joe, awesome post - I'm a little late to the party here but I'm in the middle of getting WHS running on Hyper-V as well. Quick question for you: so you are using RAID-1 for your "system" drive which contains the Hyper-V host OS as well as the guest VM's (.vhd's) right? Have you seen any performance impact as a result of doing this? I'm interested in using RAID-1 myself but am concerned that if I'm running 3 or 4 VM's that I might see some slow performance due to the mirroring. Thanks for any info!!

    -alex
    ReplyDelete
  51. JoeNov 30, 2010 09:17 PM
    Alex,

    You've got it right. I'm getting over 100mb/s writes so the RAID-1 doesn't seem to be an issue. I usually have between 6-10 VM's going and performance is more than adequate. The VHD's actually don't change that much anyway once they're setup.
    ReplyDelete
  52. AnonymousDec 14, 2010 01:56 PM
    Great write-up! Thanks!

    I've never done anything like this but your instructions make me want to try.

    Here is my situation, I've got a WHS and a Win7/Media Center PCs. Both run constantly because I record TV on my WMC7 machine using a 4 tuner Ceton PCI-e card.

    Would it be possible to host both of these machines within Hyper-V? Would the WMC7 machine be able to make use of the card?

    Thanks!
    ReplyDelete
  53. Peter BentonDec 24, 2010 02:24 PM
    Joe,

    found your answer to my plans! I am looking at a similar project, what whardware did you use in that case? MB, memory drives etc...

    Thanks!
    ReplyDelete
  54. AnonymousFeb 26, 2011 07:21 AM
    Joe, how do you enable USB passthrough? I'm using a full WinServer install hosting Hyper-V, so maybe I need to switch to a setup like yours? Anything I can find on Google explicitly says that USB passthrough isn't supported by Hyper-V.
    ReplyDelete
  55. AnonymousApr 19, 2011 11:45 AM
    Any reason you went with hyper-v only and didn't do S2008R2 with hyper-v role. I am currently using the latter but am curious why you selected the method you did.
    ReplyDelete
  56. AnonymousMay 17, 2011 02:29 PM
    To those wondering about the "USB passthrough": For USB drives, just put them offline (just like internal drives) with using the Disk Management MMC snap-in of your remote computer, and they'll be available to add to the VM. For USB printers, forget about it -- you can spin your wheels for days on this. (If you want to have a common shared printer on your machine because it's on all the time, go the full Windows Server 2008 R2 with Hyper-V Role, and share your printer from the host partition. But it's not free.)

    This blog is a great Hyper-V/WHS case study that helped me implement virtualization, but I had to go to numerous sources to get the management of Hyper-V worked out smoothly -- lots of firewall rules to touch, above and beyond the script that's available if you want to see the Hyper-V host and share its drives on your network and control remotely with MMC, etc.

    Also, be aware that you might need to disable the TCP Checksum Offload (IPv4) in the VM's Network Adapter to get reasonable throughput for large file copying and computer backups. Experiment with Net Meter.

    If anybody figures out how a WHS 2011 VM can use WHS v1 VM DE shares for storage, please let me know -- might be the best of both worlds.

    Mick Mickle
    ReplyDelete