Jon Galloway : Virtual PC

My Boot-to-VHD experiment: found some tips, like it, but still haven’t found VM nirvana

Jon Galloway — Sun, 25 Oct 2009 22:57:00 GMT

Summary

Windows 7’s Boot to VHD works as advertised – native speed, virtual machine flexibility.
I came up with some tips and tricks which you might find useful
Having to reboot without hibernate to switch to the VHD machine means it’s a lot less useful than I’d hoped.

Background

I’ve recently been running some early releases developer tools which came with the “install on VM’s if you don’t want your computer to catch fire” warning. That seemed like a good time to back off on my “VM’s are for sissies” stance and get my VHD on. After verifying that there wasn’t already a suitable VHD available for download, I decided to follow Scott Hanselman’s directions to set up a Boot To VHD instance.

Here’s a very high level overview:

Download the wim2vhd script from CodePlex
Install the Windows 7 Automated Installation Kit (AIK)
Copy ImageX.exe from the the AIK install into the same folder as the wim2vhd script
Run the script
Mount the VHD in the Windows 7 disk management screen
Run some funky commands to make the disk bootable

There's a little goofing around at the command line, but it's only a few minutes if you follow the directions. Then your new VHD shows up as an option on boot.

BUT WAIT! I have a few minor modifications. Rather than write a complete walkthrough of the process – since Scott and others have done such a good job there – I’m just going to list some footnotes to the process. I’ve very roughly outlined the steps above; I recommend you read through the following list of tips, then go and follow Scott’s walkthrough and use any of the below tips you think are helpful.

Mod #1: Getting ImageX.exe without installing the AIK

There are copies of ImageX.exe floating around on the internets. I normally wouldn’t recommending using them, I only mention that because the AIK is 1.5 GB. If you’re one of those irrational people that thinks downloading 1.5GB to get a 471KB program, you could search around for “download imagex.exe” If you do that, the CRC for my my 32 bit ImageX.exe – version 6.2.7600.16385 - is 54 BF FA D5. Not recommended, but it is an option.

Note: Dear Microsoft folks that make gigantic SDK’s, please stop. Those might have made sense before the internet, but… c’mon now. Here’s how utilities should be done: live.sysinternals.com.

If you end up downloading the entire (1.5GB!!!) AIK ISO but hate installing a bunch of junk just to use one thing, you can open the ISO in 7-zip and find the find the ImageX.exe file, by looking for F1_imagex in Neutral.cab, like so:

Then you can just extract that file and rename it to ImageX.exe.

Mod #2: Changing the VHD size

The wim2vhd script defaults to a 40GB dynamic disk. Normally, I don’t really care much about the size of a dynamic disk, because the actual size of the VHD is only as big as the actual used space, and you can compact a disk to recover space as needed. However, when you mount a dynamic drive, the boot manager and host filesystem appear to reserve the maximum possible size of the of the disk – 40GB. In my case (on a laptop), that wasn’t going to work.

It’s not just a convenience thing, either – if you have a VHD whose maximum size exceeds the physical disk space available, you’ll get a blue screen of death:

(photo credit: Bart Lannoeye, see his post about the BSOD issue)

You can change the created VHD disk size using the /size parameter. For my Windows 7 + Visual Studio 2010 testing purposes, a 16GB disk seemed to work well. To do that, you’d call wim2vhd with this command:

cscript wim2vhd.wsf /wim:e:\sources\install.wim /sku:ultimate /size:16384

The size is calculated in MB, so you calculate it as 1024 * number of GB. A 20GB would use /size:20480

Mod #3: Rearming Windows to extend the evaluation time

If you’re using a virtual machine installation of Windows for temporary testing, you can use it without activation for 30 days. However, you can extend that evaluation period 3 times, giving you a total of 4 months, which is plenty of time for most evaluation purposes. It’s really simple:

Run "slmgr -rearm" from a command prompt with admin rights

This isn’t a hack – it uses a command that’s been shipped with Windows since Vista first came out. It’s not really news – Jeff Atwood wrote about it a while ago, and Ed Bott followed up with a cool tip on scripting that command to run every 30 days so you don’t forget. But it’s a really handy note, and it bears repeating.

Note: Apparently you can use the SkipRearm registry setting to extend that even further, but I don’t know if that’s covered by the EULA. I haven’t had the need to use a VM that long, so I’ve never run into that.

Mod #4 (untested): EasyBDC

You can apparently skip the rigmarole with BCDEDIT by using EasyBCD, because beta builds of EasyBCD 2.0 support the Windows 7’s VHD features.

Note: I haven’t done this. I’ve used previous releases of EasyBCD and haven’t had a problem, but I haven’t used EasyBCD 2.0 as it wasn’t out when I set up my VHD.

Mod #5 (untested): Disk2vhd

The SysInternals team recently released Disk2vhd, which can capture a disk image (while running) and create a VHD. I believe that in order to use the created VHD on the same machine it was created from, you’d first need to sysprep it, since otherwise you’re essentially trying to run two identical copies of the same operating system on the same computer, and you run into problems with drive paths. I haven’t tried this yet.

Gotcha #1: Go with Windows 7 Ultimate

Boot to VHD only works in Windows 7 Ultimate or Enterprise, not Windows 7 Professional. But you can’t use the Windows Activation re-arm trick we just talked about in Enterprise (since it uses a different licensing model). So I recommend that you go with Windows 7 Ultimate Edition.

Gotcha #2: Dual Boot means no hibernation

I use hibernation quite a bit, and only do a full reboot when I have to. So, for me, dual booting was inconvenient. It meant shutting everything down – including saving any tabs I happened to have open in IE8 (since tab saving in IE8 has been pretty unreliable for me) – in order to use the VHD partition, then shutting everything in the VHD partition down to switch back to the main one. That’s a lot of friction, and it ended up that I don’t use it as often as I thought I would.

As I write this, I’m setting up a VHD which I’ll just run under Virtual PC, because I can start it up without shutting everything else down. I still feel like it’s a great feature, just one that I’ll use less often than I thought.

Side note: Fast switching between boot instances would be a killer feature for Windows. I’d settle for multiple hibernation instances. I’ve read that it’s not enabled because of concerns over invalidating one hibernation instance while running the second machine instance, but I disagree – let me make that decision. At least give me a registry setting or something to enable it.

Running the Mono VMWare image in VirtualPC

Jon Galloway — Sun, 30 Mar 2008 09:22:42 GMT

A lot of people love VMWare, but it bugs me. I think the main reason is that it installs with a bunch of Windows services that run automatically, every second my computer is running, whether or not I use VMWare. Since I don't use virtual machines every day, that's annoying. So, after running the free VMWare Player for a few times times, I banished it from my computer.

The Mono team publishes a VMWare image which makes it really easy to check out Mono on Linux without installing or building anything. If you're happy with VMWare, you can grab their free player and run the image. If, like me, you'd rather run this on Virtual PC, follow along and I'll show you how.

If you don't use VirtualPC, or if you don't mind having several virtual machine hosts on your computer, this post obviously isn't for you. Thanks for stopping by, though!

Grabbing the VMWare Image

You can download the Mono VMWare image from the Mono Downloads page. They offer both are http and torrent download options.

Converting the Virtual Hard Disk image

I use Vmdk2Vhd from vmToolkit to convert the hard disk images. It's free, although there's an annoying registration step (which you can probably bypass with BugMeNot). The interface is really simple - browse for the vmdk file, enter the destination filename, and click the "convert" button.

Creating the new VirtualPC Machine

Start up the VirtualPC console and go through the following steps:

Select "Create A Virtual Machine"
Set the Operating System to "Other"
You'll have to decide how much memory you can allocate to the virtual machine; on my workstation with 2GB RAM, I allocated 768MB RAM.
Select "Use an Existing Virtual Hard Disk" and browse to the location of the VHD file you created earlier using Vmdk2Vhd. I left "Use undo disks" unchecked.
Hit "Finish" and you should have a new virtual machine shown in your VirtualPC console.

Setting up the Video Driver

VirtualPC and Linux have a long standing feud over video drivers. VirtualPC emulates an S3 Trio video card with 16 bit color depth, while the Linux X Server defaults to 24 bit color depth since that's what a real S3 Trio supports. The result is that Linux video under VirtualPC is always a pain in the neck. So keep with me as we jump through a few hoops to get Linux to recognize VirtualPC's video card.

When you start the virtual machine we created earlier, hit the down arrow to select failsafe defaults and hit enter.

Note: Previous openSUSE images required me to also fix the mouse by adding "i8042.noloop" to the failsafe boot parameters, but this appears to have been fixed for the 1.9 image.

After the boot process completes, you should be looking at a console window. You'll need to log in (root / mono) and enter "sax2 -m 0=fbdev" like so:

SaX2 churns for a bit, then comes up in graphics mode. You can change the configuration, but I just clicked on OK.

At this point, you'll need to reboot. You can do that in the VirtualPC menu (Action / Reset), or you can just type "shutdown -r 0" at the command prompt. When it comes back up, you can boot to Mono 1.9 / openSUSE 1.3 without using the Failsafe options.

We've got one more thing to set up - the network connection is disabled by default. You can connect by clicking on the network icon in the lower right-hand corner, then on "Wired".

Stay tuned - in the next post, we'll make use of this image to take a look at Moonlight, the open source implementation of Silverlight on Linux.

We should be virtualizing Applications, not Machines

Jon Galloway — Tue, 04 Sep 2007 20:49:00 GMT

One of the benefits of my new job at Vertigo Software is that I have more frequent opportunities to talk with my co-worker, Jeff Atwood. If everything goes right, we argue... because if we agree, neither of us is going to learn anything. Recently, we argued about virtual machines. I think machine virtualization is hugely oversold. We let the technical elegance (gee whiz, a program that lets me pretend to run another computer as another program!) distract us from the fact that virtual machines are a sleazy, inelegant hack.

I was a teenage VM junkie

I'm saying this as someone who used to be a big VM advocate. A few years ago, I noticed that it took new developers as much as a week to get set up to develop on some of the company's more complex systems, so I worked with application leads to set up Virtual PC images which had everything pre-installed. Sure enough, I had new developers working against complex DCOM / .NET / DB2 / "Classic" ASP environments with tons of dependencies in the time it took to copy the VPC image off a DVD-ROM.

One problem: it was a lousy developer experience. Most of the developers kept quiet, but a new developer let me have it: "This stinks! It's totally slow! You really expect me to show up to work every day and work eight hours on a virtual machine?" He was right, it did stink. Even with extra memory and a beefy machine, virtual machines are fine for occasional use, but they're not workable for full time development. For instance, I frequently had to shut down all other applications (Outlook, browsers, etc.) so that the virtual machine would be responsive enough that I could get work done.

Let's admit that the virtual machine emperor has no clothes on. Virtualizing machines is a neat parlor trick. It's tolerable for a few short-term uses (demonstrations, training, playing with alpha software, or testing different operating systems) but it's not a viable solution for real development.

Let's take fast computers and make them not

Resources

The problem is that virtualizing a computer in software is just incredibly inefficient. They waste system resources like crazy - they're so inefficient that they make today's multi-core machines packed full of RAM crawl. Their blessing - they let us simulate an entire machine in software - is also their curse. Simulating an entire machine in software means they're running a separate network stack, drawing video on a pretend video card, managing pretend system resources on a pretend motherboard, playing sound on pretend soundcards, etc. Even if you spend some time and / or money optimizing your virtual machine image, virtual machine images are still pigs compared to working on real machines.

Note: Machine Virtualization is a complex subject, and I'm glossing over things a little here. Virtual PC and VMWare both use native virtualization combined with virtual machine additions to leverage physical hardware as much as possible, trapping and emulating only what's necessary. So, VM's don't emulate all hardware resources, but the fact that they need to essentially filter all CPU instructions isn't a very efficient use of system resources.

Size

Then there's the problem of space. Even an optimized image with a significant amount of software - say, a development environment and a database - runs a few GB. While we can make space for them on today's large drives the problem there is in transportation and backup. Multi-GB virtual drive files are a pain to move around. And even if you do optimize them, you're keeping multiple copies of things like "C:\WINDOWS\Driver Cache\", "C:\WINDOWS\Microsoft.NET\", etc.

Updates and virus scanning

Many people mistakenly believe that they don't need to worry about safety (patches, automatic updates, virus scanners) on virtual machines since the host machine is taking care of those things. Not so. Each virtual machine with internet or network access is more than capable of becoming infected or compromised in the browser over port 80, or of spreading viruses which exploit network issues (e.g. SQL Slammer on port 1433). A virtual machine needs to be patched and protected, but since it's not on regularly it's not as likely to have been Auto-Updated. So, if you're planning to work on a virtual machine, you should plan to spend twice as much time and effort on operating system maintenance.

Host machine services

A personal pet peeve is the vampire services that VMWare runs. Yes, their VMWare Player is free, but just by installing it you install a bunch of Windows Services which autostart with your computer and run until you uninstall the VMWare Player, even if you never open a single VM.

Isolating your work environment makes it harder to get work done

By its very nature, virtualizing an environment means that your files run on what might as well be a separate computer. That means that it's difficult to take advantage of programs on the base machine. Yes, you can share clipboards and map drives to a virtual machine (while it's running), but that's about it. If, for instance, you run Outlook on the base machine and Visual Studio in a virtual machine - you end up jumping through some hoops to send the output of a program via e-mail, or view tabular data from SSMS in Excel, or add an emailed logo to your web application. These are all simple enough to get around, but add significant friction to your daily work.

Why are we virtualizing entire machines again? Isn't there a better solution?

I sure think so, or this post would just be a useless rant. I try to avoid those.

How about running unvirtualized software?

For instance, I've been developing on Visual Studio 2008 since Beta 1, and I've got it installed side by side with Visual Studio 2005. No problems. I recently upgraded to Visual Studio 2008 Beta 2 with no problems. Truth be told, this wasn't my gutsy idea - Rob Conery did it, and (as with a few other things) I followed him over the cliff - fortunately in this case it was not a real actual cliff. As software consumers, we should be able to expect software that we can trust enough, you know, actually install and use. I understand that beta software is beta software, and I'm a fan of releasing software early and often, but I should be able to expect that beta software won't trash my machine. As a side note here, I continue to be really impressed by the ever-increasing quality of Visual Studio releases.

The best way for developers to better the situation is to write software which is well compartmentalized, so we don't need to put it in an artificial container. This is commonly done by making good use of an application virtual machine (like the common language runtime, the Java language runtime, etc.). These technologies tend to steer you towards writing well isolated software, but for the sake of legacy integration they don't prevent you from going back to old, bad habits like writing to the Windows registry, modifying or installing shared resources, installing COM objects, or storing files and settings in the wrong place. Software written to run on a software virtual machine isn't guaranteed to be isolated, but it's generally more likely to play nice.

Folks talk about running under a non-administrator account, but I'm not completely convinced. While that solution does prompt you before modifying your system, it hasn't really been that helpful in my experience. Too much software just stops working under a lower privileged account. At that point you're back to the choice - install the software as an administrator or don't install it, but once you install as administrator you've pretty much said "Sure, do whatever you want to my system, please don't trash it." Hopefully software and users will both move together into an "administrator not required" world, but it's difficult as a software user to make that move before software vendors have fully embraced it.

Those other applications that won't play nice.

So what do you do with applications that insist on modifying your system? Short of the Virtual Machine Nuclear Option, is there a way to handle software which, by design, makes fundamental changes to your system?

Yes, I'm thinking of you, Internet Explorer, and I'm shaking my head. You don't play nice - IE7 won't share a computer with IE6 without a fight. I know you've got a lot of excuses, and I still don't buy them (check out this video at 1:05:30, that's me bugging the IE team about this at MIX06). To my way of thinking, IE is a browser, and if the .NET Framework folks have figured out how to run entire platforms side by side, you can manage to do that with an application which renders web pages. I felt strongly enough about this to put a significant amount of time (okay, a ridiculous amount of time) into developing and supporting workarounds - not really for my use, but for thousands of web developers (judging from the downloads and hits). I thought the IE team's suggestion to go and use VPC was... well, pretty unhelpful, especially considering that the need to run IE6 and IE7 on the same machine was to try to write forward compatible HTML which still worked with IE6's messed up rendering. I'll move off the IE example before this turns more rantlike...

The point there, I guess, is that virtual machines are sleazy hacks which users can decide to use, but software vendors should be ashamed to require. So... no matter how much we wish we could install all our applications on one machine, some applications won't play nice. As it is now, the only out we've got is to virtualize the entire machine. Is there a way to sandbox applications which won't sandbox themselves?

Waiting for an Application Virtualization host

Yes - there's a better solution which has gone almost completely unnoticed: application virtualization. I've been reading about application virtualization technology for a while, but have been frustrated with how long it's taken to actually see it in practice. The idea is to host a program in a sandbox that virtualizes just the things we don't want modified - for instance, registry and DLL's.

GreenBorder and SoftGrid are two commercial solutions to this problem. The good news and bad news about both of these programs is that they've been bought in the past year - GreenBorder was bought by Google, and SoftGrid was bought by Microsoft. The good news part of that is that the technology is backed by major, established companies; the bad news is that the technology may be folded into larger product offerings (Hello, FolderShare? You guys got eaten by SkyDrive, but the best features haven't made it over yet... And Groove, sure miss the file preview in Groove 2007...).

SoftGrid

Here's a diagram from Microsoft's SoftGrid documentation:

Pretty cool, that's exactly what I'd like to be able to do. The problem (at least as I see it) is that this cool SystemGuard stuff is wrapped up inside a bigger Application Streaming system. The benefit is that existing applications can be wrapped up into packages which deploy kind of like ClickOnce applications - they're sandboxed, automatically updated, and centrally managed:

That's neat if you're looking at this as a complement to SMS from an enterprise IT point of view, but it's not helpful for individuals who just want to be able to install software on their local machine. I'm especially un-excited about application streaming after having to use Novell ZENWorks at a former job at a financial company. Virtualized Microsoft Office must sound great to IT folks, but it wasn't very productive for end users.

GreenBorder

GreenBorder takes a smaller scope - it just sandboxes browsers, making the border of the "safe" browser green (get it?).

That's a cool idea, but it's completely web-centric. I'd expect Google's use of it to stay that way. So, it's nice for safe browsing, but not helpful if you want to sandbox an arbitrary application on your computer.

Sandboxie

UPDATE: I just remembered another application virtualization program I've been meaning to look into: Sandboxie. The unregistered version of Sandboxie is free, and registration is only $25. From their site:

Sandboxie changes the rules such that write operations do not make it back to your hard disk.

The illustration shows the key component of Sandboxie: a transient storage area, or sandbox. Data flows in both directions between programs and the sandbox. During read operations, data may flow from the hard disk into the sandbox. But data never flows back from the sandbox into the hard disk.
If you run Freecell inside the Sandboxie environment, Sandboxie reads the statistics data from the hard disk into the sandbox, to satisfy the read requested by Freecell. When the game later writes the statistics, Sandboxie intercepts this operation and directs the data to the sandbox.
If you then run Freecell without the aid of Sandboxie, the read operation would bypass the sandbox altogether, and the statistics would be retrieved from the hard disk.
The transient nature of the sandbox makes it is easy to get rid of everything in it. If you were to throw away the sandbox, by deleting everything in it, the sandboxed statistics would be gone for good, as if they had never been there in the first place.

I'll have to give that a try the next time I want to test something... but since I'm completely sure it'll work well, should I test it in a Virtual Machine? Hmm...

MORE UPDATES:

Xenocode - I got an e-mail from Kenji Obata at Xenocode, letting me know that Xenocode virtualizes applications inside of standalone EXE's. It looks like a well thought out solution, but a single developer license costs $499. This is worth a look for professional or enterprise application virtualization without buying into the whole SoftGrid style application streaming deployment thing.

In the funny timing department, my copy of Redmond Magazine just arrived today. The September issue has an overview of application virtualization, and compares Altiris Software Virtualization Solution (SVS) with LANDesk Application Virtualization.

Let me know if there are more Application Virtualization systems I've missed, or if you've used any of these please comment with your experiences.

Can Operating Systems tell if they're running in a Virtual Machine?

Jon Galloway — Sat, 28 Oct 2006 06:06:00 GMT

Or, do androids know they're dreaming of electric sheep...

There was some recent news on Windows Vista EULA restrictions relating to Virtual Machines. Vista Home Editions aren't allowed to be run inside a Virtual Machine, and Vista Ultimate in a VM will restrict access to applications which use DRM. We're still waiting for clarification from Microsoft, but it seems like the popular interpretations are basically right.

This raises the question - is this a EULA restriction, or is it going to be enforced. Can it be enforced? Can an operating system tell if it's running in a Virtual Machine?

That's really two questions:

Can Operating Systems currently detect if they're running in a VM?
Will Operating Systems always be able to detect if they're running in a VM?

Well, I only know what I read. Let me know if you disagree...

Can Operating Systems currently detect if they're running in a VM?

Yes, they can. Right now they do it through a couple of techniques - direct hardware fingerprinting and inferred hardware fingerprinting.

Direct hardware fingerprinting is pretty straightforward. Virtual Machines have predictable hardware profiles, so you can just query for "virtual hardware" that's only available in VM's and can't easily be changed. The Virtual PC Guy describes this approach here:

The easiest way to detect that you are inside of a virtual machine is by using 'hardware fingerprinting' - where you look for hardware that is always present inside of a given virtual machine. In the case of Microsoft virtual machines - a clear indicator is if the motherboard is made by Microsoft... [WMI Script to check the motherboard vendor]
If the motherboard is made by "Microsoft Corporation" then you are inside of one of our virtual machines.

The inferred hardware fingerprinting approach is a bit more dodgy. It works by making direct machine level calls to the virtualized CPU that will reveal if the CPU is real or virtual. Some of these call instructions that the VMM's don't currently support. Others make system calls that will only succeed on specific virtual hardware, usually because of special machine calls the VM's implement to allow communication with the host OS and optimize use of host OS resources (e.g. the Virtual Machine Additions for Virtual PC / Virtual Server , or VMWare's VMware Command Line Tools). This kind of stuff is pretty slick, but it makes "undocumented system calls" look boring.

Here are some examples of indirect hardware fingerprinting:

A program on CodeProject that can detect if it is running in either VPC or VMWare.
More information on detecting the VMWare host through the presence of special IO ports implemented as system calls
The Red Pill approach, which exploits the fact that the interrupt descriptor table registor (IDTR) is relocated by VMM's. It writes to the IDTR via an SIDT instruction, the reads from the ITDR. If the values don't match, the code is executing in a VM.

Of course, this approach is subject to the whims of each VMM release, and it may vary from host OS to host OS.

These two approaches remind me of the two ways to target CSS to different browsers - ask them nicely, or beat it out of them.

Will Operating Systems always be able to detect if they're running in a VM?

Of course, that's not a question I can answer with certainty until I can get my hands on a flux capacitor and 1.21 gigawatts. That won't keep me from speculating, though...

Let's step back a second and think about whether or not we want Operating Systems to know if they're running in a virtual environment. In the context of the recent Vista EULA flap, we might want to say no - the EULA restriction is stupid, and it's a good thing that they can't enforce it.

But let's talk about The Blue Pill. It's a theoretical malware application of VM technology in which a rootkit consumes the host operating system and runs as a hypervisor (a hardware virtual machine manager). Once it's done that, it can do whatever it wants without the operating system knowing it's been compromised:

The idea behind Blue Pill is simple: your operating system swallows the Blue Pill and it awakes inside the Matrix controlled by the ultra thin Blue Pill hypervisor. This all happens on-the-fly (i.e. without restarting the system) and there is no performance penalty and all the devices, like graphics card, are fully accessible to the operating system, which is now executing inside virtual machine. This is all possible thanks to the latest virtualization technology from AMD called SVM/Pacifica. [via invisiblethings.blogspot.com]

It's mesmerizing and scary at the same time, kind of like BooBah. There's some doubt as to whether it's code or just talk at this point:

However, there is great doubt throughout computer security circles as to whether blue pill is real or a mere stunt, since details and a working sample of the source code have not been made available, contravening the industry wide standard of full disclosure. [via Wikipedia]

Regardless, the concept has been validated. Microsoft Research and a group from University of Michigan proposed SubVirt (pdf), a VMM rootkit, in May 2006. Their paper is a fascinating schizophrenic game of cat and mouse: well, you could detect this by blah, but then we could zhoop, and even if you flurped we could just breeble. The SubVirt rootkit doesn't take advantage of hypervisor technology and requires a reboot, but on the other hand it seems to be more mature.

We built VMBRs (Virtual Machine Based Rootkits) based on two available virtual-machine monitors, including one for which source code was unavailable. On today’s x86 systems, VMBRs are capable of running a target OS with few visual differences or performance effects that would alert the user to the presence of a VMBR. In fact, one of the authors accidentally used a machine which had been infected by our proof-of concept VMBR without realizing that he was using a compromised system! [Subvirt paper pdf]

The point remains, though - we probably want our operating systems to know if they're running on virtual machines. It sounds like they should always be able to do that. Anthony Liguori, and IBM software engineer who has worked on the Xen hypervisor for two years, says:

Hardware virtualization requires a technique know as "trap and emulation". The idea is that the hardware traps certain instructions and the VMM emulates those instructions in such a way as to make the software believe it is running in a virtual machine. Software emulation implies that these instructions take much longer to complete when executed under a VMM then on normal hardware. This fact is what can be used to detect the presence of a VMM. [via virtualization.info]

You may have noticed that I jumped from talking about software VMM's (VMWare, VirtualPC) to both software and hardware VM rootkits. From what I've read, it looks like this is going to be a cat and mouse game, but the VM rootkits will always need to deal with the timing issues that Anthony mentioned. The SubVirt authors discussed this, too:

A VMBR adds CPU overhead to trap and emulate privileged instructions, as well as to run any malicious services. These timing differences can be noticed by software running in the virtual machine by comparing the running time of benchmarks against wall-clock time. A VMBR can make the detector’s task more difficult by slowing down the time returned by the system clock, but the detector can overcome this by using a clock that can be read without interference from the VMBR (e.g., the user’s wristwatch). [Subvirt paper pdf]

Well, I hope we can do better than wristwatch checks. I'd hope that an OS could check the time of day once an hour and notice a 1% drag due to VM hosting, or at least pick it up over the course of a full day. Not great, but at least it'd be detectable.

There's one more secret weapon against bad VMM's. It's probably the best defense, but you probably aren't going to like it. I'm talking about the TPM, the Trusted Platform Module. Microsoft's Next Generation Secure Computing Base Digital Rights Management (DRM) technology (called Palladium back when Vista was Longhorn) ran on the TPM. Trusted computing works by using a hardware crypto chip which verifies hardware and software loaded by the hypervisor (which runs above the hardware virtualization layer, which runs above the good old CPU's... sheesh, this is getting complicated...).

It's as if an OS running on a Trusted Computing platform was using HTTPS (SSL) to talk to hardware and trusted software like DRM software, but with much stronger crypto. That's a good thing from the point of view of safeguarding against rootkits. It's bad news if you want to use software that works by virtualizing hardware (such as virtual soundcards which record streaming music like TotalRecorder, or virtual DVD drives which let you read ISO images like Daemon Tools or Alcohol 120). It's also bad news if you want full access to DRM protected content, since DRM processing protected by a TPM is quite a bit more robust than the flimsy DRM stuff they're using today. DRM'd media running on a Trusted platform could be sent from disk to soundcard with the same kind of anti-tampering assurance you'd expect when you connect to your bank's website across the big, bad internet. Hmm. Well, we've got a little while to think this through, since it's mostly been removed from Vista and won't ship until future versions of Windows.

Installing Windows Vista October CTP (Build 5231) on VPC with VM Additions

Jon Galloway — Thu, 20 Oct 2005 07:55:00 GMT

I installed the Windows Vista October CTP (Build 5231) on Virtual PC yesterday. There were a few gotchas that I thought I'd share in case it saves anyone else some time.

Virtual PC can't capture an ISO image greater than 2.2 GB, so you need to use something like Daemon Tools on your host machine to mount the ISO as a virtual drive, then capture it as a physical drive in VPC.
Vista won't install on a RAW partition, which is what an unformatted VPC Virtual Hard Drive (VHD) image gives you. You can do some tricky commandline DISKPART stunts, or you can just create the partition in the install process knowing that Vista still won't like it, reboot the VPC instance, and this time VPC will recognize the partition it created. I went with the second option.
This Vista build can't access Windows Update. You just get a message saying you need to use Automatic Updates and tells you how to set that up. Knowing that unpatched machines on the net can get haxxor'd in 20 minutes on average, I set the Auto Update time to the next hour and disabled the VPC network connection until that time. Silly me, there weren't any Auto Updates since it had just been released, but I still think minimizing Internet time before patching a machine is the best plan.
The VPC VM Additions Installation hangs on this Vista build, and the graphics are absolute hell without them (640x480 at 8 bit, I believe). The trick is as follows: leave the hung installation running, go into the Control Panel and remove the VGA Display Driver, and say Okay when prompted to reboot. When the system comes back up, you'll have the state of the art S3 Trio display driver. It got wiggy on me when I tried to change the Display Resolution, so I rebooted again and was able to change the resolution and things were great. Dog slow, but great. If you cancel the install or kill the wrong msiexec process, the install rolls back and the drivers get removed, causing the magic not to happen.
I kind of lied about the VM Additions. The video drivers are installed, and they make a big difference, but the rest of the VM Additions aren't installed. That means no folder sharing, etc. They might release VM Additions for Vista Beta 1, but I'm not holding my breath.

I'll leave it to the pundits to describe what's new in this build, but the most interesting things for me so far are WMP 11, Media Center Vista (which it sounds like is going to be inclueded in Vista, no longer a separate product), and the latest IE7 build.

MSDN Subscriptions - Visual Studio 2005 Team System Beta 2 VPC

Jon Galloway — Thu, 21 Jul 2005 22:31:00 GMT

Scott told us it was coming, and here it is - a VS 2005 Virtual PC image with Team System installed. It's on MSDN Subscriber Downloads (at subscription levels that will get VSTS when released, I think) under:

- Developer Tools

- Visual Studio 2005

- Visual Studio 2005 Beta 2

- English

- Visual Studio 2005 Team System Beta 2 VPC (English)

"Visual Studio 2005 Team System Beta 2 VPC (English) "

This just popped up in the MSDN Subscription RSS feed .

At 2.6GB (compressed) it contains:
"Microsoft Windows Server 2003 Standard Edition (expires September 16, 2006)
Microsoft Visual Studio 2005 Team Suite Beta 2 (expires May 1, 2006)
Microsoft Visual Studio 2005 Team Foundation Server Beta 2
Microsoft .NET Framework 2.0 Redistributable Package Beta 2
Microsoft SQL Server 2005 Community Technology Preview
Microsoft Office 2003 Standard Edition
Microsoft Live Communication Server 2003"

This SO rocks. I dig it when MS releases stuff as VPC images. It saves SO much time...

Downloading now.

Update #1, 7/20/2005 @ 5:01PM PDT:
Downloaded, unrar'ed (8.6GB uncompressed), VPC Differencing Disk created and now it's running.

Much faster than if I did all the installs myself.

The only issue I've run into so far is the Live Communications Service would not start. The Service's logon account being used, LCService, password had expired. All I had to do was that account's password to never expire and then the service started with no problems.

Even with only giving the VPC 512MB RAM it seems to be running pretty fast...

This really does rock. Good job Microsoft.

Source: MSDN Subscriptions - Visual Studio 2005 Team System Beta 2 VPC

Your system has no paging file...

Jon Galloway — Mon, 13 Jun 2005 11:45:00 GMT

My computer crashed hard a week ago. I repaired the Windows install, but since then I've been getting the following message when any of the three interactive users logs on:

Limited Virtual Memory
Your system has no paging file, or the paging file is too small. To fix this problem, go to System in Control Panel, click the Advanced tab, and under Performance, click Settings. On the Advanced tab, click Change. Click Custom size, and then type an initial or maximum page file size.

Unfortunately KB 316528 doesn't apply and KB 31270 didn't work. Frustrating. Further Googling just gave me that creepy thing where several people report a problem and no one answers it. Any ideas?

Looks like time to repave. I was going to take this opportunity to upgrade to XP Pro 64 Bit (CPU is an AMD64, Virtual PC won't run on XP 64 (yet - although I'm sure it's in the works). Drat.

[VPC] Compressing VHD files

Jon Galloway — Sat, 07 May 2005 15:51:00 GMT

As mentioned some time ago, we're shuttling VHD files around the office to share preconfigured developer environments. Even precompacted, VHD files are still pretty huge - 5 to 10 GB. They can compress down quite a bit, though - as much as 70% or so.

The built in "compressed file" support in Windows can choke on files this size. I've been using 7-zip, a free compression program, to shrink mine. It includes support for ZIP, but self-extracting SFX's can cut a few hundred more MB's off the file size. That can save several minutes on a network file copy, or make the difference between whether the file will fit on a DVD. For example:

VHD - 5.4GB
Zipped VHD - 2.1GB (38%)
SFX VHD - 1.7GB (31%)

Here are some batch files which I'm using to compress the images. They use the command line version of 7-Zip. I'm sure these batch files can be improved, or I could add a right click option to VHD files or something - I'm open to suggestions. The main thing I wanted to share, though, is the command line settings for max compression SFX and ZIP compression:

SFX (Self Extracting EXE):

d:\
cd "d:\virtual machines\VS2005\"
c:\Utils\File\7zip\7za.exe a -sfx -mx=9 VS2005Beta2.exe VS2005Beta2.vhd

Zip:

d:\
cd "d:\virtual machines\VS2005\"
c:\Utils\File\7zip\7za.exe a -tzip VS2005.zip VS2005.vhd -mx=9

[VPC] Using Virtual PC for developer portability

Jon Galloway — Sat, 07 May 2005 15:30:00 GMT

I've been using VPC quite a bit lately. At work, we've got a few applications which can take days to get set up for development - one app is ASP.NET mixed with VB DCOM on DB2, another is even more confusing. Both take a few days to configure, and that's if you follow the directions and ask the guys who built it. That kills developer portability - a 30 minute bug fix becomes a 30 hour bug fix if the developer hasn't gone through the installation initiatiation on that system yet.

So when our groups officially merged recently and cross-training was looking unavoidable, I worked with the leads on each of these systems to configure a VPC development image. The nice thing with this is that the system is correctly configured by someone that knows what they're doing. The image is fully patched and includes Visual Studio, configured websites, appropriate database client tools, virus scanning software, and source control client software.

Now, moving a new developer onto a project is as simple as installing VPC (about 5 minutes), copying the image (DVD / network share / USB2 external hard drive), and starting it up. Of course, the developer needs to check out the latest code from source control, but they just need to pull down the deltas since the image was built.

I know this is old news and elementary VPC usage for some developers. At Microsoft, for instance, QA tests on VPC's so they can give an image in the error state to the developer to debug. However, it's saved our group so much time lately that I wanted to share our success story. More VPC goodies and links to follow.