At ORCS Web, we’re using System Center Configuration Manager (SCCM) with Microsoft Deployment Toolkit (MDT) and have it set to deploy new images to physical or virtual hardware.
Most everything works great, but the one thing I’ve wanted to get working is to add the Hyper-V Integration services to the Boot Image (WinPE in this case) so that 1) the regular Network Adapter (not the Legacy adapter) works and 2) there is mouse support right from the beginning. Once the boot image is created, the Hyper-V virtual machine can have the .ISO image mounted as bootable media.
Note that PXE boot with Hyper-V still requires the legacy network adapter (as far as I’ve been able to determine anyway). The better way to go is to use bootable media that can be mounted to the VM as a virtual CD/DVD drive. I’ll set out to cover how to do that here.
The following three links got me going in the right direction. However, it appears that the vmguest.iso file changed from when they wrote their blog posts until now, so I used different files then they did:
Here are the key steps:
- Go to an existing Hyper-V VM and insert the Integration Services Setup Disk. This is easiest with a server which you have already installed the Integration Services on so that you will have your mouse for the remaining steps.
- Don’t run the installation now. Instead, use Windows Explorer and navigate to the new drive and the Support folder. There are 2 subfolders: amd64 and x86. To create your 32-bit image, use the x86 folder. To create your 64-bit image, use the amd64 folder (even if it’s Intel 64-bit). I created two folders on my SCCM server. One for x86 and one for x64.
- Simply double click on the .cab file to view the contents. The file will be named Windows6.0-HyperVIntegrationServices-x64.cab or Windows6.0-HyperVIntegrationServices-x86.cab.
- Copy the contents of that cab file to your SCCM server. I do it over a UNC network path, but you can use whatever means of copying that you prefer. You don’t need all of the files in the cab, but it’s a small cab and easiest just to copy everything over.
Note: Now that you have the drivers, add them to your boot image. You can use whatever method that you use for your boot image, but I’ll include instructions for SCCM here.
- Fire up Configuration Manager Console (ConfigMgr Console).
- Expand Site Database (your instance) / Computer Management / Operation System Deployment.
- Right-click on Drivers and click Import. The Import New Driver Wizard appears.
- In the Locate Driver step, point to the folder where you copied the cab contents to. Run this twice, once for 32-bit and once for 64-bit.
- In the next step I imported all drivers even though I only use 4 on a later step. I figured that I may use them for something else in the future. So, leave everything checked.
- Assign a category that mentions the bitness of the drivers so that you can tell the difference later.
- In the Add Driver to Packages step, create a new package or assign to an existing package.
- Don’t do anything on the Add Driver to Boot Images step since we don’t want all of the drivers assigned to the boot images. Click Next.
- Click Next to finish the Driver import.
Now that the drivers are imported into SCCM, you need to add them to your boot image
- In the Drivers section of SCCM, find and select the following drivers for either x86 or x64:
- Microsoft Virtual Machine Bus Network Adapter (for network support)
- Microsoft Virtual Machine Bus Input Device Miniport (for mouse support)
- Virtual Machine Bus (I’m not sure if this is needed, but I added it anyway)
- Right-click and click Add or Remove Drivers to Boot Images.
- Check the boot image that you want to add to and click Update distribution points when finished.
- Finally, recreate your boot media.
Now you will be able to install an OS on a Hyper-V VM using the regular network adapter and mouse support.
As a side, I hoped that I could use x64 boot media and install both x64 and x86 operating systems. That would save me from choosing between the images as long as the hardware supports x64. Unfortunately it didn’t appear to work. I ran into driver issues later in the install process. I’m not saying that someone else can’t get it figured. It may be possible to embed the x86 drivers into the x64 boot image, but I didn’t take it that far. What I did prove is that configuring the x86 and x64 images identically wouldn’t allow the x64 boot image to deploy an x86 OS.
When it comes to deploying websites, many developers and companies have unique and creative ways to handle deployment. While some have fully workable solutions, I believe that there is a lot of room for growth to bring powerful and straight forward publishing tools to the masses and to support mature publishing methods for simple and complex sites alike.
Visual Studio 2010 introduces 1-Click Publishing which, along with Web Deploy, makes some nice strides in this area.
VS 2010 supports 4 methods of publishing content. They are: MSDeploy Publish, FTP, File System, FPSE. Let’s talk briefly about each:
MSDeploy Publish: this is the subject of this blog post and will be covered in more depth below. At it’s core, MSDeploy, aka Web Deployment Tool, is a command line tool used for deploying websites and settings. It can deploy or migrate website content, website settings, ACLs, COM, GAC and registry settings. As you can assume, the administrator on the target machine can specify which settings are allowed to be pushed. This is the foundation of many impressive deployment tools that we should expect to see coming out of Microsoft in the coming months and years. Visual Studio takes away the command line complexity of MSDeploy and provides the ability to deploy with a single click.
FTP: File Transfer Protocol has been around since the early days of the internet but has always had one major shortcoming—security. There are ways to make it security using SFTP (SSH FTP), FTPS (FTP over TLS/SSL) and some other methods, but the FTP protocol at its core is a plain text protocol, including the authentication (username/password). Recently we’ve seen a lot of progress for FTPS within Microsoft so that servers and tools support it, so be sure to use a secure method of FTP whenever possible.
File System: If you are on the same network or using a VPN connection to the destination web server then you can use the File System method of publishing. This uses straight xcopy to push the content directly to the destination location.
FPSE: FrontPage Server Extensions. FPSE has 2 aspects. One is the extensions part which were introduced with the Internet was fairly young, to support things like hit counters, email components and other bots. Some developers that develop with FrontPage use these. The other aspect of FPSE is publishing. FPSE offers a secure method of publishing content to a web server. However, FPSE has been plagued with security and difficult configuration issues over the years and, to my relief, I believe it’s on the way out. However, VS 2010 still offers support for pushing with FPSE, so if your destination servers supports FPSE, you do have this publishing technology available to you.
The remainder of this blog post is a simple walkthrough on publishing using MSDeploy, which can be done over HTTPS, providing much better security than plain text FTP. The host or production servers need to support MSDeploy.
Playground
ORCS Web has worked with Microsoft to provide a testing account so that you can try out MSDeploy completely free of charge. This promotion is available for a few months starting in June 2009. Vishal Joshi has formally announced this plan also. If you want to try out the publish feature covered here, but don’t want to setup and configure a production server, sign up for one of these accounts.
Publish Web Tool (using MSDeploy Publish)
You can access the Publish Web Tool either by the top menu: Build –> Publish {project name} or from the Publish menu:
Open the Publish Web tool to get started.
The tool should look like this, but the fields will change with the different Publish Methods. We’ll just cover MSDeploy Publish here.
If you don’t have the information from your host or server admin team, make sure to obtain that now. I’ll cover the fields and what they mean here:
Profile Name: (at the top) This is the friendly name that you provide to reference your deployment profile. Visual Studio 2010 supports up to 50 profiles.
Publish Method: These are the 4 methods mentioned above: MSDeploy Publish, FTP, File System and FPSE.
Service URL: This is the MSDeploy URL that your administrator or host provides. It will be something like https://ServerNameOrIP:8172/MsDeploy.axd
Site/Application: This is the IIS site name, but it’s more than that. Within this field you can specify a sub folder(s) where your project will be deployed. Your host or administrator will likely provide the root path that you have access to, for example SiteName or SiteName/vdir. But you can optionally be more specific in the path, like so: SiteName/ProjectName. The root Site/Application name must be exactly the name of the destination IIS Site (plus optionally the application).
Mark as IIS application on destination: If you set a sub folder for the “Site/Application” field and check this checkbox, it will mark the folder as an application root when publishing. Your host/administrator must support this on the destination server.
Do not delete extra files on destination: If this is unchecked it will first delete everything from the destination folder before publishing the new content. You will likely only want this unchecked for the initial deployment and then have it checked every time after that, until you want to start fresh.
Allow Untrusted Certificate: The certificate that your host/administrator uses may be a self-signed certificate or it may be for a different name than the URL. Only check this if your host or administrator instructs.
User Name and Password: This is the username and password used to connect to IIS, likely provided by your host/administrator.
Save password: This goes without saying. It’s up to you whether you have Visual Studio save your password or not.
Once you’ve filled out the fields, you can Publish immediately or Save so that you can publish later.
If all is setup correctly, you can publish by clicking the Publish button, or from the 1-Click Publish button. The 1-Click Publish feature is literally that: after setup, when you make a change that you want to publish to the web, just click that one icon and it will take care of the deployment.
In my opinion, the most impressive feature of MSDeploy is that it only publishes the changes between the source and destination location. If you have a 2GB site but only changed 2 files, when you click Publish, it will deploy only the changes, very rapidly. Additionally, from a network and server utilization perspective, MSDeploy is much faster and efficient than IIS FTP.
Shortcoming – error reporting
The biggest shortcoming that I find is regarding the error reporting when something goes wrong. VS 2010 doesn’t expose to you very much information about the errors. Most error information is available to the host administrator on the destination server. If you receive an error that isn’t clear, double check each of the fields. The most common failure is related simply to incorrect information.
There’s more, but not here and now.
Visual Studio’s Publish feature has a lot more. The two other most significant features are the Database Publishing and MSDeploy’s Transform feature so that your web.config settings can be changed as they are published.
I’m not going to cover them here and no though, but I’ll provide some useful links below which cover all of these features in more detail.
Links.
The best walkthroughs and information on MSDeploy and Visual Studio’s 1-Click Publish feature likely comes from Vishal Joshi. He’s a program Manager with Microsoft for these technologies and is an authoritative source of information. Start with this blog post: Web 1-Click Publish with VS 2010
Brady Gaster, lead developer at ORCS Web, has also put together an excellent walkthrough on the same topic.
And of course the ORCS Web VS 2010 Beta testing account offer is a useful link.
Keep an eye open for more publishing tools and progress in the future. I believe we’re just at the beginning of great things to come.
With Microsoft Visual Studio 2010 Beta 1 released and available as a free download, we at ORCS Web have joined with Microsoft to provide a free testing account so that you can see for yourself how the Publish feature works for remote publishing.
As of today, you can setup a new account free of charge here. Note that this account is setup with ASP.NET 4.0 by default, which does not have a go-live license, so this account is for testing purposes only and cannot be used to host a production website.
Visual Studio 2010 now supports Web Deployment (MSDeploy) which is an impressive new technology from Microsoft, allowing rapid deployment of websites. It’s fully secure since it runs over SSL, and yet it’s lightening fast. It only copies the changed files and only the necessarily files, making deployment a breeze.
VS 2010 totes a single-click publish option now. Once you set it up, which is easy, you can deploy changes with one click.
I’ll follow up with a walkthrough with more information later this week, but I wanted to mention our hosting offer today. We didn’t launch on the same day as Bing on purpose, but we can pretend that it’s all part of planned launch to keep up with the hype. :)
Here’s the link to setup an account: http://www.orcsweb.com/hosting/vs2010beta.aspx
Sometimes I want to place a shortcut or file on the desktop of a server and want all users of the server to see it. By default, if you save a file or folders to the desktop, it is placed on only your desktop.
Prior to Vista/2008, you could manage this by copying or moving a file/folder directly to
C:\Documents and Settings\All Users\Desktop
On Vista and Windows Server 2008, there are some changes to the structure. You may see the Documents and Settings folder, but if you double click on it, you will likely get an Access Denied error. Have no fear, you can still access the user data. Rather than using C:\Documents and Settings, the User data is in C:\Users now. C:\Documents and Settings is now just a redirect for legacy code. Actually, it’s not always on the C drive, but it usually is. You can see where your profile is by typing the following in the command prompt:
echo %userprofile%
The other gotcha comes with trying to access the shared desktop. Previously it was in the “%userprofile%\All Users\Desktop” folder. If you try to access that folder, you will likely get an Access Denied error there too. The issue isn’t with access here either. It’s that the folder has been renamed.
Here’s the kicker. The new path to the shared desktop is now:
C:\Users\Public\Desktop
(Since shortcuts are used, it may show as C:\Users\Public\Public Desktop)
Note: It is a hidden folder, so if you haven’t already, make sure to set windows explorer to show hidden folders.
Additionally, you can redirect this folder or any of the user profile folders to another path or drive. This is easy to do. Simply right-click on the folder and select the Location tab.
That’s all there is to keep in mind. Remember the new folder name and you should be set.
Outlook is one of those programs that's easy to have a love-hate relationship with. It offers so much, but the earlier versions have been plagued with stability and performance issues.
With the recent updates to Outlook 2007, stability has gotten substantially better. In fact, I don't remember Outlook crashing on me in months, which says a lot to the improvements that have been made. Performance on a tuned Outlook is very comfortable now too, so with the right love and care, Outlook 2007 can function very well for you.
At the time of this writing, two performance upgrades to Outlook were recently released and are well worth installing. They are http://support.microsoft.com/?kbid=968009 (cumulative update) and http://support.microsoft.com/?kbid=961752 (hotfix). There are a bunch of exciting performance enhancements there.
However, even with the latest of everything, Outlook got REALLY slow for me last week and needed some housekeeping. It was taking me 4 – 5 seconds to view each email, which for handling any amount of email in a day, is pretty much unusable.
I’m sure there are many additional ways to improve performance in Outlook, but I thought I would explain the couple that helped me this weekend.
First and foremost, any time I hear people complain about a slow Outlook, I always tell them to clean up their email and don’t store it all in their primary mailbox. Once they clean it up, it almost always returns to a good speed again. A bloated primary mailbox causes slow load times, slow running times and slow (or failed) shutdown. Additionally the search indexer works extra hard, causing the the entire computer to slow.
I checked the size of my Outlook.ost file and it was over 9GB! Oops. Note that I use Exchange Server with caching enabled. You may have a PST as your primary mailbox.
I used a 3-step approach to get my Outlook back in-line again.
- Clean up the obvious folders
- Find the non-obvious folders
- Compact the mailboxes
1. Clean up the obvious folders
There are many different ways to work with emails. I know that a lot of people like to delete their emails when they have finished reading them. I don’t work that way. I find that I go back to emails fairly often, so I keep pretty much all email except for spam or obvious junk. This causes my email count to climb at a fast rate, and years of email really adds up. I do, however, delete old email lists and newsletter emails since they are archived online or elsewhere.
If you’re one of the people that delete your read email, this first step may simply be to get to Inbox zero and delete your emails in the process.
For me, I create a “Saved” mailbox (PST file) and I drag and drop email older than a couple months there. I like to keep a couple months worth since I’m more likely to reference recent email, and it takes more effort to search for an email thread in my Saved mailbox than in my active mailbox.
2. Find the non-obvious folders
This step is what really helped me this time. I thought I had all of my normally large folders taken care of, but it was still taking me 4 – 5 seconds to view each email. I knew something was up, so I used Outlook’s handy Mailbox Cleanup tool to find out what I was missing. You can access this from Tools –> Mailbox Cleanup…
Mailbox Cleanup has a number of tools that come in handy for cleaning up your mailbox and speeding up Outlook. In this case, it was the View Mailbox Size tool that I used.
I was in for 2 surprises after running this. The first: I came to find out that I have a lot of emails in my Deleted Items folder. I had just emptied that folder so I wasn’t expecting to find anything. It turns out that I deleted everything in the root of the Deleted Items folder, but I missed the little + showing that there were subfolders that I overlooked.
Normally I delete using “Shift Delete” which bypasses the Deleted Items folder, but occasionally I don’t, and it looks like some folders I had deleted a long time ago were still sticking around.
I didn’t save a screenshot, but it was the Folder Size tools in the screenshot above that gave me what I needed.
The second surprise was the Sync folder. I had over 61,000 items in the Sync Issues/Conflicts folder! That was the greatest cause of performance issues. I did this over the weekend and I knew that I should be spending time with my family anyway, so I did a Select All on that folder (waited a long time) and then did a Shift Delete and left the laptop to do its stuff. A few hours later my laptop was available for use again.
Performance still wasn’t resolved, so I had more work ahead.
I had to ask myself why the Conflicts folder had grown like this. I don’t have it fully figured yet, but I found that part of it had to do with NOD32, my anti-virus program. It was touching the files on the way through and adding a signature to the files to say that they are scanned. That change caused Outlook to flag the message as a sync issue. I tweaked that setting and the new Conflicts are minimal now. I’ll continue to watch this and find out the cause for the rest.
3. Compact the mailboxes
Even though the mailboxes were down to a reasonable size, the performance issue remained. Outlook still needed to compact the mailboxes to reclaim that space.
The compact option is hidden away somewhat. You can get to it from Tools –> Accounts –> Data Files
Here if you want to look at the file itself, select the mailbox name and click on Open Folder…. The Compact Now button is in the mailbox settings.
For PST files, the button is on the first screen:
Click on Compact Now and let it do it’s stuff. Note that if it’s taking a while and you want to use your computer again, you can safely cancel it at any time and it will continue next time where it left off.
An exchange mailbox hides the Compact Now button even further. It’s in the Advanced tab –> Offline Folder File Settings …
It took another few hours for the compact to complete for me. I may have been faster just deleting the mailbox and creating it again, but I let it do its thing and it went from the bloated 9GB down to around 1GB. If I really wanted, I could get it smaller, but with it performing very fast again, there was no need.
That did it. My mouse clicks are measured in milliseconds now rather than seconds!
Outlook really has improved over the years and when looked after properly, it can serve you well.
If you haven't properly utilized compression in IIS, you're missing out on a lot! Compression is a trade-off of CPU for Bandwidth. With the expense of bandwidth and relative abundance of CPU, it's an easy trade-off. Yet, are you sure that your server is tuned optimally? I wasn't, which is why I finally sat down to find out for sure. I'll share the findings here.
A few years ago I wrote about compression on IIS 6. With IIS 6, the Microsoft defaults were a long ways off of the optimum settings, and a number of changes were necessary before IIS Compression worked well. My goal here is to dig deep into IIS 7 compression and find out the impact that the various compression levels have, and to see how much adjusting is needed to finely tune a Windows web server.
Note: If you don't care about all the details, jump right down to the conclusion. I've made sure to put the key review information there.
What's my purpose here?
To find out the bandwidth savings for the different compression levels, contrast them against the performance impact on the system and come up with a recommended configuration.
Understanding IIS 7 and its Differences from IIS 6
IIS 6 allowed compression per extension type (i.e. aspx, html, txt, etc) and allowed it to be turned on and off per site, folder or file. Making changes wasn't easy to do, but it was possible with a bit of scripting or editing of the metabase file directly.
IIS 7 changes this somewhat. Instead of compression per extension, it's per mime type. Additionally, it's easier to enable or disable compression per server/site/folder/file in IIS 7. It's configurable from IIS Manager, Appcmd.exe, web.config and all programming APIs.
Additionally, IIS 7 gives the ability to have compression automatically stopped when the CPU on the server is above a set threshold. This means that when CPU is freely available, compression will be applied, but when CPU isn't available, it is temporarily disabled so that it won't overburden the server. Thanks IIS team! These settings are easily configurable, which I'll cover more in a future blog post.
Both IIS6 and IIS7 allow 11 levels of compression (actually 'Off' and 10 levels of 'On'). The goal for this post is to compare the various levels and see the impact of each.
Objectives and Rules
The first thing I had to do was determine what tests I wanted to run, and how I could achieve them. I also wanted to ensure that I could clearly see the differences between the various levels. Here are some key objectives that I set for myself:
- Various file sizes: All tests had to be applied to various sizes of files. I initially tested using the default IIS7 homepage (689bytes), a 4kb file, a 28kb file and a 516kb file. I figured that would give a good range of common file sizes.
As it turned out, part way through I discovered that performance is severely affected on files of about 200kb and greater, so I did another series of tests on 100kb, 200kb, 400kb and 800kb files.
- Test compression only: It was important to me that compression was the only factor and that other variables didn't cloud the picture. For that reason, all of my test pages generate almost no CPU on their own. Almost all of the CPU increases in my tests are from compression only.
- Compression ratio: I used Port 80 Software's free online web tool to get the compression amount.
- Stress Testing: I used Microsoft's free WCAT tool. For the heavy load testing I used two WCAT client servers so that I can test the IIS server to the limit. One client couldn't quite bring the IIS server to 100% CPU.
- Baseline Transactions per Second: After testing for a while, I realized that the most useful data was with a fixed trans/sec goal. Otherwise, if I tested with a virtually instant page and thousands of transactions per second, the compression resource usage is unrealistic and the data confusing. Very few real-life IIS servers handle thousands of very-fast pages per second. So, to make the numbers more useful, I added a 125ms delay to each page so that my server could serve up 120trans/sec with 0% CPU when compression is turned off.
- Compression at all CPU Levels: I had to turn off the CPU roll-off so that all pages are compressed, even at 100% CPU. IIS 7's default setting of CPU roll-off is great, but it gets in the way of this testing.
- Solid Computer: Since compression is CPU bound, it's important that the server isn't a dinosaur. The test server is a Dell M600 with a Quad Core Intel Xeon processor (E5405 @ 2.00Ghz). The server only has 1GB of RAM but the physical memory was never used up for these tests, so it had all the memory that it needed. The disks are SAS, 10K RPM in a RAID 1 configuration.
I'll include the raw data at the bottom of the post for anyone that is interested.
Let's take a look at the results.
Compression Levels
As shown in the graph here, the largest increase was between compression level 3 and 4. After that, the compression improvements were very gradual. However, if you check out the raw data below though, every level offered at least some incremental benefit.
Time to First Byte
Here's a fun test I've always wanted to do, but never got around to it. I was curious how quickly a compressed vs. a non-compressed page would load under minimal server load. In other words, if there is plenty of CPU to spare, does a compressed page load at near line-speed? While both Time to First Byte (TTFB)--when the page starts to load--and Time to Last Byte (TTLB)--when the page finishes loading--are valuable, I felt that the TTFB gave the picture that I needed.
As you can see, all but the 516kb came in at about 0 milliseconds, even compressed. Note that the 28kb and 4kb lines are hidden behind the 689 bytes line. Even the 516kb file came in at under 80ms. What's 80ms?? That's very low unless you have a really busy site. As far as I'm concerned from this data, even a 516kb file loads at line speed as long as the server isn't bogged down.
Let's think about the 80ms a bit more. Dynamic pages, database calls and other factors play a much bigger role. Plus, the <80ms saves over 300kb on the page transfer, which is a savings of 2.34 seconds on a 1Mbps line! (Calculation based on: 300kbytes savings. A 1Mbps line will transfer 1024Kbps/8 = 128KB per second. 300/128KB/s = 2.34 seconds)
All of the smaller pages load almost instantly, so the impression that the end user gets from a non-pegged server is going to be better with compression on. Note that my testing doesn't take Internet latency and limits into account. So, as long as the server can compress it at near line speed, it's going to benefit the site visitor all the more.
CPU Perspective
The following two charts are from a second round of testing that I did, so the file sizes are different than in the previous test. In my first round of testing I found that the file size makes a huge difference. With anything under 100kb, the compression overhead is almost non-existent. However, once you have file sizes of a couple hundred kb or greater, the CPU overhead is significant.
I ran the tests on 100kb, 200kb, 400kb and 800kb files. To put the size in perspective, I spent a good 10 minutes getting content from various sites to come up with 800kb of text. I used pages like this from Scott Guthrie's blog, and it still took multiple pages to collect 800kb of text. That particular page only has 145kb of text on it, but it's 440kb in size because of the HTML mark-up. So, a 400kb file isn't overly common (not many people have as many blog comments as Scott Gu), but it's certainly possible.
To obtain a good CPU chart, I had to use some trickery. I wanted to create a test where I could hit the server as hard as a heavily utilized server is hit, but somehow keep non-compression related CPU to 0. I achieved this by using System.Threading.Thread.Sleep(125) in the pages. This sets a 125ms delay that doesn't use any CPU. Then I set the WCAT thread count to 30 per server (2 WCAT client computers). This became my baseline since, with these settings, the IIS server handled 120 Transactions/second. From looking at some busy production servers here at ORCS Web, I concluded that that was a reasonable level for a busy server. The CPU load without compression was nearly zero, so I was pleased with this test case.
I found this the most interesting. Notice that each file size hit a sweet spot at a different compression level. For example, if you figure that your files are mostly 200kb and smaller, you may want to use Compression Level 4, which uses almost no CPU for a 200kb file.
This is on a Quad Core server, targeting 120 transactions/second. As you can see, if you plan to have that level of traffic on your server, and your file sizes are into the hundreds of kilobytes, you may want to watch the compression utilization on the server. It may be using more resources that you realize. I don't think most people have much to worry about though as the average page size is less than 100kb, and the load on the average server is often much less than 120 transactions/second.
I'll mention more in the conclusion, but it's worth briefly mentioning now that you should consider having different compression levels for static and dynamic content. Static content is compressed and cached, so it's only compressed once until the next time the file is changed. For that reason, you probably don't care too much about the CPU overhead and can go with a setting of 9.
On the other hand, dynamic pages are compressed every time (for the most part, although further details on cached dynamic pages can be found here). So, the setting for the dynamic compression level is much more important to understand.
This also lets us realize that it would be foolish to turn on compression just for the fun of it. Formats that don't compress much (or are already compressed) like JPG's, EXE's, ZIP's and the like, are often large and the CPU overhead to try to compress them further could be substantial. These aren't compressed by default in IIS 7.
Transactions per Second
Just as a reminder, my goal was to start with 120 transactions/sec. The server could handle much more, but this was controlled so that 120 was considered the base. That was achieved with <1% CPU when non-compressed.
Notice that 100kb and 200kb files can still be served up at nearly the same rate. Once you get to 800kb file sizes though, the server spends massive computing power to compress each page. Part of that isn't just compression related. Notice that even with compression turned off, IIS could only serve up 80 Transactions/sec using the same WCAT settings. However, the transacctions/sec drops off considerably with each compression level.
Conclusion
So, what is my recommendation? Your mileage will vary, depending on the types of files that you serve up and how active your involvement in the server is.
One great feature that IIS 7 offers is the CPU roll-off. When CPU gets beyond a certain level, IIS will stop compressing pages, and when it drops below a different level, it will start up again. This is controlled by the staticCompressionEnableCpuUsage and dynamicCompressionDisableCpuUsage attributes. That in itself offers a large safety net, protecting you from any surprises.
Based on the data collected, the sweet spot seems to be compression level 4. It's between 3 and 4 that the compression benefit jumps, but it's between 4 and 5 that the resource usage jumps, making 4 a nice balance between ‘almost full compression levels' and ‘not quite as bad on resource usage'.
Since static files are only compressed once until they are changed again, it's safe to leave them at the default level of 7, or even move it all the way to 9 if you want to compress every last bit out of it. Unless you have thousands of files that aren't individually called very often, I recommend the higher the better.
For dynamic, there is a lot to consider. If you have a server that isn't CPU heavy, and you actively administer the server, then crank up the level as far as it will go. If you are worried that you'll forget about compression in the future when the server gets busier, and you want a safe setting that you can set and forget, then leave at the default of 0, or move to 4.
Make sure that you don't compress non-compressible large files like JPG, GIF, EXE, ZIP. Their native format already compresses them, and the extra attempts to compress them will use up valuable system resources, for little or no benefit.
Microsoft's default of 0 for dynamic and 7 for static is safe. Not only is it safe, it is aggressive enough to give you ‘most' of the benefit of compression with minimal system resource overhead. Don't forget that the default *does not* enable dynamic compression.
My recommendation is, first and foremost, to make sure that you haven't forgotten to enable dynamic compression. In almost all cases it's well worth it, unless bandwidth is free for you and you run your servers very hot (on CPU). Since bandwidth is so much more expensive than CPU, moving forward I'll be suggesting 4 for dynamic and 9 for static to get the best balance of compression and system utilization. At this setting, I can set and forget for the most part, although when I run into a situation when a server runs hot, I'll be sure to experiment with compression turned off to see what impact compression has in that situation.
Disclaimer: I've run these tests this last week and haven't fully burned in or tested these settings in production over time, so it's possible that my recommendation will change over time. Use the data above and use my recommendations at your own risk. That said, I feel comfortable with my recommendation for myself, if that means anything. Additionally, at ORCS Web, we've run both static and dynamic at level 9 for years and have never attributed compression as the culprit to heavy CPU on production servers. My test load of 120 transactions/sec is probably more than most production servers handle, so even 9 for both static and dynamic could be a safe setting in many situations.
The How
Here are some AppCmd.exe commands that you can use to make the changes, or to add to your build scripts. Just paste them into the command prompt and you're good to go. Watch for line breaks.
Enable Dynamic Compression, and ensure Static compression at the server level:
C:\Windows\System32\Inetsrv\Appcmd.exe set config -section:urlCompression -doStaticCompression:true -doDynamicCompression:true
Alternately, apply for just a single site (make sure to update the site name):
C:\Windows\System32\Inetsrv\Appcmd.exe set config "Site Name" -section:urlCompression -doStaticCompression:true -doDynamicCompression:true
To set the compression level, run the following (this can only be applied at the server level):
C:\Windows\System32\Inetsrv\Appcmd.exe set config -section:httpCompression -[name='gzip'].staticCompressionLevel:9 -[name='gzip'].dynamicCompressionLevel:4
IIS 7 only sets GZip by default. If you use Deflate, run the previous command for Deflate too.
Note that when changing the compression level, an IISReset is required for it to take effect.
Data
In case the raw data interests you, I've provided it here.
Compression ratio
| Level |
400kb Text |
28kb Text |
1kb Text |
516kb text |
|
Size |
% |
Size |
% |
Size |
% |
Size |
% |
| Off |
3904 |
0% |
28274 |
0% |
689 |
0% |
528834 |
0% |
| 0 |
1713 |
57% |
13878 |
51% |
594 |
14% |
238630 |
55% |
| 1 |
1636 |
59% |
12825 |
55% |
588 |
15% |
221751 |
59% |
| 2 |
1597 |
60% |
12342 |
57% |
587 |
15% |
213669 |
60% |
| 3 |
1592 |
60% |
11988 |
58% |
587 |
15% |
206795 |
61% |
| 4 |
1434 |
64% |
11400 |
60% |
457 |
34% |
190362 |
65% |
| 5 |
1428 |
64% |
11228 |
61% |
457 |
34% |
184649 |
66% |
| 6 |
1428 |
64% |
11158 |
61% |
457 |
34% |
181087 |
66% |
| 7 |
1428 |
64% |
11151 |
61% |
457 |
34% |
180620 |
66% |
| 8 |
1428 |
64% |
11150 |
61% |
457 |
34% |
180485 |
66% |
| 9 |
1428 |
64% |
11150 |
61% |
457 |
34% |
180481 |
66% |
CPU Usage per Compression Level
| Level |
100kb |
200kb |
400kb |
800kb |
|
CPU |
T/sec |
CPU |
T/sec |
CPU |
T/sec |
CPU |
T/sec |
| Off |
1% |
120 |
1% |
119 |
3% |
103 |
6% |
81 |
| 0 |
1% |
125 |
1% |
115 |
11% |
117 |
65% |
83 |
| 1 |
1% |
112 |
1% |
113 |
42% |
103 |
59% |
73 |
| 2 |
1% |
126 |
1% |
123 |
48% |
103 |
65% |
76 |
| 3 |
1% |
122 |
2% |
125 |
44% |
107 |
79% |
75 |
| 4 |
1% |
120 |
2% |
122 |
43% |
95 |
79% |
62 |
| 5 |
1% |
120 |
44% |
111 |
68% |
79 |
90% |
47 |
| 6 |
6% |
117 |
47% |
111 |
83% |
75 |
95% |
33 |
| 7 |
27% |
105 |
57% |
117 |
83% |
70 |
98% |
31 |
| 8 |
45% |
105 |
60% |
120 |
84% |
66 |
98% |
29 |
| 9 |
45% |
90 |
60% |
90 |
84% |
66 |
98% |
29 |
Time to First Byte (TTFB) - minimal load. In milliseconds.
| Level |
4kb |
28kb |
689bytes |
516kb |
| Off |
0 |
0 |
0 |
0 |
| 0 |
0 |
0 |
0 |
15 |
| 1 |
0 |
0 |
0 |
24 |
| 2 |
0 |
0 |
0 |
26 |
| 3 |
0 |
0 |
0 |
33 |
| 4 |
0 |
0 |
0 |
40 |
| 5 |
0 |
0 |
0 |
57 |
| 6 |
0 |
0 |
0 |
78 |
| 7 |
0 |
0 |
0 |
79 |
| 8 |
0 |
0 |
0 |
78 |
| 9 |
0 |
0 |
0 |
78 |
Whether programmer or administrator, we're always looking for details on certain ASP.NET or IIS configuration settings, and sometimes we come up wanting because of lack of good documentation.
The IIS team released an impressive IIS 7 reference a couple nights ago. You can find it here: http://www.iis.net/configreference. This is a reference definitely worth bookmarking. You can take every IIS configuration item, click on it on the side and be provided with detailed information written directly from the IIS team. Robert McMurray has worked hard to survey the product team directly to obtain a wealth of detailed information about every configuration setting, and Pete Harris and others have been hard at work helping make this happen.
Along with most sections is a full description, info on where the setting lives, a full list of Attributes with description of each one, often setup information and configuration samples and code/command samples for appcmd, Microsoft.Web.Administration and AHAdmin code samples for .NET (C#, VB.NET), JavaScript or VBScript.
I used it yesterday already to refresh my memory about responseMode for httpErrors: http://www.iis.net/ConfigReference/system.webServer/httpErrors.
Bill Staples announced it on his blog yesterday too.
This reference is well worth bookmarking. It will come in handy.
I work with web farms on a daily basis, and one of the requirements that I run into, for testing, troubleshooting and logging, is to tell which node is handling a specific request.
To use a current example, at ORCS Web, we're testing Microsoft's new Application Request Routing Module (ARR) for performance, stability and features to see if it offers value along-side our other load balancing solutions. (Note: ARR is still in RC1 at the time of this writing) During testing, I want to be able to tell how ARR is distributing the load under different settings. Does a custom round robin setting really work as promised? Does the least current requests work? How can I tell and test? Etc, etc.
One simple but important piece of information that I want to see over and over is which server node is being served. Without knowing the server name of the web node, it's just a guessing game, and the testing is pretty useless.
Fortunately the solution is trivial in ASP.NET. There are multiple ways to do it, but two namespaces that expose the machine name are System.Environment and System.Web.HttpContext.Current.Server.
To create a testing page that exposes the server name in VB.NET, create a file called CurrentNode.aspx with the following in it:
<%
Response.Write (System.Environment.MachineName)
%>
Or, if you're using C#, the only difference is the ; (semi-colon) requirement on the end:
<%
Response.Write (System.Environment.MachineName);
%>
That is literally all that I put in the file, or else I tack this on the bottom of an existing page. You'll notice that it's not valid HTML and nothing fancy, but no common browsers will complain, this is something that I can memorize and quickly type, and it's fully functional.
The computer name is obtained by Windows on system boot-up by reading the registry value.
I'm not aware of any differences in functionality, but you can use the following instead:
<%
Response.Write (HttpContext.Current.Server.MachineName)
%>
Hit that page from the load balancer's IP (often called a VIP [virtual IP]) and you'll know which node is handling the request.
This comes in useful for troubleshooting failures too. You may have users saying that they are receiving errors on your site, but you're unable to reproduce it consistently. Expose the server name in your error logging so that you know if it's something specific to particular nodes, or if it's global across all nodes.
I don’t tend to blog too much about myself personally, but I thought I would break my mold and start off the new year with an exciting non-technical project I worked on over Christmas. This will hopefully kick off a year of more consistent blogging since I’ve been pretty quiet the last year.
Being from Canada (I’ve lived in the States for 5 years) my family is a long way away and is only able to visit every couple years. This year we planned a 2 week vacation. My parents came down from Northern Canada (in the Sub-Arctic) and my Sister and family came down from Eastern Canada.
My father’s thought of a great vacation is one where he works on a large unique project. I need to find something to build or do before he comes to visit. Note that he’s also in the computer industry, but \isn’t scared to take on any type of project. When we visited my sister’s family a few years ago, he rented a back-hoe, and we tore up her lawn and replaced the drainage along their house. When I visited my parents 2 years ago, he bought a cow, and we killed and butchered it. So, as you can guess, he was anxious for something along those lines. He suggested a treehouse, which we all thought would be very exciting, and we started looking at pictures online and dreaming up different ideas of what we would like. Of course, being the non-experts that we are, we dreamed really big, thinking it would be easily doable.
Well, to make a long story short, we had a great vacation, working until about 2:00AM most days by the time we drew out our plans for the next day, created the bill of materials and all of that. We stopped sometimes to do something non-tree-house-related with the rest of the family, and of course the rest of the family joined in also. For the most part we worked around the clock. I’ll let the following pictures tell the rest of the story.
How does a computer geek with a Prius transport lumber? Use the Prius of course! That’s me standing there. (We got a truck for some of the materials, but the Prius worked for many of our small trips.)
Here’s my son Joel after we got the main beams in place. The design allows the trees to move in the wind without stressing the treehouse. The main platform sits on the cross beam that Joel is leaning against with some metal plates to allow movement as needed. It is amazing the force of the tree movement when it moves.
My daughter Alisha figured that even 2 beams was enough to sit in the ‘treehouse’ and read a book. She’s a real outdoors person.
Here we are with the 6x6 posts were in place and we started to lay the 2x6’s for the floor. The weather was fairly nice. It was warm some days and cold other days and often rainy, but it was never too cold or wet to keep us from working. Got to love North Carolina.
Here’s my father in the garage, with the back wall built and leaning up against the garage to mostly hide it from the rain.
Here’s the same view from the other side.
Alisa did most of the platform screws.
After getting going, we realized that leg bolts screwed into the tree wouldn’t be enough, so I had to get better drilling equipment to drill a hole right through the Oak trees and use threaded rods instead. Getting through the trees was not easy!
Getting ready to put up the walls, which we build in advance in the garage.
Further progress.
Front view
My sister trying out the view.
Here I am, working on the edge. This is the floor of the loft that I’m on.
Here’s my wife, Melissa, and kids in the partially finished loft. It’s 4 feet high, so not meant for the big people except sleeping.
Working on the roof was a lot of fun.
And for Christmas, we also got a zipline. Alisha is trying it out. We all love this.
My wife was the architect and builder for the rope bridge. She’s working on it now.
We got the mailbox for Christmas too!
Here’s the rope bridge. It worked out really well.
The patio
The inside isn’t finished yet, but here’s a view of the entry to the loft. The kids built the ladder by themselves!
Here’s the treehouse with the four of us hidden in it. Can you find us?
I’m sure we could have gotten a better picture, but here’s me on the zipline.
And, for New Year’s Eve, we had it ready enough to put up some lights for a party at our house.To wrap up, here’s a video my wife put together of the building: www.forsythfamily.com/treehouse-movie.wmv
This was a rewarding project for the family. I learned a ton and enjoyed the process. There are some loose ends to finish since we’re running electricity and insulating it so that it’s usable throughout all but the hottest months of the summer. It was definitely a much larger project that we originally expected, but having a usable treehouse in the backyard is worth it all!
In IIS, you run into an interesting situation when you need to access another resource off of the IIS server and certain fairly common situations occur. When using Integrated Security, anonymous access is disabled, and impersonation is turned on, a security measure kicks in and doesn't allow your site to access resources on any network servers. This includes access to a UNC path directly from IIS or SQL Server using Windows authentication.
The reason is because of a 'double hop' that authentication is doing. When you authenticate to the IIS server using Integrated Authentication, that uses up your first 'hop'. When IIS tries to access a network device, that would be the double or second hop which is not allowed. IIS cannot in turn pass on those credentials to the next network device, otherwise the developer or administrator could abuse your credentials and use them in ways that the site visitor didn't anticipate.
This doesn't occur with anonymous access or with impersonation off because in that case IIS takes care of authenticating you and then it uses a different user for local or network access. This means that the app pool identity or anonymous user can make a network call as the first hop.
Anyway, I didn't mean to even write that much since the following blog post answers this well already, and includes the three most common solutions. I wanted to blog this as a reference point for anyone running into this authentication issue, and so that I have a reference to it myself. I'll let Numos complete the story:
http://blogs.msdn.com/nunos/archive/2004/03/12/88468.aspx
More Posts
Next page »