Scott Forsyth's Blog

Flush IIS HTTP and FTP Logs to Disk

2012-02-04T04:11:13Z

Today I wanted to find a way to flush the IIS FTP logs on-demand. The logs for IIS FTP flush to disk every 6 minutes, and the HTTP logs every 1 minute (or 64kb). This can make troubleshooting difficult when you don’t receive immediate access to the latest log data.

After looking everywhere I could think of, from search engine searches to perusing through the IIS schema files, I figured I had better go to the source and ask Robert McMurray.

Sure enough, Robert had the answer and even wrote a blog post in response to my question with code examples for four scripting/programming languages (C#, VB.NET, JavaScript, VbScript).

There is not a netsh or appcmd solution though, so the scripting or programming options are the way to do it. Actually, you can also flush the logs by restarting the Microsoft FTP Service (ftpsvc) but, as you would assume, it will impact currently active FTP sessions.

This blog post serves three purposes.

It’s a reference pointing to Robert’s examples
I’ll include how to do the same for the HTTP logs
I’ll provide a PowerShell example which I based on Robert’s examples

1. The reference is mentioned above already, but to give me something useful to write in this paragraph, I’ll include it again. Programmatically Flushing FTP Logs.

2. For HTTP there is a method to flush the logs using netsh.

netsh http flush logbuffer

This will immediately flush the HTTP logs for all sites.

3. The FTP logs can be done from PowerShell too. Here’s a script which is the PowerShell equivalent of Robert’s examples. Just update $siteName, or pass it as a parameter to the script.

    Param($siteName = "Default Web Site")

    #Get MWA ServerManager

    [System.Reflection.Assembly]::LoadFrom( "C:\windows\system32\inetsrv\Microsoft.Web.Administration.dll" ) | Out-Null

    $serverManager = new-object Microsoft.Web.Administration.ServerManager 

    $config = $serverManager.GetApplicationHostConfiguration()

    #Get Sites Collection

    $sitesSection = $config.GetSection("system.applicationHost/sites")

    $sitesCollection = $sitesSection.GetCollection()

    #Find Site

    foreach ($item in $sitesCollection){

        if ($item.Attributes.Item($elementTagName).Value -eq $siteName){

            $site = $item

        }

    }

    #Validation

    if ($site -eq $null) { 

        Write-Host "Site '$siteName' not found"

        return

    }

    if (!($ftpServer.ChildElements.Count)){

        Write-Host "Site '$siteName' does not have FTP bindings set"

        return

    }

    #Flush the Logs

    $ftpServer = $site.ChildElements.Item("ftpServer")

    $ftpServer.Methods.Item("FlushLog").CreateInstance().Execute()

I hope one of these programming/scripting options come in handy for times when you want immediate access to the latest FTP log data.

IIS FTP User Isolation-Week 46

2012-01-23T16:34:37Z

You can find this week’s video here.

I’ve been looking forward to releasing this week’s video. IIS FTP User isolation is an interesting topic because it offers a lot of power and flexibility but it’s not very intuitive because of how it’s managed.

This week we walk through the five isolation modes to gain a full understanding of the IIS FTP method of configuration for user isolation.

IIS FTP is a powerful application, but some of the flexibility is hidden through a unique convention based method of management. It’s easy to miss the fact that IIS FTP allows the ability to have multiple users who can be directed to different folders and be fully isolated from each other. For example, you can have a designer1 who has access to the whole site while designer2 has access to just project1 only, while—if you set it up correctly—you can feel confident that designer2 can’t gain more access than they are allowed.

IIS FTP requires understanding a few core principles to manage it effectively and to ensure that you don’t overlook key security settings that would allow users to gain more access than they should. IIS FTP 7.5 offers five different isolation modes, each of which targets a different situation.

This is now week 46 of a 52 week series for the web pro, and the 3rd of a 5 week mini-series on IIS FTP. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

Also, if you’re reading this early enough, I’m taking questions for the last couple weeks of the series. Read more about it here.

You can find this week’s video here.

Redirecting Non-www to Domain Equivalent

2012-01-05T15:00:19Z

A question came up on a blog post of mine on how to redirect a domain name that doesn’t have www in it while retaining the original domain name. Basically, to have a generic redirect to add on a www if it’s not already there.

For example, how could you have something.com always redirect to www.something.com where something could be any of a number of domain names that you manage.

This is fully possible with URL Rewrite. Here’s what the config looks like. If you want a full walkthrough using the setup wizard then refer to my previous post for the details.


<rule name="non-www to www" enabled="true" stopProcessing="true">
 <match url=".*" />
 
<conditions>
 <add input="{HTTP_HOST}" pattern="^[^\.]+\.[^\.]+$" />
 
</conditions>
 <action type="Redirect" url="http://www.{HTTP_HOST}/{R:0}" />
</rule>

This rule watches for all 2 level domains and redirects to the same domain name and tacks on the www to the beginning. So something.domain.com won’t be redirected, but domain.com will.

Alternately, if visitors will come in through either http or https, you can ensure that this retains the protocol with 2 rules, one for each:

<rule name="non-www to www http" enabled="true" stopProcessing="true">
    <match url=".*" />
    <conditions>
        <add input="{HTTP_HOST}" pattern="^[^\.]+\.[^\.]+$" />
        <add input="{HTTPS}" pattern="off" />
    </conditions>
    <action type="Redirect" url="http://www.{HTTP_HOST}/{R:0}" />
</rule>
<rule name="non-www to www https" enabled="true" stopProcessing="true">
    <match url=".*" />
    <conditions>
        <add input="{HTTP_HOST}" pattern="^[^\.]+\.[^\.]+$" />
        <add input="{HTTPS}" pattern="on" />
    </conditions>
    <action type="Redirect" url="https://www.{HTTP_HOST}/{R:0}" />
</rule>

IIS FTP and IIS Manager Users-Week 45

2012-01-04T14:27:49Z

You can find this week’s video here.

IIS FTP supports a new type of user called IIS Manager Users. This type of user can be used instead of Windows Local or Active Directory Users.

They are useful if you want to manage a number of FTP users but don’t want to use Windows Local Users or you want to keep IIS FTP separate from the operating system users. They are also compatible with Shared Configuration so you can create the users once and they will work on all servers in a web farm.

To properly manage IIS Manager users with IIS FTP, there are a few things to consider for security and configuration. This week I cover everything you need to know to properly support IIS Manager Users with IIS FTP. This is week 45 of a 52 week series, and the 2nd of a 4 week mini-series on IIS FTP.

Also, if you’re reading this early enough, I’m taking questions for the last couple weeks of the series. Read more about it here.

You can find this week’s video here.

Accepting Questions for Web Pro Series Q&A

2011-12-19T03:36:51Z

If you’ve been following my 52-week series you most likely noticed that we’re nearing the end. This evening, for example, I’m recording week 47.

Predrag Tomasevic (see developer related articles he’s written) sent me an email and made a suggestion to make the last couple videos as a Q&A. I thought that would be a great idea. Let’s do it! I’ll accept questions for the next month or so and toward the end of January I’ll record the final two weeks as Q&A sessions.

Please use the comments section below to ask IIS or IIS related questions which you would like to see answered. Note that I approve all comments before they show up, otherwise the spammer overwhelm the comments, so your question may not show up immediately.

I’m targeting quicker Q&A questions rather than full walkthroughs, but feel free to ask any type of question you want and I’ll take on whichever questions that I’ll have time for, or which I hope to be able to give a reasonably worthwhile answer for.

IIS FTP Basics–Week 44

2011-12-13T17:51:45Z

You can find this week’s video here.

The FTP (File Transfer Protocol) protocol has been in wide use for years and is still widely used today. It's used for transferring files between locations and practically useful for deployment websites from a dev machine or continuous integration server to staging or production websites.

Microsoft IIS FTP has evolved over the years and the latest versions are particularly powerful. It now supports all standard functionality that you would expect from an FTP server, plus FTP over TLS (secure FTP), host headers, IIS Manager user permissions, IP restrictions, greatly improved isolation options, and more.

This week's video lesson is the start of a 4-week mini-series on IIS FTP. The four weeks will include:

Week 1: IIS FTP Basics
Week 2: IIS FTP and IIS Manager Users
Week 3: IIS FTP and User Isolation
Week 4: IIS FTP Firewall settings and Troubleshooting

This first week I cover installing IIS FTP on IIS 7.0 and 7.5, setting up the site, authentication and authorization, security, and other general FTP discussion.

This is now week 44 of a 52 week series for the web pro. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.

IIS 7.x Classic and Integrated Pipeline Modes-Week 43

2011-12-05T23:34:24Z

You can find this week’s video here.

IIS 7 has two application pool pipeline modes: Classic and Integrated. Using Integrated mode is the ideal since it allows forms authentication for all types of page requests—not just ASP.NET pages—has tighter integration with the entire pipeline, and will be leveraged by more native and 3^rd party apps over time.

However, you may run into a “HTTP Error 500.22 - Internal Server Error” when trying to run your site in Integrated mode. This week we look at the differences and find out how to migrate a configuration file from IIS6 or IIS 7 Classic mode to IIS 7 Integrated mode.

The article referenced in the video can be found here. This gives a list of breaking changes with integration mode.

This is now week 43 of a 52 week series for the web pro. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.

Setting an aspnet.config File per Application Pool

2011-12-01T17:32:59Z

The aspnet.config file is a little known config file which is supported by ASP.NET 2.0 and greater. Generally it lives in the root of the framework folder, for example:

     C:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet.config

    C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet.config

    C:\Windows\Microsoft.NET\Framework64\v2.0.50727\aspnet.config

    C:\Windows\Microsoft.NET\Framework\v2.0.50727\aspnet.config

This config file is further leveraged in ASP.NET 4.0 for concurrency and threading. For example, you can set maxConcurrentRequestsPerCPU, maxConcurrentThreadsPerCPU and requestQueueLimit, in addition to the previous runtime settings.

In Windows Server 2008 R2 (IIS 7.5) support was added to allow different settings per application pool. Where previously the settings had to be applied to the whole framework version, now they can be specific to each app pool. It does this by allowing you to create a custom aspnet.config file per app pool. You can save them wherever you want on disk and IIS will pick them up when the app pool starts.

This is supported by a new attribute for the IIS app pool called CLRConfigFile mentioned in this MSDN article. There is no default value which means that if you don’t set it, it will read the framework’s aspnet.config file and not look for additional files.

It doesn’t appear to be configurable from the application pool settings in IIS Manager so you must use any of the manual ways to set it. appcmd is probably the easiest. Here’s what it will look like, notice the two variables are the path and the application pool name:

%windir%\System32\inetsrv\appcmd.exe set config  -section:system.applicationHost/applicationPools /[name='DefaultAppPool'].CLRConfigFile:"c:\inetpub\AppPoolClrConfig\DefaultAppPool_aspnet.config"  /commit:apphost

Below is another copy of the same, but as a template. Just swap out the {AppPoolName} and {FilePath} with your settings. (run appcmd from %windir%\System32\inetsrv)

appcmd.exe set config -section:system.applicationHost/applicationPools /[name='{AppPoolName}'].CLRConfigFile:"{FilePath}" /commit:apphost

Note that the framework aspnet.config file is still used so only differences per app pool need to be set here.

Here’s an example of what the file can contain:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <runtime>
        <legacyUnhandledExceptionPolicy enabled="false" />
        <legacyImpersonationPolicy enabled="true"/>
        <alwaysFlowImpersonationPolicy enabled="false"/>
        <SymbolReadingPolicy enabled="1" />
        <shadowCopyVerifyByTimestamp enabled="true"/>
    </runtime>
    <startup useLegacyV2RuntimeActivationPolicy="true" />
  <system.web>
    <applicationPool 
        maxConcurrentRequestsPerCPU="5000"
        maxConcurrentThreadsPerCPU="0" 
        requestQueueLimit="5000" />
  </system.web>
</configuration>

As for disk permissions, the app pool will try to read the file using the app pool’s identity. Since this file is probably not a security risk you can grant Users with read permissions and it will work. Alternately, you can properly lock down the file like so:

icacls c:\inetpub\AppPoolClrConfig\DefaultWebSite_aspnet.config /grant "IIS APPPOOL\DefaultAppPool":(R)

Be sure that the file doesn’t have inheritance or other more permissive permissions already set.

Now you can make adjustments per app pool in each of the corresponding config files, and it’s also fine to point multiple app pools to the same config file.

Changes are picked up when the app pool is created, so after applying any changes be sure to recycle the app pool.

Note also that this is for IIS 7.5 and greater, and the applicationPool settings only apply in Integrated Pipeline mode.

Here are some good links with further reading:

<system.web> applicationPool settings: http://msdn.microsoft.com/en-us/library/dd560842.aspx
ASP.NET Thread Usage article: http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx
applicationPools config settings in IIS 7.5: http://msdn.microsoft.com/en-us/library/aa347554(VS.90).aspx

Change Password vs. Reset Password-Week 42

2011-11-30T14:19:59Z

You can find this week’s video here.

The differences between change password and reset password are not well known. This week's video walks through the differences and shows them in action. Tune in to find out more about password management.

It wasn’t until fairly recently that I realized that there is a difference between a change password and a reset password. One is safe, while the other not so much. I remember when Windows Server 2003 was first released and resetting a user’s password had a distinct warning about irreversible loss of information. I wondered why it wasn’t mentioned in previous operating systems, but I also wondered if it was true since I never personally noticed any impact.

It wasn’t until about a year ago when I really dug in to understand this topic better. This week’s lesson covers the differences between a change password and a reset password. In this video we also take a look at it in action so that we have a solid understanding of the topic, and briefly discuss how it works for programming APIs too.

This is now week 42 of a 52 week series for the web pro. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.

URL Rewrite city.domain.com to domain.com/city

2011-11-23T19:09:59Z

An IIS 7.x URL Rewrite question that comes up often is how to redirect something.domain.com to domain.com/city.

Here’s an example URL Rewrite rule to accomplish that:

     <rule name="CName to URL" stopProcessing="true">

        <match url=".*" />

        <conditions>

            <add input="{HTTP_HOST}" pattern="^(?!www)(.*)\.domain\.com$" />

        </conditions>

        <action type="Redirect" url="http://domain.com/{C:1}/{R:0}" />

    </rule>

This will redirect http://anything_except_www.domain.com to http://domain.com/anything_except_www.

It will also maintain the URL and querystring, so http://subdomain.domain.com/aboutus?more=info will redirect to http://domain.com/subdomain/aboutus?more=info.

It’s also possible to do this with a rewrite rule instead so that the original URL is maintained while the server sees the rewritten path. That rewrite rule will look like this:

    <rule name="CName to URL - Rewrite" stopProcessing="true">

        <match url=".*" />

        <conditions>

            <add input="{HTTP_HOST}" pattern="^(?!www)(.*)\.domain\.com$" />

        </conditions>

        <action type="Rewrite" url="/{C:1}/{R:0}" />

    </rule>

Basically this rule will look like http://subdomain.domain.com/aboutus/ to the site visitor but it will really hit siteroot/subdomain/aboutus/ on the server.

Note that there are other considerations for rewrites which are covered in more depth here.

Line Numbering in Notepad-Week 41

2011-11-21T15:28:32Z

You can find this week’s video here.

Notepad is so simple, yet so useful. Yet, at times the "Go To" appears to break and doesn't work as expected. This week's video is short and sweet. Learn about line numbering in notepad.

One of my all-time favorite applications is notepad. You may think I’m joking, but I’ve grown quite fond of notepad over the years. Like a faithful friend, always there for you when you need it. Whether it’s an old computer or new, it opens instantly. I can’t remember notepad ever crashing. Wish I could say that for most other applications.

This week’s lesson is a quick one, but if you’ve ever run into issues with line numbering in notepad, I hope you find it useful. I remember the first time the “Go To” feature didn’t work in notepad for me. It took me a while to figure it out so I hope to save you the grief that I went through.

Watch this week’s video for a couple quick tips on the tried and true notepad.

This is now week 41 of a 52 week series for the web pro. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.

Resolving Blocked Ports Through Windows Firewall-Week 40

2011-10-28T22:07:51Z

You can find this week’s video here.

This week's video shows how to easily and quickly track down IPs and ports that need to be opened to allow legitimate traffic through the firewall.

Part of being a web server administrator is keeping your firewalls properly configured and troubleshooting to track down the necessary ports and IPs that have legitimate reasons to pass through your firewall. It’s not unusual to run into difficulty trying to find out which ports are required by a particular application. Sometimes you can determine which ports to allow by reading the specifications of the application. However, there are times when documentation is limited or you deal with a custom app.

This is now week 40 of a 52 week series for the web pro. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.

Introduction to Log Parser-Week 39

2011-10-17T15:22:45Z

You can find this week’s video here.

Log Parser is a powerful tool that every web administrator should become familiar with. With Log Parser we can do deep troubleshooting and data mining of the IIS Logs, Event Viewer, the File System, other file types and more.

Useful examples of parsing the IIS logs include: finding long running pages, finding all error pages with a 500 status code, all requests from a particular IP (potential hacking attempt), and much more.

This week is an introduction to Log Parser with walkthroughs covering Event Viewer, IIS logs, various filtering, different output formats, how to find your own way around, Log Parser Lizard GUI, and more. Log parser is a tool that no web administrator should be without.

This is now the 39th week of the entire series. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.

High Availability for ARR-Week 38

2011-10-07T16:44:53Z

You can find this week’s video here.

Note: This week’s video got stuck in the publishing process so sorry for the delay, but here it is. Next week I’ll move on to some other topics besides web farms, to mix it up.

Application Request Routing (ARR) does not have high availability built in, so it's up to us to implement a solution to ensure that ARR doesn't become a single point of failure. Options include hardware load balancers, other software load balancers, or two Microsoft options-namely NLB and Clustering Services.

Today's video discusses these options and then takes us on a walkthrough of setting up Microsoft's Network Load Balancing(NLB) solution. There are certain considerations to ensure that NLB works with your network and that it properly health tests to ensure that a failed node is automatically taken out of rotation.

For your reference, the following four links were mentioned in the video:

This is now the 13th week in a mini-series on web farms, and the 38th week of the entire series. You can view past and future weeks here:http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.

ARR Tricks to Share with Web Server-Week 37

2011-09-21T22:19:32Z

You can find this week’s video here.

This video covers two specific tricks to allow the Application Request Routing (ARR) and Web roles to share the same server. This is useful if you have multiple servers and can't setup dedicated servers for ARR, or even if you want to leverage ARR's functionality on the same server as the website.

In recent weeks we’ve been looking at Application Request Routing (ARR) and how to use it in your environment. Since it is a service that lives on a Windows server, it is possible to have the ARR role on the same server(s) as the web role.

My recommendation is to give dedicated servers to the ARR role if possible. The two reasons why I recommend this is for 1) troubleshooting reasons since it’s a bit more difficult to troubleshoot URL Rewrite rules when both roles are on the same server and 2) to protect the ARR servers (very low failure rate + low performance requirements) from the web servers (higher failure rate + higher performance requirements).

However, there are still valid reasons to collapse the two roles in a single server, or into multiple ARR+WEB servers. This video discusses this option and shows two tricks on how to do this successfully.

This is now the 12th week in a mini-series on web farms, and the 37th week of the entire series. You can view past and future weeks here: http://dotnetslackers.com/projects/LearnIIS7/

You can find this week’s video here.