2003/2008 Server ASP.NET slow refresh - TCP Chimney Offload
Ran into this issue with a SharePoint server running on 2008 Server. However the issues can affect 2003 Server SP2 and non-SharePoint sites as well.
The basic problem description is slow page refresh times. This is the point where you say "SharePoint slow? No Way!" ;-) Anyway, this was more than slow startup time. In fact, the initial page load on a site that had already been JITed was pretty reasonable. On subsequent page refreshes, not just re-navigating to the page using the links but an actual "F5" refresh, the load time was abysmal. Initial load time of a few seconds was now averaging over 30 seconds a page. Most noticeable were the image files in my custom master page header. They would be last to render on the page.
Analyzing the problem I opened up Fiddler to see what requests were taking so long and hopefully get some clues. I notice these different behaviors:
Caching Setup:
I have the aspx pages all set to expire immediately, this is set in SharePoint master page
Page load with browser cache emptied:
- With no pages in cache all files are requested for first time
- All files (aspx,js,CSS,images) requested return HTTP status of 200
- Load time was reasonable
Subsequent page loads using navigation links:
- cached files are not checked with server, only the aspx page is requested.
- HTTP status of 200.
- Load time reasonable
Subsequent page loads using refresh (F5):
- aspx page is requested, returns HTTP 200
- All other resource files are checked on server to see "if-modified-since" last known change date
- Some of these files come back with HTTP 401
- this kicks off a new NTLM negotiation handshake
- Handshake succeeds, returns eventually a 304 "not modified" as expected
- These request take a "huge" amount of time. in my scenario 5+ seconds
- The rest of the resource files return a 304 on first request and return in less than a 10th of a second
- The files that return 401's are somewhat random
- consistent offenders are in my custom SharePoint layout images folder ("_layouts/clientname/projectname/images/")
- other offenders are the Ajax library scripts ("/ScriptResource.axd.....")
- However not the same files on each refresh, somewhat random even including core SharePoint CSS files on certain loads
- Load time abysmal
Subsequent page loads after resetting IIS on server:
- With items still in the cache the behavior on the browser side is the same as above
- aspx page and all resource files are checked
- however in this scenario all, or most all, files succeed with 304's on initial request
- no extra NTLM handshake, no performance lags
Subsequent page loads with caching disabled at client (using Fiddler):
- With client caching disabled (using Fiddler)
- all files need to be requested each time the page loads
- all requests come back HTTP 200, no exception, every time.
Thanks to fellow Infusion-ite Nadeem Mitha for pushing me in the right direction even though I had disregarded the blogs and white papers I had found on the subject of TCP Offloading issues with certain network adapters. The links I had found pointed only to 2003 server and did not seem to apply to my situation.
I modified the Broadcom NetExtreme properties on my 2008 server and disabled what it referred to as TCP checksum Offloading and my page load time were back down to expected.
However, I still am dealing with the somewhat random 401's being returned for some resource files. I had originally assumed they were one and same problem, which proved to be a red-herring. So now I have to revisit that on its own....
Online resource dealing with this issue:
First blog post I found on the subject by Greg Galipeau
2008 Server TCP Chimney Offload configuration
2003 Server Network Connectivity Issues related to TCP Chimney Offloading
2003 Server Hotfix for TCP Chimney Offloading related issues
© Copyright 2009 - Andreas Zenker