Andreas Zenker

Failing the Google debug test

So, here is the scenario, you have what you think must be a clever, hard to diagnose bug in the framework. You have hunted the beast thru debug sessions, Reflector, you may have even downloaded the .NET symbol files so you can step thru the framework. You are stumped, you are pretty sure what the bug is or, at least, is related to, it just doesn't add up. So, you do what any self-respecting developer would do, you Google.

You try every combination of symptoms, supposed causes that you can think of and what you get are the Google equivalent of crickets.

At this point you have two possibilities:

So, at least I blogged about it.

© Copyright 2009 - Andreas Zenker

My SharePoint "Soylent Green" Moment

When you first start working with SharePoint, like any other new (to you) technology, you turn on the fire hose and start drinking down all the books patterns, samples and articles you can find. The SharePoint learning curve, which forms roughly a 90 degree angle to your former path of progress, is bigger than most. Perhaps because of that, or perhaps because it provides a convenient excuse, you tend to swallow down whole patterns and ways of thinking about web development that may be contrary to how you would approach normal ASP.NET development.

One case in point: SharePoint web parts. To get your first web part up and running you will likely look to samples and documentation on line. MSDN has lots of good samples to choose from like this one. In this sample you will be shown how to add controls to your web part by overriding the CreateChildControls() method like so:

   1: protected override void CreateChildControls()

   2: {

   3:      //Create text box

   4:      newTitle = new TextBox();

   5:      newTitle.Text = "";

   6:      Controls.Add(newTitle);

   7:  

   8:      //Create button

   9:      saveTitle = new Button();

  10:      saveTitle.Text = "Set Web Part Title";

  11:      saveTitle.Click += new EventHandler(saveTitle_click);

  12:      Controls.Add(saveTitle);

  13: }

Kind of feels strange at first, not like adding web controls to page markup in ASP.NET, or dragging one on to the form using the visual designer if that's the way you roll. But, then, you suck it up and soldier on, "this is SharePoint development men, it's just different". After all, the options presented at this point like creating custom user controls which then you have to deploy to SharePoint don't make your life, or the learning curve, any easier so you accept and move on.

Then, at some, not-too-distant, future point you get a bug report that goes something like this: "The grid sorting is flaky, sometimes it works and sometimes it loses its sort and goes back to default." Or: "when I select an item in the grid to delete, an item gets deleted, but it's not the same one I clicked on". I tripped over these bugs myself as I was preparing to sign off on dev and hand off to UAT. As I traced the problem, stepped thru the code I realized what was happening. I was storing my current sort order and expression in ViewState to persist the data between calls, familiar approach from ASP.NET projects of the past. ViewState, however was not consistently available to the code that needed it to re-set the sort on post back. That is when it all hit me, in a true Soylent Green moment of stupid realization of the obvious. Only, instead of "It's made of people!" my shocked moment of clarity was: "It's made of dynamic web controls!"

I clearly remember occasions on previous projects where fellow devs came to me with odd problems with ViewState, either not returning what it should or the page post failing validation, or controls not persisting, or posting back, the values that were expected. I would ask if they were dynamically loading controls and, when they replied they were, I would say: "don't do that", or something "clever" like "just because it's possible doesn't make it a good idea". Now here I was being bitten by the same stupid bugs. Truth was, in 95% of the cases we didn't really need to load controls dynamically, it was far easier to solve the problem another way then to take on the challenges of managing dynamic controls such as making sure the same exact controls were populated on each post back in the Page.Init() method, or else ViewState and Postback data wouldn't get populated correctly etc. In the rare occasions we deemed it necessary to load controls dynamically, we were, at least, fully aware of the ramifications and dealt with them as needed.

So, I don't know if it's just me, or if others have experienced during the SharePoint learning "wall", that you almost forget that it's all ASP.NET under the hood, and the "best practices" and lessons you have learned thru the years just get chucked out the window because this is SharePoint.

© Copyright 2009 - Andreas Zenker

2003/2008 Server ASP.NET slow refresh - TCP Chimney Offload

Ran into this issue with a SharePoint server running on 2008 Server. However the issues can affect 2003 Server SP2 and non-SharePoint sites as well.

The basic problem description is slow page refresh times. This is the point where you say "SharePoint slow? No Way!" ;-) Anyway, this was more than slow startup time. In fact, the initial page load on a site that had already been JITed was pretty reasonable. On subsequent page refreshes, not just re-navigating to the page using the links but an actual "F5" refresh, the load time was abysmal. Initial load time of a few seconds was now averaging over 30 seconds a page. Most noticeable were the image files in my custom master page header. They would be last to render on the page.

Analyzing the problem I opened up Fiddler to see what requests were taking so long and hopefully get some clues. I notice these different behaviors:

Caching Setup:

I have the aspx pages all set to expire immediately, this is set in SharePoint master page

Page load with browser cache emptied:

• With no pages in cache all files are requested for first time
• All files (aspx,js,CSS,images) requested return HTTP status of 200
• Load time was reasonable

Subsequent page loads using navigation links:

• cached files are not checked with server, only the aspx page is requested.
• HTTP status of 200.
• Load time reasonable

Subsequent page loads using refresh (F5):

• aspx page is requested, returns HTTP 200
• All other resource files are checked on server to see "if-modified-since" last known change date
• Some of these files come back with HTTP 401
• this kicks off a new NTLM negotiation handshake
• Handshake succeeds, returns eventually a 304 "not modified" as expected
• These request take a "huge" amount of time. in my scenario 5+ seconds
• The rest of the resource files return a 304 on first request and return in less than a 10th of a second
• The files that return 401's are somewhat random
• consistent offenders are in my custom SharePoint layout images folder ("_layouts/clientname/projectname/images/")
• other offenders are the Ajax library scripts ("/ScriptResource.axd.....")
• However not the same files on each refresh, somewhat random even including core SharePoint CSS files on certain loads
• Load time abysmal

Subsequent page loads after resetting IIS on server:

• With items still in the cache the behavior on the browser side is the same as above
• aspx page and all resource files are checked
• however in this scenario all, or most all, files succeed with 304's on initial request
• no extra NTLM handshake, no performance lags

Subsequent page loads with caching disabled at client (using Fiddler):

• With client caching disabled (using Fiddler)
• all files need to be requested each time the page loads
• all requests come back HTTP 200, no exception, every time.

Thanks to fellow Infusion-ite Nadeem Mitha for pushing me in the right direction even though I had disregarded the blogs and white papers I had found on the subject of TCP Offloading issues with certain network adapters. The links I had found pointed only to 2003 server and did not seem to apply to my situation.

I modified the Broadcom NetExtreme properties on my 2008 server and disabled what it referred to as TCP checksum Offloading and my page load time were back down to expected.

However, I still am dealing with the somewhat random 401's being returned for some resource files. I had originally assumed they were one and same problem, which proved to be a red-herring. So now I have to revisit that on its own....

Online resource dealing with this issue:

First blog post I found on the subject by Greg Galipeau

2008 Server TCP Chimney Offload configuration

2003 Server Network Connectivity Issues related to TCP Chimney Offloading

2003 Server Hotfix for TCP Chimney Offloading related issues

© Copyright 2009 - Andreas Zenker