Latest Microsoft Blogs

I get several emails every day (through the blog) with urgent requests to help troubleshoot memory issues or memory related performance issues. Some just say “we have a memory issue, how do I troubleshoot it?” and others are far far more detailed with links to dumps and everything:) I love getting emails, but since I have a day job too (troubleshooting such issues:)) I unfortunately have to ignore most such requests through the blog:( to have time to help the customers that contact me through the normal support channels, and therefore I would recommend that if you do have a pressing issue, that you create a support case on http://support.microsoft.com/ to make sure that you get a dedicated support engineer and that your issue gets...

I was helping out on an issue the other day where the process would stall if they added enough users in their load tests.  Btw, serious kudos to them for making load tests, so much nicer to work with a problem in test rather than when it is getting critical on a production machine. We gathered some memory dumps with debug diag of the asp.net process (w3wp.exe) and found that most of the threads were waiting in this type of callstack:  NOTE:  I have changed a lot of function names and code snippets since it is not relevant for the post. 0:071> !clrstack OS Thread Id: 0x21ac (71) ESP       EIP     1c8ceb88 7c8285ec [HelperMethodFrame_1OBJ: 1c8ceb88] System.Threading.ReaderWriterLock...

A couple of weeks ago I helped out on a high CPU issue in an ASP.NET application. Problem description Every so often they started seeing very slow response times and in some cases the app didn’t respond at all and at the same time the w3wp.exe process was sitting at very high CPU usage 80-90%.  This started happening under high load, and to get the application to start responding again they needed to restart IIS. Debugging the problem They gathered a few memory dumps during the high CPU situation for us to review and when running the sos.dll command ~* e !clrstack (in windbg) to see what all the threads were doing we found that they were all stuck in callstacks similar to this one: OS Thread Id: 0x27dc (124) ESP      ...

I had an interesting case today where an asp.net app was stuck in a true deadlock.  In other words two threads were both waiting for resources that the other thread owned. The scenario in these cases usually goes something like this: FunctionA (locks on ResourceA) and calls FunctionB where it needs ResourceB FunctionB (locks on ResourceB) and calls FunctionA where it needs ResourceA And is typically pretty easy to spot, understand and fix… However, in this particular case things were a bit different. We had gotten memory dumps of the process and after loading them up in windbg, loding sos.dll, and running ~* e !clrstack to see what the threads were working on, we found that a lot of the threads were sitting in System.Threading.Monitor.Enter...

The other day I came across an issue where an ASP.NET site stopped responding and didn’t start serving requests again until the W3WP.exe process was restarted. We grabbed some memory dumps with debug diag before restarting the process to see what was going on. Debugging the dumps: I opened the dump in windbg, and loaded up sos (.loadby sos mscorwks). As with all hang dumps I usually check what the threads are doing (~* kb) and I found that one thread (10) was initiating a garbage collection. 0:010> kb ChildEBP RetAddr  Args to Child              01cceeb4 7c827d0b 77e61d1e 00000280 00000000 ntdll!KiFastSystemCallRet 01cceeb8 77e61d1e 00000280 00000000 01cceefc ntdll!NtWaitForSingleObject...

Is it a plane, is it a bird, is it a UFO? Before you can delve into any kind of troubleshooting of an issue you need to thoroughly define it.  If you don’t you’ll probably end up spending a lot of time randomly gathering and looking at data that is probably not even relevant to the issue at hand.  More importantly, how do you even know that your problem is fixed if you don’t have a good definition of the problem? This might sound like common sense, but having worked with troubleshooting the better part of the last 10 years I can tell you that it is extremely common that people start looking at data before really understanding the problem.  More often than not this is because people are stressed and want to fix the problem fast...

Recently I had a case where the customer had an issue with session variables.  The claim was that if they use in-proc session state their session variables would work just fine, but if they changed the session mode to stateserver in web.config their sessions were lost. <sessionState mode="StateServer" cookieless="UseCookies" stateConnectionString="tcpip=127.0.0.1:42424"/> When looking at bit closer at the repro, it wasn’t really all session variables that were lost, it was only ones set in Page_Error, and when they were read in an error page they redirected to (with the customErrors setting in web.config) would be null. Why is there a difference in this case between in-proc and out of process session...

I know that for a lot of you this is a bit far into the future, but I thought it would be nice to run through a few of my lab scenarios to show off some of the new features in Visual Studio 2010.  I am sure that I will miss a bunch of cool features here as this post is just based on my first experience with this, but I will likely post more about these features as we get closer to RTM.  As we are only in Beta, anything and everything may change before RTM and the below is just based on my own impressions so there may be other better ways of doing things. Why debug dumps with Visual Studio .NET I love WinDbg and will probably continue using it for many years to come since it is a very simple and clean and powerful debugger.  I...

Bret wrote a post about Managed Stack Explorer , a really nice tool that was developed in 2006 but since then seems to have been forgotten. It’s excellent if you want to troubleshoot hangs/performance issues in a process but don’t want to go through the hassle of getting and analyzing a dump.   What it will do is basically attach, snap the .net call stack and detach so you can see what all the .net threads are doing without using a debugger. What is even cooler is that you can copy/run the tool from anywhere which means that you can copy it to a clients machine if you have a runaway thread in a win forms app for example.  Have a nice weekend, Tess Read More...

A few weeks back me and Micke (one of our Architect Evangelists) had a session at TechDays where we talked about “things that looked good on paper” i.e. things that sound pretty ok in the design/development phase but sometimes turn out to be a disaster in production. We are both pretty passionate about making the lives of the ops. people easier by thinking about the maintenance of the sites/apps at design time, rather than having it be an afterthought.  I stole the title of this post from one of Mickes talks about bridging the gap between dev and ops.  The topics we brought up are based on issues that we commonly see in prod. environments and we started off each section with a quote and dissected the pros and cons and what we think...