Managed Threads (MT) vs. ThreadPool Threads (TP)

As I was first building my Web Spider, i figured that the easiest thing to build the spider with would be the TP.  So based on my previous ramblings, I was disappointed by the fact that the WebClient also used the TP to retrieve its results, even when used in a synchronous fashion.  This effectively cut my possible performance in half.  Add to this the fact that the TP in .NET only supports 25 threads per cpu at any one moment and I was double frustrated.  The result was that I could only fire up 12.5 threads per cpu on my development system.  I just knew that if I could just switch to managed threads, I would be able to pull in 25 threads per cpu (based on the WebClient in System.NET).  While I am also constrained by the bandwidth at my office, I knew that the addition of more threads would allow me to “smooth” in the waves when the TP version wasn't able to access the network due to other work that was going on.  I just knew that I could outsmart the TP scheduling mechanism that will only allocate a specific number of threads based on system resources.

Given the above, I worked this morning on implementing MTs.  I got through my bugs, set the system to run with 20 MTs, hit the start button, and watched with excitement as....................the performance of the app went in the toilet compared to using the TP.  How in the world could this happen?  Well, as I scaled back the number of threads, I watched performance increase.  It appears that the fact that I had too many threads attempting to run across my limited bandwidth was causing too many problems.  It looks like, given the amount of bandwidth at my office that 8 MTs is the appropiate number of threads.  Maybe I am not smarter than the TP manager in .NET..............

Just remember folks, throwing more threads at a solution does not make that solution run better. 

Wally

3 Comments

  • too much context switching is bad for performance

  • Stefan,



    I think the problem is that I am out of bandwidth and once I fill the bandwidth that I do have up, the system performs worse as I add threads. Yes, I agree, too much context switching is bad for things.



    Wally

  • David,



    Great suggestion. My next problem is going to be getting enough bandwidth to try this out. I found that even my cable modem at home with the equivalent of almost two T-1s is not enough, but then again, I was running on my laptop and it was a while ago before I made a big set of changes and got my sql indexes straight. Man, being out of town sure does throw my memory off.



    Wally

Comments have been disabled for this content.