The "Error creating window handle" exception and the Desktop Heap

When a big Windows Forms application I'm working on for a client is used actively, users often get "Error creating window handle" exceptions.

Aside from the fact that the application consumes too much resources, which is a separate issue altogether that we are already addressing, we had difficulties with determining what resources were getting exhausted as well as what the limits are for these resources.
We first thought about keeping an eye on the Handles counter in the Windows Task Manager. That was because we noticed that some processes tended to consume more of these resources than they normally should. However, this counter is not the good one because it keeps track of resources such as files, sockets, processes and threads. These resources are named Kernel Objects.

The other kinds of resources that we should keep an eye on are the GDI Objects and the User Objects. You can get an overview of the three categories of resources on MSDN.

Window creation issues are directly related to User Objects.

We tried to determine what the limit is in terms of User Objects an application can use.
There is a quota of 10,000 user handles per process. This value can be changed in the registry, however this limit was not the real show-stopper in our case.
The other limit is 66,536 user handles per Windows session. This limit is theoretical. In practice, you'll notice that it can't be reached. In our case, we were getting the dreaded "Error creating window handle" exception before the total number of User Objects in the current session reached 11,000.

We then discovered which limit was the real culprit: it was the "Desktop Heap".
By default, all the graphical applications of an interactive user session execute in what is named a "desktop". The resources allocated to such a desktop are limited (but configurable).

For more information about the Desktop Heap, you can refer to the very good articles published on the NTDebugging MSDN blog:

• Desktop Heap Overview
• Desktop Heap, part 2

It's possible to monitor the Desktop Heap usage thanks to a command line tool: Desktop Heap Monitor (dheapmon.exe).

It would be interesting to monitor this usage directly from within applications to prevent crashes. We could let users know that all the resources are about to be exhausted, and ask them to close windows and prevent them from opening new screens. This would help to avoid the "Error creating window handle" errors. When these exceptions occur, it's difficult to handle them gracefully and it's often too late to react because the application is in an unstable state.

Unfortunately, it's not possible to consult the usage of the desktop heap programmatically from an application. The dheapmon.exe tool is based on a kernel mode driver (a .sys file) for collecting the data it returns.

A solution could be to create a new desktop, dedicated to the application. In practice, this is not viable though because only one desktop can be visible at a time.

Another solution is to increase the size of the Desktop Heap. We can, for example, replace the second value by default of SharedSection (3072) by 4096. Yay, more resources to waste!

Increasing the Desktop Heap is an effective solution, but that's not the ultimate one. The real solution is to consume less resources (less window handles in our case). I can guess how disappointed you can be with this solution. Is this really all what I can come up with??
Well, there is no big secret here. The only way out is to be lean. Having less complicated UIs is a good start. It's good for resources, it's good for usability too. The next step is to avoid waste, to preserve resources, and to recycle them!

Here is how we're doing this in my client's application:

• We use TabControls and we create the content of each tab on the fly, when it becomes visible;
• We use expandable/collapsible regions, and again fill them with controls and data only when needed;
• We release resources as soon as possible (using the Dispose method). When a region is collapsed, it's possible to clear it's child controls. The same for a tab when it becomes hidden;
• We use the MVP design pattern, which helps in making the above possible because it separates data from views;
  
• We use layout engines, the standard FlowLayoutPanel and TableLayoutPanel ones, or custom ones, instead of creating deep hierarchies of nested panels, GroupBoxes and Splitters (an empty splitter itself consumes three window handles...).