CRMAppPool number of worker processes and plugins

I recently came across some strange behaviour in dynamics crm. It seems that if you increase the number of worker processes from 1 (default) to n and you have plugins running synchronously crm will throw access denied exceptions randomly when the load is high on the server.

Unhandled Exception: *.CrmException:
  0x80048405
  Access is denied.
  Platform

What seems to happen is a request comes into crm, it fires a plugin then when the plugin tries to make another call to crm (using context.CreateCrmService()) it fails, the reason I believe is this request gets routed through a different w3wp which is a whole different app domain.



Workaround
    Keep the worker process count at 1 on the CrmAppPool