Jason Clark's Blog

Software, Hardware, whatever...

WCF Streaming Issue Under IIS

Recently, I've been doing a lot of work with streaming under an IIS hosted WCF service (using the BasicHttpBinding).   I ran into a rather strange issue, that appears to be an oversight deep inside the internals of WCF.   Enabling streaming on a binding is typically set when you anticipate sending large objects across the wire, and do not want to buffer up the entire message prior to sending.  When streaming mode is enabled, the headers of the message are buffered and the body is exposed as a stream and sent across the wire.   When you host your service inside a console app or windows service, everything works as expected (the body is indeed streamed no buffering at all).    Now comes the issue...

When you host your service under IIS, no matter if you enable streaming or not, your service will buffer the entire message prior to sending it.   The reason for this, is that it appears as though WCF does not set the Response.BufferOutput to "false" (default is true), when streaming is enabled on a service.   This seems to be an oversight in my opinion, that could be rectified in the framework code.   So the good news is, there is a way around this issue:

Since we want to somehow set that Response.BufferOutput to false, we need to get at the HttpContext.  The flexibility of WCF comes to our aid here with the ability to enable AspNet Compatibility Mode, there are 3 changes we need to make to the service, to work around this issue.  

1)  Add the following attribute to your service: [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]

2)  Inside the default constructor for your service, or wherever you want to set the bufferoutput property paste this code:

HttpContext httpContext = HttpContext.Current;

if (httpContext != null)
{
 httpContext.Response.BufferOutput = false;
}

3)  Inside <system.serviceModel> place the following tag

  <serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>

What we've done here is get access to the HttpContext and manually set the flag, which isn't wonderful, but works around the problem.

Comments

Alexander Gornik said:

i've tried this and got a hang up of iis or wcf =[

the response just does not come out =[

i dunno. IIS7 / Vista. WCF .net 3.5

if you had some problems please, can you contact me by email:

agornik (at) gmail.com ?

# March 7, 2008 7:13 AM

jclarknet said:

A hang up?  If you mean that IIS terminates the connection, no I have not seen that.   From my testing,  this work around worked fine.   Maybe some more details about what your particular scenario?

Cheers

# March 7, 2008 7:22 AM

John Samson said:

Hi,

I am trying to stream PDF but i cant get it. It waits for the PDF to download then it displays it. Is there are way of doing such that it loads page by page until it finished loading the full document(pdf)?

Thanks,

John.

# April 25, 2008 12:27 AM

BryanW said:

Thanks for the tip on the Response Buffer & BufferOutput.  This was blocking me as well.

Also, I found that Mtom will stream but the client will reconstitute the entire buffer so the benefits are lost at the client when stream processing is desired.

# April 27, 2008 10:39 PM

aconway said:

Your article hit the nail on the head for exactly the issue I am having.  However I implemented the solution you suggested (httpContext.Response.BufferOutput = false) and it did not solve my problem.

The following exception message in my situation is I believe in direct correlation to the 'maxRequestLength' property in the .config file:

"The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '10:00:00'."

In the System.Web tag the maximum value for the 'maxRequestLength' needs to be set as follows:

<system.web>

<!--2097151 = 2GB This is the maximum value allowed-->

<httpRuntime maxRequestLength="2097151" />

</system.web>

When streaming data using WCF the file is temporarily cached to the following directory on the client:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\MyWCFProject\b1a1a32e\6d915d8a\uploads

Once the file being cached hits the 2GB limit (this can be seen by hitting the F5 key to watch the file grow as it is streamed), the exception above is thrown.

Well as long as the files being streamed are 2GB or less this is not a problem.  However, I want to stream files larger than 2GB.  I do not understand the purpose of being able to stream if we are still bound by the 2GB ASP.NET 'maxRequestLength'.  This does not make any sense anyways because the file is being sent in chuncks much smaller than 2GB anyway.  My guess is that the caching process is what is causing this limitation.  If there was a way the file could just be streamed directly to its destination without caching 1st, this might solve the problem.  This is possible if the WCF service is hosted by a console application.  I moved a 3GB file no problem in less than 5 minutes using a console hosted service with the exact same code.

I thought with your solution maybe the file would not be cached, but it still is.  Do you have any reccomendations?

Thanks!

# May 14, 2008 9:18 AM

CuriousGeorge said:

You are a life-saver. I hit exactly this issue and your work-around solved it. I find it hard to believe this made it through Microsoft QA.

# July 9, 2008 3:18 PM

Jason Clark said:

aconway:

I'm quite certain 2GB is all your allowed over http in a browser.  

# July 9, 2008 6:17 PM

kevin Mocha said:

# August 28, 2008 11:39 PM

Yatendra said:

Thanks

It's a simple and elegant description for a HTTPContext in WCF.

# January 14, 2009 8:44 AM

atconway said:

As a follow up to my earlier post, I ended up hosting the WCF service in a Windows Service, but it still communicates via HTTP, so I got the best of both worlds.  By hosting in a Windows Service I did not have to be bound by the 2GB IIS limitation.  For security, I applied a SSL certificate to the Windows Service on the port the service communicates on.  I streamed a 13GB file across servers in about 5 minutes.

# February 5, 2009 12:00 PM

Sepp Renfer said:

Hi Jason, I have problems by hosting my WCF-Service under IIS.

If I change the transferMode on the Client Side, from "Buffered" to "Streamed", I always get a curiouse Exception:

The remote server returned an unexpected response: (400) Bad Request.

If I host my Service in a other host, everything works fine.

Could you send me a running Solution of a Streamed WCF Service hosted in IIS? If You want I can send you my Solution...

Thanks Sepp

# May 5, 2009 11:22 AM

Madhu said:

Hi Jason, Your post is similar to issues we have been encountering with our WCF web service. The client is however, Windows 2003 + IIS 6.0. TransferMode is streamed.

So we have a web page that lists links to PDFs. Each link invokes the web service that streams back the PDF files.

The PDF files are between 30KB - 1.2 MB.

The first few calls work just fine. Subsequently however, we stop receiving a response and IIS indicates a "Buffer Overflow". Closing the window and re-trying another link starts the delivery again for the next few links.

Is the temporary cache mentioned in C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\MyWCFProject\b1a1a32e\6d915d8a\uploads

used for all web users. Please note that this website is a secure extranet used for a lot of users.

I can send the client and service code if reqed. A quick response is very much appreciated.

# June 29, 2009 10:16 AM

danang said:

Hi,

I have tried to implement your solution. but it seems that the streaming is still not working.

Do you have an example application for this?

Thanks.

# August 12, 2009 5:59 AM

Haripraghash said:

Hi Jason,

Does this work?I am confused as the default service constructor will never be called until we implement an IInstanceprovider!

# September 30, 2009 12:38 PM
Leave a Comment

(required) 

(required) 

(optional)

(required)