Tales from the Evil Empire

Bertrand Le Roy's blog

News


Bertrand Le Roy

BoudinFatal's Gamercard

Tales from the Evil Empire - Blogged

Blogs I read

My other stuff

Archives

The fastest way to resize images from ASP.NET. And it’s (more) supported-ish.

Thumb I’ve shown before how to resize images using GDI, which is fairly common but is explicitly unsupported because we know of very real problems that this can cause. Still, many sites still use that method because those problems are fairly rare, and because most people assume it’s the only way to get the job done. Plus, it works in medium trust.

More recently, I’ve shown how you can use WPF APIs to do the same thing and get JPEG thumbnails, only 2.5 times faster than GDI (even now that GDI really ultimately uses WIC to read and write images). The boost in performance is great, but it comes at a cost, that you may or may not care about: it won’t work in medium trust. It’s also just as unsupported as the GDI option.

What I want to show today is how to use the Windows Imaging Components from ASP.NET APIs directly, without going through WPF.

The approach has the great advantage that it’s been tested and proven to scale very well. The WIC team tells me you should be able to call support and get answers if you hit problems.

Caveats exist though.

First, this is using interop, so until a signed wrapper sits in the GAC, it will require full trust.

Second, the APIs have a very strong smell of native code and are definitely not .NET-friendly.

And finally, the most serious problem is that older versions of Windows don’t offer MTA support for image decoding. MTA support is only available on Windows 7, Vista and Windows Server 2008. But on 2003 and XP, you’ll only get STA support. that means that the thread safety that we so badly need for server applications is not guaranteed on those operating systems. To make it work, you’d have to spin specialized threads yourself and manage the lifetime of your objects, which is outside the scope of this article.

We’ll assume that we’re fine with al this and that we’re running on 7 or 2008 under full trust.

Be warned that the code that follows is not simple or very readable. This is definitely not the easiest way to resize an image in .NET.

Wrapping native APIs such as WIC in a managed wrapper is never easy, but fortunately we won’t have to: the WIC team already did it for us and released the results under MS-PL. The InteropServices folder, which contains the wrappers we need, is in the WicCop project but I’ve also included it in the sample that you can download from the link at the end of the article.

In order to produce a thumbnail, we first have to obtain a decoding frame object that WIC can use. Like with WPF, that object will contain the command to decode a frame from the source image but won’t do the actual decoding until necessary.

Getting the frame is done by reading the image bytes through a special WIC stream that you can obtain from a factory object that we’re going to reuse for lots of other tasks:

var photo = File.ReadAllBytes(photoPath);
var factory =
(IWICComponentFactory)new WICImagingFactory(); var inputStream = factory.CreateStream(); inputStream.InitializeFromMemory(photo,
(uint)photo.Length); var decoder = factory.CreateDecoderFromStream(
inputStream, null,
WICDecodeOptions.WICDecodeMetadataCacheOnLoad); var frame = decoder.GetFrame(0);

We can read the dimensions of the frame using the following (somewhat ugly) code:

uint width, height;
frame.GetSize(out width, out height);

This enables us to compute the dimensions of the thumbnail, as I’ve shown in previous articles.

We now need to prepare the output stream for the thumbnail. WIC requires a special kind of stream, IStream (not implemented by System.IO.Stream) and doesn’t directlyunderstand .NET streams. It does provide a number of implementations but not exactly what we need here.

We need to output to memory because we’ll want to persist the same bytes to the response stream and to a local file for caching. The memory-bound version of IStream requires a fixed-length buffer but we won’t know the length of the buffer before we resize.

To solve that problem, I’ve built a derived class from MemoryStream that also implements IStream. The implementation is not very complicated, it just delegates the IStream methods to the base class, but it involves some native pointer manipulation.

Once we have a stream, we need to build the encoder for the output format, which could be anything that WIC supports. For web thumbnails, our only reasonable options are PNG and JPEG.

I explored PNG because it’s a lossless format, and because WIC does support PNG compression. That compression is not very efficient though and JPEG offers good quality with much smaller file sizes. On the web, it matters. I found the best PNG compression option (adaptive) to give files that are about twice as big as 100%-quality JPEG (an absurd setting), 4.5 times bigger than 95%-quality JPEG and 7 times larger than 85%-quality JPEG, which is more than acceptable quality.

As a consequence, we’ll use JPEG. The JPEG encoder can be prepared as follows:

var encoder = factory.CreateEncoder(
Consts.GUID_ContainerFormatJpeg, null); encoder.Initialize(outputStream,
WICBitmapEncoderCacheOption.WICBitmapEncoderNoCache);

The next operation is to create the output frame:

IWICBitmapFrameEncode outputFrame;
var arg = new IPropertyBag2[1];
encoder.CreateNewFrame(out outputFrame, arg);

Notice that we are passing in a property bag. This is where we’re going to specify our only parameter for encoding, the JPEG quality setting:

var propBag = arg[0];
var propertyBagOption = new PROPBAG2[1];
propertyBagOption[0].pstrName = "ImageQuality";
propBag.Write(1, propertyBagOption,
new object[] { 0.85F }); outputFrame.Initialize(propBag);

We can then set the resolution for the thumbnail to be 96, something we weren’t able to do with WPF and had to hack around:

outputFrame.SetResolution(96, 96);

Next, we set the size of the output frame and create a scaler from the input frame and the computed dimensions of the target thumbnail:

outputFrame.SetSize(thumbWidth, thumbHeight);
var scaler = factory.CreateBitmapScaler();
scaler.Initialize(frame, thumbWidth, thumbHeight,
WICBitmapInterpolationMode.WICBitmapInterpolationModeFant);

The scaler is using the Fant method, which I think is the best looking one even if it seems a little softer than cubic (zoomed here to better show the defects):

Cubic
Cubic
Fant
Fant
Linear
Linear
Nearest neighbour
Nearest neighbor

We can write the source image to the output frame through the scaler:

outputFrame.WriteSource(scaler, new WICRect {
X = 0, Y = 0,
Width = (int)thumbWidth,
Height = (int)thumbHeight });

And finally we commit the pipeline that we built and get the byte array for the thumbnail out of our memory stream:

outputFrame.Commit();
encoder.Commit();
var outputArray = outputStream.ToArray();
outputStream.Close();

That byte array can then be sent to the output stream and to the cache file.

Once we’ve gone through this exercise, it’s only natural to wonder whether it was worth the trouble. I ran this method, as well as GDI and WPF resizing over thirty twelve megapixel images for JPEG qualities between 70% and 100% and measured the file size and time to resize. Here are the results:

Size of resized images
Size of resized images
 Time to resize
Time to resize thirty 12 megapixel images

Not much to see on the size graph: sizes from WPF and WIC are equivalent, which is hardly surprising as WPF calls into WIC. There is just an anomaly for 75% for WPF that I noted in my previous article and that disappears when using WIC directly.

But overall, using WPF or WIC over GDI represents a slight win in file size.

The time to resize is more interesting. WPF and WIC get similar times although WIC seems to always be a little faster. Not surprising considering WPF is using WIC. The margin of error on this results is probably fairly close to the time difference. As we already knew, the time to resize does not depend on the quality level, only the size does. This means that the only decision you have to make here is size versus visual quality.

This third approach to server-side image resizing on ASP.NET seems to converge on the fastest possible one. We have marginally better performance than WPF, but with some additional peace of mind that this approach is sanctioned for server-side usage by the Windows Imaging team.

It still doesn’t work in medium trust. That is a problem and shows the way for future server-friendly managed wrappers around WIC.

The sample code for this article can be downloaded from:
http://weblogs.asp.net/blogs/bleroy/Samples/WicResize.zip

The benchmark code can be found here (you’ll need to add your own images to the Images directory and then add those to the project, with content and copy if newer in the properties of the files in the solution explorer):
http://weblogs.asp.net/blogs/bleroy/Samples/WicWpfGdiImageResizeBenchmark.zip

WIC tools can be downloaded from:
http://code.msdn.microsoft.com/wictools

To conclude, here are some of the resized thumbnails at 85% fant:

IMG_2734_85 IMG_2744_85 IMG_2228_85
IMG_2235_85 IMG_2300_85 IMG_2305_85
IMG_2311_85 IMG_2317_85 IMG_2318_85
IMG_2325_85 IMG_2330_85 IMG_2332_85
IMG_2346_85 IMG_2351_85 IMG_2363_85
IMG_2398_85 IMG_2443_85 IMG_2445_85
IMG_2446_85 IMG_2452_85 IMG_2462_85
IMG_2504_85 IMG_2505_85 IMG_2525_85

Comments

Bertrand Le Roy said:

# May 3, 2010 12:22 PM

DBJ said:

On the first glance it seems that CallStream-ing kind-of-a interface would elegantly hide these complexities ?

--DBJ

# May 5, 2010 4:21 AM

Bertrand Le Roy said:

@DBJ: he he. Yes, a fluent interface (callstream or more conventional) for image processing would be nice...

# May 5, 2010 4:29 AM

Bertrand Le Roy said:

@Kevin: The latest WIC is not available for XP.

# May 14, 2010 6:10 PM

Brian Brewder said:

Thanks for the info. This will come in very handy for us.

# May 19, 2010 2:03 PM

ASPGuy said:

Great article, this was our strigle for a while as GDI is not officially supported on ASP.NET.

# November 5, 2010 9:00 PM

Ries Spruit said:

Thanks for the great sample Bertrand, for me it works like a charm on Windows 7, but I can't get the sample to compile on Windows 2008 Server.

I think I am missing the point with the wrappers. (Been a while since I wrapped anything..).

I run into the following compilation error:

Foutbericht van compiler: CS0234: Het type of de naam van de naamruimte Test bestaat niet in de naamruimte Microsoft (ontbreekt er een assembly-verwijzing?)

Fout in bron:

Regel 7:  using Microsoft.Test.Tools.WicCop.InteropServices.ComTypes;

(you understand Dutch right? ;-)

I can run all the WICTools samples so I assume the components are correctly installed. I just don't know how to make sure the WIC components are correctly referenced..

All help is greatly appreciated!

# February 1, 2011 2:45 PM

Bertrand Le Roy said:

@Ries: the wrappers are necessary because we are calling into unmanaged Windows APIs directly here. It seems like the problem here is in the WIC APIs so I think your only reasonable option is to contact support.

# February 1, 2011 2:55 PM

Bertrand Le Roy said:

@Brian: I've heard good things about that library but haven't tried it myself. That's a great idea for a future post though.

I think Bing is using pretty much this method above for its image resizing needs. That's as big as it gets.

# July 13, 2011 4:00 PM

Alexandre said:

Hi,

Great article, I'm trying to keep my IPTC metadata during the process.

I successfully retrieved my app13 block using :

PropVariant metadataValue = new PropVariant();

frame.GetMetadataQueryReader().GetMetadataByName("/app13", metadataValue);

But I'm not able to set it back in the resized frame.

outputFrame.GetMetadataQueryWriter().SetMetadataByName("/app13", metadataValue);

OR

outputFrame.GetMetadataQueryWriter().SetMetadataByName("/", metadataValue);

If I put this line before the WriteSource then I get a InvalidCastException or another error, if I put it after the WriteSource I don't have any metadata set in resulting picture.

Can you help me with this ?

# July 28, 2011 7:52 AM

Bertrand Le Roy said:

@Alexandre: I'm afraid not. You might want to try the MSDN forums.

# July 28, 2011 8:31 PM

Chris Hynes said:

Any idea how to performantly crop and scale the image at the same time, so that the thumbnail can be a full frame square regardless of the source size?

I popped a WICBitmapClipper in the pipeline before the scaler, which seems to work. Problem is, it doubles or triples the rendering time. It seems like cropping should reduce the amount of data the scalar needs to go through, so I'm rather stumped as to why it takes longer to scale a smaller image.

Any thoughts or can you point me to someone that might know?

I tweaked your resize code like this:

uint frameSize = Math.Min(width, height);

IWICBitmapClipper clipper = factory.CreateBitmapClipper();

clipper.Initialize(frame, new WICRect() { X = (int)((width - frameSize) / 2), Y = (int)((height - frameSize) / 2), Width = (int)frameSize, Height = (int)frameSize });

IWICBitmapScaler scaler = factory.CreateBitmapScaler();

scaler.Initialize(bmp, size, size, WICBitmapInterpolationMode.WICBitmapInterpolationModeFant);

# December 1, 2011 4:57 PM

Bertrand Le Roy said:

@Chris: just use the ImageResizer library.

# December 4, 2011 9:33 PM

Bertrand Le Roy said:

@Alexandre: I would recommend you contact support if your forum post doesn't receive answers from the WIC team.

# February 15, 2012 2:30 PM