Gunnar Peipman's ASP.NET blog

ASP.NET, C#, SharePoint, SQL Server and general software development topics.

Sponsors

News

 
 
 
DZone MVB

Links

Social

ASP.NET 4.0: Writing custom output cache providers

Until now we can use ASP.NET default output cache. We don’t have any option to write our own output cache mechanisms. ASP.NET 4.0 introduces extensible output cache – programmers can write their own output cache providers. In this posting I will show you how to write your own output cache provider and give you my sample solution to download.

Preparing

Before we begin with caching we need Visual Studio 2010 solution that has two projects – one web application and one class library. Class library is for output cache provider and web application is for testing purposes. My solution uses following names:

  • Solution: CacheProviderDemo
  • Web application: CacheProviderApp
  • Class library: MyCacheProviders

Before moving on let’s modify web application and add project reference to class library. In this case we don’t have to copy our cache provider to GAC after each build.

Configuring output caching

Now let’s change our web.config and add there the following configuration block under system.web section.


<caching>

<outputCache defaultProvider="AspNetInternalProvider">

  <providers>

   <add name="FileCache"

    type="MyCacheProviders.FileCacheProvider, MyCacheProviders"/>

  </providers>

</outputCache>

</caching>


This configuration block tells ASP.NET that there is output cache provider called FileCache available but we want to use default provider calles AspNetInternalProvider. We will change this block later.

Writing output cache provider

Now let’s start with output cache provider. From output cache configuration block you can see that Class1.cs in our class library must be renamed to FileCacheProvider.cs. Don’t forget also change the class name. Add reference to System.Web to class library and open FileCacheProvider.cs. In the beginning of this file add using line for System.Web.Caching namespace.

Our output cache provider class must extend OutputCacheProvider base class. This base class defines all methods that output cache providers must implement. Our output cache provider uses file system for cache storage. Output cache entries are serialized to binary format and back when writing and reading objects from files.

Classes for our custom output cache provider are here.


public class FileCacheProvider : OutputCacheProvider

{

    private string _cachePath;

 

    private string CachePath

    {

        get

        {

            if (!string.IsNullOrEmpty(_cachePath))

                return _cachePath;

 

            _cachePath = ConfigurationManager.AppSettings["OutputCachePath"];

            var context = HttpContext.Current;

 

            if (context != null)

            {

                _cachePath = context.Server.MapPath(_cachePath);

                if (!_cachePath.EndsWith("\\"))

                    _cachePath += "\\";

            }

 

            return _cachePath;

        }

    }

 

    public override object Add(string key, object entry, DateTime utcExpiry)

    {

        Debug.WriteLine("Cache.Add(" + key + ", " + entry + ", " + utcExpiry + ")");

 

        var path = GetPathFromKey(key);

 

        if (File.Exists(path))

            return entry;

 

        using (var file = File.OpenWrite(path))

        {

            var item = new CacheItem { Expires = utcExpiry, Item = entry };

            var formatter = new BinaryFormatter();

            formatter.Serialize(file, item);

        }

 

        return entry;

    }

 

    public override object Get(string key)

    {

        Debug.WriteLine("Cache.Get(" + key + ")");

 

        var path = GetPathFromKey(key);

 

        if (!File.Exists(path))

            return null;

 

        CacheItem item = null;

 

        using (var file = File.OpenRead(path))

        {

            var formatter = new BinaryFormatter();

            item = (CacheItem)formatter.Deserialize(file);

        }

 

        if (item == null || item.Expires <= DateTime.Now.ToUniversalTime())

        {

            Remove(key);

            return null;

        }

 

        return item.Item;

    }

 

    public override void Remove(string key)

    {

        Debug.WriteLine("Cache.Remove(" + key + ")");

 

        var path = GetPathFromKey(key);

 

        if (File.Exists(path))

            File.Delete(path);

    }

 

    public override void Set(string key, object entry, DateTime utcExpiry)

    {

        Debug.WriteLine("Cache.Set(" + key + ", " + entry + ", " + utcExpiry + ")");

 

        var item = new CacheItem { Expires = utcExpiry, Item = entry };

        var path = GetPathFromKey(key);

 

        using (var file = File.OpenWrite(path))

        {

            var formatter = new BinaryFormatter();

            formatter.Serialize(file, item);

        }

    }

 

    private string GetPathFromKey(string key)

    {

        return CachePath + MD5(key) + ".txt";

    }

 

    private string MD5(string s)

    {

        var provider = new MD5CryptoServiceProvider();

        var bytes = Encoding.UTF8.GetBytes(s);

        var builder = new StringBuilder();

 

        bytes = provider.ComputeHash(bytes);

 

        foreach (var b in bytes)

            builder.Append(b.ToString("x2").ToLower());

 

        return builder.ToString();

    }

}


With cached item we have to keep date and time that say us when cached object expires. I write simple CacheItem class that is marked as serializable (otherwise it is not possible to serialize it and therefore we cannot use binary formatter). CacheItem class is simple.


[Serializable]

internal class CacheItem

{

    public DateTime Expires;

    public object Item;

}


I made this class internal because we have no reason to show it to the world. We only use this class to store and restore cache entries with expiry date.

Some notes for FileCacheProvider class.

  • You must have appSetting called OutputCachePath that contains path to cache files folder. This path may contain ~ because in the case of web application this path goes through Server.MapPath() method.
  • File names are made of MD5 hashed cache entry key and .txt extension. You can use whatever file extension you like.
  • I am using binary formatter because it workes better for me than XmlSerializer (Add() method is used with internal classes that have no public constructor with empty argument list).
  • My code uses calls to Debug.WriteLine() method. When you run web application with Visual Studio then you can see output cache messages in output window. I found that it is very easy and powerful way to debug provider because these messages show you what is going on.

Preparing web application for testing

Let’s prepare now web application for testing. As a first thing we must make some changes to web.config. We have to add appSetting for cache folder and we have to make our file cache provider as default output cache provider. Here are XML-blocks for configuration.


<appSettings>

  <add key="OutputCachePath" value="~/Cache/" />

</appSettings>

 

<caching>

  <outputCache defaultProvider="FileCache">

    <providers>

      <add name="FileCache" type="MyCacheProvider.FileCacheProvider, MyCacheProvider"/>

    </providers>

  </outputCache>

</caching>


I expect you know where to put these blocks in web.config file. We have one more change to make before running our web application. Let’s turn output caching on for it. Although there should be providerName attribute available it doesn’t work yet. At least for my installation of Visual Studio 2010 and ASP.NET 4.0. Add the following line to Default.aspx.


<%@ OutputCache VaryByParam="ID" Duration="300" %>


I added VaryByParam for one reason: to test caching with different ID-s. Cache duration is 300 seconds (5 minutes). You can change to attributes if you like.

Testing output cache provider

Now let’s test our output cache provider. Run web application.

Web application: we are using it to test output cache provider

When you see default page in your browser check out output window in Visual Studio 2010. You should see there something like this.


Cache.Get(a2/default.aspx)
Cache.Add(a2/default.aspx, System.Web.Caching.CachedVary, 31.12.9999 23:59:59)
Cache.Set(a2/default.aspxHQNidV+n+FCDE, System.Web.Caching.OutputCacheEntry, 19.11.2009 12:20:56)

After refresh you should see something like this in output window.


Cache.Get(a2/default.aspx)
Cache.Get(a2/default.aspxHQNidV+n+FCDE)
Cache.Get(a2/styles/site.css)
Cache.Get(a2/webresource.axd)
Cache.Set(a2/styles/site.css, System.Web.Caching.OutputCacheEntry, 20.11.2009 12:17:09)
Cache.Add(a2/webresource.axd, System.Web.Caching.CachedVary, 31.12.9999 23:59:59)
Cache.Set(a2/webresource.axdHQNdVjnaEMMY-tPD2-oCbZFp7vHhVoQiB8SS98KHQMNKjUfs1FCDE, System.Web.Caching.OutputCacheEntry, 19.11.2010 12:17:09)
Cache.Get(a2/webresource.axd)
Cache.Get(a2/webresource.axdHQNdVqx_c6oOlTqymCAsGZh_8JA2FCDE)
Cache.Add(a2/webresource.axd, System.Web.Caching.CachedVary, 31.12.9999 23:59:59)
Cache.Set(a2/webresource.axdHQNdVqx_c6oOlTqymCAsGZh_8JA2FCDE, System.Web.Caching.OutputCacheEntry, 19.11.2010 12:17:10)

You see that second request caches some more data. There requests to FileCacheProvider are made by ASP.NET. During additional refresh some resources get cached and the other resources are removed from output cache.

Here is one simple screenshot fragment of file cache folder.

List of files that contain cache entries

You can open these files to see what is inside them. You should see there cache items in binary format as binary formatter serializes them.

Conclusion

Although there are not much informative and guiding documentation about custom output cache providers I got mine working seemlessly well. I am not sure about some aspects of my provider and I will consult with Microsoft about my doubts. Of course, I will let you know if I get some valuable feedback from Microsoft.

The point of this posting was to show you how to write custom output cache provider. As we saw it is not something crucial or complex – we have to extend OutputCacheProvider class and implement some methods that store, edit and clear cached items. You can write your own implementations of output cache provider and keep your output cache wherever you like.

Sample solution

Visual Studio 2010 Solution CacheProviderDemo.zip
Visual Studio 2010 Solution | Size: 181 KB

kick it on DotNetKicks.com pimp it 顶 Progg it Shout it

Comments

azamsharp said:

I am trying to create Outputcacheprovider and for some reason when I return the object back from Get it throws that the object (whatever type) cannot be converted to System.Web.Caching.CachedRawResponse.

Any ideas?

# March 9, 2010 12:41 PM

azamsharp said:

This will not handle the situations when the varybyid is from querystring.

# March 9, 2010 4:30 PM

DigiMortal said:

Well... for caching you will get the cache object that ASP.NET gives you. It creates it and when needed it gives it to your code. You have to take this object and save it. If this object is asked then you have to return it. If you start your projects in debugging mode and put break point on the first line of some of these methods then you can check out the type of object it gives you.

Take a look at this class:

[Serializable]

internal class CacheItem

{

   public DateTime Expires;

   public object Item;

}

And check out how it used in code. Specially look at attribute called Item. It is never casted to any type. It is evaluated and returned but never casted.

If you have still problems with your code then please post it somewhere (or even here) so I can see it.

# March 13, 2010 1:14 PM

wesnur said:

I just wanted to point out that if your application is running in a multi-server environment or even a web garden, ASP.NET Cache will not work because it is a stand-alone InProc cache. What you need is a distributed cache to allow you to scale up your application. NCache is an in-memory distributed cache for .NET. It has a very similar API to ASP.NET so migration to NCache is very easy. Check it out at

www.alachisoft.com/.../index.html

# March 26, 2010 7:04 AM

.NETMaster said:

I want to point out that there are significant errors in this code. Specifically, the Add method is incorrectly implemented in two ways.

First, when it checks to see if the entry already exists, it does not check to see if an existing entry has expired. If it has expired, it must be replaced with the new entry and new expiration time.

Secondly, the Add method is clearly defined as returning the CACHED entry if the key is already in the cache, not the entry for which the Add method was called. The implementation in this article does not do that.

# May 31, 2010 9:30 AM

DigiMortal said:

Thank you for your valuable feedback, .NETMaster. This may be key to solution of some other problems I ran into when dealing with caching.

# May 31, 2010 9:44 AM

OSP said:

Thankx .NETmaster, this is the key issue.

The code in Add method should be as you pointed, otherwise this provider does not work at all:

<pre>

public override object Add(string key, object entry, DateTime utcExpiry)

{

   var path = GetPathFromKey(key);

   if (File.Exists(path))

   {

       object res = Get(key);

       if (res == null)

           return entry;

       return res;

   }

   using (var file = File.OpenWrite(path))

   {

       var item = new CacheItem { Expires = utcExpiry, Item = entry };

       var formatter = new BinaryFormatter();

       formatter.Serialize(file, item);

   }

   return entry;

}

</pre>

# June 1, 2010 8:24 AM

murat yüksek said:

this method not work!

# July 10, 2010 1:46 PM

murat yüksek said:

sorry, my mistake it is working

# July 10, 2010 2:29 PM

Joe Hendricks said:

Excellent tutorial/example and followup corrections - thanks!

# August 3, 2010 10:49 AM

Neil Gallagher said:

Hello,

Thank you for the article. I have used your provider but only the Get is called. Add or Set is never called. I have a Response.Filter set. Is this likely to be the reason? If it should work with this is the data cached pre or post the Response.Filter processing?

Thanks

# October 14, 2010 10:06 AM

DigiMortal said:

Thanks for question, Neil! My implementation has some little bugs in it that I have to fix. As soon as I have time enough I will add correct implementation of output cache provider to my GitHub repository (http://bit.ly/gunhub)

# October 14, 2010 4:34 PM

Alex said:

For VaryByParam is not working. if you change the querystring only the first page is cached.

# October 19, 2010 9:00 AM

ajaxnewbee said:

Hi Gunnar Peipman,

Awesome code... wanted to know if this is thread safe, if there are multiple requests to the cache, since the files are read, will there not be a access denied error / exception

Thanks & Regards

Mujahid

# July 24, 2011 9:50 AM

Ioannis Dontas said:

Hello

The keys that the Add, Get, Set, Remove methods receive in my implementaion are Hash values. How is it possible to receive the file path as a key?

Regards

Ioannis Dontas

# October 30, 2011 4:48 PM

Dan Pop said:

Hi,

I implemented this caching solution in an ASP.MVC 2 Project, but unfortunately I get this error: "When using a custom output cache provider like 'FileCache', only the following expiration policies and cache features are supported:  file dependencies, absolute expirations, static validation callbacks and static substitution callbacks."

The way I'm using this is :

[OutputCache(Duration = 2592000, VaryByParam = "posttags", VaryByCustom="user")]

public ActionResult Index(string posttags)

Can you tell me please what I'm doing wrong?

Thanks

# January 16, 2012 5:21 AM