The 'Reluctant Cache' Pattern

Caching is one of the greatest strategies for improving the performance of our applications. Operations such as database access and web service calls can take time, require network hops and consume valuable server resources such as processor cycles.

A common caching pattern is as follows:

   1:  public static List<Customer> GetCustomers() {
   2:      string cacheKey = "Customers";
   3:      int cacheDurationInSeconds = 5; //an artificially low number for demonstration
   4:   
   5:      object customers = HttpRuntime.Cache[cacheKey] as List<Customer>;
   6:   
   7:      if (customers == null) {
   8:          customers = CustomerDao.GetCustomers();
   9:   
  10:          HttpRuntime.Cache.Insert(cacheKey, customers, null, DateTime.Now.AddSeconds(cacheDurationInSeconds), System.Web.Caching.Cache.NoSlidingExpiration);
  11:      }
  12:   
  13:      return (List<Customer>)customers;
  14:  }

If the item is not present in the cache, we fetch it from the data access layer and insert it into the cache for quick access next time (assuming that there is a next time). This works well, but gives us little control as everything will be cached, regardless of how often it is accessed. Imagine how a cache will grow as google indexes our website - the majority of cached items will be thrown away after a period of time without ever been accessed a second time.

The following pattern provides a simple solution for caching the most frequently accessed items, while ignoring items that are seldomly accessed.

The Reluctant Cache Pattern

   1:  public static List<Customer> GetCustomers() {
   2:      string cacheKey = "Customers";
   3:      int cacheDurationInSeconds = 5; //an artificially low number for demonstration
   4:   
   5:      object customers = HttpRuntime.Cache[cacheKey] as List<Customer>;
   6:   
   7:      if (customers == null) {
   8:          customers = CustomerDao.GetCustomers();
   9:   
  10:          if (new ReluctantCacheHelper(cacheKey, cacheDurationInSeconds, 2).ThresholdHasBeenReached) {
  11:              HttpRuntime.Cache.Insert(cacheKey, customers, null, DateTime.Now.AddSeconds(cacheDurationInSeconds), System.Web.Caching.Cache.NoSlidingExpiration);
  12:          }
  13:      }
  14:   
  15:      return (List<Customer>)customers;
  16:  }

As you can see, it only adds one extra line to our cache helper class. The ReluctantCacheHelper class keeps a count of any requests for a particular item over a period of time. If this request count reaches a configurable threshold, it inserts the item into the cache.

A simple demonstration of this illustrates the point. If the list of customers are requested more than once in a five second period, they are placed in the cache. Source code is also available.

The ReluctantCacheHelper class listing is as follows:

   1:  public class ReluctantCacheHelper {
   2:      private short _requestCountThreshold;
   3:      private ReluctantCacheRequestToken _cacheRequestToken;
   4:   
   5:      public ReluctantCacheHelper(string cacheKey) : this(cacheKey, 60) {}
   6:      public ReluctantCacheHelper(string cacheKey, int cacheDurationInSeconds) : this(cacheKey, cacheDurationInSeconds, 2) {}
   7:      public ReluctantCacheHelper(string cacheKey, int cacheDurationInSeconds, short requestCountThreshold) {
   8:          this._requestCountThreshold = requestCountThreshold;
   9:   
  10:          //get the token
  11:          string cacheRequestTokenKey = cacheKey + "_token";
  12:          this._cacheRequestToken = (ReluctantCacheRequestToken)HttpRuntime.Cache[cacheRequestTokenKey];
  13:   
  14:          if (this._cacheRequestToken == null) {
  15:              //create and insert a new token
  16:              this._cacheRequestToken = new ReluctantCacheRequestToken();
  17:              HttpRuntime.Cache.Insert(cacheRequestTokenKey, this._cacheRequestToken, null, DateTime.Now.AddSeconds(cacheDurationInSeconds), System.Web.Caching.Cache.NoSlidingExpiration);
  18:          } else {
  19:              //increment the request token
  20:              this._cacheRequestToken.Increment();
  21:          }
  22:      }
  23:      
  24:      public bool ThresholdHasBeenReached {
  25:          get {
  26:              if (this._cacheRequestToken.RequestCount >= this._requestCountThreshold) {
  27:                  return true;
  28:              } else {
  29:                  return false;
  30:              }
  31:          }
  32:      }
  33:  }

It stores a light-weight CacheRequestToken in the cache and increments it as each request for the item is made. If the count reaches the threshold level, the item will be inserted in the cache for quick retrieval in subsequent requests.

The CacheRequestToken code listing is as follows:

   1:  public class ReluctantCacheRequestToken {
   2:      private short _requestCount = 1;
   3:      public short RequestCount { 
   4:          get { 
   5:              return this._requestCount;
   6:          } 
   7:      }
   8:   
   9:      public void Increment() {
  10:          this._requestCount++;
  11:      }
  12:  }

View Demo - Download Source -


Published Tuesday, May 23, 2006 11:51 PM by gavinjoyce
Filed under: , , ,

Comments

# re: The 'Reluctant Cache' Pattern

Nitpicking here, but why do you declare 'customers' as object?

You're already using a defensive cast with the 'as' keyword...

So having:

List<Customer> customers = HttpRuntime.Cache[cacheKey] as List<Customer>;

Will save you the cast at the end of the method:

return customers;

Wednesday, May 24, 2006 4:39 AM by Wim

# re: The 'Reluctant Cache' Pattern

Hi Wim,

I wanted to extend the pattern outlined by Steven Smith : http://weblogs.asp.net/ssmith/archive/2003/06/20/9062.aspx.

I agree that :

List<Customer> customers = HttpRuntime.Cache[cacheKey] as List<Customer>;

will save a cast and it also reads better.

Cheers,
Gavin

Wednesday, May 24, 2006 8:07 AM by gavinjoyce

# re: The 'Reluctant Cache' Pattern

I dig the pattern, but I feel I need to clarify its primary motivation.  Does the pattern work to keep only frequently-accessed items in the cache, or does it work to limit the server memory used by the cache.

If I have a dedicated web server for my application, then I don't think I would mind letting the cache grow as much as it likes.

I realize now that I don't quite know the details of setting memory limits in ASP.NET caching.  If I'm missing something, let me know.

Wednesday, May 24, 2006 2:50 PM by John Bledsoe

# re: The 'Reluctant Cache' Pattern

John,

This could be used for both ends.

dotnetkicks.com has an article cache which is currently configured with a threshold of 2 requests every 15 minutes. An article is placed in the cache on the second request. This allows the cache size to remain small even when google indexes the site. Every time an article is kicked or commented on, it is removed from the cache. If two search spiders often indexed our site at the same time, we could raise the threshold.

The threshold logic could also take into account memory availability, and adjust its decisions accordingly.

Gavin

Wednesday, May 24, 2006 9:20 PM by gavinjoyce

# re: The 'Reluctant Cache' Pattern

тема не раскрыта.. может есть ещё информация по этому поводу?

Tuesday, June 15, 2010 3:57 PM by kikus

# re: The 'Reluctant Cache' Pattern

отлично написано, у автора прям талант

Tuesday, June 15, 2010 4:06 PM by kikus

# re: The 'Reluctant Cache' Pattern

God save me from my friends, I can protect myself from my enemies.

Sunday, February 6, 2011 7:59 PM by Single Kontakt aus Bavaria

# re: The 'Reluctant Cache' Pattern

I always was concerned in this topic and still am, regards for posting .

Monday, February 7, 2011 9:58 AM by katrina kaif

# re: The 'Reluctant Cache' Pattern

I don't commonly comment but I gotta say thankyou for the post on this great one : D.

Thursday, February 10, 2011 2:49 AM by katrina kaif

# re: The 'Reluctant Cache' Pattern

I'd constantly want to be update on new content on this web site , saved to favorites ! .

Thursday, February 10, 2011 5:17 AM by led grow lights

# re: The 'Reluctant Cache' Pattern

I appreciate your work , appreciate it for all the good content .

Wednesday, February 16, 2011 12:02 AM by how to become an actor

# re: The 'Reluctant Cache' Pattern

Dig the well before you are thirsty.

Saturday, February 19, 2011 4:12 AM by male enhancement pills

# re: The 'Reluctant Cache' Pattern

Which came first, the chicken or the egg?

Wednesday, February 23, 2011 4:52 AM by Tadalis tabletten bestellen

# re: The 'Reluctant Cache' Pattern

The Reluctant Cache Pattern.. May I repost it? :)

Tuesday, March 29, 2011 2:20 AM by weblogs.asp.net

# re: The 'Reluctant Cache' Pattern

Terrific blog! I genuinely really like how it really is uncomplicated on my eyes along with the info are well written. I'm questioning how I might be notified whenever a brand new post has been created. I've subscribed to your rss feed which should do the trick! Have a nice day!

Friday, July 1, 2011 6:29 AM by Kacie Teabo

# re: The 'Reluctant Cache' Pattern

Caching greatly increases the speed at which your computer pulls a bit and bytes from memory. It turns out that caching is an important computer-science process that appears on every computer in variety of forms. There are memory caches, hardware and software disk caches, pages caches and more. Virtual memory is even a form of caching. In this article, we will explore caching so you can understand why it is so important.

Tuesday, February 7, 2012 12:27 AM by Jackey Worden

# re: The 'Reluctant Cache' Pattern

I always appreciate topics like this being discussed to us. Thanks for sharing.

Sunday, May 20, 2012 7:36 PM by man and van London

# re: The 'Reluctant Cache' Pattern

Went to this Blog just to read this article, great piece of writing right there. Very informative.

Sunday, May 20, 2012 7:40 PM by vulkao

# re: The 'Reluctant Cache' Pattern

50 % discounts on all airtickets flights and hotels as well as all football matches

email us at  makitelove@yahoo.com

no advance payment needed

Tuesday, May 29, 2012 11:38 PM by FoeproloWer

# re: The 'Reluctant Cache' Pattern

I’d must verify with you here. Which isn't one thing I often do! I take pleasure in reading a put up that can make individuals think. Also, thanks for permitting me to comment!

Wednesday, June 27, 2012 2:18 AM by computer repair service

# re: The 'Reluctant Cache' Pattern

Wow… Fantastic Info. I am rather glad to you for supplying this helpful details.

Tuesday, July 24, 2012 9:32 AM by carpet cleaning

Leave a Comment

(required) 
(required) 
(optional)
(required)