When Side Effects and Laziness Collide

While working on a side project recently, I came to rediscover some of the consequences of one of my earlier posts on “Side Effects and Functional Programming”.  It’s important that we realize that when we are creating our programs to beware of lazy evaluation and side effects. 

Adding a Wrinkle

Consider the following example.  We need to be able to get the stream data from a given web request for a given URL.  From there, we will take the stream of data and read each line and return it as a collection.  Typically if we’re going to do this through F#, we’d use a sequence expression to lazily read through each line and return it as an IEnumerable<string>.  But, let’s add an extra wrinkle to this.  What if we want to do this in a non-thread blocking  asynchronous scenario?  This adds an extra wrinkle to this in terms of managing the lifetime of our objects.  We might write the code such as the following:

open System.IO
open System.Net

let getStreamData (uri:string) =
  async { let request = WebRequest.Create uri 
          let! response = request.AsyncGetResponse()
          
          return seq { use stream = response.GetResponseStream()
                       use reader = new StreamReader(stream)
                       while not reader.EndOfStream do yield reader.ReadLine() } 
        }

What we’re doing is putting a sequence expression inside of an asynchronous expression.  We must bind the response outside of the sequence expression due to the fact that we are doing this in an asynchronous fashion.  Once inside a sequence expression, a bind has a different meaning.  But, there’s a problem with this code.  Can you spot it? 

The problem lies in the fact that we’re not explicitly disposing the response, which implements IDisposable.  My rule of thumb when flushing out my design is that if a class implements IDisposable and I ultimately control its lifetime, then I must wrap it in a using block.  If you read the documentation, you’ll realize that it’s not totally necessary when dealing with an HttpWebRequest to call Dispose, but it is when you’re dealing with an FtpWebRequest.  Because I don’t make any distinction in this code, calling Dispose is a must.  We might be tempted then to change the code to the following:

open System.IO
open System.Net

let getStreamData (uri:string) =
  async { let request = WebRequest.Create uri 
          use! response = request.AsyncGetResponse()
          
          return seq { use stream = response.GetResponseStream()
                       use reader = new StreamReader(stream)
                       while not reader.EndOfStream do yield reader.ReadLine() } 
        }

Running through a quick example using F# interactive, we’d get the following result:

> Async.RunSynchronously (getStreamData "http://www.bing.com");;
val it : seq<string> =
  Error: Cannot access a disposed object.
Object name: 'System.Net.HttpWebResponse'.

The problem now becomes that our response binding now will be disposed by the time that I iterate the results of the sequence expression.  Remember that our evaluation of the sequence expression is lazy, so by the time I’m ready to iterate, the response has already had its Dispose called.  So, where does that leave us? 

Well, there are several things that we could do here.  First, we could copy the response inside of our sequence expression and bind that to a using block.  This would ensure that by the time we’re finished, we would properly dispose the response.  That code might look like the following:

open System.IO
open System.Net

let getStreamData (uri:string) =
  async { let request = WebRequest.Create uri 
          let! response = request.AsyncGetResponse()
          
          return seq { use response' = response
                       use stream = response'.GetResponseStream()
                       use reader = new StreamReader(stream)
                       while not reader.EndOfStream do yield reader.ReadLine() } 
        }

This approach will work, unlike the previous example.  By copying in this reference, we can ensure that the response will be disposed by the time we’re finished with our sequence expression.  Although this works, it has a certain code smell to me.  This code smell revolves around copying these references and there is a certain danger in doing so in terms of managing the lifetime.  Instead, could we try another approach, perhaps more eager?

From Lazy to Eager

Another approach we could take with this is to do away with the idea of lazy I/O in this solution altogether.  Instead, how about something as simple as a list comprehension instead of a sequence expression?  The code might then look like the following:

open System.IO
open System.Net

let getStreamData (uri:string) =
  async { let request = WebRequest.Create uri 
          use! response = request.AsyncGetResponse()
          
          return [ use stream = response.GetResponseStream()
                   use reader = new StreamReader(stream)
                   while not reader.EndOfStream do yield reader.ReadLine() ] 
        }

Now what we have is that we have the use binding on our response and stream.  At the end of our function, we simply use a list comprehension instead of the sequence comprehension which in this case gives us the benefit of eager evaluation.

Conclusion

When dealing with asynchronous expressions mixed with sequence expressions can cause problems due to object lifetimes.  As I’ve had in my earlier post, it’s best to avoid such scenarios if possible and instead, focus on an eager approach.  The savings you get here is you have to worry less about the resource lifetimes which may or may not be in scope by the time you’re ready to execute the sequence expression.  The approach using the list comprehension should be the preferred solution.