Performance and Memory: StreamWriter/StreamReader/FileStream...
Alrighty, this is going to be a short. First, I'd like to point out the default memory requirements for each of these constructs.
- StreamWriter/StreamReader - character array /* 2K bytes */, byte buffer to hold chars /* 1K or 2K bytes */, FileStream if in use /* 4K */
- FileStream - input-output buffer /* 4K bytes */
Those are the defaults. Now, that puts the StreamWriter and StreamReader taking twice the memory footprint in some cases over the basic FileStream. You really only need the StreamWriter and StreamReader if you are doing textual processing, so if you aren't then you might want to leave them out. All of these guys have parameters to customize the size of buffers.
- StreamWriter/StreamReader - min size is 128 characters or /* 256 bytes + 128 or 256 bytes for byte array */, 4K file stream if you don't use the overloads that take a Stream by default so you can pass in a better FileStream.
- FileStream - Apparently there is no minimum size.
That leads me to some tips.
- StreamWriter/StreamReader - Tune your buffer sizes and see if you can't eek extra performance out based on your data. There are so many optimizations buried in this code with regards to specific scenarios (namely scenarios where a user buffer is used for decoding rather than the internal buffer), that each scenario needs to be examined. Don't play with the FileStream unless you are trying to extend the user buffer optimization to the file IO layer as well. Doing this can be potentially dangerous.
- FileStream - Use this in place of the stream encoding classes if you can. Users often make use of the wrong stream type when working with things. For instance, users will load binary readers and writers when copying data from network streams down to files. This extra level of indirection is unnecessary.
- FileStream - Make use of the user buffer optimization. If the user buffer is the same size or larger than the buffer allocated internally, then the user buffer is used in place of the internal buffer. This removes a memory copy. This is most readily apparent when using the Read/Write methods that take a byte array. The internal buffer is delay allocated, so I think under some circumstances you can prevent it from ever being allocated even though you set an internal buffer size when creating the FileStream.
Honestly, the file IO stuff is one area that I've neglected because the performance is always suitable. Some new methods that I'm working on require much higher performance bars so hopefully I'll get a chance to look at these classes in more detail and provide some source code demonstrating and even testing various performance characteristics. At the end of the day, the MS guys have tuned their classes to work best in the general circumstances, so only in edge cases should you consider trying to optimize for better performance. The question of the day would be, “What are the edge cases?”, and hopefully I can start to answer that as I find them.