[Terrarium] How do we pass creature assemblies around?
This is the primary feature of the .NET Terrarium. The ability to safely and easily pass around creature assemblies and run them on end user machines. There are several considerations for this process including security, transport, loading, and storage. I've already talked about the Terrarium Model and security over in the Plug-In Framework (Foreward): Defining the end goal, so I'll be talking about the later three now.
Transport
Transport can be very easy (WebRequest), or fairly difficult (Socket). Today, you can easily use web services or web requests to grab information off of a central server, and the Terrarium does this all the time. We have several services that go out and grab extinct creature assemblies off of the web server. This is probably the easiest way to get a creature assembly.
The P2P methods are a bit more difficult. We actually created a miniature web server HttpWebListener (thanks to Lance Olson) and use that for P2P interaction. So are we really using Socket? Yeah, but not the way you would think. Each client still issues basic WebRequests for data from each peer, but we host those little miniature HTTP servers in order to intercept the commands. This is actually pretty cool and I'm sure everyone will enjoy having the source code at their fingertips for this little gem pretty soon.
Since HTTP is a stateless protocol I can tell you now that we have to implement a message based system whenever we communicate with other peers. All of the messages originate on one side of the pipe (the client) and will change depending on the response from the listener. This is a fairly easy process, made quite complicated by requring multiple connections, one for each message, but overall it isn't that bad. Sample messages would be, do you have assembly A? Here is assembly A, did you get it okay? Here is a new creature, are you receiving it okay?
Storage
Now that we have the assemblies getting tossed around we have to store them somewhere. We can't store them in the same place as our executable. We can't store them in Program Files and still act under “Run as Normal User”. We can't really store them anywhere, except for the users local settings. Ah, we found a spot. Since we have to store assemblies off in some remote directory, how do we go about loading creature assemblies when they are requested?
That is where our assembly caching logic comes in. The basic process starts inside of the game logic. You see the game logic knows about the assembly caching logic, and so rather than calling an Assembly.Load on a creature's full name, we simply call into your assembly cache and pull out an assembly. This works great. However, there are still times when an Assembly.Load might get called outside of our code (deserialization) and we need to be aware of that so we can return the assembly when probing fails. For that we hook AssemblyResolve, verify that the assembly being requested isn't one of our own, and finally look things up in the assembly cache.
There is one hang-up here. We never make use of private paths, so the run-time doesn't control the probing, we do. We get a lot more storage potential out of this, including monitoring how many assemblies are currently loaded, how many are in the assembly cache, and finally the ability to run clean-up logic if we want to. We also don't automatically download the assemblies if they aren't already in the assembly cache. We could make a web request and search for the assembly on the server, however, we rely on other game logic to ensure the assembly is there.
To note some additional security we've layered on our cache. Each time the Terrarium starts, the cache directory is dynamically changed to some random directory name. No brute force, I know where your assemblies are hidden, attacks. This prevents users from storing things in assemblies and then taking advantage of them later through email, URLs on their website, or any number of other hacker routes.
Loading
This is the most complex aspect of assembly use that we have. Not only do we have many layers of code access security protection, but we grep the assembly IL for bad constructs, load the assembly in multiple ways to prevent context caching of assemblies, and a few other goodies. I've detailed the code access security items and IL grepping within Plug-In goals document, so you can see the real deal there. I'll talk a bit about loading assemblies for the first time and the process they go through.
Loading an assembly for the first time is a pretty big deal. We obviously run the IL grep over the assmebly before we do anything, but once that process happens, we have a number of other checks that have to be run. Namely, we have to get a bunch of assembly attributes off of the type, make sure the image isn't bad, etc... Simply enumerating the IL won't give us this full verification that everything is kosher, and we don't want to lock the file on disk. So we use the byte[] overloads of the Load method. The method supports loading of just the assembly, or what most people don't know, debug symbols as well. Took some time to get all of this code written correctly to allow all of that precious debug information to make it to it's final location in the assembly cache.
Once the assembly is verified, we check it against the Species service to make sure it is unique, add it to the list of creatures, and store it's assembly remotely. If this is a new creature for sure, we save the assembly into the assembly cache (copying symbols if necessary), and load the assembly again. This time for real, since this will be the assembly codebase that we use for creating creatures.
The entire process is fairly quick. Later as the application shuts down and restarts, we'll be responsible for loading creature assemblies over and over again. They'll always come out of the assembly cache from there on out. Some hang-ups do occur. If the original assembly (before copy) was located in the probing path, a call to Assembly.Load by the serialization code might load the wrong assembly. We prevented this by some ingenious serialization, but I'll talk about that later. You also have to make sure all of your code loads from the assembly cache as well, and doesn't call Load, else that probing issue will catch you sooner or later.