Here's links to a couple of interesting articles which I've seen today:
This is a common question and Jason provides a nice, understandable, definitive answer as to why you cannot Unload Assemblies from an appDomain.
This is interesting although I still haven't found a definitive guide as to what Assembly attributes can cause an IsolatedStorage path to change... I'd be interested to know though!
Update: I found this description about Assembly Identity. It is the Assembly Identity which is used to help derive the IsolatedStorage path. http://blogs.msdn.com/suzcook/archive/2003/07/21/57232.aspx. It appears that, you wouldn't want to use IsolatedStorage if the location of your application doesn't remain constant (i.e.: each version has it's own sub-folder).