Justin Rogers

There have been a number of complaints over the past year about the Terrarium documentation.  Some people claim inaccuracies, others point out spelling errors, and others can't get over the grammatically incorrect text that is often found.  I just have to say, I'm a developer, and certainly not a UE guy.  I never really meant for the Advanced Developer's Documentation to be a literary masterpiece.

Now, being that some things are changing in the world of Terrarium, I've gone through and updated the documentation.  In doing so I've taken the time to improve the grammar and spelling, as well as double checking all of the text for inaccuracies.  If you have any specific gripes on items you feel are inaccurate, please let me know.  I know at least a couple of the readers here play Terrarium or did in the past, so speak up and let me know.  If you don't, then the next version of the documentation, while grammatically accurate, may still contain perceived inaccuracies (differences of opinion primarily, since I am pretty certain the code and documentation is accurate as applied to the engine, but may be confusing in some way).

Back to the grindstone for me with a special Post-It(tm) to thank the Word team for such a great grammar checking tool.

I recently blogged about a trend in the game market where women form one of the largest segments.  To prevent reiterating the entire posting you can view it here http://weblogs.asp.net/justin_rogers/archive/2004/02/18/75377.aspx.  Since that posting, I've gone ahead and tried to find the various books that I was thinking of when making the post.  Reason being that I wanted to lend some credence that game design for the casual versus core market is actually a good thing.

Note that I'm referencing the core market in bold type.  The reasoning behind this is to define the industry standard core game play group.  One such book, http://www.amazon.com/exec/obidos/tg/detail/-/1556229739/, by Roger Pedersen tends to think this group is the male segment ages 13-25.  At least that is what I got out of the book when doing a quick perusal.  Another book, http://www.amazon.com/exec/obidos/tg/detail/-/1556229518, by Erik Bethke, defines the core group as the male population from 18-45.  I have to point this out, because normally when you refer to a core group in any other situation you pick the largest market segment, not one of the smallest market segments.  Take for example the core consumer of the .NET Framework.  You might say, C# programmers, and probably be very wrong, since there are millions of VB/VB.NET developers consuming the .NET Framework compared to the much smaller C# segment.  VB/VB.NET books also result in much larger sales than C# books in general and simply putting a book in the more widely used VB can double or better your sales.

So what books are actually focusing on the larger and yet untapped gaming market segments?  Well, Game Programming for Teens, http://www.amazon.com/exec/obidos/tg/detail/-/1592000681, by Maneesh Sethi seems to be leading the way for designing games targeted to teenagers.  Teenagers account for a good deal of retail sales in the US and current retail analysis shows that teenage male/female buyers have a good deal of disposable income from summer time jobs or part-time jobs during school.  While they spend most of the money on cell-phones and trendy clothes, some of that disposable income is making it's way to buying games and purchasing other forms of entertainment (music and DVDs are big on the list).  The teeenage market is definitely a good place to focus your game design, and certainly not a waste of time as some game publishers/development companies might think.  A recent advertisement in Game Developer Magazine for a game developer/designer was asking for individuals that could create games targetted to the female age groups from 8-18.  These guys found a niche and are making money off of a large untapped market.

The next book, Gender Inclusive Game Design: Expanding the Market, http://www.amazon.com/exec/obidos/tg/detail/-/1584502398, by Sheri Graner Ray, focuses on game design that targets both the male and female audiences.  By gender inclusive, the book generally focuses on creating gameplay elements that are either gender neutral or creating pairs of gameplay elements that will apply to the male and female market independently.  The book notes hundreds of small things that a game design can include to help bridge the gap between male oriented game design (or focusing on the core market) and the much more casual female gaming audience.  I personally think this book is a must read for any small production game company looking to grab a large market share by focusing on disparate market groups, or for any larger game companies that are hoping to get more than just the core market into their game.

Hopefully these books help to point out that existing game design truly isn't focusing on the proper markets as demonstrated by the first two books.  A step is being taken in the right direction by some forward thinking individuals.  Hopefully these trends continue and we start to see more games that are less focused towards a specific group and more geared towards general consumption by the entire gaming audience as a whole.

So the method in question is Queue.Clone().  Ideally, Queue.Clone() would return an exact copy of the Queue being cloned.  IMO that should mean array density and size, current head/tail pointers, and current versions.  However, the Clone method does a couple of strange things.  Look below at what the code does actually do.  It creates a new queue of the current _size, not the currently allocated internal array length.  Uhm, kay.  Then it sets the private _size field, uhm kay.  Then it does an array copy, which is where things get really messed up.  First, it copies from array element 0 to array element size.  This means that elements of the queue at a location beyond _size (perhaps nearer _array.Length), won't even get copied.  So elements appear to be lost.  That isn't right.

Note also that _head and _tail are never copied.  So if I Enqueue a bunch of elements, then Dequeue one, then Enqueue one.  The _head element gets reset to 0 in the cloned queue.  So does the _tail element.  Hell, the only reason a new call to Enqueue doesn't overwrite the entire collection is because of a call to SetCapacity when you Enqueue the next element.  Secretly, the _tail element gets updated to the new _size of the array.  SetCapacity is called because the new queue length was perfectly sized to the number of existing elements.  Algorithmically having _tail set to 0 isn't a bad thing though, because logically if you have a queue and you resize it down to the number of elements in the queue, the tail wraps back around and should be pointing at the first element in the list.  I can't fault the programmer her for finding a cool shortcut, I guess, just in their array copying implementation.  (note a switch to CopyTo would appear to do the *right* thing).

  /// <include file='doc\Queue.uex' path='docs/doc[@for="Queue.Clone"]/*' />
  public virtual Object Clone() {
   Queue q = new Queue(_size);
   q._size = _size;
   Array.Copy(_array, 0, q._array, 0, _size);
   q._version = _version;
   return q;
  }

So what might fix this problem?  Well, first, using the CopyTo method of the Queue would have done the right thing.  Even better would have been to simply use the ToArray method.  I've included what I think would be the appropriate code at the bottom, along with a sample program that demonstrates *losing* elements because of the way the Clone is done.  Note a Clone of an unused queue would work as expected.

If I recall there was some validation behind why the Clone() method didn't really work correctly, but I can't for the life of me remember.  Hopefully someone chimes in and gives us some validation for this problem.  Special thanks to Christoph Narh for pointing out some of these issues on the msnews.microsoft.com newsgroups.

Here is the recoded Clone method using ToArray:

/// <include file='doc\Queue.uex' path='docs/doc[@for="Queue.Clone"]/*' />
public virtual Object Clone() {
    Queue q = new Queue(); // We don't care about initial settings
    q._array = this.ToArray();
    q._size = q._array.Length;
    q._version = _version;
    return q;
}

Here is the sample program that demonstrates losing elements:

using System;
using System.Collections;

public class QueueCloneHack {
    private static void Main(string[] args) {
        Queue queue = new Queue(50); // Initial Capacity
       
        /*
            1   2   3   4   5   6
            h                       t
        */
        queue.Enqueue(1);
        queue.Enqueue(2);
        queue.Enqueue(3);
        queue.Enqueue(4);
        queue.Enqueue(5);
        queue.Enqueue(6);
       
        /*
            1   2   3   4   5   6
                h                   t
        */
        queue.Dequeue();
       
        /*
            1   2   3   4   5   6   7
                h                       t
        */
        queue.Enqueue(7);
       
        Queue queue2 = (Queue) queue.Clone();
        Console.WriteLine(queue2.Count);
        while(queue2.Count > 0) {
            Console.WriteLine(queue2.Dequeue());
        }
    }
}

While this game used to have a very good following surrounding it, with lots of chat about various cool goings-ons, etc..., it has lost all of its mystique for me.  I recently sat through a 2 hour nightmare while *observing* (clench your first against your chest, since that is the observer hand signal).  While I was expecting a rambunctious younger crowd to show up (18-25), it turned out to lure older lonely individuals that don't have a better way to spend their time.  Nothing of note really happened, nothing interesting at least, since the game is primarily social.  The groups were very quiet about their gatherings and didn't really exude any sort of presence as I would expect from a purely social game.

So what is the problem here?  Well, Vampires are granted supernatural powers, period.  Most of these powers are clearly outside of the understanding of humans in general.  Flying, extreme charisma, powers of persuasion beyond reason, and extremely acute senses.  While these can be theoretically applied to a human role-playing a vampire, the human has to actually step up and play the role.  However, how do you play the role of something outside of your own abilities.  How do you appear as charismatic and interesting as a Vampire when the whole mystique of Vampires is that they have these powers and we don't?  Well, everyone else around you has to pretend as well. I'm sorry, I can't really get into a game where the individuals are so deluded as to convince themselves that another individual has features about them that they truely are incapable of displaying and/or role-playing.

Needless to say, I won't be doing the vampires thing again.  I missed out on a truly interesting trip to a new bar, Cow Girls, down in Seattle to go to this meeting of the vampires.  Another excellent option for the night would have been to head down to the firwood in Fife and watch an excellent band, Kameleon, previously known as KamelToe or CamelToe (not sure how they previously spelled it).  The lead guitarist is most excellent, and their lead female vocalist is also extremely good on the guitar.  If you get the chance check them out at www.kameleonrocks.com.

I don't know why I figured the StringCollection might actually be more performant than the ArrayList.  I guess I assumed this was the one case where they would emit a strongly typed collection that was based on ArrayList, but was strongly typed on the back-end.  However, looking at the rotor source, all it does is inherit from ArrayList and really doesn't provide any added value (except that you don't have to do the string cast when pulling elements out).

Doing a basic timed iteration over a StringCollection using a large number of elements (in the millions), it was almost twice as fast to just use the ArrayList and forget about StringCollection altogether.  Unless you want a collection that is easy to use from the standpoint of not having to do the string casts, I would highly recommend against the StringCollection.  It just doesn't buy you anything.  I would highly recommend using the ArrayList source code from the rotor code-base to write your own strongly typed collection though.  You have to remember how much time went into setting the initialCapacity and determining the growth metrics for the ArrayList thanks to the performance team at MS.  They made the ArrayList one of the best collections available.

I've heard of some decent tools for developing strongly typed collections as well.  CodeSmith comes to mind, so you might want to check those out.  All of these things become null and void (for the most part with some exceptions) when generics hit the shelves since generics make it easy to write strongly typed collections and most of the collections classes/interfaces have already undergone a generics overhaul and are quite performant.  If you'd like to run the numbers for yourself of ArrayList versus StringCollection you can just compile the below test class.  Enjoy.

using System;
using System.Collections;
using System.Collections.Specialized;

public class StringCollectionTest {
    private static void Main(string[] args) {
        DateTime start, end;
        int iterations = int.Parse(args[0]);

        /*
            Get all the strings into a strongly typed
            array so we don't incur the hit of placing
            them into the string table during the actual
            usage of the collections.
        */
        string[] strings = new string[iterations];
        for(int i = 0; i < strings.Length; i++) {
            strings[i] = i.ToString();
        }

        /*
            Create a string collection and add all of
            the strings.  Note this is just adding the
            strings to an ArrayList underneath.
           
            Once done, do a CopyTo our newly allocated
            strongly typed string[].
        */
        StringCollection coll = new StringCollection();
        start = DateTime.Now;
        for(int i = 0; i < strings.Length; i++) {
            coll.Add(strings[i]);
        }
        string[] collStrings = new string[strings.Length];
        coll.CopyTo(collStrings, 0);
        end = DateTime.Now;
        Console.WriteLine("String Collection: {0}", end - start);

        /*
            Create an ArrayList.  Add all of the strings
            to the collection.
           
            Once done, do a ToArray, which internally creates
            a newly allocated array of the appropriate length
            and type and returns it after copying all elements
            into place.
        */
        ArrayList coll2 = new ArrayList();
        start = DateTime.Now;
        for(int i = 0; i < strings.Length; i++) {
            coll2.Add(strings[i]);
        }
        string[] coll2Strings = (string[]) coll2.ToArray(typeof(string));
        end = DateTime.Now;
        Console.WriteLine("ArrayList        : {0}", end - start);
    }
}

Alrighty, so the algorithm of the day is going to be NNTP, since I happen to have some NNTP classes on my disk that demonstrate the issue at hand.  Note there are several ways to read from sockets, which I'll quickly enumerate, along with their pitfalls.

1.  Receive.  This is a blocking method that doesn't return.  If you aren't running things on a separate thread then this method will hang until some data comes in.
2.  Receive using AvailableBytes.  First you check to see if data exists, then you read it in.  This is a bit better, since you can control the read operation so it doesn't block.
3.  BeginReceive/EndReceive.  Okay, this one requires callbacks to process the incoming data as it becomes available.  Callbacks can take a linear read/response algorithm and turn it into a bunch of smaller functions that control the asynchronous processes.  At times this can obfuscate the underlying algorithm.

So, ideally, we'd like to establish an algorithm, make it look good and still use some form of asynchronous programming so we can easily cancel out of operations that appear to be taking too long.  To do this, I'll take a single NNTP command LIST OVERVIEW.FMT, and demonstrate a method of using Receive in order to process the commands incoming data.  What is important here is the while loop and the initial if statement used to control the processing of data.  In a Network environment you can't use the 0 bytes available on the stream to determine if it is complete or not, and you can't use the stream closed condition either.  You have to actually process the incoming data and determine based on the underlying protocol that the message has completed.  That makes all of the looping constructs and data checks very important.  Let me just toss the code in for the command:

public class NewsReaderCommand_ListOverviewFormat : NewsReaderCommand {
    protected string[] headers = null;
   
    public override void RunCommand(Socket newsSocket) {
        newsSocket.Send(System.Text.Encoding.ASCII.GetBytes("LIST OVERVIEW.FMT\r\n"));
        bool complete = false;
        string groupText = "";
        ArrayList groups = new ArrayList();
       
        while(!complete) {
            if ( newsSocket.Available > 0 ) {
                byte[] b = new byte[newsSocket.Available];
                newsSocket.Receive(b);
                groupText += System.Text.Encoding.ASCII.GetString(b);
            } else {
                System.Threading.Thread.Sleep(50);
            }
           
            if ( groupText.Length > 0 ) {
                while(groupText.IndexOf("\r\n") > -1 ) {
                    string response = groupText.Substring(0, groupText.IndexOf("\r\n"));
                    groupText = groupText.Substring(groupText.IndexOf("\r\n") + 2);

                    if ( message == null ) {
                        message = new NewsResponse(response);
                    } else {
                        if ( response == "." ) {
                            success = true;
                            complete = true;
                        } else {
                            try {
                                groups.Add(response);
                            } catch {
                                Console.WriteLine(response);
                                throw;
                            }
                        }
                    }
                }
            }
        }
       
        headers = (string[]) groups.ToArray(typeof(string));
    }
   
    public string[] Headers {
        get {
            return headers;
        }
    }
}

Okay, so we poll the news socket for available data and if it is available we actually read it in.  Note this could be 1 byte or a thousand bytes so we just allocate a buffer based on what is available.  We then translate the bytes into ASCII (1 byte per character, so hopefully no worries of truncated characters), and append it to our string buffer for later processing.  If no data is available we simply sleep the current thread giving that time back for othe threads in our application to run.

Next we process any lines that have been made available.  Each line in the response should either be an NNTP response message (first line), part of the response, or the end of message terminator the period.  Processing the text of the message isn't hard and we keep on going until we find the end of message terminator making sure to set some flags to break out of our data reading loop.

If you look at the code the algorithm for this particular command is readily apparent.  There isn't an overshadowing by the data reading code or the message processing code that obfuscates the algorithm and makes it hard to read or understand.  At this point we have a pseudo-asynchronous way to process this particular NNTP message.  The algorithm processes the returned data in real-time, meaning we don't read the entire message into a buffer and then start processing it later (the class handles both the IO and the processing as a pair rather than an IO first, processing later deal).  Porting this into asynchronous sockets code won't look pretty I'm thinking.  My original implementation was based around asynchronous IO and because of the ugliness (I consider it ugliness) I resorted to helper classes for reading an entire message before having it processed.  That way I wrote the IO code as a black box that just gave me some bytes back.  This resulted in some large byte arrays getting passed back (listing groups on msnews.microsoft.com for example), and so memory was becoming an issue.

I'm not nearly as stupid this time though, and I'm not just hacking something together like I was when I originally wrote the code.  I have a purpose.  To clearly write the above NNTP command algorithm using asynchronous sockets without obfuscating the algorithm itself by making it seem less apparent amongst a bunch of asynchronous IO code.  I'll also be adding in the timeout code which for controlling command processing time.  You'll notice it doesn't exist in the code above.  This is because the LIST OVERVIEW.FMT command doesn't really return a good deal of information.  I didn't feel the need for a timeout.  However, if I never receieve a response, then a timeout would be appropriate.  It just wasn't high on the list of commands that needed the extra control code.

Well, keep your eyes peeled for Part 2 and a fully asynchronous version of the LIST OVERVIEW.FMT command ;-)

Well, I recently found the need to use the command line in order to get some information (file names) that needed to be processed by my application.  My purpose was to find all non read-only files (files that had changed) that matched a set of criterion.  You see I'm working on a source controlled project outside of source control, so it becomes hard to remember which files I've changed, especially in a VS environment where multiple files that I'm not actually using get changed (though they do warn me before changing the read-only flag).

Normally, I would just use Perl, since it has a qc// syntax which allows you to run commands and pipe the output into a Perl array.  I love this syntax because it is so fast and easy.  .NET simply doesn't have these in-grained features unless of course you are using a .NET language that contains them (Perl.NET anyone?).  So what goes into a command like qc// anyway?  Well, a bunch of IO redirection and that is pretty much it, so that is what I wanted to implement in .NET.  I came up with two classes, ProcessSnoop and CommandLineSnoop.  ProcessSnoop lets you work with any process you want, while CommandLineSnoop has command line only features like automagically building your command line string for you.  Let's check these out:

public class ProcessSnoop {
    protected string stdOut = null;
    protected string stdErr = null;
    protected ProcessStartInfo psi = null;
    protected Process activeProcess = null;
   
    public ProcessSnoop(ProcessStartInfo psi) {
        this.psi = psi;
    }
   
    public int Run() {
        Thread thread_ReadStandardError = new Thread(new ThreadStart(Thread_ReadStandardError));
        Thread thread_ReadStandardOut = new Thread(new ThreadStart(Thread_ReadStandardOut));
   
        Console.WriteLine(psi.FileName);
        Console.WriteLine(psi.Arguments);

        activeProcess = Process.Start(psi);
        if ( psi.RedirectStandardError ) {
            thread_ReadStandardError.Start();
        }
        if ( psi.RedirectStandardOutput ) {
            thread_ReadStandardOut.Start();
        }
        activeProcess.WaitForExit();
       
        thread_ReadStandardError.Join();
        thread_ReadStandardOut.Join();
       
        return activeProcess.ExitCode;
    }
   
    public void Thread_ReadStandardError() {
        if ( activeProcess != null ) {
            stdErr = activeProcess.StandardError.ReadToEnd();
        }
    }
   
    public void Thread_ReadStandardOut() {
        if ( activeProcess != null ) {
            stdOut = activeProcess.StandardOutput.ReadToEnd();
        }
    }
   
    public string StandardOut {
        get {
            return this.stdOut;
        }
    }
   
    public string StandardError {
        get {
            return this.stdErr;
        }
    }
}

You start with the basics.  You need to redirect the IO and you need to read it out.  ProcessSnoop assumes you've set up a ProcessStartInfo with all of the vitals, and it will redirect either StdOut and StdErr depending on whether you've marked them as redirect in the ProcessStartInfo.  We have to start threads for each read since there is a blocking type race condition mentioned in the documentation.  I think my implementation meets the specs and overcomes this condition.  I simply spawn and wait for a thread for each IO queue.  Works out fine as long as I wait for the threads to finish using a Join at the end (you could use another sync routine, but I find Join's to work fine for this process).

The IO queues are simply read into strings, which can later be processed by your application.  The next step is the command line implementation that takes some of the work out of the process of setting up a ProcessStartInfo.  I was surprised at how easy this step was, since it really enables a lot of power with access to any command line program, with relatively few lines of code.  I could probably set up a cool event driven system to allow for StdIn overrides as well, but I don't need that feature set at this time, and if I did, I'd probably just use a form of pipe redirection since there are few times where you actually need to respond to prompts and actually need an event driven input mechanism.  So here is the derived CommandLineSnoop class.

public class CommandLineSnoop : ProcessSnoop {
    private static string program = "\"%COMSPEC%\"";
    private static string args = "/c [command]";
   
    public CommandLineSnoop(string command) :
        base(
            new ProcessStartInfo(
                Environment.ExpandEnvironmentVariables(program),
                args.Replace("[command]", command)
            )
        )
    {
        if ( this.psi != null ) {
            this.psi.CreateNoWindow = true;
            this.psi.UseShellExecute = false;
            this.psi.RedirectStandardOutput = true;
            this.psi.RedirectStandardError = true;
        }
    }
}

I generally compile this into a library, however, I realized recently that I actually use unit testing in nearly all of my applications.  How do I do this you might ask?  Well, I simply include a LibTest class in all of my library code that has a Main method that matches the requirements for an executable.  That way if I compile the code as an executable then we can test the basic operation of the library to make sure it works.  My implementation focuses on doing a file listing of all non read-only files (excluding directories), recursively from the current directory and using the bare format so I can easily process the results.  You can pass a new command line in via the args, but there is a basic command line in case you don't specify one.  Then I process the output and print out very specific files that don't match additional search criterion.  It isn't a very powerful application or use of the CommandLineSnoop class, but it does the job of testing the functionality.

public class LibTest {
    private static void Main(string[] args) {
        string commandLine = "dir /a-r-d /s /b";
        if ( args.Length > 0 ) {
            commandLine = string.Join(" ", args);
        }
       
        CommandLineSnoop cls = new CommandLineSnoop(commandLine);
        cls.Run();
       
        ProcessReturn(cls.StandardOut);
        // Console.WriteLine(cls.StandardOut);
        // Console.WriteLine(cls.StandardError);
    }
   
    private static void ProcessReturn(string output) {
        using(StringReader sr = new StringReader(output)) {
            string line = null;
            while((line = sr.ReadLine()) != null) {
           
                if ( ValidExtension(line) && ValidPath(line) ) {
                    Console.WriteLine(line);
                }
            }
        }
    }
   
    private static string[] extensionExclusionList = new string[] { ".exe", ".pdb", ".dll", ".user", ".InstallLog", ".ncb", ".suo", ".webinfo" };
    private static bool ValidExtension(string line) {
        for(int i = 0; i < extensionExclusionList.Length; i++) {
            if ( line.EndsWith(extensionExclusionList[i]) ) {
                return false;
            }
        }
       
        return true;
    }

    private static string[] pathExclusionList = new string[] { "\\bin\\Debug\\", "\\bin\\Release\\", "\\AsmCheck\\Debug\\", "\\AsmCheck\\Release\\", "\\obj\\" };
    private static bool ValidPath(string line) {
        for(int i = 0; i < pathExclusionList.Length; i++) {
            if ( line.IndexOf(pathExclusionList[i]) > -1 ) {
                return false;
            }
        }
       
        return true;
    }
}

Well, since I feel bad for being extremely sick this week, I figured I'd share some code that everyone might find useful.  Hopefully I'm over this stupid flu and I'll be able to get back to my normal blogging and coding patterns.

I've included the code below.  Notice that the IPAddress can be constructed using a long for the address.  Also note there is an *internal* version that only uses an int.  An IPv4 address only contains 32 bits, so it makes sense to use the int.  Hell, they even make sure more than 32 bits aren't specified in the long address that they consume.

So why did this happen?  You might say, “Hey, IPv6“, but then you wouldn't be knowing much about IPv6 since it is 128 bits.  I haven't seen any Int128's laying around, else it would be nice if a long mapped to a 128 bit integer in a 64 bit platform (maybe it does and maybe that is the reason.  I don't play with 64 bit so I just don't know).

I've wondered this question for quite some time.  Thankfully the methods are being deprecated in favor of byte array versions that can take either 32 bits or 128 bits depending on the address you are trying to specify.  This allows you to use either IPv4 or IPv6 which is kinda nice.  Well, I'll chalk this up to my rotor FUBAR of the day, unless one of the System.Net guys wants to jump in and provide an explanation.  The code comments sure don't explain things.  They just mention the 32 bit version is for Winsock compliance.  Again, I don't know where this long is coming from, and I've never worked with a long anything when working with sockets before.  Enlighten me someone, please!

        /// <include file='doc\IPAddress.uex' path='docs/doc[@for="IPAddress.IPAddress"]/*' />
        /// <devdoc>
        ///    <para>
        ///       Initializes a new instance of the <see cref='System.Net.IPAddress'/>
        ///       class with the specified
        ///       address.
        ///    </para>
        /// </devdoc>
        public IPAddress(long newAddress) {
            if (newAddress<0 || newAddress>0x00000000FFFFFFFF) {
                throw new ArgumentOutOfRangeException("newAddress");
            }
            m_Address = newAddress;
        }

        //
        // we need this internally since we need to interface with winsock
        // and winsock only understands Int32
        //
        internal IPAddress(int newAddress) {
            m_Address = (long)newAddress & 0x00000000FFFFFFFF;
        }

Well, I'm back to hammering away at the Terrarium code base.  I have to say, it is probably a better gaming code-base than I would care to admit, and it appears we actually did a good job in most places at making it extensible and customizable.  That isn't really the big push right now though.  The big push is to get the source code into the hands of the community (that would be you and I), so that there might possibly be some derivative works.

Right now I'm working through most of the small issues that are left in the Terrarium.  Actual code issues that need to be cleaned and not gameplay functionality.  So the source release won't have much over on the previous public release.  So what kinds of things are happening?

Well, the largest issue we've tackled is the DeserializeAnimal with a null array.  While this wasn't really an engine issue, like most people thought, it was causing some grief in the community.  I guess you could say the gameplay around this member has been improved to avoid developer confusion.  I'll talk more about this little guy in a separate blog.  I have to get some small related code-bits approved before I can give you any real meat.

There was another issue with a floating point truncation issue when converting to int that was plaguing some players (but not others because it isn't consistent).  Some users were finding they had to put one extra point into a stat to get the benefit since numbers were being represented in floating point as (3.999999...), and then cast down to 3.  Adding an extra point (and getting a warning from the introduction system) would put you over the top with something like 4.2499999, and you'd actually get that extra umph you were looking for.  This was an unfortunate issue, and only some users ran into the problem.  I have 3 machines right now that I can't repro the issue on, and another that I actually can, so it is very frustrating.

Maybe not such a big issue, but a real big push, has been the ability to run under lessened OS privs when running the Terrarium.  We want people to be able to run without having Admin/Power User privs and instead just run as a normal user on the machine.  This was probably one of the most daunting issues in the Terrarium codebase to fix because games generally own their directories and we don't get to.  All of our IO has to happen under the AppData section of the disk.  We had lots of code that broke this rule, some in the graphics engine, some as part of the updater features, some logging functionality, etc...  This should all be sorted out now, and after installation, you should be able to launch and effectively run the Terrarium under normal privs.  Enjoy!

Jeez, all the stuff I get to do sounds so boring already.  The man with the real plans is Mitch Walker.  Check out his blog, since he is the guy responsible for the cool new graphics displayed at the PDC, and he'll be responsible for adding/approving any cool tweaks we do to the engine before the source release hits the web.

I still have a few tricks up my sleeve though with cool additions to the Terrarium code base after it is released.  Hopefully, we can get a good amount of community excitement and build an external support structure for new code additions, new graphics, and a public array of game servers for users to connect to.  As I get some free time, I'll start talking a bit more about what I have planned and how it might suck away your free time in the months after the codebase is released.

If you actually want to look at the puzzle, then why not head on over to the original blog entry.  If you are into algorithms and figuring out loop issues, etc...  then you'll enjoy the puzzle.  It hinges on the concept of how many bytes it takes to actually encode a full sized integer of varying bit lengths 32, 64, 128, etc...

I had the answer typed up in my text editor with a full roll-out of the data representations as they would be laid out in memory and as they would be converted into 7 bit encoded forms.  I'll just copy this in for brevity, since I think this cached response covers all of the issues I had with the original algorithm.  Note I appear as if I'm speaking to someone, because I am.  I was responding as to why their method would actually fail.  Note that the method in the original blog entry does begin to fail at 64 bits, so I was correct in my original assumptions.

// 128 bit integer
// 0xFFFFFFFFFFFFFFFF
// 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111
// 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111

Okay, so we have the above 128 bit integer.  16 full bytes.  Now encode said
integer.  Now break it up into 7 bit sections

// 01111111 01111111 01111111 01111111 01111111 01111111 01111111 01111111
// 01111111 01111111 01111111 01111111 01111111 01111111 01111111 01111111
// 01111111 01111111 00000011

That is the expanded 7 bit version of the same number.  What does this look
like written out?

// 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111
// 11111111 11111111 11111111 11111111 11111111 11111111 11111111 11111111
// 11111111 11111111 00000011

Now read it back in using your method.  Each byte is incrementing _counter
by 1.  _counter is incremented after the byte is read, but before the
combining logic.  The exception condition is checked BEFORE the increment, so
the counter starts at 0.

What does this mean?

Well, read the first 8 bytes, _counter = 7 before and 8 after which is not greater than Marshal.SizeOf(Int128)
Well, read the next 8 bytes, _counter = 15 before and 8 after which is not greater than Marshal.SizeOf(Int128)

Now, we have three more bytes left to read.  As soon as we try to read the 18th byte, your
condition will throw an exception.  This doesn't happen with low bit integers, because your
method allows us to read 1 more byte than the natural byte precision of the underlying type.

For 32 bit numbers your method reads up to 5 bytes.  For 64 up to 9 bytes and for 128 up to 17 bytes.
I think that 64 bit integers will throw as well.  Since I think 64 bit integers will throw as well,
just change your arrays to use long types instead to simulate a 64 bit environment.

Woop dee doo, you know there is a bug.  Easy way to fix it?  Yep, get rid of the _counter altogether because it is worthless.  Instead compare the shift operator against the expected size of the type in bits.  If the shift operator extends past the types resolution then we know there is a problem.  Below is my code.  I'm checking _shift > (Marshal.SizeOf(_count) << 3) each time through the loop (I'm precomputing this into _bitSize).  In my original algorithm I was checking the shift register after I read the next byte.  This was just an algorithmic failure on my part, since I should be checking the shift register when first entering the loop.  This is the best and cleanest version of the method I've come up with yet, so enjoy..

    public int SafeRead7BitEncodedInt() {
        int _count = 0;
        int _bitSize = Marshal.SizeOf(_count) << 3;
        int _shift = 0;
        int _b = 0;

        do {
            if ( _shift > _bitSize ) {
                throw new Invalid7BitDataException();
            }

            _b = BaseStream.ReadByte();

            if ( _b == -1 ) {
                throw new Invalid7BitDataException("", new EndOfStreamException());
            }
           
            _count |= ((_b & 0x7F) << _shift);
            _shift += 7;
        } while ((_b & 0x80) != 0);

        return _count;
    }

Questions?  I am more than happy to answer questions, or have someone point out flaws in my algorithm.  That's how you learn.

Quick Tips?  Note, that I talked about this before, but if I encoded say 30 ints, I could read them back out of the stream as 30 longs quite easily.  The reason is that identical values no matter how large the bit size will always take up the same number of encoded bytes.  I really like this feature, again, as pointed out in another article, for encoding enumeration values and id fields that increment from the low numbers (say 0 or 1).  This also works well when serializing arrays and collections, since element counts are generally low.  So many ways to save space.