Published by

Comments

Monday, January 19, 2004 3:58 AM by Frans Bouma

# re: Towards a more functional TreeView

The real problem is under the hood. The .NET treeview is based on the asynchronous message model of win32. This means that when you do something synchronously in .NET, it is executed under the hood by sending win32 messages. This can result in serious issues.

.NET needs a synchronous, native gui library which is synchronous all the way till GDI+. The decision to base the gui glyphs on win32 stuff is perhaps based on teh fact that it would be 'faster' and 'easier to implement', but it also is the basis for most issues with .NET gui controls.
Monday, January 19, 2004 6:59 AM by anon

# re: Towards a more functional TreeView


Many great things about the windows forms treeview? It doesn't even support multiple selection natively. I think at least half of applications need that.


"Think about that for a minute, because the UI implications and different types of views start to become limitless with"

I don't know if this encourages consistent look and feel UIs, and how much this will hurt average people that are now used to the tree ctrl in windows explorer. In the corporate world, any change in the look and feel and UI costs millions in retraining. I would be willing to know how much corporate IS are spending to get people to upgrade to Windows XP, only for the required retraining. Interesting.



Monday, January 19, 2004 12:55 PM by TrackBack

# Basically, how basic can a Wizard in Windows Forms be?

Monday, January 19, 2004 1:20 PM by Raymond Chen

# re: Towards a more functional TreeView

If a single treenode could shared among multiple treeviews, what would it return for the Parent property or IsExpanded or any other property that can be different in the two treeviews.
Monday, January 19, 2004 8:01 PM by Justin Rogers

# re: Towards a more functional TreeView

I realize there are some architectural issues with my original comments, some things that need or should have been thought through more thoroughly. However, the base points shouldn't be lost. For instance:

While a basic TreeNode has properties that make it functional only when located in a single tree, there are things that could have been done to make multiple view synchronization easier. The current TreeNode can't even have an overriden Text property to allow treenodes to notify a container or factory that it's value has changed and that synchronization needs to occur.

I labeled the article, Towards a more functional TreeView for a reason. I think work can be done to make it better, and I've shown in at least one case exactly what can be done. I'll show the same in another case relatively soon, because the concept of a TreeNode factory that allows you to make *connected* copies that automatically synchronize with one another actually works. And it isn't hard to do at all.

And yes, the Parent property and IsExpanded properties between the various views might actually be different, but the Text property and any properties you add that should be synchronized could easily be.
Tuesday, January 20, 2004 4:49 AM by Darren Neimke

# re: Using the Security System for Defensive Coding

Thanks for the heads-up Justin :)
Tuesday, January 20, 2004 4:18 PM by DavidM

# re: Using Weblogs @ ASP .NET and begin friendly to the printer...

There is another problem - which I have flagged a number of times - if I view the weblogs homepage on my 1024 x 768 laptop - the navbar on the left covers half the posts - which is complete pants
Tuesday, January 20, 2004 4:39 PM by David Cumps

# re: Using Weblogs @ ASP .NET and begin friendly to the printer...

I have the same thing with the layout where a navbar is on the right, and it covers half of the text as well (i believe it was on a .Text site as well)
Wednesday, January 21, 2004 2:18 AM by Vance

# re: Using Weblogs @ ASP .NET and begin friendly to the printer...

That's one of the drawbacks to using div tags and css for positioning. Sure it makes it easier to completely change the look just by changing the stylesheet, but it tends to be somewhat quirky. Especially if you shrink the browser window and watch how things start overlapping.
Wednesday, January 21, 2004 8:18 AM by Rogelio Morrell

# re: Plug-in Framework (Part1): Marking Types for Consumption

Great article!
Wednesday, January 21, 2004 8:51 AM by Scott

# re: .Text offers excellent article hosting, but where is the RSS?

Sorry for the confusion. (agg view column)

The whole point of articles is they are not syndicated.

-Scott
Wednesday, January 21, 2004 9:14 AM by TrackBack

# First article in a series on plug-in frameworks in .NET

Wednesday, January 21, 2004 2:03 PM by chadb

# re: Plug-in Framework (Part1): Marking Types for Consumption

That's good stuff...
Friday, January 23, 2004 5:53 AM by Illumineo

# re: Revisiting the .NET Terrarium

Does it mean I'll be able to write my own agent system with the help of a Terrarium SDK, or something alike?

Friday, January 23, 2004 9:07 PM by Justin Rogers

# re: Revisiting the .NET Terrarium

Once the source code is released, you should be able to do some work and get your own agent system running.

You can also watch my series on Plug-Ins since I'll be discussing a generic framework for getting agents to run safely.
Saturday, January 24, 2004 3:22 PM by Chris McGuirk

# re: Game Engine Design Principles even apply to the most basic of games (or applications)...

If you want an example that is just a tad bit bigger than that, you should check out Axiom http://axiomengine.sf.net, which is quickly approaching a full blown .Net graphics/game engine.
Monday, January 26, 2004 9:34 PM by Matt Berther

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

Justin: I'm with you on this. I cant remember how many times I've seen sample code in the MSDN library where the samples don't even compile.

When I start looking for how to do something, I look towards Google rather than the MSDN, because I tend to find a lot better samples on the web.

As a last resort, Ill use an MSDN sample, but only as that. A sample. I wont cut and paste the code into my project as is.

It's really too bad, however, it has helped me to write better API documentation. :=)
Monday, January 26, 2004 9:34 PM by Edgar Sánchez

# They definetively shouldn't

Many programmers just copy and paste Microsoft samples, if the examples do something silly as using the sa SQL Server user, the programmer will feel authorized to do the same. So please go the "best practices" way.
Tuesday, January 27, 2004 3:00 AM by Scott

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

Yeah, why should Wrox books have a monopoly on errata. I think MSDN should include MORE bad code.

Now that the MVP's are allowed to write KB articles willy nilly it's just going to get worse.

Tuesday, January 27, 2004 4:28 AM by Pavel Lebedinsky

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

> 2. Not checking for file existence before creating the FileStreams.

This is actually the right thing to do. FileStream is documented to throw FileNotFoundException if the file does not exist, and this is exactly what a function like FileCompare should do anyway.

You could call File.Exists() but what would you do if it returns false? Throw FileNotFoundException? Then what about the case where the file exists but you do not have access to it? Or if the file is on a remote share and there's a network problem?

File.Exists is rarely useful in library code like this FileCompare function. It lumps all possible failures into a single bool value and it is prone to race conditions.
Tuesday, January 27, 2004 4:29 AM by Simon

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

The KB above is a prime example, the code they show is to improve the use of getting XML from an ADO stream. However they use a classic perf issue of concatenating strings.
This code can be improved greatly using an array, but they don't want to change the KB.

So don't do as they say or as they do!
Tuesday, January 27, 2004 9:05 AM by Paul Menefee

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

I've got to defend MS here, a bit. The point of the KB it demonstrate a particular code example. We harass them all the time about being verbose and for once they cut to the chase. And just tell us what we need to know. If I'm looking for a quick solution to a particular problem I don't need to see it wrapped in a bunch of production ready code. You know, and after giving this a little more thought perhaps that's their point. If they handed programmers half-baked code that forces the programmer to take ownership and wield the code ore attuned to their application.
Tuesday, January 27, 2004 9:20 AM by Derick Bailey

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

I agree with Paul Menefee.

When I'm looking up code examples in a KB article or on another website, I don't want to see it all wrapped up in best practices... I want to know how it works and why. I don't care that they missed catching a filestream exception for a null string - I know to take care of that.

If every peice of sample code was obfuscated in production ready best practices code, noone would ever learn anything because all they would have to do is cut & paste from a hundred examples and press "play" in their IDE.

I think this is more a question of properly training developers. If your coworkers or employees are dumb enough to cut & paste sample code and expect it to be production ready, maybe you owe it to them to spend some time educating them on best practices and the difference between samples / production code.
Wednesday, January 28, 2004 7:42 AM by Raymond Chen

# re: Towards a more functional TreeView

It's a question of pay-for-play. Do you want to complicate the base treeview with these features (that probably few people will actually use)? You can do a good chunk of this yourself - split your treenode into two objects, a treenode and a treenodedata. create multiple treenodes that share the same treenodedata.
Friday, January 30, 2004 5:55 PM by Justin Rogers

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

Everyone focused on my assertion that file checking should exist, but that wasn't the biggest problem with the code. The largest problem is that their code paths left an open file stream if the first file opened but the second file didn't. I was pointing out ALL of the issues with the sample to demonstrate how many they were, and by pointing out that they left open file streams I also pointed out an extreme example. I honestly think you should thoroughly read the entire message before making comments on only part of it.

As it stands I'm just guessing everyone that defended MSDN here agrees that resources like file streams don't really need to be closed explicitly. And the fact that if you called this method twice before the GC kicked in you'd get an access violation on the file in question because it is already open.
Sunday, February 01, 2004 8:11 AM by TrackBack

# [Article] Game Engine Design: A series of abstractions for better engines.

Tuesday, February 03, 2004 5:29 AM by TrackBack

# Multi-Threaded applications and Abort, careful not to kill your statics...

Tuesday, February 03, 2004 8:00 PM by Brian Grunkemeyer

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

Aborting your threads is one of the things that shouldn't be used by the vast majority of applications. As we started into our reliability discussions for SQL Server, we realized that a thread abort can happen basically between any two machine instructions in managed code. This means you can't guarantee you won't leak handles, and you can't guarantee that you won't corrupt static state. You could try writing finally blocks for the second case, but then we had the possibility of a host injecting a rude thread abort on your thread while running the finally, because the finally was taking too long.

(We invented SafeHandle to fix the resource leakage case, which required two additional pieces of infrastructure - critical finalizers and "must run" code, though we've replaced the second with something way better, mostly because must run didn't mean it must run at all. Ask about CER's when our beta is released, or better yet, don't.) I also had some thoughts on the static class constructor issue, but we opted not to fix them in this version due to resource constraints.

In short, if you get an asynchronous exception (ThreadAbortException, OutOfMemoryException, or StackOverflowException) anywhere within an appdomain, our recommendation in Whidbey is unload the entire appdomain. In the absense of some higher level application protocol where the caller and callee have perfect knowledge of one another and perhaps do some synchronization, you absolutely must pessimistically assume your appdomain is no longer usable.

You can read Chris Brumme's blog for some of his thoughts on Thread.Abort, but I have already given you the extraordinarily scary parts of the argument.

http://blogs.gotdotnet.com/cbrumme/PermaLink.aspx/dac5ba4a-f0c8-42bb-a5cf-097efb25d1a9

If you want to screw around with ResetAbort, well, good luck. I won't fix any bugs in the BCL w.r.t. ResetAbort any time soon. For example, if you get a Thread.Abort while removing an item from the middle of an ArrayList in just the right spot, you'll remove the LAST element from the ArrayList. After looking at this problem in great detail, we realized thread abort makes reliable code nearly impossible to write, if not impossible. You need our new Whidbey features to make this problem tractable.

Attempting to build anything reliable on top of Thread.Abort in anything pre-Whidbey (short of using process recycling) is crazy, and in Whidbey it will be difficult, but possible.
Tuesday, February 03, 2004 9:14 PM by Brian Grunkemeyer

# re: Static Constructors and Plug-In Security...

1) System.CompilerServices.RuntimeHelpers.RunClassConstructor(RuntimeTypeHandle). It will guarantee that the class constructor has been called, or it will throw some sort of exception.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemRuntimeCompilerServicesRuntimeHelpersClassRunClassConstructorTopic.asp

2) No. See another comment of mine on your blog about why you don't want to use Thread.Abort.
Tuesday, February 03, 2004 9:14 PM by Justin Rogers

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

So the only reliable way to run arbitrary code that may have need of being terminated before it is done, is to make liberal use of AppDomains? I realize there is a good deal of work going into cross AppDomain marshalling so the speed issues might be easy enough to overcome at this point.

We overcame many of these issues in Terrarium by hacking around a lot of potential problems and ensuring the code didn't have specific constructs. These included static fields, static methods, any static access whatsoever, making sure code has empty constructors, and preventing finally blocks. The concept being that since this was arbitrary code, it wouldn't need access to these advanced features. Unfortunately, there is no method to do this without an IL grep. There are no run-time features or enhanced security features to apply to code being loaded into an AppDomain that says a) this code can't access statics or b) this code can't use finally blocks.

Now that AppDomain's might turn out to be the only method for reliable secure plug-in usage, I'll go down that path.
Tuesday, February 03, 2004 9:18 PM by Brian Grunkemeyer

# re: Finalizers and the nasty stuff plug-ins can do if they want to...

There isn't much you can do to stop this. GC.ReRegisterForFinale does not have any security permissions on it, not even any host protection permission in our Whidbey builds.

One thing you could do is write your own "IL verifier", where you only load code that meets your programming model constraints. IE, disallow all static variables, or reject any classes that provide a finalizer (noting that Object of course has a finalizer - just test this rule on some conforming types).
Tuesday, February 03, 2004 9:30 PM by Brian Grunkemeyer

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

Our reliability story for managed stored procedures in SQL Server is hinged on being able to cleanly recycle an appdomain if any of the threads (or managed threads running on fibers within that AD) go awry. With Whidbey features like SafeHandle and some exceedingly pervasive out of memory cleanup, we can ensure we don't leak OS resources in this case. If it's good enough for SQL Server, it's probably good enough for you.

There are some guidelines for writing reliable managed code in an appdomain-recycling model like this. I'm not sure we can make them public at this point, but keep an eye out for them in our Whidbey beta release. The most notable thing you should know is lock(typeof(Foo)) is bad, for all Foo. Don't do it. An upcoming FxCop release should have some reliability rules as well that point out some of this information.

BTW, SQL will also use some kind of IL verifier to enforce programming model constraints on stored procedures. Also, check out host protection permission. If you're willing to go all-out and host the CLR in-process using our complex hosting API's, there are ways you can add link demands that not even fully trusted code will be able to satisfy. Unfortunately for you, the HostProtectionAttribute can only be applied to methods like Environment.Exit and Monitor.Enter, and is not useful for problems like "does my class have a .cctor?" Hence, a custom IL verifier.
Tuesday, February 03, 2004 9:57 PM by Brian Grunkemeyer

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

BTW, here's Chris's explanation of our reliability story, complete with a disclaimer against being prematurely slash-dotted. Read this if you want a pretty in-depth understanding of where we are on reliability, and my comments above are in essence added material. If you want more, come work on the CLR team.

http://blogs.msdn.com/cbrumme/archive/2003/06/23/51482.aspx
Saturday, February 07, 2004 1:42 AM by TrackBack

# Preamble to the Plug-In Framework series of articles...

Saturday, February 07, 2004 4:53 AM by Sami Vaaraniemi

# re: Plug-in Framework (Part1): Marking Types for Consumption

Good article!

There is another fairly common approach to plugin frameworks that I think deserves to be mentioned. You can use a sort of a catalogue service (or a configuration file) that returns a list of assembly locations and types, for example: "http://company1/plugins/plugin1.dll, Company1.Plugin1". This bypasses the need for marking types because once the loader has this list it already knows which types to load.

Doing this securely is another story though - you would probably want to require all remotely loaded plugins to be signed.
Saturday, February 07, 2004 7:48 AM by Stefán Jökull

# re: Allegiance Source Code vs Quake II, you decide...

I'd definately go for Quake II. It's a lot more popular and is still played heavily. Being able to do some improvements as well would be nice to show how easy it is using C#.

By the way, how have you been converting it. Have you been doing a straight conversion or have you been making it more object oriented then originally?
Saturday, February 07, 2004 9:28 AM by OmegaSupreme

# re: Allegiance Source Code vs Quake II, you decide...

I vote quake because its so cool. If people see a classic fps in c# they're gonna be impressed.
Saturday, February 07, 2004 11:37 AM by Dan Bright

# re: Allegiance Source Code vs Quake II, you decide...

I vote Q2 also. You can never go wrong with an old standard like that.

Saturday, February 07, 2004 1:36 PM by Shannon J Hager

# re: Allegiance Source Code vs Quake II, you decide...

I would say Allegiance for all the reasons above. "Classic"? "Old standard"? That's going to impress someone? I've seen the old standards: tetris, pacman, etc, ported to C#. If you want to show the power of C# then show a great modern game, not an old standard because it is easier. Going the easy route makes me think that C# can't handle a real modern game.
Saturday, February 07, 2004 10:43 PM by Justin Rogers

# re: Allegiance Source Code vs Quake II, you decide...

To answer everyone's questions:

This is not a straight conversion, this is an object oriented rearchitecture of the original game. There are quite a few areas were it makes sense and some where it has appeared pretty stupid to do this, but hopefully the end result will be a better demonstration of .NET's power than a simple port.

I love the fact that it is an FPS. Would be the first fully managed FPS? Quake .NET?

Shannon makes some good points too though. The Allegiance codebase is much larger and more complex than the Q2 codebase. The time to conversion would be a bit longer, but the Allegiance code makes use of more recent techniques that more properly translate into fully object oriented code than does the Q2 code.

The biggest addition to the Q2 code will be an MDX for DX9 display filter. Rather than using the crappy old OGL or Software Rasterizers we'll go straight out to MDX. Should provide a huge frame-rate increase (over 250 FPS)
Sunday, February 08, 2004 7:03 AM by Nicko

# re: Allegiance Source Code vs Quake II, you decide...

Vertigo Software have already ported Quake II to the .net runtime using Managed C++

See http://www.vertigosoftware.com/Quake2.htm

This was linked from the msdn.microsoft.com home page for a while.

Looks like you are porting to C# which is a bit different, but I doubt that if you just port Quake II it will have that much of an impact, so I suggest that you port the Allegiance engine.
Sunday, February 08, 2004 9:11 PM by Justin Rogers

# re: Allegiance Source Code vs Quake II, you decide...

Vertigo software's port of the Quake II engine is kind of a joke. Quake II can be recompiled onto the .NET Framework using managed code within a few hours, I know, because I've done that already.

The upgrades they did are also a joke using overlaid forms.

This is a true port, with emphasis on redesigning the engine to work at maximum potential under the managed run-time. It also emphasizes a Managed DirectX rendering layer, which I pointed out in previous comments. I don't see how a C# application running at twice the framerate of the C++ version of the application is not much of an impact. I also don't see how the first FPS running fully managed using C# isn't much of an impact.
Sunday, February 08, 2004 9:13 PM by Shannon J Hager

# re: Allegiance Source Code vs Quake II, you decide...

Either way, Justin, keep us posted.
Monday, February 09, 2004 1:53 AM by TrackBack

# Plug-In Framework (Part 1), Update covering configuration based enumeration

Monday, February 09, 2004 5:02 AM by stefan demetz

# re: Plug-in Framework (Part1): Marking Types for Consumption

Excellent
Monday, February 09, 2004 9:23 PM by Rick Ross

# re: Extending the security system with new permissions...

>Do CAS permissions have to be located in strongly named assemblies?

Yes. Any security classes used to extend CAS must be configured as a fully trusted assembly.
Tuesday, February 10, 2004 4:34 PM by Brian Grunkemeyer

# re: Preamble to the Plug-In Framework series of articles...

Look at the CLR's host protection permission in Whidbey. You'll see it labeled as "HPA's" often, since you can only apply this permission by using the HostPermissionAttribute. This allows the host to do two things:

1) Disable user code from calling certain API's for programming model reasons
2) Disable even fully trusted code (like System.dll) from calling certain API's for reliability reasons

Unfortunately, a method or property must be explicitly marked with an HPA for this to take effect. So you can't make an arbitrary set of programming model restrictions, but only choose whether to restrict access along some of the axes we've provided (such as self-affecting threading vs. external threading, synchronization, UI, etc.) Hopefully that's somewhat interesting.
Thursday, February 12, 2004 3:34 AM by Stephane Rodriguez

# re: On the new Longhorn SDK, a designer asks, "Does size matter"...


Reindexing MSDN content usually takes several minutes. Are you ready to pay for this every other week?
Thursday, February 12, 2004 5:21 AM by Justin Rogers

# re: On the new Longhorn SDK, a designer asks, "Does size matter"...

I'm not suggesting the re-indexing of MSDN content. I'm suggesting that the very latest index files be present on my machine based on an automatic from-web update straight from Microsoft. Worried about long downloads, then use a diff merge instead of downloading the entire thing each time.

The indexes will only change when new content is added or old content is modified. I don't think the churn will be enough that new index downloads would be an issue.

Don't use the existing system as an example, it isn't nearly the system that would catch my attention for both performance and extensibility features.
Thursday, February 12, 2004 6:06 PM by stefan demetz

# re: Using the Security System for Defensive Coding

http://dotnetjunkies.com/WebLog/stefandemetz/archive/2004/02/06/6748.aspx
Friday, February 13, 2004 10:03 PM by TrackBack

# PermitOnly is excellent for security, but can be a bane to programming..., enter PermissionSet

Friday, February 13, 2004 11:39 PM by Shannon J Hager

# re: Movie recommendations, what in the heck are they thinking...

Do you want to help make a better recommendation system? I've had one rolling around in my head for a few weeks, if you're interested in hearing about it, shoot me an email via the contact form on my blog.
Saturday, February 14, 2004 12:25 AM by Jason Mauss

# re: Movie recommendations, what in the heck are they thinking...

I've "rated" like over 300 movies on NetFlix, thinking it would produce better reccomendations for me and it still doesn't even come close. I've resorted to clicking on the actors names on the movie detail pages to get a list of films for an actor, then scanning those for the actor's latest films.
Saturday, February 14, 2004 3:51 PM by John

# re: Movie recommendations, what in the heck are they thinking...

Certainly it's not ideal. Over time it does start to help. I've noticed that the recommendations come in different places by different criteria, and ultimately you start feeling, "I've seen all this!" but give it a few weeks and more recommendations and suddenly you find listing you never saw before. Or maybe Netflix expanded... I just know it's reasonably effective, though not ideal.
Sunday, February 15, 2004 5:49 AM by TrackBack

# MatchSparse? An implementation detail.

Sunday, February 15, 2004 5:53 AM by TrackBack

# MatchSparse? An implementation detail.

Tuesday, February 17, 2004 4:06 AM by stefan demetz

# re: Needed a Crc32 class for some code, figured I'd share with the rest of you.

thanks
Tuesday, February 17, 2004 6:49 AM by Darren Neimke

# re: BinaryFormatter can hang within modified stream?

Justin... did you find a suitable hack, err, *workaround* :-) for this?
Tuesday, February 17, 2004 10:24 AM by bilbo

# re: Using the FilePlanet.com model with regards to Web Services and availability...

perhaps look at existing web service models for reference. For instance, eBay's web service has different payment subscription levels. Google limits the number of times you can query the service, etc. I'm having the same problems figuring out how I want to offer my web service. Free to the public hurts my pocket book!
Wednesday, February 18, 2004 3:48 AM by Justin Rogers

# re: BinaryFormatter can hang within modified stream?

I never did work through to the root cause, reason being, I don't have access to the code for the BinaryFormatter itself to figure out why changing the length of a type name in the stream would cause such a hang.

However, examining the types involved, it would take a good deal of inspection through IL in order to get to the root of the problem. The BinaryFormatter appears to be one of the more complex pieces of code present in the framework.

Maybe I'll head on over to the Rotor source in the next week and use that to discover the root issue.
Wednesday, February 18, 2004 5:50 AM by Ricky Dhatt

# re: Causal Gamers, Women over 40, (aka PopCap games and MS Gaming Zone ;-)

Your "unhackable" comment caught my eye. My girlfriend and her friends all play The Sims. Now they all use the cheat to get unlimited cash so they can build the home exactly the way they want. This drives me nuts, not enduring the "challenge" of the game; but she's perfectly content with it.
Wednesday, February 18, 2004 7:21 PM by Jason Olson

# re: Allegiance Source Code vs Quake II, you decide...

Hey Justin, I would vote for Quake 2.
Wednesday, February 18, 2004 8:43 PM by Justin Rogers

# re: Allegiance Source Code vs Quake II, you decide...

Yep, I'm still working on the Quake II source code. Most of the process is done. Rewriting the graphics engine right now is the largest chore because of the GL processes that don't translate well into DirectX 9. I'm also taking the time to improve the code-base using modern techniques and batching.

Stay tuned, because this is going to be one hell of a sample.

(latest development: Adding the ability to import HL models ;-)
Thursday, February 19, 2004 1:10 AM by Darren Neimke

# re: BinaryFormatter can hang within modified stream?

Cool, thanks Justin. I'll check back here from time-to-time as I remember to but, if you do find out anything interesting inside the next 6 months (say), and you remember, can you give me a ping?

Cheers :-)
Thursday, February 19, 2004 5:26 AM by Darren Neimke

# re: BinaryFormatter can hang within modified stream?

got it! thanks mate :-)
Thursday, February 19, 2004 5:32 AM by Frans Bouma

# re: Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

Aren't string in .NET Unicoded?
Thursday, February 19, 2004 5:33 AM by Justin Rogers

# re: Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

They appear to be UTF8 encoded as per the default BinaryWriter would do.
Thursday, February 19, 2004 8:12 AM by TrackBack

# Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

Thursday, February 19, 2004 8:31 AM by TrackBack

# Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

Thursday, February 19, 2004 8:41 AM by Ken Cox [MVP - ASP.NET]

# re: Pointing out the Control Gallery's...

Hi Justin,

Good timing! I've been doing some housekeeping in the Control Gallery this week and thinking that it should be offering new items and categories as RSS feeds. That would be a valuable addition. (There are also some small formatting bugs that need swatting.)

The problem is that we need someone who knows the existing code (hint, hint) and who could implement these few changes. (Okay, I'm lame in C#.)

The best course would be to contact Rob Howard at Microsoft about getting private access to the current code to make the changes.

Thursday, February 19, 2004 9:46 AM by TrackBack

# Final complaints about BinaryFormatter and the persistence format...

Thursday, February 19, 2004 9:46 AM by TrackBack

# Final complaints about BinaryFormatter and the persistence format...

Thursday, February 19, 2004 3:07 PM by Bill

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

<<
Should KB articles that attempt to create non-existent technology do so with the mindset of LCS...
>>

In my opinion, kb articles, msdn docs and all of the sample code need to demonstrate their associated concepts. Production quality code is, in my opinion, not what these things should be expected to provide. They're not supposed to be teaching people how to write code. They aren't code repositories.

<<
This is actually the right thing to do.
>>

I disagree. The "right thing" to do is to validate input.

<<
As it stands I'm just guessing everyone that defended MSDN here agrees that resources like file streams don't really need...
>>

Your logic is flawed.
Friday, February 20, 2004 5:23 AM by Darren Neimke

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

I really don't have much to add in terms of your questions but, it always interests me to see people who are concerned enough to put conditions at the bottom of loop structures like that (i.e. do...while's as opposed to while loops). It's a good reminder that there's more than one way to do stuff but, left unchecked it opens up another possibility for failure.
Friday, February 20, 2004 5:31 AM by Justin Rogers

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

For the most part, this is straight out of the rotor code, except for our updates. I have to say, putting the condition at the end of the loop here is IMO the right thing to do. It makes the code much more elegant than it would otherwise be.

Now, would I have written the loop the same way had I not been looking at the rotor code when writing my method? Maybe, maybe not. It would certainly have taken me some thought to come to this method, because it is not an oft used method for me.
Friday, February 20, 2004 7:00 AM by Darren Neimke

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

So, what does:

( ( _b & 0x80 ) != 0 )

.. test for? As in, what is 0x80?
Friday, February 20, 2004 7:08 AM by Justin Rogers

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

_b & 0x80 tests the highest bit of the byte.

So basically:
0x80 translates to 10000000 binary

The way 7 bit encoded integers work, is this. Read a byte, take the bottom 7 bits and add it to your accumulator, shifting if necessary (by a multiple of 7). Then check the highest bit, or the 8th bit, and see if it is set. If it is set, then we aren't done yet. We need to read another byte from the stream and continue the decoding process.
Friday, February 20, 2004 12:56 PM by gilad g

# re: Awesome Windows Forms message pump trick...

...And with the power of anon. methods in C# Whidbey, there won't be any need for pre-defining a "Method Invoker" :)
Friday, February 20, 2004 1:31 PM by Bill

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

<<
do...while's as opposed to while loops
>>

In the Write... code a while loop is used because you don't always want to enter the loop. In the Read... code, a do loop is used because you always want to go through the loop at least once. You could re-code each method to use the other construct, but you'd have to add more code to handle special cases.

<<
left unchecked it opens up another possibility for failure.
>>

Not any more than other constructs. Unless I'm missing something about do..while loops.
Saturday, February 21, 2004 10:03 AM by TrackBack

# Answers to the rotor code algorithm puzzle...

Monday, February 23, 2004 10:58 AM by Jerry Dennany

# re: I promised I would talk a bit about Terrarium, so here goes...

Ah, but it isn't boring. Many, many applications make the same mistakes about writing to the current directory, and you help bring focus to this issue.
Monday, February 23, 2004 3:16 PM by Justin Rogers

# re: I promised I would talk a bit about Terrarium, so here goes...

Yes, most applications do make the mistake of using their own directory as a stomping ground for file IO code. It is hard to decide how to implement your file IO restrictions though, and I think this will be the interesting facet of the Terrarium changes. Currently the code changes just do the right thing, however, there are some sticky issues with when we really do need to write files in user selectable locations.

I'm thinking a change to our permissions code to deny File IO at a very early stage in the loading process might be worth looking into. I'll definitely blog about that kind of change because it starts to get interesting.
Tuesday, February 24, 2004 2:34 AM by Thomas Tomiczek

# re: Rotor FUBAR of the day: Why did IPAddress take a public long constructor?

One World:

CLS.

CLS does not allow usigned variables. So UInt (which IPV4 ACTUALLY is does not work). So - in rder to handle all "official" representations of IP in single int form, they had to make the variable an Int64.

You get the same thing with port numbers on sockets, interesting enough, although there they do not get into compatibility mode. he IpEndPoint has an int as port number - interesting in that port numbers run up 65536 and int is 32 bits. But heck, then: CLS does not allow an ushort to be used :-)
Friday, February 27, 2004 8:38 PM by Justin Rogers

# re: Rotor FUBAR of the day: Why did IPAddress take a public long constructor?

Darn, sorry for the long delay in reply, the flu seems to have found me.

This is a perfect response, since it really points out why certain designs were adopted. I'm glad someone had the insight to point this out, since I probably would have overlooked this.

Why? Well, because having used sockets for so long, the sign of a 32-bit integer has never really mattered. Behind the scenes a negative integer is simply mapped to an appropriate IP, and you never find yourself typing the integer format of an IP address anyway.

I think this FUBAR has been reversed and I'll graciously accept that the *right thing* was done.
Sunday, February 29, 2004 4:15 AM by Kartal Guner

# re: My god what was I thinking, using StringCollection over ArrayList?

I assumed the same thing. Good to know this though. I haven't gotten to the point where I am trying to fine tune the performance yet. The StringCollection has a Contains function which is nice though.
Sunday, February 29, 2004 5:36 AM by Justin Rogers

# re: My god what was I thinking, using StringCollection over ArrayList?

Contains is part of IList, and so ArrayList also has a Contains method. It uses an Equals override to determine equality.

if (item.Equals(_items[i]))

For strings, Equals is an internal call. It looks something like the following code which is a basic C++ string comparision function, with the exception of there being a couple of additional items to handle managed specifics.

//
//
// COMPARATORS
//
//
bool WcharCompareHelper (STRINGREF thisStr, STRINGREF valueStr)
{
DWORD *thisChars, *valueChars;
int thisLength, valueLength;

//Get all of our required data.
RefInterpretGetStringValuesDangerousForGC(thisStr, (WCHAR**)&thisChars, &thisLength);
RefInterpretGetStringValuesDangerousForGC(valueStr, (WCHAR**)&valueChars, &valueLength);

//If they're different lengths, they're not an exact match.
if (thisLength!=valueLength) {
return false;
}

// Loop comparing a DWORD (2 WCHARs) at a time.
while ((thisLength -= 2) >= 0)
{
if (*thisChars != *valueChars)
return false;
++thisChars;
++valueChars;
}

// Handle an extra WCHAR.
if (thisLength == -1)
return (*((WCHAR *) thisChars) == *((WCHAR *) valueChars));

return true;
}
Monday, March 01, 2004 2:02 AM by TrackBack

# Game design books focused on non *core* markets, a short resource list...

Tuesday, March 02, 2004 8:27 PM by Addy Santo

# re: Trials and Tribulations of UI design: An examination of the Focus SVT dashboard...

A friend of mine got a panicky call from his wife a while back, saying "I'm in my car and a red icon just came on with a picture of an explosion with a mushroom cloud. I stopped on the side of the road and I'm afraid to turn the engine back on"

A quick phone consultation with the dealership revealed that the icon she was referring to was "low windshield fluid" and the mushroom cloud was really an attempt to show fluid spraying on the windshield (think of a whale-blowing-air caricature)

Friday, March 05, 2004 1:19 AM by TrackBack

# re: New Terrarium UI

Saturday, March 06, 2004 12:55 AM by TrackBack

# String::Join vs StringBuilder::Append vs StringBuilder::Append /w capacity...

Sunday, March 07, 2004 1:15 AM by Richard Tallent

# re: Doing some perf testing, String::Join vs StringBuilder::Append...

This may be a bit unfair to StringBuilder: after all, it is dynamically resizing its internal structure each time you double the number of characters after 16, whereas your Array has already been instatiated, sized, and populated.

Better apples to apples:
1. Use StringBuilder.EnsureCapacity to overcome the advantage Array has by having a preset allocated size.
2. Move the starting time for Array to above the For loop to remove the bias of overhead from the loop and population effort.

Notwithstanding the above, you've made a good point: if you already have an array, String.Join makes a lot more sense than iterating through the array again and copying it to a StringBuilder.
Sunday, March 07, 2004 2:56 AM by Justin Rogers

# re: Doing some perf testing, String::Join vs StringBuilder::Append...

I accounted for the resize in a follow-up post. If you click on the trackback link for the first comment you'll see the follow-up. While there is stil some incorrect information in the follow-up regarding string interning which turns out not to be happening at all, the timing values are spot on and shows String::Join to be only 25% faster than StringBuilder::Append when using capacity planning.
Saturday, March 13, 2004 8:57 AM by Winnifred Alberts

# re: I promised I would talk a bit about Terrarium, so here goes...

Is there any way you can give us eager little coders an approximate date as to when the terrarium source code is going to be released? I am so excited about the prospects of getting my hands on the code! :-)
Saturday, March 13, 2004 8:08 PM by Justin Rogers

# re: I promised I would talk a bit about Terrarium, so here goes...

Eager coders should take this to heart... Soon. And this time we mean it, and are very close.

Since I won't tell you when I can tell you a little more about what. The Terrarium source release is going to be a true experience. Starting with the hopefully flawless and easy to use install process of all Terrarium components down to the refined look and bug fixes to the game play that have been plaguing some developers.

The source code and documentation are going to be indispensable for reference when programming similar applications in the future. In other words, it is worth a small wait and a bit of uncertainty ;-)
Saturday, March 13, 2004 8:14 PM by Koji Ishii

# re: String::Join vs StringBuilder::Append vs StringBuilder::Append /w capacity...

I don't remember how I get this, but if I remember correctly, what string.Join does is:
1. Create a StringBuilder with capacity strings.Length * 16 (this 16 is a fixed number)
2. call sb.Append
This explains why your SBWithCapacity is a bit slower than string.Join, because string.Join doesn't iterate through the array to detect capacity of StringBuilder.
string.Join is faster if average of the length of the strings are less than 16, but I guess yours is faster if longer than that.
Saturday, March 13, 2004 9:08 PM by Justin Rogers

# re: String::Join vs StringBuilder::Append vs StringBuilder::Append /w capacity...

This is incorrect, String::Join as defined inside of the rotor code-base, iterates and creates a precisely sized string. It does not use StringBuilder internally.

The answer is inside of clr\src\vm\comstring.cpp as ConcatenateJoinHelperArray which is called by String::Join to help stitch the string together.

And here is the code and comments that defines precisely that String::Join allocates a perfectly sized memory area for the concatenation of the strings:

//Figure out the total length of the strings in (*value).
elemCount = startIndex + count;
for (numElems=startIndex; numElems<elemCount; numElems++) {
if (((*value)->m_Array[numElems])!=NULL) {
newLength+=((STRINGREF)((*value)->m_Array[numElems]))->GetStringLength();
}
}
numElems=count;

//Add enough room for the joiner.
joinerLength = (*joiner)->GetStringLength();
newLength += (numElems-1) * joinerLength;
Sunday, March 14, 2004 7:42 PM by Darren Neimke

# re: Testing your tests. Ensuring test validity and preventing common mistakes...

Justin, thanks for this post it cleared up a lot of things for me. One thing though - just for accuracy's sake - when you say:

"In Darren's case he thought he was fixing the problem and that the 30 was the real result he was looking for and that the 4000 had to be inherently flawed"

That's not quite right. I blog'ged them because I knew something was wrong somewhere but just didn't really know where/how to look for it. I actually posted a copy off to the VB guys as well just for them to look at.

Another quirk is that I knew that what I was testing probably wasn't functionally == I was just looking for the quickest way to break strings into smaller pieces (for example I'm sure that my Regex.Split tests are not == to String::Split()).

The biggest surprise for me so far is that the String::Split semantics is different to what I had pre-supposed; and you're right, I would have picked that up if I'd checked the outputs.

Thanks again for taking the time to look into this stuff!
Sunday, March 14, 2004 8:29 PM by Justin Rogers

# re: Testing your tests. Ensuring test validity and preventing common mistakes...

Well, even though you weren't thinking that way, you still made a great example ;-) A lot of people do think that way, especially when the initial tests were showing such a huge performance issue with the VB version.

Many developers would have assumed they had the *fix* after the changes you made were implemented because it appears to rationalize the results. I mean, 4000 milliseconds is a long time compared to the results of the other tests you ran.

As for the String::Split semantics, the only way to get something functionally equivalent that splits on a string rather than a character basis, is to code your own (hence the VB guys Split function implementation). However, their method isn't truly optimal, and I think you could write a faster Split() function implementation. I might just try this.
Sunday, March 14, 2004 9:40 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Sunday, March 14, 2004 9:40 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Sunday, March 14, 2004 9:42 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Sunday, March 14, 2004 9:42 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Monday, March 15, 2004 12:24 AM by TrackBack

# re: String::Split - unexpected behaviour?

Sunday, March 21, 2004 4:46 AM by Darren Neimke

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

Justin, just for laughs can you show the metrics for a test where you *do* the ToLower() on the input string. In other words:

private static Regex ignoreCase = new Regex("hello".ToLower(new CultureInfo("en-US", false)), RegexOptions.Compiled);

One of the things that interested me while peeking at Rotor source is that, one of the overloads of AddConcatenate takes a third boolean parameter which stops the ToLower() call from happening when doing a replacement - even if using IgnoreCase. I couldn't really work that one out.
Sunday, March 21, 2004 5:35 AM by Frans Bouma

# re: Performance: If vs Switch in what we might call an extended usage scenario?

Actually the compiler optimizes the switch statement: if it can, it will create a hashtable with name-value pairs. The 'name' part is the value used in the code for the case (such as case "Foo"), the value is the value used in hte optimized case (such as case 4). The hashtable is static.

With a few iterations, this is slower, as it will have to build up the hashtable. For more iterations this is very fast. Check a release build compile of a switch case with strings in reflector :)
Sunday, March 21, 2004 5:39 AM by Darren Neimke

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

BTW... good luck digging through the source to see what IL is produced; have you ever seen the stuff which is Emit'ted from the RegexCompiler module ? :-)
Sunday, March 21, 2004 5:59 AM by Justin Rogers

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

I know why the one is faster versus the other now... I'm pretty adept at going through Rotor ;-)

A character class uses a specialized binary search algorithm across a specialized string of characters that exists as a result of RegexCharClass. This becomes a single call within the compiled binary into a helper function.

With an alternation construct, a butt-load of instructions are emitted that trace each path in the alternation and backtrack if necessary to start on the next item. These are called groups and use something called the grouping stack.

At least this is what I'm gathering right now. I'll probably keep doing work on the source, however, running simulations seems to be turning out better numerical results, even if those results aren't being firmly rooted.
Sunday, March 21, 2004 6:31 AM by Justin Rogers

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

Here are the results of doing a ToLower before calling the expression:

C:\Projects\CSharp\Utilities\PoliCheck>WordIgnoreCase.exe
Ignore Case By ToLower Sanity Check
True
End Ignore Case By ToLower Sanity Check

Ignore Case Sanity Check
True
End Ignore Case Sanity Check

Explicit Case Sanity Check
True
End Explicit Case Sanity Check

Character Class Sanity Check
True
End Character Class Sanity Check

Ignore Case By ToLower, 00:00:02.8440896
Ignore Case, 00:00:01.7625344
Explicit Case, 00:00:01.2518000
Character Class, 00:00:01.1716848
Sunday, March 21, 2004 7:13 AM by Justin Rogers

# re: Performance: If vs Switch in what we might call an extended usage scenario?

However, I wasn't using strings in the example, and instead characters. If you blast out the resulting IL there is very little difference between the switch method and the if statement method in terms of IL and no Hashtable is used (I was already using /optimize+). At this point, I have to continue to assume that there is something happening at the JIT level to optimize the things.

However, I will go ahead and play the part of the diligent worker bee. I'll dump the IL and do a sample walk-through based on data and try to count the number of IL instructions that will be encountered by each process. There are some small differences in the switch vs if, just none that would immediately qualify as providing the speed increase.
Sunday, March 21, 2004 10:23 AM by Duncan Godwin

# re: Making the Regex engine choke... Ever wanted to? I'll show you how...

Hi,

I've been reading Advanced .NET recently and it mentions you can use peverify to tell how the IL is invalid, does this give anything of interest?

Sunday, March 21, 2004 6:07 PM by Justin Rogers

# re: Making the Regex engine choke... Ever wanted to? I'll show you how...

Using PEVerify results in what you would say is a valid assembly. The PEVerify tool must be a loose subset of all rules that can be implemented, and at least with the version I'm running there aren't any flags to specify how strict PEVerify should be since there are four levels of IL safety.

I'm not all that surprised that PEVerify doesn't find anything wrong. However, I am surprised to wake up after nearly 12 hours and find the ILAsm compilation still running.
Sunday, March 21, 2004 8:19 PM by TrackBack

# Monolithic patterns can cause exceptions

Sunday, March 21, 2004 8:21 PM by TrackBack

# Monolithic regex patterns can cause InvalidProgramException to occur

Sunday, March 21, 2004 10:24 PM by TrackBack

# Take Outs for 21 March 2004.

Take Outs for 21 March 2004.
Sunday, March 21, 2004 10:43 PM by TrackBack

# The root of an InvalidProgramException and a possible JIT bug?

Monday, March 22, 2004 1:25 PM by Kit George [Microsoft]

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

Justin, I thougt I would respond to some of your queries here.

I did want to confirm that the added pressure affects everything in Gen-Zero, so it is not tied to your instance per-se. The GC simply alters its collection strategy for all Gen-Zero items as a result of the added pressure, it does not keep a close eye on one particular item.

As for your concern about what happens if you forget to remove the pressure: I have to stress, DON'T forget. We do not try to remove the pressure if you have forgotten. Therefore there is a danger that you can leave the GC in a state where its collection strategy has changed and is no longer optimal. You really do have to make sure that if you add <x> pressure, you later remove <x> pressure when the object is finalized. We like to keep the GC instance agnostic which is why we avoid tieing it to specific items.

As a note: FxCop will be a great help here, since we have rules in place to help ensure that if you AddPressure somewhere, that you also remove the pressure in the finalizer for the class.

As for HandleCollector: you're right, in that it is simply a helper class. It was something which we were using internally, and believe made sense to expose, since people were probably writing their own anyway.

I am sure we'll continue to expand on what's available in this area, and we're always interested to hear your ideas/suggestions for improvements. For example, I have taken note of the request to consider adding an 'AvailableForGC' method in a future release.
Monday, March 22, 2004 8:21 PM by Justin Rogers

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

If you might be able to comment on what the usage of AvailableForGC might be if there isn't any way to actually collect the object after making the examination? What can we do with this information if we do get it.

You also mention again, using the finalizer to make sure the memory gets removed. However, are we instead going to see the Dispose pattern being more important here? AKA, does it matter if the managed object is collected if the unmanaged memory has already been released. The only time the finalizer for a bitmap might be called is if the programmer is relaxed in his disposal of the resource. Now, in the Dispose pattern the finalizer does make sure to call the Dispose methods, so maybe that is what you guys are meaning.
Tuesday, March 23, 2004 4:26 AM by TrackBack

# Discussion on GC in Whidbey

Tuesday, March 23, 2004 4:58 AM by Iain

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

It occurs to me that sure, you could use this stuff to leak resources. However, the only real applicability is for stuff that wraps unmanaged resources - so utilising these could leak memory, but you will only have cause to use these if you already have the ability to leak memory.
Tuesday, March 23, 2004 9:30 AM by Doug McClean

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

To me, an AvailableForGC method doesn't make sense because it would be almost as expensive to implement as an actual garbage collection (e.g., you need to do the full marking pass of the garbage collector to figure out what the answer to this method is), so if you are going to see if something is AvailableForGC you might as well just do the GC and be done with it. Am I missing something?
Tuesday, March 23, 2004 4:44 PM by Kit George [Microsoft]

# re: Making the Regex engine choke... Ever wanted to? I'll show you how...

Justin, this is interesting. We'll be taking a look at this to see what the problem is, since this does look problematic.

Thanks!
Kit
Tuesday, March 23, 2004 9:22 PM by Justin Rogers

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

To Iain: Currently you can leak unmanaged resources, you are right. However, this leakage doesn't affect the GC at all, it has no impact.

Now, if you fail to call RemoveMemoryPressure, you are going to permanently (application lifetime I'm assuming) affect the GC and possibly cause more GCs than are necessary.

So rather than leak a single resource now, you are leaking a new pseudo resource. The pseudo resource is the additional memory pressure placed on the GC that is causing it to change it's heuristics and collect more often because it has been told some object consumes more memory than the GC can deduce.

Thinking really hard on this matter, I really think the memory pressure should be assigned an object instance, or at least have that option. If that object instance is ever collected, it is assumed that the memory pressure, if not already released by the object, gets released. I realize FXCop has rules for this type of thing, but for every developer using FXCop, there are 10 that don't.
Wednesday, March 24, 2004 9:45 AM by Matt Hawley

# re: Installer Products and why I'm not sure I like them very much...

I enjoy using NSIS...its more developer friendly because you get to write the script yourself :)
Thursday, March 25, 2004 3:25 AM by Matt Berther

# re: Installer Products and why I'm not sure I like them very much...

Justin: Regarding item 4, DevStudio 9 does support C# installers. These are loosely similar to C# custom actions, except for the fact that you can not control where in the sequence they occur.

Look in the System.Configuration.Installers namespace for how to author them.

When you want to place one in a DevStudio project, select the component that contains the Installer class and select the option about it being a .NET installer. Arguments can be passed to the installer via the command line arguments option in the same grid.
Thursday, March 25, 2004 3:35 AM by Justin Rogers

# re: Installer Products and why I'm not sure I like them very much...

Yes, we actually run a number of installers, however, I wasn't aware this was as powerful as the Visual Studio custom actions. For instance, I'm able to run my Installer class, but not able to get to needed MSI properties.

Also, yes, not being able to control where they occur is yet another issue with them. I've looked at these, they simply aren't powerful enough to take the place of custom actions, and really only have their place if you don't need the installers to actually interact with the installer process. At least that is what I've taken away from the whole experience.
Thursday, March 25, 2004 7:56 AM by Matt Hawley

# re: More InstallShield DevStudio 9 issues...

PS - its NSIS, not NSSI (http://nsis.sf.net)
Thursday, March 25, 2004 8:36 AM by Justin Rogers

# re: More InstallShield DevStudio 9 issues...

Ah, thanks for the link. Looks powerful and the source is available. I like it. I'll have to code something up now so I can package it using this installer.
Friday, March 26, 2004 8:51 AM by Josh Baltzell

# re: Placing vegetarianism on the chopping block...

I was a veggie in HS and a little while after. I started ebcause someone told me I couldn't do it for all day on Thanksgiving. I was a vegetarian for like 2 or 3 years after that.

I didn't eat healthy or anything, I just didn't eat meat. Then I got a girlfriend (my fianceé now) slowly but surely I was eating fish, then chicken and then after a while I was eating all meats again.

So if you really want to get back in to eating meats for some reason (Don't forget that meat meals are generally more expensive on top of worse for you) then maybe just working up from fish is the way to go.
Friday, March 26, 2004 2:21 PM by Christoc

# re: Placing vegetarianism on the chopping block...

Yummmmm Steak-Ums!

Good luck to you, hope you don't get sick for too long due to this.
Friday, March 26, 2004 3:25 PM by Justin Rogers

# re: Placing vegetarianism on the chopping block...

I do eat fish at times. I'm classified as a lacto/pesco/ovo vegetarian. I guess people weren't comfortable with the whole black and white you are or you aren't, so they had to come up with additional classifications.

Most of my protein comes from soy sources and eggs, so I'm not sure what I'd do without the ovo part. The lacto I could probably do without, but I love coffee drinks and soy milk is still extra (darnit). I still like a piece of fish at times, but does one piece of fish a month still take on the pesco? I don't know. I've seen vegans pile through some things not on their recommended eating list at a doctor's requirement.
Friday, March 26, 2004 5:01 PM by Rj

# re: Placing vegetarianism on the chopping block...

Good luck eating meat again. Yours seems to be one of the most rational reasons for not eating meat.

btw, Irregardless is not a word, except through continued missuse. Regardless works just fine.
Friday, March 26, 2004 5:06 PM by Shannon J Hager

# re: Vegetarian rating of 5 types of meat, a follow-up to the original, all too tempting test.

I went a couple years without meat, but whenever anyone asked if I was a vegetarian, I replied "huh? I don't even know where Vegetaria is. I'm American."


Now that I am an omnivore, I am still down with the veggies. I swear I make the best tofu I've ever had. Vegetarians are cool with me, but humanitarians scare me.
Friday, March 26, 2004 6:22 PM by TrackBack

# Vegetarian rating of 5 types of meat, a follow-up to the original, all too tempting test.

Saturday, March 27, 2004 11:51 AM by Brian Desmond

# re: Installer Products and why I'm not sure I like them very much...

I've never used the VS Installer. All of the installer's I've built have been in DevStudio 9 or Developer 8. Probably 30 or 40 MSIs by now.

It's really not *that* bad, just takes some getting used to. I don't use any of the wizard crap. just skip to the treeview with everything in it & find what I want.
Sunday, March 28, 2004 8:50 AM by Justin Rogers

# re: Performance and Insight: For versus Foreach over a strongly typed array... Some things you might not know.

Someone asserted that the JIT might optimize out the entire loop and make it *not happen* because the resulting value isn't being used. I think this is a highly unfounded assertion, but just in case, you'd think we might be able to save the JIT the trouble of optimizing things out by appending the bar variable onto the Console.WriteLine call. Once you do that, the variable bar becomes *used* and so the JIT couldn't possibly optimize it out.

However, I know the JIT wasn't optimizing things out because I was viewing the resulting assembler for each of the above methods and watching it run. I think that asking the JIT to optimize all of the above code out would be a rather tall order, but I'm assuming the metrics could be there for it. As a human compiler I would optimize the code out since I would realize the value of bar was never used and so the iteration was for naught.

I guess the days of using looping constructs to count time away are over. No more: "Please insert an integer between 1 and 10000: " when you start running a game under the GW BASIC interpreter. And I thought I was so slick when I wrote my own custom timing function to get rid of that message for good, so no user ever had to feel the wrath of submarines and torpedos at warp speed on their ultra fast new 386.
Sunday, March 28, 2004 9:00 AM by RossJ

# re: Found the creator of Ping while doing graphics research on Cone Tracing, go figure!


Do you have more information on your photon mapping software? One of our student developers is working on some photon mapping solution for his Masters dissertation and I thought it would be nice if I could help him find more (not that he really needs more) solutions doing the same thing.
Sunday, March 28, 2004 9:10 AM by Justin Rogers

# re: Found the creator of Ping while doing graphics research on Cone Tracing, go figure!

I'll be releasing more information soon. I just sat down and wrote the software over the past couple of days. I'm still deciding if I want certain math libraries to be written in Managed C++, or whether I should leave the entire thing in C#.

I also plugged in my own partioning library, and I'm finding it to be inadequate. I'll probably plug through another 2-3 different types of partioning trees this weekend.
Sunday, March 28, 2004 9:12 AM by RossJ

# re: Found the creator of Ping while doing graphics research on Cone Tracing, go figure!


He did describe in (excrutiating) details how he was doing this and it *sounded* impressive (I'm not oreinted that way) .. I'll just point him at your blog instead :)
Monday, March 29, 2004 1:25 AM by Omar Shahine

# re: Dynamic versus Cached Thumbnailing in GDI+

I have an application which does just this (http://workspaces.gotdotnet.com/jpeg) however, the perofmance of creating the bitmap just sucks (IMHO). Basically I get what you say, 2 images loaded per second. Applications like PhotoShop can do a whole folder of photos (50 or so) in about the same time.
Monday, March 29, 2004 3:22 AM by Justin Rogers

# re: Dynamic versus Cached Thumbnailing in GDI+

Omar, the reason for their abilities is in how they process the images straight off the file system, rather than first loading the bitmap into memory and then converting it down to a thumbnail.

I will say, however, that on my most recent revisions for the Async thumbnailing code, I've been able to process closer to 5 or 6 thumbnails per second rather than the original 2. I attribute this to some issues with my async code that I've since worked out.

While Photoshop may be able to do an entire folder in about the same time you load only a couple, I would instead point you towards an Explorer folder where thumbs.db or thumbnail caching has been turned off. Thumbnailing images in this fashion operates at about the same speed as the GDI+ code.

So I'm not saying we couldn't get better performance using enhanced caching, or better methods for thumbnailing images without loading the entire bitmap, or even a better thumbnailer that operated faster. What I would say, is that programs that use the same methods we are using, run at the same speed, and I find it an acceptable speed.
Monday, March 29, 2004 5:18 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

The sample program had a single naming issue. I fixed this, but only after there was a reported view on the source sample. If that someone has tried to run the code, sorry for the mix-up, the new code in the entry should run as planned.
Monday, March 29, 2004 5:38 AM by David Levine

# re: Performance: Different methods for testing string input for numeric values...

I haven't done any performance testing on this, but you should add double.TryParse() to your test suite - it avoids the problem that int.Parse() has of throwing an exception if it fails.
Monday, March 29, 2004 5:48 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

Adding performance output for everything except the double.TryParse()

IsNumber String 1: False
IsNumber String 2: True
IsDigit String 1: False
IsDigit String 2: True
HandCodeSwitch String 1: False
HandCodeSwitch String 2: True
HandCodeIf String 1: False
HandCodeIf String 2: True
RegexDigit String 1: False
RegexDigit String 2: True
IntParse String 1: False
IntParse String 2: True
IsNumber 00:00:03.2146224
IsDigit 00:00:02.4635424
HandCodeIf 00:00:00.7410656
HandCodeSwitch 00:00:00.7811232
RegexDigit 00:00:13.3391808
IntParse 00:08:32.7372800
Monday, March 29, 2004 5:50 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

Adding code and timing for TryParse on System.Double.

start = DateTime.Now;
for(int i = 0; i < iterations; i++) {
makeSureTheJitDoesNotOptimizeMeOut = double.TryParse(testStrings[i%2], NumberStyles.Number, null, out result);
}
end = DateTime.Now;
Console.WriteLine("DoubleTryParse {0}", end - start, makeSureTheJitDoesNotOptimizeMeOut);


DoubleTryParse 00:00:13.2490512
Monday, March 29, 2004 7:02 AM by TrackBack

# Terrarium: How do we punish creatures that take too much time?

Monday, March 29, 2004 7:10 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

Adding code and timing for Microsoft.VisualBasic.Information.IsNumeric.

start = DateTime.Now;
for(int i = 0; i < iterations; i++) {
makeSureTheJitDoesNotOptimizeMeOut = Information.IsNumeric(testStrings[i%2]);
}
end = DateTime.Now;
Console.WriteLine("Information.IsNumeric {0}", end - start, makeSureTheJitDoesNotOptimizeMeOut);

Information.IsNumeric 00:00:17.1746960
Monday, March 29, 2004 5:33 PM by Josh Baltzell

# re: Vegetarian rating of 5 types of meat, a follow-up to the original, all too tempting test.

I made it all the way through over a year of being a vegetarian without ever being able to make anything that actually tasted good out of tofu.

People generally refer to my prior vegetarian self as a hippie. Now granted I may have been slightly hippieish, but being a vegetarian has nothing to do with being a hippie. Thank you for telling people that.
Monday, March 29, 2004 8:35 PM by TrackBack

# Various things around the web today

Wednesday, March 31, 2004 1:19 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

I've always wondered why this wasn't built into the framework... It's such a common task. I guess Rico didn't stop by this team and talk to them about performance issues with exceptions...
Wednesday, March 31, 2004 1:33 AM by Jesse Ezell

# re: Why is photon mapping popular in gaming right now, and will it stay?

Very cool stuff. Have some demos?
Wednesday, March 31, 2004 1:53 AM by Justin Rogers

# re: Why is photon mapping popular in gaming right now, and will it stay?

Soon. I've been researching photon mapping for a while, and all of my cool stuff is theortical and based on numerical analysis. My first true scene should be the club scene with about 30 static colored lights, 10 floor standing stage lights, and a single strobe.

I'm doing all of the work in C# and Managed DirectX, so there is a curve where-in my performance knowledge is having to be augmented. Namely in the random number generators (the default is too slow), and in spatial storage tress (hard to make these fast in .NET).
Wednesday, March 31, 2004 2:33 AM by Alex Campbell

# re: Following up on the benefits of continued use of int.Parse...

Hi Jason,

Thanks for the feedback. As usual, I agree with almost all of what you say. Your alternative approaches are certainly faster. However, I think in some particular cases, try... int.Parse()... catch still has a place.

Your description of my (not even remotely original) approach as "admirable" is hard to judge. Did you mean "admirable" in the same way that I often describe really stupid ideas as "creative", "novel", "unique", "fascinating", "breathtaking" etc? :-)

I also agree with Jesse that it would have been great if the .Net Framework team had included this functionality in the framework.
Wednesday, March 31, 2004 2:47 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

Admirable in that you were willing to defend the method based on it's additional power to differentiate int's from non-ints. None of the original routines that I tested were truly capable of discerning ints and it was good that you pointed that out.

Even my newer techniques have issues in that they aren't globalizable since they aren't based on culture variances (I pointed this out above).

However, even if the BCL team had provided better options for programmers, my method would still be faster. Unless they dynamically compiled, based on the culture info, a string to integer parser, it wouldn't be faster than my implementation that is hard-coded to work only in the US english case. I am currently in the process of discovering how this might be done, since fast string->number conversions are very important for several of my own applications.
Wednesday, March 31, 2004 3:17 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

If you really want a flexable implementation that matches the .NET framework's, you could probably just use reflector to get the source and then remove the exception throwing stuff (or get the mono source or rotor source).

In any case, Whidbey adds a TryParse method, so at least you won't have to wait too long for this functionality.
Wednesday, March 31, 2004 3:23 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

PS: If you don't mind casting in .NET 1.0/1.1, you can use Double.TryParse and convert to an integer...
Wednesday, March 31, 2004 3:28 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

As for a flexible implementation, a good deal of the code for numeric types is implemented as part of the VM or virtual machine. There are ee calls that take place (InternalCall) that actually handle the conversion of the string to a number. These calls happen to be where the exceptions are thrown from, so reflector doesn't really help.

To answer your Double.TryParse comments. Double.TryParse is slow. Very slow in fact. It is much slower than int.Parse assuming all numbers are valid. You can read my previous blog entry that covers the speed of many other parsing methods.

<a href="http://weblogs.asp.net/justin_rogers/archive/2004/03/29/100982.aspx" title="Justin Rogers">Performance: Different methods for testing string input for numeric values...</a>
Wednesday, March 31, 2004 3:30 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

A second assertion would be that a user can input non integer values into Double.TryParse, that could then be, validly cast into an integer. This would also enable loss of data, and a bunch of other *bad* things. I honestly don't think that using Double.TryParse and casting down is a valid way to test a string for integer value.

Wednesday, March 31, 2004 3:41 AM by Andy Smith

# re: Following up on the benefits of continued use of int.Parse...

Double.TryParse takes, as one of the parameters, a NumberStyle. One of those NumberStyles is Integer. using the integer numberstyle, you are guarenteed an int in the value parameter when it returns true.

And as for your perf argument... You've said yourself that your custom methods only work on US English. I'm sorry, but sacrificing localization for perf is not acceptable. "Anybody can write a fast program that does the wrong thing."

And as for the perf gain over Int32.Parse, well, that throws exceptions on bad data, which is not what you want for perf.
Wednesday, March 31, 2004 3:57 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

How often do you use a localized parse? You aren't using it in WebForms applications, since in most cases the locale is locked down to that of the server. You most of the time aren't using it in WinForms applications, and your configuration files/files being parsed are most likely in some set format. Not to mention unless you store the locale in the file, it will be mis-parsed if you send it across a locale.

My argument would be that sacrificing perf for localization isn't acceptable in most cases. Presenting a reverse argument never really accomplishes much though.

I will note that NumberStyles.Integer is not linked to a specific bit length. So you can still improperly decode a 64 bit integer into a 32 bit integer. You can also decode a 32/64 bit integer into a 16 bit integer. I really don't see what this Double.TryParse really buys, except for a bit of added and additional complexity.

Hell, you can decode an infinite length string using Double.TryParse. Check the following program.

using System;
using System.Globalization;

public class HackDoubleTryParse {
private static void Main(string[] args) {
double retVal;
string[] integers = new string[] {
"7290847123984721093749812374092137498217309821098472198347",
"-210943712908347091823409812734982370947213098412093741924"
};

Console.WriteLine(integers[0]);
Console.WriteLine(Double.TryParse(integers[0], NumberStyles.Integer, null, out retVal));
Console.WriteLine((int) retVal);
Console.WriteLine();

Console.WriteLine(integers[1]);
Console.WriteLine(Double.TryParse(integers[1], NumberStyles.Integer, null, out retVal));
Console.WriteLine((int) retVal);
Console.WriteLine();
}
}
Wednesday, March 31, 2004 4:05 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

It appears to accept up to 308 characters before it returns false. So:

new string('5', 308) would succeed, while
new string('5', 309) would fail.

However, characters may not be the order of the day, but instead a huge value. Since:
new string('1', 309) appears to work, but
new string('1', 310) doesn't.

Very strange indeed. I won't bother looking at the Rotor code for this method, since I'm not sure it would help stave off any afronts.

Wednesday, March 31, 2004 4:33 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

I just ran your test with the Double.TryParse code inserted... 6 secs for 10 mill. conversions really ain't that bad for most apps. If you pass a pre-constructed NumberFormatInfo into the method call it completes the loop a bit faster than RegEx on my CPU (especially if the numbers get large, like 8 digits or so). I don't think the extra time is really all that bad, considering that you are getting a more powerful parsing algorithm that deals with culture and other things like exponents and commas (2.2 vs 6.6 for 10 mill ints isn't too bad). For 99.9% of all apps that kind of perf is still in the acceptable range (beats IntParse by 50x on my machine... I had to go down to 100000 just to make it complete in a reasonable amount of time with the IntParse method included).

Adding a few sanity checks after the double conversion could easily mitigate any conversion issues that TryParse might lead to.
Wednesday, March 31, 2004 4:37 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

Explain your sanity checks, and then examine the timing with your sanity checks added. Since those would count as part of your *algorithm* for parsing integers.
Wednesday, March 31, 2004 5:05 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

Passing in integer as the number style should take care of the loss of any decimal data issues at almost no cost. Then, all you have to do is check to make sure the double is in range of Int32.MaxValue and Int32.MinValue.
Wednesday, March 31, 2004 5:12 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

It actually runs faster with the sanity checks added.

6.0 secs vs 6.6
Wednesday, March 31, 2004 5:24 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

You'd actually have to post the code so we can make sure the time testing is all in the proper places, and that the JIT isn't optimizing out your checks. You also need to come up with some way to process the method returning 0 and True, when the value really isn't 0. See my above code sample for cases where the string is clearly not an Int, but the method returns True and 0.

Wednesday, March 31, 2004 5:36 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

Just to let you know, if you only pass in Integer as your number format style, then my method actually is doing everything that double.TryParse will do for you. Note that Integer actually maps to AllowLeadingSign, AllowLeadingWhite and AllowTrailingWhite. So go ahead and use that method if you would like, but it is quite a bit slower than mine.

I guess the Rotor code examination really does make the difference here since it points out the lack of additional processing that TryParse is doing, and that my algorithm should work on any culture for the purposes of processing integers. If you'd like I can even add the flags for turning on parsing of the thousands separator.

Wednesday, March 31, 2004 5:37 AM by nfactorial

# re: Wayne's World: People actually ask about the ellipses character?

<Quote>
many games use pre-generated graphics for printing text since it looks *cool*
</Quote>

Actually, games use bitmapped characters because they render extremely quickly (two stripped triangles as opposed to an outline\fill algorithm). Assuming I take your meaning correctly :)

n!
Wednesday, March 31, 2004 5:48 AM by Justin Rogers

# re: Wayne's World: People actually ask about the ellipses character?

Taken slightly out of context, but not such a big deal. I should have clarified.

Most games do appreciate the performance increase of indexing into a single texture filled with pre-gen characters and overlaying that on top of an arbitrary quad.

However, you can offer many of the same performance benefits, by dynamically rendering and caching your alphabet using GDI to blit said letter onto a texture surface somewhere.

With this in mind, recent usages of a bitmapped alphabet overlay are more for coolness, IMHO, than they are for necessity of speed. Even with the recent usage of the pre-canned font class delivered as part of the DirectX SDK, the proliferation of textured alphabets does persist, along with the lack of an ellipses usage even though the pre-canned font class should be capable of rendering any character in the given character set.

I'm not sure about the performance numbers here, but I've had the experience that it is faster to render an entire text buffer to texture using GDI and overlay the texture on a single quad, than it is to build up my display using many small quads per character. It also presents the benefits of the font layout manager for character spacing, line spacing, and other features that don't exist when you are using a textured alphabet.

Wednesday, March 31, 2004 8:48 AM by nfactorial

# re: Wayne's World: People actually ask about the ellipses character?

I'm not sure I'm talking about the same thing now :) I class dynamically rendering and caching an alphabet, using GDI+, to a texture the same thing as 'pre-generated graphics'. Even if this is done when the application is started.

Certainly rendering a single message to a texture (using the above pre-generated font) and rendering that message will be faster, but I'd still consider that the same thing.

Games (that I'm familiar with, at least) do generate their 'generic' fonts using GDI (or GDI+). I suppose you're recommending this is actually done during run-time initialization rather than pre-generated and packaged on the CD\DVD? I originally thought you meant games chose not to render fonts in the same way as windows for aesthetic reasons.

There are also other bitmap fonts that are created specifically for a game (which are all 'dolled up' to look pretty and fit with the game atmosphere). These are 'just to look cool', but then the OS (if you're lucky to be on an OS in the Win32 sense of the word) doesn't provide such fonts so there isn't much choice. Though I find these types of font are on the decline as they're more difficult to localize and read.

Sorry, I didn't mean to hi-jack the posting based on a single line. The ellipsis being a single character was a surprise to me too! :)

n!
Wednesday, March 31, 2004 9:41 AM by Justin Rogers

# re: Wayne's World: People actually ask about the ellipses character?

Nothing better than talking about gaming on a blog that contains good things on gaming.

Yeah, I think we are confusing different meanings since there are so many reasons for the way text is handled in gaming. In general, you can pre-gen a font before shipping and include that on the DVD, you can pre-cache the font at run-time initialization (which is actually a method I prefer because you can render the font for the target resolution), or you can dynamically render all of your fonts.

Recently, I think most games choose to use the DirectX font class (less pre-gen, more dynamic), or a variation there-of. But they still keep the old pre-gen stuff around for other cool features as you suggest. They might be on the decline, or they might just be getting started. Games are pushing the DVD capacity limits as we speak, and a few megs worth of extra jpegs for different alphabets, coupled with the write once, use many code required to display localized resources, really isn't that bad.

I might have to do some performance work on text engines in games. I know, for instance, that Longhorn will have one of the best text compositing engines ever written. And they are saying the new text compositing has very little performance impact on the system. That leads me to believe that current text rendering systems really aren't a viable bottleneck, especially considering the small amount of text that can be fit on the screen at one time. Just doing basic tests, I can fill a form using GDI+ flow and layout in just a few milliseconds. Since in the DirectX world that render would get re-used for a second or two before I had to re-render. And I get the font rendered for the target resolution, rather than having it shrink to unseeable at super high resolutions, and have it fit the DPI of the target device in the case of 300 DPI screens.

I guess the console market still has a good reason to use pre-gens, as would small gaming devices. I'm betting they'll be in use there for a long time as a performance increase. Maybe rendering three dots is less than the cost of an extra glyph. I'm just not sure. I just know the ellipses at this point, is a character that is either often used (insert ellipsis character) or often mis-used (insert three periods). You decide!
Wednesday, March 31, 2004 11:17 AM by Ben Hutchings

# re: Wayne's World: People actually ask about the ellipses character?

"The most readily apparent to those of us that blog, would be to use the ellipses character directly from the HTML view &#0133. Just make sure the character set being used has such a character."

No, this is wrong. Numeric character references always refer to code points in the document character set (for HTML this is ISO 10646/Unicode) and not in the file's character set.
Wednesday, March 31, 2004 12:00 PM by Justin Rogers

# re: Wayne's World: People actually ask about the ellipses character?

Ben, if you come back, can you clarify this? I've found that using &#0133 here in the .Text blogs system while in HTML view allows me to insert an ellipses. Or I can use Character Map to simply copy/paste one into the Design view. I was just trying to provide a method for individuals to insert such a character (which does work for me at least), but you seem to have additional insight. Can you perhaps demonstrate what you mean?

Wednesday, March 31, 2004 6:38 PM by Jason Olson

# re: Why is photon mapping popular in gaming right now, and will it stay?

Keep the info and posts comin' Justin! I look forward to every new post of yours.

Is there a chance that you will release the source code with the demos? That would be super cool. And I think it would help out the Managed DirectX community in general by seeing some good applications of Managed DirectX and C#.

Keep it up :).
Thursday, April 01, 2004 1:46 AM by Justin Rogers

# re: Why is photon mapping popular in gaming right now, and will it stay?

Since it'll probably be the weekend before I get to format another decent photon mapping article, I'll whet your appetite.

I definitely want to release my code with the demos. That is why I'm not releasing right now. I want my own personal examination of each of the methods and algorithms to be rigorous, to make sure I'm not producing an algorithm, that while nice to look at, has issues with physical law.

I definitely agree that we need some nice MDX/C# samples, and that is why I've chosen to use these technologies instead of the more rapid development I'd achieve by simply using C++. While C# generally provides for rapid development, performance heuristics are new territory and take some time to fully explore. Structures that work in the unmanaged world, sometimes don't work anymore in the managed world.

Glad to see I'm making a good impression and gaining good readership. I've been disappointed lately in the game development websites and their lack of content. Too much in the gaming industry is closed doors or theoretical, and there is a large gap between hobbyist and professional with not much opportunity in between. If I can help bridge that gap just a bit, I'll be happy.
Thursday, April 01, 2004 4:06 AM by TrackBack

# re: System.Drawing.Image Performance

Thursday, April 01, 2004 8:36 AM by Paul Nicholls

# re: Wayne's World: People actually ask about the ellipses character?

On the character entity references page for HTML
http://www.w3.org/TR/html4/sgml/entities.html
it has the ellipses as &hellip;
Thursday, April 01, 2004 9:22 AM by Oleg Tkachenko

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

Completely agree with you, Justin!
Thursday, April 01, 2004 9:32 AM by Steve

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

I couldn't agree more... XSLT is very, very powerful. I try and use it all the time.
Thursday, April 01, 2004 9:42 AM by Jeff

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

I agree with you, but I'm just as guilty. I've used XSLT once for something simple, and that's it. It is very powerful, but like regular expressions, I frequently can't relax and concentrate long enough on it to really get the most out of it.
Thursday, April 01, 2004 11:25 AM by Henry P. Erich III

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

I love xslt.

XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(URL_MAIN_BLOG);

XslTransform xslt = new XslTransform();
xslt.Load(Server.MapPath(PATH_MAIN_BLOG_STYLE));

StringWriter writer = new StringWriter();
xslt.Transform(xmlDoc, null, writer, null);

mainContent.InnerHtml = writer.ToString();
Thursday, April 01, 2004 12:10 PM by Omar Shahine

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

I'll try this out today and let you know the results.
Thursday, April 01, 2004 3:54 PM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

in your original code, you can get rid of 1 multiplication by doing

value = value * 10 + val

you don't need to explicitly keep track of a multiplier through each loop.
Thursday, April 01, 2004 4:17 PM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

However, you'd have to work through the number from GSD to LSD that way. We are currently working on the assumption that you are working from LSD to GSD. This allows for us to prevent checking overflow logic for every digit counting GSD to LSD.

Here is a sample:

"123". With the current loop we are doing
3*1+0 = 3,
2*10+3 = 23,
1*100+23 = 123

You see, your method assumes that we go in reverse.

"123".
0*10 + 1 = 1
1*10 + 2 = 12
12*10 + 3 = 123

Now, when you get to the higher numbers, namely the maximum limits. You have to check for overflow conditions manually. The reason for this is that checked arithmetic throws exceptions (bad), so we can't use them. Something along the lines of Int32.Max*2+4, would actually wrap around and become a positive number again, and we would incorrectly process it. This means we have to check digits at the upper extremes. Doing this with a GSD algorithm, would mean checking at EACH digit, or at least doing some stranger logic.
Thursday, April 01, 2004 4:27 PM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Note: If you have an algorithm that works GSD and handles overflow conditions without using checked arithmetic let me know and I'll test it out. While I still think the unrolled code currently being used by the constantly updated source sample is going to be faster, I'll definitely put the methods side by side and run some speed tests. I want the fastest algorithm possible, and right now I don't think I can get it any faster without using pointers (unsafe code), and even if I do use pointers I don't think it will be faster, only more compact and elegant.
Thursday, April 01, 2004 8:13 PM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

well, probably won't be faster than unrolled loop, who knows.

const int tenthBound = Int32.MaxValue / 10;
const int digitBound = Int32.MaxValue % 10;

//
// only for Int32
//
public static bool TryParseInt(string s, out int value)
{

value = 0;

int digit;

//
// since we are being silly, this should be less costly
// than making multiple calls to s.Length. maybe?
//
int max = s.Length;

if( max == 0 || max > 10 )
return false;

bool checkEnd = max == 10; // is this max allowable length?

max = checkEnd? 9 : max; // we only loop 9 times if it's max length

for( int i = 0; i < max; i++ )
{

digit = s[i] - '0';

if( digit < 0 || digit > 9 )
return false;

value = value * 10 + digit;

}

if( checkEnd )
{

if( value > tenthBound )
return false; // x10 will result in overflow
else
{

digit = s[9] - '0';

if( digit < 0 || digit > 9 )
return false;

if( value == tenthBound && digit > digitBound )
return false; // x10 + digit will result in overflow
else
value = value * 10 + digit;

}
}

return true;

}

since this is not really heavy on array access, i doubt unsafe would provide much benefits.
Thursday, April 01, 2004 8:15 PM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

sorry, should have used <pre> to format. I hate HTML.
Thursday, April 01, 2004 8:47 PM by TrackBack

# re: Programming Challenge: Phraser

Thursday, April 01, 2004 10:28 PM by TrackBack

# Entries to my programming challenge

Friday, April 02, 2004 12:25 AM by Omar Shahine

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

So, it seems just as fast. Sweet!!!
Friday, April 02, 2004 2:15 AM by TrackBack

# Take Outs for 1 April 2004

Take Outs for 1 April 2004
Friday, April 02, 2004 5:55 AM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

DJ, here is a small test suite. Your's fails quite a few of the tests. Here is your output followed by expected output, followed by some test code. I'm definitely digging your approach, since it is radically different from the methods I've been using, and your code is a bit smaller than my final code.

<pre>
*** Community.TryParseInt32 ***
False
False
False
True
True
False
False
False
False
False
False
False
False
False
False
False

/* expected output
EC1: False
EC1: False
EC1: False
MAX1: True
MAX2: True
MIN1: True
MIN2: True
EC2: False
EC2: False
EC2: False
PC1: True
PC1: True
PC2: True
PC2: True
PC3: True
PC3: True
*/


Console.WriteLine("*** Community.TryParseInt32 ***");
Console.WriteLine(Community.TryParseInt("9999999999", out retValInt32));
Console.WriteLine(Community.TryParseInt("8888888888", out retValInt32));
Console.WriteLine(Community.TryParseInt("7777777777", out retValInt32));

Console.WriteLine(Community.TryParseInt(Int32.MaxValue.ToString(), out retValInt32));
Console.WriteLine(retValInt32 == Int32.MaxValue);
Console.WriteLine(Community.TryParseInt(Int32.MinValue.ToString(), out retValInt32));
Console.WriteLine(retValInt32 == Int32.MinValue);

Console.WriteLine(Community.TryParseInt("-7777777777", out retValInt32));
Console.WriteLine(Community.TryParseInt("-8888888888", out retValInt32));
Console.WriteLine(Community.TryParseInt("-9999999999", out retValInt32));

Console.WriteLine(Community.TryParseInt("-999999999", out retValInt32));
Console.WriteLine(retValInt32 == -999999999);
Console.WriteLine(Community.TryParseInt("-99999999", out retValInt32));
Console.WriteLine(retValInt32 == -99999999);
Console.WriteLine(Community.TryParseInt("-9999999", out retValInt32));
Console.WriteLine(retValInt32 == -9999999);
</pre>
Friday, April 02, 2004 5:59 AM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

It looks like it may fail for all negative numbers, but I also didn't try it for all positive numbers. The real test suite runs as follows. If you need a positive number test suite that is comprehensive I suggest using:

for(int i = 0; i < Int32.MaxValue; i++) {
int retVal = 0;
string tryParse = i.ToString();
if ( Community.TryParseInt(tryParse, out retVal) ) {
if ( i != retVal ) {
Console.WriteLine("Equality failure in TryParseInt: {0}, {1}", i, retVal);
return;
}
} else {
Console.WriteLine("Processing failure {0}", i);
return;
}
}
Friday, April 02, 2004 6:31 AM by TrackBack

# System.Drawing.Image Performance Update

Friday, April 02, 2004 8:13 AM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

sorry, should have commented "it only parses positive Int32's". I didn't take negatives into consideration.

did you test how my code fair on performance? (on positives of course). it would be an interesting test (in general) to see how JIT optimizes loops vs. manually unrolling a loop.
Friday, April 02, 2004 9:04 AM by Jon Hanna

# A completely different approach

Why use 1 to 10 comparisons per character rather than one comparison on a result from a lookup multistage table (or single-stage table if you want to have a single dereference but not be able to reduce the size of the data).

This would particularly helpful if you came to support characters like the Arabic-Indic digits or the Bengali digits since there are currently 268 characters in the UCS with a non-null Decimal Value property.
Friday, April 02, 2004 2:40 PM by ed

# re: Trying my hand at the old Phone number to Words teaser project!

I came up w/the same solution you did, even using c#. You wrote a couple things better than I did, so I'm not going to submit mine. The solution to the problem was the same though.
Friday, April 02, 2004 4:45 PM by the1

# re: Trying my hand at the old Phone number to Words teaser project!

Justin,

I posted my own solution at http://blogs.msdn.com/the1/archive/2004/04/02/106691.aspx
Saturday, April 03, 2004 5:11 PM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Just to show I'm still alive on this (I have been super busy with some work), I had some algorithms that were loop based and process pos/negs... I've shown them as commented out in the lib code.

As for the multi-stage table lookup, I would say try to examine the slowness of using arrays. They are super slow to say the least.

Since we are only processing up to Int64 using these methods, we could come up with an array based implementation that used pre-multiplied digits. This is completely possible and would probably be a bit faster than any of the methods I'm using right now, if we went to unsafe pointer arithmetic. I'll see if there is a method for doing the same and having it be fast without the pointer arithmetic.

Note: If we create the pre-multiplied table it would be of size 20*10 or 200 elements. That isn't too big is it? We could optimize out 0's and it would only be 20*9 (20 digit locations are needed to process a 64 bit integer, and 9 digits 1 through 9 have to be premultiplied by 1*10^0...1*10^19.

Saturday, April 03, 2004 5:58 PM by temptress

# re: FunHi.com, a dating service, a new game, a bit over the top?

been a member for bout 4 weeks now. honeslty never had more fun, and i hate the web and never had any fun on dating sites or rating sites. not sure what makes it so much fun but it is!
Saturday, April 03, 2004 6:11 PM by Justin Rogers

# re: FunHi.com, a dating service, a new game, a bit over the top?

I'm definitely looking for feedback here, so I'll make some quick observations.

1. I'm interested in why it is so fun for those that do join. I know it is probably hard to explain, but if you can, I'm willing to hear you out in as much depth as you wish to use.

2. Embarassed to admit you are part of the site? I'm impartial, so feel free to contact me privately if you think you have important comments that would help better define exactly what this site is and why it appeals to people.

3. Note, I signed up just today. However, a short disclaimer. I've signed up on nearly every single dating site known to man. Not to use them, but to figure out more about the services they offer. I'm always researching various web ideas looking for that next exciting niche (unfortunately I'm a researcher and I give all my findings away for free, but that is the fun for me).

4. HotOrNot.com was the last site to do something similar and it was definitely *hot* for it's time. Is FunHi the next logical step drawing research from HotOrNot?
Monday, April 05, 2004 6:01 AM by Jon Hanna

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Arrays are slow? Is that a C# thing? In C and C++ we tend to use arrays for this kind of work to get a speed increase (constant arrays in static memory, creating them dynamically would indeed be super slow). In the case of going from a character stream to integers it wouldn't be that much of an increase (if you ignore the non-ASCII digits), but in other cases the difference is immense.

The table doesn't have to be an array, it could be a binary resource, though whether that's efficient or not in C# I couldn't say.
Monday, April 05, 2004 10:43 AM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Well think about it for a second. Array access in C# has to be safe, so they have to perform a bounds check (this can be optimized out in some cases). Then you have to do some memory manipulation to get the indexed location (multiplication of type size or stride by index?). Now compare this to popping a constant on the stack and doing a single multiplication. Anyway, I really loved your ideas. Here are the links where I talk about implementing both your ideas and some unsafe code using arrays. I just can't get anything faster.

http://weblogs.asp.net/justin_rogers/archive/2004/04/03/107177.aspx
http://weblogs.asp.net/justin_rogers/archive/2004/04/03/107200.aspx
Monday, April 05, 2004 12:21 PM by Matt Hawley

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

[Standing Up & Clapping] Very well said, and I completely agree.
Monday, April 05, 2004 12:57 PM by AndrewSeven

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

Why doesn't the enum itself support this?
enumName.Parse(string enumName) or .FromString


Most MS help suffers an amost unavoidable situation : The better you know the answer, the easier it is to find.

If you don't already know the answer, better Google it. Google is only concensus, not truth, so then you need to go back and try to understand some more.

http://weblogs.asp.net/andrewseven/articles/Ramblings.aspx



Monday, April 05, 2004 1:07 PM by Justin Rogers

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

I think the disconnect in the case of finding ways to parse strings to enums is probably that individuals think the enum is a language feature, and not backed by a system feature (System.Enum). I can definitely see that. Having the instance method as you point out would fix the problem since it would appear that the *language feature* had a *code feature* for parsing enumerations in some way.

I guess in the end, they wanted to leave enum semantics on the enum itself to the language (enum as a lightweight constant), and special enum processing outside of the language (System.Enum statics) so it didn't interfere with the basic operation of the language.

I guess enums are a mixed bag of nuts, but this happens all the time. Just today someone asked how to show a Form fullscreen in one of the newsgroups (tell me you can't get that information with a 5 second search), how to get a textual substring (bejeebus, they don't even look in the docs), and my pet peeve the illustrious "Does anyone have a comparision between managed and unmanaged DirectX", because we don't answer that one 50 times a week.
Monday, April 05, 2004 2:14 PM by Panos Theofanopoulos

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

> Has it been well answered? Yes it has,
I eval all those answers close to 5 (with 10 as the perfect).

I see nowhere an answer that doesnot use catch (thus very slow) and is case insesitive, something similar to double.TryParse
Monday, April 05, 2004 3:24 PM by Adam Hill

# re: A mention of C# in a mainstream, widely accepted game programming series book?

We should submit a few MDX articles next year :)
Tuesday, April 06, 2004 3:50 PM by Justin Rogers

# re: A mention of C# in a mainstream, widely accepted game programming series book?

They've never invited me into the process of article submission. However, I'd be more than happy to submit articles for both Game Programming Gems and AI Game Programming Wisdom.
Tuesday, April 06, 2004 8:44 PM by Funhi guy

# re: FunHi.com, a dating service, a new game, a bit over the top?

I think what puts the Fun in Funhi is the social networking that becomes very apparent once you start to get hookups and get to join some funhi vip forums. this all happens behind the scenes. there are many ways to communicate. I suppose some people might think they are going to get some gorgeous girl to fall in love with them by buying them a virtual 'diamond ring', however that happens on dating sites that offer no gift exchange. Yes, it is a game, I have inquired about other features, and when i first signed up, it said 'FUNHI BETA' I have been since notified that there are more additional gaming elements coming. maybe the idea of donations are what will drive the R&D for the next phase of this site? Another aspect of what makes it interesting and a time waster, is my curiousity. its almost a voyeuristic approach to seeing the next person that will signup. I have bought credits, which is the funhi currency, and actually returned gifts to people that for no apparent reason to me other than, i interested them in some way, bought me a gift. Afterwards, i noticed my status changed from being a mere "Playa" to a "VIP" wow.. heeh.. They have a moto that says 'Hater Free'. This probably makes people feel much safer knowing someone is actually looking out for thier well being.
Tuesday, April 06, 2004 10:50 PM by TrackBack

# Solutions to the Phraser programming challenge

Tuesday, April 06, 2004 10:50 PM by TrackBack

# Solutions to the Phraser programming challenge

Wednesday, April 07, 2004 10:44 AM by Ron

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

So far, my uses of Enum.Parse are in evaluating the selection of an item in a listbox previously populated with the enum's values via GetNames(). I don't expect a failure as the listbox is limited to just the enum's values.
Wednesday, April 07, 2004 3:31 PM by Panos Theofanopoulos

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

In my case it was the result of a refactoring of some big switches with string values. I decided that i will use enum(s) and then switch on them (after i succesfully parse them).
Wednesday, April 07, 2004 3:37 PM by Justin Rogers

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

C# refactors large switches with strings into internally managed hashtables for quick switch jumps. Did you do the refactoring for readability or for performance? Did you find the performance better on your refactored sample? If you did this for performance reasons, then do you parse strings into enumerations a lot? Basically, would any of the enhancements above be of use to you?

Also, if you could generalize your refactoring into a repro case that would be very awesome. If not, if you could provide numbers for how large the switch statement actually was, so we might examine what is more performance, the privately used hashtable or the overhead of the enum parse and switch.
Wednesday, April 07, 2004 4:42 PM by Panos Theofanopoulos

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

Too many questions !!!!

1) C# switch is not case insesitive, you have to do a ToUpper on the string before the switch.

2) Both

3) Yes

4) Yes

5) I already use reflection :(
to avoid the catch

6) i refactored switches that had from 10 up to 50 strings, but the factor parses/switches is not always 1/1 if that's what you are asking. the result of the parsing may stored and switched several times in my case.
Friday, April 09, 2004 12:16 AM by Chris Martin

# re: McWhat? McVeggie? Tastes good and has a low carb content. Can't be true, can it?

Now this is something that I'd thought I'd never see. A veggie recommending McDonalds. If I were still a veggie, I'd virtually smack you across the face. :)
Friday, April 09, 2004 12:47 AM by Justin Rogers

# re: McWhat? McVeggie? Tastes good and has a low carb content. Can't be true, can it?

Yves is virtually the best vegetarian food product company in the US. After making vegetarian food products for 60 years now, I think Mc D's did the *right thing* here and is actually selling a worthy product.

I understand the disgust. I tend to be a *puritan* vegetarian, eating very few of the meat-like foods. However, I'm going to make an exception in this case, probably about once a month.

I still need to figure out if the fries are truly Vegan... (check out the three period ellipses ;-)
Friday, April 09, 2004 2:39 AM by TrackBack

# re: Mutable reference types should not be read-only fields

Friday, April 09, 2004 3:39 AM by Shannon J Hager

# re: McWhat? McVeggie? Tastes good and has a low carb content. Can't be true, can it?

I began the path back to eating meat with sushi. It was as pure as you can get with meat. I have come to the conclusion that the measure of quality of food is the measure how how raw you can eat it. If a food can't be eaten raw, I have to say something is wrong with it.

But the major lesson I learned upon growing a love for sushi, which lead to a love for good, rare quality steaks, was that I was able to give up meat because my whole life I had eaten badly cooked pieces of mediocre-to-bad meat. I think I was 24 before I had a great steak. I actually thought that the stuff I grew up on was as good as it gets, boy was I wrong.

I still say that the major cause of vegetarians is bad cooking. I know people that still put steak sauce on steaks. If you have to put sauce on it, you cooked it wrong or it was a piece of meat that isn't really worth eating.
Friday, April 09, 2004 4:24 AM by TrackBack

# .NET Immutability Tip #1: Nothing is immutable.

Friday, April 09, 2004 11:12 AM by Henry Erich III

# re: Rob Rylea on autocompilation of XAML containing code within IE

I see this compilation issue as alot like Web Forms are done now. You have the "HTML" file and the "Code-Behind" file. Of course you must compile the code. But like now, you can always go and change the .aspx "HTML" without recompiling.
Did I just ramble too much?
Also correct me if I am wrong about being able to change the XAML "markup" file after compilation please. Right now that is just my theory because Im at work and cant test it =]
Friday, April 09, 2004 9:33 PM by TrackBack

# A fun lunch with a couple of Rotor fans

Sunday, April 11, 2004 6:28 AM by Frans Bouma

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Hardcoded ordinals are error-prone. You can however cache the ordinals: in the first record, grab the ordinals for each name, and store these in a table (hashtable or other). Every next row, you use the ordinals cached. You can implement even more faster constructs. (However a hashtable is pretty fast).

What's often forgotten though is that dr.IsDBNull() is very very very slow. Better do:
bool isNull = (dr[ordinal]==System.DBNull.Value);

Saves 20% of speed.
Sunday, April 11, 2004 6:37 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

My point for enumeration based ordinals was in production level applications. Once you've finalized a database interface, the error level of using enumerations to identify columns starts to get continuously smaller. Again, the point in case for the ASP .NET Forums system, of a 7% increase was HUGE.

Code spit automation programs take even more of the error metric away because you can use the results of a data reader to spit the relavent code in terms of enumeration for ordinal mapping, mapping of data to strongly typed object properties, and expanding out DBNull checks as you display above.

Honestly, saying it is error prone is like pointing out that the usage of pointers is error prone. While I agree highly that it CAN be, there is definitely a time to user pointers, and certainly a time to use enumeration backed ordinals for the data reader.
Sunday, April 11, 2004 7:10 AM by RichB

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

7% is an OK increase. It's not great though.

Prior to spending the time replacing the code with ordinals, I'd first look at the SQL I was using. It's quite likely I'd get 50% perf gains from tweaking the SQL. I would also look at internal algorithms - I once re-wrote a complex algorithm which reduced accuracy in the algorithm, but changed it from a O(n^2) to an O(n) algorithm, reducing calculation times from minutes down to seconds. Luckily the reduced accuracy wasn't important.

Only after considering all this would I then look at replacing ordinals.
Sunday, April 11, 2004 7:18 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Depends on the context of 7%. In this case the context of 7% was on the fully optimized SQL set used for the ASP .NET Forums system. This was after massive architectural changes to the database, index tuning, upgrading RAID hardware, etc...

The point here is that thousands of calls per second are being wasted on FieldNameLookup, period. The majority of people may not need real performance, others do. For those that do, or simply want to ship a leaner component, then enumeration backed ordinal lookups are simply that much faster and that much leaner. Can't beat a relatively free performance gain.

So no more assumptions here people. No more spending hours looking at your SQL, just spend 10 minutes and make a frigen enum ;-)
Sunday, April 11, 2004 7:40 AM by Plip

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Two words: Magic Numbers.

*Shudder* ;)

I do not want to maintain code that's littered with Ordinals, especially on Data Schema's I'm not currently familiar with.

http://www.programmingresearch.com/solutions/QAJDOCS/Patterns/no-magic-numbers.html

If you *MUST* user Magic Number please please please use Enums like Justin suggests, or use declared variables: -

int FieldNameThatMakesSense = 9;
string moo = DR[FieldNameThatMakesSense].ToString();

I wasn't aware of the performance loss though, thanks for that, I'll certainly be passing this information on to the rest of the Developers on my team.

And absolutely no need to apologise, it was bad code to make my point look prettier ;)
Sunday, April 11, 2004 7:45 AM by Plip

# re: Interesting article listing game genres, including Programming Games, Demos, Utiltities, etc...

What's scary is that there are games without Genre, they're mixes of several Genre's. I saw that when Gladiator: Sword of Vengence was pushed to Sony and Microsoft for sumission whilst I was at Acclaim.

None of us could agree on what Genre it was in.
Sunday, April 11, 2004 7:49 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Yep, magic numbers makes it sound so dirty. Definitely write yourself a tool that runs your query or SPROC and uses the results to create the enumeration. If you create all of your enumerations using an identical process you can even validate over time that people aren't screwing things up by changing column ordering. This is definitely a concept of performance versus resilience. Depending on your comfortability with your own setup, you'll pick one over the other.
Sunday, April 11, 2004 7:54 AM by Justin Rogers

# re: Interesting article listing game genres, including Programming Games, Demos, Utiltities, etc...

Yes, in those cases you have to classify each game or playability element as a separate genre for purposes of classifying the game. When you do that, you start to define a new genre, thus creating a Venn diagram of the intersections of many and all genres.

We had a thought about something like Pac Man. For the time it was a kind of action game, but would probably be considered a puzzle game today. What about Pong? I would classify that as a sports game, but others would classify it differently. The old games are interesting because each new game created it's own temporary genre since the playability, graphics and techniques were so new.

At the end of the day Andy Smith defined a game genre in a manner I find befitting. He noted that a game's genre is defined solely on the skills it takes to play the game. While we left the definition there, I'm thinking it would be nice to define the skills for each genre to see how well this stands up.
Sunday, April 11, 2004 8:14 AM by Plip

# re: Interesting article listing game genres, including Programming Games, Demos, Utiltities, etc...

Yeah it must be hard for people today though, the genre they decide to slot their game in to predetermines their audience.

I'm sorry I missed the chance to chat with you over the subject, sounds like it was a good night! :)
Sunday, April 11, 2004 9:55 AM by Patrick Santry

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

I like the use of enums for managability. Another reason for the use of ordinals was on one project I worked on where I had a search query, and then I wanted to bind an entirely different search to the same UI. Since all of the field names were referenced in the new query I had to write it so it would match up with the original query, for example:
Select thisfield As theOtherFieldName, thatField As thatOtherField, etc...

I had to do a lot of writing in my sproc to make it match up with what the code was expecting. Whereas by using ordinals, it wouldn't be a big deal and save me some time writing.
Sunday, April 11, 2004 10:35 AM by Jim Ross

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Hurray for Plip! Magic numbers, indeed. Interesting that this young guy from across the pond uses the same term I've been using (with equal scorn) for years.

Re: addressing via an enumeration. I always start out that way. In a typical project, once you've coded it with strings, it's going to be a rare thing if you have the time to go back and replace them. Not to mention the additional debugging cycle you introduce. Ugh!. Start out defining your enumeration, and code using it from the start. For me, it's a whole bunch easier to change an enumeration than embedded magic numbers. And when you're done, you're done.
Sunday, April 11, 2004 11:51 AM by Phil Winstanley

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

I still have a concern here that there is a definate contract between the DAL and the Stored Procedures, if a DBA goes in and moves a column they will break your code, most likly causing Conversion Exceptions in your DAL/BAL.

This isn't the case with Named columns, (mind you there is nothing to stop renaming a Column however it's more likly a new Column will be added to a select, if that goes at the beggining everything else is out by one ... I see gremlins on the horizon)
Sunday, April 11, 2004 12:04 PM by TrackBack

# some .NET blog tips

Sunday, April 11, 2004 3:10 PM by Dave Burke

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Justin, Thanks for pointing me in the direction of using Enums for column ordinals. Never thought of it, but will definitely pursue.
Sunday, April 11, 2004 5:15 PM by Nat Luengnaruemitchai

# Nahhhh dr[dataColumn] is faster

When you populate the data into dataset or whatever, you can assign the columns that you want to use in DataColumn variable and use it from there.... It is even faster than dr[intOrdinal]

DataColumn dcUserName = dt.Columns["userName"];

for(... ;... ;...)
{
blablabla....(dt.Rows[i][dcUserName]);
}

this way, you will achieve both performance and compatibility if you change the SQL statement in the future
Sunday, April 11, 2004 5:30 PM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

That would appear exceptionally intelligent if we weren't talking about the DataReader class. The DataReader class doesn't have a Rows collection, nor does it accept DataColumn as a default indexed property overload.

We are talking about some raw performance here, not the case where you are using the higher level APIs.
Sunday, April 11, 2004 10:28 PM by TrackBack

# Developing a Windows Forms Wizard...

Sunday, April 11, 2004 10:28 PM by TrackBack

# Developing a Windows Forms Wizard...

Sunday, April 11, 2004 11:30 PM by Jacob Morgan

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

I know we have beat O/R mappers to death around here, but this discussion really exemplifies their usefulness.

I don't use O/R mappers to avoid writing SQL. I don't use them to avoid writing domain objects.
I don't use them to avoid building database tables.

I use them to avoid writing hundreds of redundant assignment statements that lead to synchronization hell. Any O/R mapper worth a flip will give you ordinal based lookups based on the explicit list of queried fields. Optimizations such as this can counter some of the lost optimization of dynamic SQL.
Monday, April 12, 2004 5:08 AM by TrackBack

# re: IsDBNull() alternative

Monday, April 12, 2004 2:09 PM by Henry Erich III

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Excelent post and argument!

My Opinion:
Code Gen: Numeric Ordinals
Hand Gen: String Ordinal Lookup

My Opinion Also:
Enums or Constants would be a waste of time for readability gain.
Tuesday, April 13, 2004 12:14 AM by Adam Kinney

# re: Curious about the lives of up and coming game developers and their schooling?

No RSS :(
Tuesday, April 13, 2004 12:49 AM by John Winkle

# re: Curious about the lives of up and coming game developers and their schooling?

Yeah, eventually I will have to find somewhere with RSS but for now this will work. Once I get into a good job again I will find a nice hosting solution and get a good website with blog up and running.

John
Tuesday, April 13, 2004 4:41 AM by Justin Rogers

# re: Curious about the lives of up and coming game developers and their schooling?

Darnit John, you just had to pick something that didn't allow RSS feeds eh... I'll work on getting .Text up on Games4dotnet.com so we can just host the gaming stuff there.

Now we just need Tobi on so we can get some graphics done ;-)
Tuesday, April 13, 2004 5:09 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

The actual framework issue behind IsDBNull being as slow as it is.

http://weblogs.asp.net/justin_rogers/archive/2004/04/13/112100.aspx
Tuesday, April 13, 2004 5:27 AM by TrackBack

# IsDBNull

Justin points to Frans Bouma noting the performance hit using IsDBNull() in .NET and a workaround....
Tuesday, April 13, 2004 6:35 AM by Andy Smith

# re: Found a relatively accurate comparison of game world size to real world size. Games are small!

That's some crazy stuff, but I wonder how size is measured...
Do they measure by getting the size of a person in the world and compare that to use in our world? Or do they measure by walking speed in their world compared to average human walking speed in our world? The two different metrics could give vastly different results.
Tuesday, April 13, 2004 6:40 AM by Justin Rogers

# re: Found a relatively accurate comparison of game world size to real world size. Games are small!

Most games have published metrics. AC for example had published metrics as to how large zones were, how many zones, etc... Obviously the games that encompass actual real world arenas can easily be measured by simply mapping their bounds as compared to an actual map.
Tuesday, April 13, 2004 6:55 AM by kvr

# re: General texture mapping resources... Some good math explanations.

Don't forget this one (from Chris Hecker
of Game Developer Managazine)

http://www.d6.com/users/checker/misctech.htm
Tuesday, April 13, 2004 8:10 AM by TrackBack

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Tuesday, April 13, 2004 8:10 AM by TrackBack

# re: IsDBNull() alternative

Tuesday, April 13, 2004 9:50 AM by Brad More

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Might I suggest Convert.IsDBNull? Performs similarly to the alternative mentioned in one of your references and was graciously provided by our friends in Redmond just for this purpose.
Tuesday, April 13, 2004 12:23 PM by Drew Marsh

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Actually, the fastest code I can imagine would be:

reader[ordinal] is DbNull

Because there is only ever one instance of DbNull you don't need to check against Value, you ca just test the type. Plus, when you check against DbNull.Value you end up re-boxing whatever comes out of the reader to pass it to DbNull's Object::Equals override.
Tuesday, April 13, 2004 12:26 PM by Drew Marsh

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Sorry, small correction... you don't have to box because it comes out as Object already. However, I'd bet dimes to dollars that using "is" is faster than calling Equals.
Tuesday, April 13, 2004 2:42 PM by Shane O'Dell

# re: A slightly better WinForms wizard, and slightly more work.

One thing about adding panel navigation instead of a series of forms is that I don't see a good way to use the VS Designer to create the different panels. Any thoughts on this?
Tuesday, April 13, 2004 4:28 PM by Justin Rogers

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Convert.IsDBNull looks really good. Under the covers the only additional operation it performs is checking for an implementation of IConvertible and using that to possibly determine if the value is Empty. I'm inclined to worry about this extra goo.

The is keyword is backed by a not so fast crawling instruction isinst. isinst has to do a lot of work, so I could never imagine it being faster than the equality methods.

Be careful of small instructions that hide large details like walking down an object's derivation table.
Tuesday, April 13, 2004 4:37 PM by Justin Rogers

# re: A slightly better WinForms wizard, and slightly more work.

I was hoping someone would point this out. Form navigation does have the benefit of being designable, but at a pretty large cost. The size, location, and navigation UI all have to be replicated in some form or another. Of course you could use the reverse theme that each panel support a UI root where the navigation could be tossed into.

The biggest issue with form wizards is that you lose the forms on the back-end if the get closed. You see, it is easy to Close a form. They provide UI for it even. However, it isn't easy to Close a panel and so you are pretty much ensured that your panel with the controls that contain your data will be there.

That brings us back to designability. The only true designability would be to implement our own designer. That is for both Wizard Forms and the Panels. That might be something I get to in the article series. Another option is to write generic layout code that follows the design guidelines from MS for wizards and dialogs. Not even the WinForms designer does this for you, with it's goofy grid system. You'd have to manually type the stuff in or be a diligent drag and drop artist.

I think this layout designer might be just what is needed for the panels. As for the Form, you can design that in VS (I'll pop out a better version for the 4th article) if you'd like.
Tuesday, April 13, 2004 5:40 PM by Justin Rogers

# re: General texture mapping resources... Some good math explanations.

Yeah, that is a set of ancient articles. Good job pulling that link out of the swirling mass that is the web.
Tuesday, April 13, 2004 6:00 PM by Mickey Gousset

# re: Virtual sales now linked to grief playing? This argument always turns into a mud slinging contest.

You hit the nail right on the head with this one! Good Job!
Tuesday, April 13, 2004 9:34 PM by Roger Heim

# re: A slightly better WinForms wizard, and slightly more work.

Wouldn't UserControls give you the best of both worlds? You would get the benefit of designability and avoid the problem with forms being closed.
Tuesday, April 13, 2004 9:49 PM by Justin Rogers

# re: A slightly better WinForms wizard, and slightly more work.

Yep, with the current framework there isn't any reason you can't use Inherited Controls (to inherit from Panel). Not having time to test how easy this would be I chose not to throw it out there.

Since people really want designability, I'll mark that as my next article and talk about how to make this framework truly designable. Perhaps using UserControl's, perhaps using some other feature.

Wednesday, April 14, 2004 12:17 AM by Scott Mitchell

# re: [Rant] Getting rid of tele-marketers permanently, a case by base.

I like to play songs using the assorted tones of the touch-tone phone before hanging up. One time, though, a telemarketer called me back after I hung up, and mashed her hand into the keypad and uttered a few choice words herself. :)
Wednesday, April 14, 2004 2:08 AM by Justin Rogers

# re: [Rant] Getting rid of tele-marketers permanently, a case by base.

This begs a response from me. The legality of the callback in this case is quite questionable. Just the idea that they would be allowed this sort of option is outside my grasp. So much for my call being monitored for quality control ;-)
Thursday, April 15, 2004 3:22 AM by John Winkle

# re: Curious about the lives of up and coming game developers and their schooling?

Ok, got an RSS feed setup through Feedster.com for those interested.

John Winkle
Thursday, April 15, 2004 9:15 PM by TrackBack

# re: The SLAR on System.Array

Friday, April 16, 2004 3:32 AM by Tommy

# re: Performance: Different methods for testing string input for numeric values...

for(int i=text.Length-1; i>=0; i--) is faster than for(int i=0; i<=text.Length; i++) because it has to access the .Length-property only once. And in this case, the direction isn't really important.
Friday, April 16, 2004 6:08 PM by ChessMess

# re: Tips for noobs at E3! Based on a request by a reader.

Sweet! Thanks for the tips this is mundo helpfull!!

Some followup questions...

1. I've reserved a hotel near the airport, any possible problems with that?

2. Is there fee's to park at the convention center?

3. I plan on taking a digital camera, digital camcorder and laptop with me into the exhibits (I don't have workshops)... probably get a backpack to put them in. Any problems with that much stuff? Could I get 'online' with my laptop there to report back news and info?

4. How much swag does one actually get? lol ... do I need to take a duffle bag as well? :)

5. How close can you get to a booth babe before they hit you with pepper spray and start yelling for security?

6. Is it usually warm enough for shorts around that time or pants? What attire do people generally wear (Jeans vs Casual Dress)?

7. Parties... are there any or is everyone way to tired by the end of the day to do nothing more then hit the hotel and crash?

All I can think of at the moment... Thanks for the great info!

ChessMess
Friday, April 16, 2004 9:37 PM by Justin Rogers

# re: Tips for noobs at E3! Based on a request by a reader.

1. Shouldn't be. Make sure you can get a shuttle service to the convention center. A lot of places have them.

2. I don't recall a fee the last time I went, but it doesn't mean there wasn't one. Keep your info ready at all times. This is a trade show (not open to the public) and they will check it.

3. I've never gotten *on-line* at E3. I guess it is my chance to get away from all that stuff. However, as long as you don't mind carrying all of those items around, take em.

4. Depends on your character I'd assume. The first time I went I walked away with a ton of t-shirts, little toys, etc... Last time I went, I walked home with 1 t-shirt. I didn't *make the most* of the exhibitions last time. Dumb me.

5. As close as they let you get man! A lot of times you can get up close pictures, extra swag, and all kinds of bonuses from talking with the booth babes.

6. People wear all kinds of stuff. Most people try to wear the more stylish clothes, especially the exhibitioners. I've seen people wearing themes for their favorite games, suits, or casual. I'd say wear something comfortable. Hell, I went in wearing fuzzy slippers and my pajamas one time.

7. Again, up to your kind of personality. There are lots of impromptu dinners and such. You are definitely going to be in a party town. Make the most of the night-life there too if you want. I generally use the night after the last day and the next day for any hard partying.

I have to say this is kind of unorthodox for me to write about. The concept that people have questions about trade-shows is pretty strange to me. The first time I went was in 99 I believe and I literally just drove down and solo'ed the whole event walking around going to workshops and attending sessions. I got loads of swag the first time I went. Have to say it's been my most successful trip to E3.
Saturday, April 17, 2004 11:55 PM by Andy Smith

# re: Fast absolute, and statistical line counting algorithms for use with progress notification

great post justin. if you are going to add a toc, maybe you should make em linky? :)
Sunday, April 18, 2004 3:53 AM by Justin Rogers

# re: Fast absolute, and statistical line counting algorithms for use with progress notification

I tried doing linkies before, but the system doesn't like me and it renames a bunch of stuff between HTML and design view. Namely the hyperlinking system fully qualifies the path, which is erroneous.
Sunday, April 18, 2004 11:51 AM by TrackBack

# Early Morning News 3

Sunday, April 18, 2004 8:13 PM by TrackBack

# re: Efficiency of iteration over arrays?

Monday, April 19, 2004 11:59 PM by Lavos

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Interesting stuff. It's late to add this comment to the party, but I will anyway.

Using enumerations is great, but as was illustrated within the first month of my current job, someone at our provider inserted a column into a single resultset that broke a lot of our production applications that had code dependant on ordinals.

We can blame that on poor testing on our part but it still follows that DBAs and upgrades are going to break old applications that had been chugging along since forever if you depend on ordinals, and thus increases maintenance costs.

With that said, ordinals really do seem to help you out with speed :)


My personal preference if the gains were that important, would be to have a singleton class that more or less has fields/properties/whatever that I can reference by name and do runtime checks (via string names or whatever) to populate with the proper ordinals.

(Or, have all your enumerations defined in a seperate assembly that you can "upgrade" independantly to keep pace with the latest changes.)
Tuesday, April 20, 2004 12:57 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

It is equally easy to remove a single column from a result set as add one. Removing a single column would break any form of column indexing method, just adding one happens not to.

I'd argue that any competent DBA would create a new procedure before he'd add columns to an existing result set. If he does add columns why would he add it to the beginning of the column set or in the middle rather than append it to the end.

A small investment in user education for a 7-10% performance gain is well worth it. And since the same DBA could just have easily removed a column, I can't see a valuable reason behind not using the oridinals.

Proper versioning of enumerations is important in any case. Having singleton classes and levels of abstraction does nothing but hurt performance. In many cases the extra indirection of using properties is going to hurt perf a bit, as is doing runtime checks for proper ordinals. There are some solutions, but they all involve dynamically generating optimized code using a CodeDOM or Reflection.Emit and I don't think the majority of users are ready for that step (though I use it all of the time for my own stuff).
Tuesday, April 20, 2004 12:58 AM by JollyGreenGiant

# re: Making money in commercial games Part 2: The programmer.

Intersting aspect and so true. Like with anything that requires human creation, humans break it or make it better. I see the potential here for a little profit myself. (after all, I told my wife I was gonna make some money playing this thing 24 hours a day, I better start)

But, as with anything on this earth, man is not created to design permanent things.. but created to destroy himself and all his surroundings. Eq is just a hyper version of what is to come for all mankind. We will build it and destroy it very rapidly..

I would prefer to play eq on an automated bases, checking in with it daily and seeing my work in progress.. Perhaps i could then have time to give love and time to my family, while i still enjoy my sucess in the game.

Plese let me know of any programmers interested in making money selling their works and creations that would make eq soooo much funner for me and my family.
Thursday, April 22, 2004 2:39 AM by tracdalat

# re: Adding a design time dialog and creating a VS project sample

hey, I've follow these articles from the beginning. Thanks a lot, it's very helpful; and great (of course)
Thursday, April 22, 2004 5:11 PM by TrackBack

# Loopy Decisions

<p>In his blog, <span style="font-style: italic;">Better Living Through Software</span>, Joshua Allen looks at three different ways to loop in C# from an optimizing-for-performance point of view:</p><p><code>foreach (int i in foo) {}<br>
<br>
for (int i = 0; i &lt; foo.Length; i++) {}<br>
<br>
int len = foo.Length; for (int i = 0; ...
Saturday, April 24, 2004 10:17 PM by TrackBack

# Take Outs for 24 April 2004

Take Outs for 24 April 2004
Monday, April 26, 2004 8:27 AM by Justin Shen

# re: Code-Only: Winforms Wizard Series Article 5 (Managed C++)

the syntax of managed C++ is being refined, the new syntax looks more elegant.
Monday, April 26, 2004 8:27 AM by Justin Shen

# re: Code-Only: Winforms Wizard Series Article 5 (Managed C++)

the syntax of managed C++ is being refined, the new syntax looks more elegant.
Monday, April 26, 2004 8:36 AM by Ron

# re: For those using the Wizard Framework, do you want a solid framework, or solid articles?

I've primarily used it to help refactor a wizard I'd already written. I was already in the process of abstracting out a lot of the internal workings, but your articles pointed out a few things I was missing.

As for how I was using the articles, I've primarily built them as-is and then took the time to study what each part is doing. I could then add in features I wanted in a consistent manner. For example, I wanted the panels to trigger when the Next button is available (to require user input before proceeding).

I've enjoyed the articles. Thanks for writing the series.
Monday, April 26, 2004 8:37 AM by Justin Rogers

# re: Code-Only: Winforms Wizard Series Article 5 (Managed C++)

Don't know if I'd call it more elegant. I kind of like the new property syntax, but for the managed pointer changes (namely ^), I can't say that I like that at all. I could have made the above code much more *elegant* as well, but I chose for the quick port, and not necessarily the most elegant port.
Monday, April 26, 2004 10:08 PM by TrackBack

# Wizards in .NET with C#

Justin Rogers is the wizard man. He already wrote his sixth article in the series, I've just started with the first one, basically copying and pasting his code into VS.NET, and got a simplistic wizard app to work right away....
Tuesday, April 27, 2004 12:58 AM by Jesse Ezell

# re: I'm going to do something unfair to make a point: int.TryParse versus DWC.NumberUtilities.

Yah, too bad they left them out of the first release... which is really strange, considering that they remembered to put in double.tryparse...
Tuesday, April 27, 2004 10:13 AM by Daniel Jin

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

I've learned that optimization is actually performed to JIT away the bound checks. For example, in the following code

for( int i = 0; i < array.Length; i++ )
{
// do something with array[i]
}

JIT compiler does a pattern recognition on that and JIT away the bound check typically occurs on array[i]. I don't know how ArrayList is handled since there's a bound check on ArrayList indexer in addition to the underlying array. but I've learned now to trust that JITer will (or at least eventually will) do the right thing. as an outsider, you never know what JIT is designed to do, some manual optimization could actually be counter productive and hurt the performance.

about stack, queue, and list. theoratically, they should all be linked list based, not array based. I sure hope the queue is not implemented with an array, otherwise popping off the head is going to be really costly on every call.
Tuesday, April 27, 2004 10:17 AM by Raymond Chen

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

If access to the internal array were made public, then that would require List<T> (for example) to use an array forevermore. Do you really want to lock in the internals of List<T>? Suppose that somebody discovers that linked arrays are faster than a single array. Do you want to prevent them from using that new faster data structure?
Tuesday, April 27, 2004 12:56 PM by TrackBack

# re: The change from Hashtable to Dictionary

Tuesday, April 27, 2004 2:41 PM by Drew Marsh

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

I agree with Raymond and would point out that if your Decompose method returned the actual array, then it would have empty slots. Remember the array is grown dynamically, but it is grown according to an algorithm that will (hopefully) reduce the number of times that it must be grown. If you've added 75 items to the List<T> and it decided that it had to grow a few times, the internal array might actually be 200 items. How would you know where to stop if you got your hands on that array? Length would be 200, so you can't just go by that. If you're storing a reference type, sure you could stop when you find a null, but what if you're storing a value type such as Int32? Would you stop when you find Int32.default? Well what if 0 (which is Int32's default) is a valid value??
Tuesday, April 27, 2004 3:50 PM by Jerry Pisk

# re: Darnit, they break the Hashtable (err Dictionary) then ask us how we think they should fix it...

Why don't you keep using Hashtable if you don't like the templated Dictionary?
Tuesday, April 27, 2004 5:32 PM by Justin Rogers

# re: Darnit, they break the Hashtable (err Dictionary) then ask us how we think they should fix it...

Dictionary is quite a bit faster actually. So I'd much prefer to use it, avoid the boxing and casting costs, and still not have the exception.
Tuesday, April 27, 2004 5:43 PM by Justin Rogers

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

Drew: As mentioned in my post, the esoterics of length are up to you. The intelligent Decompose could have an out parameter that stores your length for you, or you could store the length before you do a Decompose (doing this is more dangerous since the length can change).

Raymond: Very interesting to point out the performance there. Of course a linked array would be exceptionally slower on indexing (one extra level of indirection), on CopyTo, marginally slower on enumeration. So yes, I'd love to lock them into using a single array. The only thing linked arrays does fix is the grow factor. So rather than guesstimate that they might make List<T> faster, go ahead and make a LinkedArrayList<T> that uses the alternate storage method and depending on the user's scenario, they'll make use of the appropriate class. Don't take away my options by limiting me.

Daniel Jin: I guess Joshua Allen was correct. The JIT optimization is for array's because Length can't change, not for collections where Count (not Length) can change behind the scenes). You can't just JIT optimize away all bounds checks my friend, else they wouldn't have put them there in the first place.

As for Stack, it is best implemented using an array, since it grows from index 0 up. As for Queue, it is also best implemented using an array with head and foot pointer indices into the array. They may find it more appropriate to use a linked list for this guy in the future. So maybe, just maybe this one could be taken off the list
Tuesday, April 27, 2004 7:14 PM by Daniel Jin

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

> You can't just JIT optimize away all bounds checks my friend

you are correct. I meant to say *sometimes* bound check can be JITted away.

> As for Stack, it is best implemented using an array, since it grows from index 0 up ...

Raymond definitely made a more convincing argument on that.
Tuesday, April 27, 2004 9:56 PM by TrackBack

# Real Time Colorizing - Some initial thoughts

Wednesday, April 28, 2004 12:41 AM by TrackBack

# Take Outs for 27 April 2004

Take Outs for 27 April 2004
Wednesday, April 28, 2004 12:41 AM by TrackBack

# Take Outs for 27 April 2004

Take Outs for 27 April 2004
Wednesday, April 28, 2004 6:27 AM by David Levine

# re: Darnit, they break the Hashtable (err Dictionary) then ask us how we think they should fix it...

I like the idea of being able to choose between a dictionary that throws and one that returns a sentinel value. But I want to be able to set the value to use so that I can use the sentinel value that is most appropriate (even for references). A null value is not always appropriate.

I haven't looked at the new language features yet so I don't know what Nullable<t> does - this may be more appropriate.

Wednesday, April 28, 2004 10:19 AM by Laurent

# re: Allegiance Source Code vs Quake II, you decide...

Why not to publish some pieces of your work already ? I am sure a lot of people would love to give you a hand.
Friday, April 30, 2004 4:13 PM by Chris Stueck

# re: DWC.Algorithms.NumberUtilities

How do I compile this under visual studio.net??I have not built a library yet.

Thanks.
Friday, April 30, 2004 7:49 PM by Justin Rogers

# re: DWC.Algorithms.NumberUtilities

You'd create what I think VS calls a new Class Library project and copy the above code into it's own .CS file and compile. It should just work from there.

If you want to include the source directly in your project (aka not as a library), you can do that also. Just make the above source its own source file and then compile your application as normal.
Saturday, May 01, 2004 12:14 AM by Daniel O'Connell

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Out of curiosity...what are the results you are seeing when you run this with int keys? I'm seeing Nullable win when the key is a string and integral win when the key is an int. It seems a strange result to me.
Saturday, May 01, 2004 12:59 AM by Justin Rogers

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Below are my results given varying input to the Dictionary. Using both String as a key and Int as a key shows that Nullable always wins still.

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.7310512

Start Dictionary<String, Int> Dictionary
Int Dictionary Complete: 00:00:00.9713968, 0

Start Dictionary<String, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.6509360, 0

Start Dictionary<Int, Int> Dictionary
Int Dictionary Complete: 00:00:00.4105904, 0

Start Dictionary<Int, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.2804032, 0

Using TryGetValue
Found: 50
Not Found: 11000001
Saturday, May 01, 2004 2:06 AM by Daniel O'Connell

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Strange, the result is still the opposite on this system. I wonder which is the odd ball.
this is my results for Int keys:

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.0156250

Start Int Dictionary
Int Dictionary Complete: 00:00:00.2343750, 0

Start Nullable<Int> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.3437500, 0

Using TryGetValue
Found: 50
Not Found: 11000001


and my results for string:

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.9062500

Start Int Dictionary
Int Dictionary Complete: 00:00:01.2656250, 0

Start Nullable<Int> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.6250000, 0

Using TryGetValue
Found: 50
Not Found: 11000001


What are you compiling with? I am doing just csc /optimize+, perhaps there is some minor difference in that?

Also, there are some interesting differences there...my speed for int dictionaries outstripes yours for the most part, however my results for string dictionaries shows your results as faster. Could this perhaps be the result of cpu specific JITing?

These results are rather stable here, I've run both multiple times...this is curious indeed.

Saturday, May 01, 2004 3:12 AM by TrackBack

# re: The change from Hashtable to Dictionary

Saturday, May 01, 2004 4:28 AM by Justin Rogers

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

The only thing I can think is that we aren't running the same test code. I've double checked mine just to make sure that I wasn't doing anything stupid. To humor me, please run the code at:

http://weblogs.asp.net/justin_rogers/articles/124403.aspx

If you are still getting the same results after running the test there then let me know. Since you coded up your int keying test separately (as I didn't supply the code for an int key above), there must be something we are doing that is different. Thanks.
Saturday, May 01, 2004 4:48 AM by Justin Rogers

# re: Nullable Versus Integral Data Types (C# 2.0 Test Code)

Running this code a few more times Dictionary<int, int> and Dictionary<int, Nullable<int>> are definitely close in performance. They've actually traded off in some tests putting one or the other at the faster speed depending on code layout within the class file (aka the JIT is having a huge impact, and is very inconsistent, so we'll have to wait for a later release for better numbers)

However, the Dictionary<String, int> simply performs poorly and I don't think there is any good reason for it. There isn't any casting, or boxing, or anything in the IL that would make it intrinsically different from the Nullable version and the Nullable version is actually doing extra work to convert the integer value into a Nullable<int>. The extra indirection through the op_Implicit call should actually make it slower.

When I started this I expected to examine how much worse the Nullable version was, and I was expecting it was only going to be a small marginal amount making the use of Nullable types acceptable. I didn't expect to find Nullable to be faster, nor should it be under any circumstances faster than the integral type.
Saturday, May 01, 2004 7:28 AM by TrackBack

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Saturday, May 01, 2004 3:01 PM by Bill Meyers

# re: Fast absolute, and statistical line counting algorithms for use with progress notification

fuck
Saturday, May 01, 2004 4:54 PM by Daniel O'Connell

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Well, my code for the int test was basically the same asyours, just with type changes(I took your code, changed stringsForKeys to an int[] and took out i.ToString() for the most part). My results from your posted code, however still show the same thing:

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.9375000

Start Dictionary<String, Int> Dictionary
Int Dictionary Complete: 00:00:01.2343750, 0

Start Dictionary<String, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.7812500, 0

Start Dictionary<Int, Int> Dictionary
Int Dictionary Complete: 00:00:00.3437500, 0

Start Dictionary<Int, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.4062500, 0

Running it several times is showing a fair amount of change on Dictionary<String,Int> sometimes its as much as 1.2 seconds, sometimes as low as .90.

Other tests I've run have shown string, int to take upwards of 12 seconds when using the Add method...its very strange indeed. This must be a manifestation of the JIT right now.

Out of curiosity, what kind of hardware are you running on?

Intel\AMD?

I'm running P4's here...
Saturday, May 01, 2004 7:31 PM by Justin Rogers

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

I ran that on an Intel P4 2.8. I added some comments to the actual test case code that explain my findings after running the tests many times. It does appear that the JITer is very unstable since code reordering, addition of code, changing reference locality (aka, adding a usage of all four of the dictionary's and the string table to AFTER the tests are complete to ensure none of them are GC'ed while the tests are running).
Monday, May 03, 2004 8:38 PM by darrenford

# re: Software Design and Home Depot, who would have thought they went together so well.

Have you ever heard of paper? They sell it at Wal-Mart.
Tuesday, May 04, 2004 2:37 AM by Justin Rogers

# re: Software Design and Home Depot, who would have thought they went together so well.

I went the paper route for a while. I bought 50 notebooks at 10 cents a piece from Target during a going back to school sale or some other similar event. I figured I'd scope each concept in it's own notebook and/or group similar concepts into the same book. Drawing on paper isn't easy. You either use pen (and it shows up well and becomes unchangeable) or you use pencil (which shows up like butt, makes the paper dirty and still doesn't erase very easily).

You could say just throw it out if it isn't good enough and start over. I'd rather not take ten pages to spec something out that I could do on one with the ability to use multiple colors, easily erase and restart portions, the abilities of whiteboard are endless.

You tend to get a small high from working in close proximity with the markers for a long period of time. Some people get headaches, I just feel a bit better and keep on writing ;-)
Tuesday, May 04, 2004 4:56 AM by Joel B.

# re: People are confused by ApplicationContext in Windows Forms, but there really isn't any magic happening.

The message pump started by Application.Run must have a mechanism for telling the OS "hey gimme just these messages -- I don't want those other ones", and a mechanism for dispatching them (i.e. code must have a way to register event handlers with the message pump).

Well, it is unclear to me how both mechanisms are implemented.
More specifically, I would like to understand how things work and why there is an Application.AddMessageFilter() method but no way to register an EventHandler

Anyone who knows, or can suggest a book that explains the mistery, could just drop me an e-mail joeblk50@microsoft.com
Thx
Joel
Tuesday, May 04, 2004 5:02 AM by nfactorial

# re: If you could pick the reason why GDI should be .NET accessible, what would it be?

I haveto say my number one reason for GDI access is speed. GDI+ is suitable for loads of cases and provides really nice output. But the old GDI kicks its butt in speed terms (due to the hardware acceleration).

ie. A recent test app I wrote, I needed to draw a background in the window to make it look like graph paper. All those lines hit GDI+ badly (even though they were simple, horizontal & vertical lines). I wrote a aimple wrapper for GDI and used that for the graph background and the speed increased dramatically.

Of course, this probably wont apply when Longhorn's available. But for now its my number one reason, when you need it to be fast and you're not bothered about the quality the old GDI rocks :)

Though I do agree your reason is perfectly valid to me too :)

n!
Tuesday, May 04, 2004 5:08 AM by Dave L

# re: .NET Immutability Tip #3: Protect your properties AND your methods.

It's an interesting pattern. Q: Why wouldn't you want it to signal callers that the object is immutable when calls are made to change its state?

I suppose you can have different categories of immutable objects, ones where the caller does not care if an operation fails and others where an inability to change its state is a "don't care", but it's difficult for me to come up with use cases where the calling code really doesn't care about the success/failure of the operation. Where would you use this?

For the pattern where the caller does care the immutable object could signal the results by throwing an exception, which is non-performant, but there aren't any good alternatives when setting property values. If performance is an issue then the caller should check the immutable flag prior to the call, but that adds overhead too.

I prefer to avoid directly accessing field1 and the flag just to avoid the perf hit of invoking the property setter before the access. Unless you are in a tight loop invoking it thousands of times per second the overhead is unlikely to be a big perf hit, and the downside is that you now have lots of places that have to be touched if changes are made to the property access. This has the feel of a false saving.


Tuesday, May 04, 2004 5:08 AM by nfactorial

# re: If you could pick the reason why GDI should be .NET accessible, what would it be?

Oh, I may have misread you. I thought you were suggesting there *should* be a System.Drawing.GDI namespace. But seeing one of your later future posts it looks like you're saying there *is* a System.Drawing.GDI namespace :) That's cool.

Sorry, I don't have access to Whidbey yet. But I've now learned to read all your new posts before I start writing replies :)

n!
Tuesday, May 04, 2004 5:17 AM by Franck

# re: Allegiance Source Code vs Quake II, you decide...

I hope you don't really expect to blow the "old crappy OGL" just by using MDX... I'm a DX user usually but your statement is plain stupid. At first you should start by hoping you can come somewhere not too far from OGL performances... when will people stop feeding the $ beast?
Tuesday, May 04, 2004 6:31 AM by Justin Rogers

# re: .NET Immutability Tip #3: Protect your properties AND your methods.

Immutability patterns come in two forms. The first is that you have a single code piece capable of updating some data, and a bunch of others that have access to it. Think about a gaming scenario where I want to share the game state between all of the AI and other code.

That code can't change the state or it would
break the game, and it shouldn't. Nor should it try to change the state.

The second pattern is that a type is mutable for some time where many code points have access to changing it, and then it becomes mutable. In this case once the object becomes immutable your algorithms should stop trying to change the value. Maybe in your case though the fact that they do try to change the value is an exceptional case and something you need to find through debugging. Rather than toss exceptions you might find that you'd rather use asserts instead.

As for knowing an object is immutable, you can always extend a read-only property that returns the immutable flag. Then code can always find out whether or not the object is mutable and you no longer need to rely on the exception.

In general, I think most programmers are to used to owning all of their code. In most cases you are going to share a lot of code, and the mutability is for protection against many sources changing data when they shouldn't. Having an exception in this case could ruin a real-time simulation. After all, once the state is complete, the user shouldn't be changing it anyway.

I'll be covering some extended scenarios soon. MS tends to adopt the private derived encapsulating class (aka ArrayList.ReadOnlyArrayList which derives from ArrayList, but is private and can only be instantiated by calling methods on ArrayList).
Tuesday, May 04, 2004 6:40 AM by Justin Rogers

# re: People are confused by ApplicationContext in Windows Forms, but there really isn't any magic happening.

You need to pull out a book on the Windows message pump. The OS really doesn't have a mechanism for saying you only want certain messages, you build that into your message pump, and pump those you don't want down to DefWndProc.

As for Dispatching, everything is based on windows handles. Controls get messages because messages are sent to the HWND. UnsafeNativeMethods.DispatchMessageW(ref msg); is responsible for this process.

As for filtering, you can always filter messages by adding a message filter on Application.AddMessageFilter. You pass it an object that implements IMessageFilter which contains the PreFilterMessage method. You can process messages here. If you want to add event based notifications, you can stack those on top of PreFilterMessage by simply returning false (that way the message is dispatched as normal), and then firing an asynchronous event massing out the Message structure.
Tuesday, May 04, 2004 6:43 AM by Justin Rogers

# re: If you could pick the reason why GDI should be .NET accessible, what would it be?

Speed is the reason I'm assuming they added it. If you look deeply into V1.1 WinForms code you'll find that at some point they traverse the GDI+ to GDI border in order to render over the screen. GDI+ is great for rendering high detail components, but works even better when you cache the results and store the final result in a native GDI format for updating the display whenver you need to refresh.

My only major issues are that the GDI namespace is lacking a great deal of features, and rather than demonstrating GDI functionality, it is actual mirroring GDI+ functionality in terms of method names and feature set. This is pretty bogus since the two APIs have different problem sets they solve.
Tuesday, May 04, 2004 11:19 AM by Ron

# re: I keep telling everyone WinForms printing gets easier in Whidbey, but that doesn't help them today...

>Maybe it is time to make a larger investment in Crystal Reports?

Or maybe you could check out Active Reports. http://www.datadynamics.com/
Tuesday, May 04, 2004 12:18 PM by Marty

# re: Software Design and Home Depot, who would have thought they went together so well.

My office has a whole wall (14' X 8') filled with the stuff from Home Depot. I am like you there is nothing like being able to draw and draw without having to crumple up paper and restart.

I then use my digital camera to take a picture of what I want to keep before erasing.

Wednesday, May 05, 2004 7:27 AM by Shannon J Hager

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

Looks like another "who wants to be the lowest bidder" site. Guru.com was like that, there was always someone more desperate than you or some kid in highschool that didn't mind working for minimum wage in his evenings.
Wednesday, May 05, 2004 7:40 AM by Justin Rogers

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

I'll find out soon enough. You have to remember the high school kid probably won't get the work done and get kicked off the site early on. Hopefully I'll get an opportunity to do some of the site work and can find out a bit more about it.
Wednesday, May 05, 2004 12:09 PM by Antwerp_Diamond

# re: FunHi.com, a dating service, a new game, a bit over the top?

What are the barriers for a neat marketing ploy, surrounded by competition?
Wednesday, May 05, 2004 4:15 PM by darrenford

# re: Software Design and Home Depot, who would have thought they went together so well.

Are you using the 1/8" hardboard with the smooth white coating that they also sell in 4X8 sheets ?
Wednesday, May 05, 2004 11:19 PM by Justin Rogers

# re: Software Design and Home Depot, who would have thought they went together so well.

It might come in larger sheets at other Home Depot's, since I think I might have seen it before in Bellevue in larger sheets. But yes, 1/8" HB MB.
Thursday, May 06, 2004 2:20 AM by Claus Konrad

# re: Plug-in Framework (Part1): Marking Types for Consumption

Hi Justin

What is happening to the rest of the articles?
We are looking forward to read the remaining 4 chapters of the Plugin-framework articles series...

Thursday, May 06, 2004 8:57 AM by Scott

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

Actually Shannon-- we have a fair bidding system, unlike our competitors, you cannot post a bid below 75% of the average bid rate for a project with 3 bids. So if the project has 3 bids averaging $1000, you cannot submit a bid below 750. We try to keep everything fair unlike the other sites! (I know from personal experience, I hate the other sites because you get undercut at the last minute and lose a 2500 job to someone who bids $500)
Thursday, May 06, 2004 12:12 PM by Shannon J Hager

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

That seems like a good idea. 3 bids at $1000 each, then 1 at $750, the next at $704, the next at $670, the next at $640, eventually it would pretty much bottom out, even moreso if someone actually dared to bid higher than average.

I never had last minute bids cut me off, personally, I just had competition from people with lower cost-of-living (or lower self-worth). The projects always seemed to be looking for lowest bidder and, in my experience, there is always another lower bidder.
Thursday, May 06, 2004 1:29 PM by darrenford

# re: Software Design and Home Depot, who would have thought they went together so well.

When you get to the point of covering your wall, how do you plan to handle the seams? I wish I had asked back when Marty was watching since it sounds like he already has dealt with that.
Thursday, May 06, 2004 2:23 PM by Scott

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

Well it's difficult from my position to be strict in bidding and appeal to the bidders, while still appealing to the job-posters that are looking for affordable, quality labor.

Personally I always go with the guy that has the best resume and feedback/stats (if available). I've posted a couple of my own personal jobs on the site and am about to choose 2 winning bids. I look for best resume, not cheapest bid, and I know a lot of job-posters are similar.

Any way you slice it, it's a fight to get work and stay employed (in the freelance world), so you just gotta keep at it and try hard to build relationships. If you do happen to sign up I'd love to hear your feedback on bidding and using the system. It's my customers' feedback that is going to shape the development path for the site.

Thanks,
Scott
Thursday, May 06, 2004 8:29 PM by Justin Rogers

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

Just for laughs. Note the Stream uses Stream.SynchronousAsyncResult for its IAsyncResult implementation. Isn't that funny. Even more funny is it's implementation of CompletedSynchronously:

IL_0000: ldc.i4.1
IL_0001: ret

In other words, it ALWAYS completes synchronously.

Damn, I just thoroughly read the docs (V1.0 docs on this machine, I'll check the V1.1 docs later) and everything in there seems to IMPLY that the methods really are asynchronous. They do state that the asynchronous versions are implemented on top of the Read/Write synchronous versions and that if you replace the Read/Write methods then things will still work for you. That's cool, IF the BeginRead/BeginWrite methods were actually doing asynchronous by loading a ThreadPool item.

They even have recommendations for multiple simultaneous asynchronous reads, and I quote "Multiple simultaneous asynchronous requests render the request completion order uncertain." Based on the fact that the implementation is SYNCHRONOUS, I don't think that will be a problem.
Thursday, May 06, 2004 9:28 PM by Chris

# re: Tips for noobs at E3! Based on a request by a reader.

Is there fee's to park at the convention center?

$10 - $15 depending on where you park.

7. Parties... are there any or is everyone way to tired by the end of the day to do nothing more then hit the hotel and crash?

Plenty... but your not invited. The big parties are usually reserved for celebrities and such.
Friday, May 07, 2004 9:48 AM by Steve

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

That is a real stinker... It sort of defeats the purpose of even using it to begin with.
Saturday, May 08, 2004 1:38 AM by Hmnt(Sayona_Hmnt@hotmail.com)

# re: Performance: If vs Switch in what we might call an extended usage scenario?

http://www.dotnet247.com/247reference/msgs/35/178987.aspx
Saturday, May 08, 2004 2:34 PM by TrackBack

# .NET Links

Well, I have a ton of links built up that I'd like to post, so instead of posting them individually, I'll just throw them all at you at once. Enjoy. :) I'm not sure where I saw it first, but...
Saturday, May 08, 2004 2:34 PM by TrackBack

# .NET Links

Well, I have a ton of links built up that I'd like to post, so instead of posting them individually, I'll just throw them all at you at once. Enjoy. :) I'm not sure where I saw it first, but...
Monday, May 10, 2004 6:21 AM by Joel B.

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

I find intriguing using message filtering to debug applications.
Now, if only there were a simple matrix (i.e. multidim multitype array) where I could just use m.Msg as an index, getting 1) a string with the message name 2) a proper type that I can pass to GetILParam to extract proper LParam.......things would be easier.
Anything to suggest?
How have you implemented message debugging in C#?

Joel
Monday, May 10, 2004 10:15 AM by Justin Rogers

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

Yep, I've done some message debugging using the methods I've detailed above. Mainly for synchronicity issues where I need ordering of messages to determine why my component isn't working the way that I think it should.

Casting between the m.Msg index and a set grouping of stuff isn't easy. For many messages the ID is static, however, a user can re-define or reallocate any value in the user range to do whatever they want.

Roving id's can be created by using RegisterWindowMessage so that each time a program launches there is the possibility of getting a different ID for the same message.

To answer any questions on how I debug things, I quick write the Message structure down to a binary file. I worry about name mapping later when I'm checking the message ordering. For more advanced debugging (delving into the structure), I use my PreFilterMessage handler as the place where I put the breakpoints. Since you don't have code for the Winforms user controls, it isn't easy to set code backed breakpoints inside of their WndProc, so breaking the debugger at your message filter is super handy. Then you can hand cast any of the parms to user defined structures in the debugger to investigate your results.
Monday, May 10, 2004 8:42 PM by Alexei

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

There are two parts to this problem one is your side the other one is on Stream class.

1) The Stream class problem is that it should be issuing a blocking call in BeginRead().

2) You problem is that on every BeginXXX call you should be checking ar.CompletedSynchronously and if so proceed on the same thread. In fact, sync completions would give you perf improvement since an async completion needs to flow execution context onto a new thread.


IAsyncResult ar = BeginDoSomething(... _MyCallback, myState);
if (!ar.CompletedSynchronously)
return;
ProcessSomething(true);

void MyCallback(IAsyncResult ar)
{
if (ar.CompletedSynchronously)
return;
ProcessSomething(false);
}

void ProcessSomething(bool onMainThread)
{
try {
EndDoSomething(ar);
}
catch (Exception e)
{
if (IsNotExpectedException())
throw; // This may and should bring your process down.

if (onMainThread)
throw; // Something on the main thread may want to handle it.

//
// If something on the main thread could handle an exception call it here.
//
HandleException();
return;
}
HandleSuccess();
}
Tuesday, May 11, 2004 6:11 AM by Joel B

# re: I've done a little experimenting and......

First of all, as you all are aware, I'm very inexperienced. This is a sort of disclaimer...

I wrote a simple Windows Forms application, to test how I could use Application.AddMessageFilter();

1) I went through winuser.h and a few other headers. I built a hashtable where the key is the msg number and the value is a string (WM_xxxxx), so I can easily understand what is going on.

2) I implemented a message filter as described, adding each message to a collection holding msg.HWnd plus the message description if a match is found in the hashtable, otherwise just the message number when the value is unknown (to me)

3) Surprise!
Very few messages have a known value. They fall into just 3 categories: mouse events including button clicks, keyboard events, WM_PAINT (only if not an indirect cause of a mouse movement). Then there are a bunch of events with value 0xc0xx (which are supposed to be private and I have no idea about how to investigate them).
Sometimes there is also a message with value 0x118 (this value falls within the system messages range, however I could not find any reference to it anywhere so I have no idea of what that means -- Anyone has a clue?)

4)I then decided to implement a void WndPro() override.
This time my collection gets all the messages that I expected to catch. It behaves pretty much like I was expecting from using Application.AddMessageFilter()


Any idea about why Application.AddMessageFilter() behaves that way? Could it be still useful in any way?
Tuesday, May 11, 2004 11:11 AM by Justin Rogers

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

Joel, I'm going to try and get around to a more broad code-path examination of this today with some sample code, since I think that will help you out quite a bit. Stay tuned!
Friday, May 14, 2004 12:05 PM by Lorenzo Barbieri

# re: Wanting to fix all my past mistakes, here is an updated ImageFast library that doesn't rely on System.Drawing for init...

The link only works in the HTML page, not in the RSS feed, it's missing the http://weblogs.asp.net/justin_rogers part...
Friday, May 14, 2004 3:56 PM by TrackBack

# Wanting to fix all my past mistakes, here is an updated ImageFast library that doesn't rely on System.Drawing for init...

Sunday, May 16, 2004 12:33 PM by Sacha De Vos

# re: WinForms Wizard Series Article 6, a quick look at implementing a meta-property storage class.

I can't wait to see the next article ! Thanks for the very good ideas ... First time I see a working real life example of the model/controller/UI approach.
I had some troubles adapting this wizard so that it uses a modal dialog within an existing application though. Anyways this saved me a whole bunch of work. thanks again...
Monday, May 17, 2004 6:33 AM by Darren Neimke

# re: BasicParse with State/Transition graphing, a much more robust parser.

Very nice Justin - do you have a full code dump of this stuff yet or will you post something when you have it completed?

Also, what language is this parser built for? I presume that it is built for a subset of a C-like language?
Monday, May 17, 2004 8:26 AM by TrackBack

# NullReferenceException = LoadFromFile( ... ) ;

Monday, May 17, 2004 1:54 PM by William Stacey

# re: BasicParse with State/Transition graphing, a much more robust parser.

Very good Justin. I am still studying it but looks like your really moving on it. Cheers!
Monday, May 17, 2004 2:49 PM by Ron

# re: BasicParse with State/Transition graphing, a much more robust parser.

Jason,

A comment on the switch in ConvertTokenToType.

Should the switch on token.TokenData evaluate case " ": (space)? Didn't the StringToTokens method toss the spaces? If you're coding for the possibility someone called StringToTokens without tossing anything, shouldn't you also test for tabs, carriage returns, and line feeds to mark them as white space?
Monday, May 17, 2004 2:58 PM by Justin Rogers

# re: BasicParse with State/Transition graphing, a much more robust parser.

[Darren] I have a full code dump for this. I'll get it up shortly.

[Ron] The Lexer and Parser are two different modules. My switch statement is not more complete because my input to the lexer is always explicit. The easy override in the lexer that just takes a source input is only there as a demonstration of how you might use it. The switch statement for that case would be much different.
Monday, May 17, 2004 3:44 PM by Nat

# re: Using PInvoke with GDI+ just isn't an easy thing to do. System.Drawing.dll must have some *magic* I don't know about.

using System;
using System.Runtime.InteropServices;
using System.Threading;

[StructLayout(LayoutKind.Sequential)]
struct StartupInput
{
// Fields
public int GdiplusVersion;
public IntPtr DebugEventCallback;
public bool SuppressBackgroundThread;
public bool SuppressExternalCodecs;

// Methods
public static StartupInput GetDefault()
{
StartupInput s = new StartupInput();
s.GdiplusVersion = 1;
s.SuppressBackgroundThread = false;
s.SuppressExternalCodecs = false;
return s;
}

}

[StructLayout(LayoutKind.Sequential)]
struct StartupOutput
{
// Fields
public IntPtr hook;
public IntPtr unhook;

}

public class LockImageAndLoad
{
[DllImport("gdiplus.dll", CharSet=CharSet.Unicode)]
public static extern int GdipLoadImageFromFile(string filename, out IntPtr image);

[DllImport("gdiplus.dll", CharSet=CharSet.Unicode, ExactSpelling=true)]
private static extern int GdiplusStartup(out IntPtr token, ref StartupInput input, out StartupOutput output);

[DllImport("gdiplus.dll", CharSet=CharSet.Unicode, ExactSpelling=true)]
private static extern void GdiplusShutdown(IntPtr token);

private static void Main(string[] args)
{
IntPtr ptr1;
StartupOutput output;
StartupInput input = StartupInput.GetDefault();
GdiplusStartup(out ptr1, ref input, out output);
try
{
IntPtr pimg = IntPtr.Zero;
GdipLoadImageFromFile(@"c:\windows\Rhododendron.bmp", out pimg);
}
finally
{
GdiplusShutdown(ptr1);
}

}
}
Monday, May 17, 2004 6:05 PM by Justin Rogers

# re: Using PInvoke with GDI+ just isn't an easy thing to do. System.Drawing.dll must have some *magic* I don't know about.

Yeah, I quickly figured that out and posted the more complete ImageFast library.

http://weblogs.asp.net/justin_rogers/articles/131704.aspx
Monday, May 17, 2004 9:07 PM by TrackBack

# re: Using PInvoke with GDI just isn't an easy thing to do. System.Drawing.dll must have some *magic* I don't know about.

Tuesday, May 18, 2004 5:01 AM by Justin Rogers

# re: Introducing symbol tables with a C# example using 'get' as the point of interest.

To shed some light on the method C# uses to process the get accessor of a property I offer up the following:

1. It appears all of C#'s symbol resolution happens after the parse tree has been created. They do have a name manager in their lexer/parser, but I think it is more to save space and resolve keywords than any other purpose.

2. For the get accessor they use a special entry in this name manager, but when the lexical token for get is returned it is returned as an identifier no matter what the scope. So they don't use a symbol table for conditional processing during the lexer/parser phase.
Tuesday, May 18, 2004 8:24 AM by Darren Neimke

# re: Introducing symbol tables with a C# example using 'get' as the point of interest.

Most of this stuff went whooooshing over my head because I haven't really done anything on symbol tables before - they are a wierd beast!

I was going to ask you how you would add keywords into your BasicParse language. Up until now you have only parsed single char tokens and strings.

I was wondering what would happen to the lexer if you add a keyword such as "var" to your grammar. Would you simply add the chars 'v', 'a', 'r' to your terminal lookup table such that it would now become:

public static Token[] StringToTokens(string tokenString) {
return StringToTokens(tokenString, " \n\r\t{}\"=;.()[],var", " \t\r\n");
}
Tuesday, May 18, 2004 7:38 PM by Darren Neimke

# re: Adding keywords to BasicParse, keeping it short and sweet.

Thanks Justin, that's really cool - I think that I'm beginning to *get* symbol tables now. Just one thing, I think that you have an error in your TransitionParseVar() method. Shouldn't the test:

if ( this.tokenStream.Current.TokenType == TokenType.Id )
...be written as:

if ( this.tokenStream.Current.TokenType == TokenType.Var )

Tuesday, May 18, 2004 9:42 PM by TrackBack

# re: Why can't I use the same variable as an inner loop does?

Tuesday, May 18, 2004 10:14 PM by Justin Rogers

# re: Adding keywords to BasicParse, keeping it short and sweet.

Nope, we've already Eaten that token. Remeber that every time control is passed back to the state loop, we advance by one token. That is why in the Open state I check for Var, and then return ParseState.ParseVar. By the time I enter the ParseVar transition method, I'm already on to the next token, which by all accounts should be Id.
Tuesday, May 18, 2004 10:46 PM by Kieu Huu Chien

# re: More InstallShield DevStudio 9 issues...

Installer
Wednesday, May 19, 2004 11:48 AM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

Very Cool Justin. Is this complete now for the bind language more steps to go? Cheers!
Wednesday, May 19, 2004 5:32 PM by Justin Rogers

# re: Adding keywords to BasicParse, keeping it short and sweet.

William, the bind language has a separate language definition from this language, though they do seem similar. I'll code up a parser for precisely the bind scheme this evening, I've kind of gotten off track a bit in writing about other areas of the compiler that are more interesting to me.

Things that need to be added in order for the bind scheme to work are:

1. TokenStream patching to allow include files (these are really cool and I'll talk about them since it allows processing of include files inline without preprocessing).
2. Changes to the language definition based on the way bind works. Bind has more structure than just a basic key/value system. They allow complex inline types with multiple name/value pairs that appear in a structured manner.

For the full bind configuration compiler, I think that a keyword based language will be important, and the parsing will be much more strict. Arbitrary identifiers won't be allowed, since bind has no use for arbitrary identifiers. On the back-end, I'll map these fixed format types directly to class definitions, so the final compilation format will be a configuration object you can use directly. This compiler will be more an interpreter, since it is going to translate, at run-time, a given configuration into some other more usable format.
Wednesday, May 19, 2004 7:06 PM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

Your the man. Are you a Mensa member? You do more stuff in an hour then I can do in a month. Cheers!
Wednesday, May 19, 2004 7:22 PM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

BTW. If/when you get a generic french brace parser to parse any config with key words (maybe your are there already), would you care it I "wrap" it in a MSH "cmdlet" and put it up at the beta place for Monad shell? Will provide proper credit.

$obj = get-config -file msomefile.txt
echo $obj.Var1
ecjp $obj.Var2
$obj | set-config file mynewfile.txt

Need to think more on property access, etc. Could also gen a Provider for access like:

>cd ./myconfig ## "CD" to Config Provider.
>ls ## get-childitems in container.
Item Value type
---- -------- -------
var1 "string" string
var2 32 int
var3 "one", "two" string[]
...
TIA
Thursday, May 20, 2004 1:11 AM by Justin Rogers

# re: Adding keywords to BasicParse, keeping it short and sweet.

We need to define semantics for a *generic* french brace parser. The reason for this is that the Bind format that started this entire discussion, has items such as the following:

acl "alias for acl list" {
...
}

How does a generic french brace parser perform on the above? Do we create an acl element, then give it an Id, or do we create an acl element then immediately nest another element based on the string (aka, default nesting of identifiers) and put any values into this second nested element.

What about strings of identifiers that exist in bind, such as:

controls {
inet * port 50 allow { address_list } keys { keys_list }
}

In the bind specification the inet command is a single command, and multiple instances of said command can exist within the controls section. Portions are optional.

Now, if I throw out the assertation that a statement terminator must exist my generic parser can turn the above into:

<controls>
<inet Value="*" />
<port Value="50" />
<allow></allow>
<keys></keys>
</controls>

Multiple instances would simply turn into linearly defined items. Another option is to demand statement terminators, and parse all you can. I could turn the above then into:

<controls>
<inet Value="*">
<port Value="50">
<allow></allow>
<keys></keys>
</inet>
</controls>

Things start to get really sticky when you parse based on arbitrary grammars. You start to lose all of the meaning between Item/Value/Type association. That means there can't truly be a 100% generic french brace parser unles you are willing to accept the results in a very non-determinate form.
Thursday, May 20, 2004 6:37 AM by Stephane Rodriguez

# re: How hard/easy is an HTML parser using the BasicLex/BasicParser design?


Me think that you have no real choice if you intend to write a parser as part of the renderer. You need to apply the same rules than IE. That's why dominant positions are such a shame, especially when the software is so much broken and full of internal choices.

Regarding the parsing service itself, although Lovett's SGML parser might be regarded as a reference, I guess it becomes completely useless in the real world, where most of the time you get invalid html. The strength on an html parser should be to have an efficient diagnostics mechanism (with a meaningful report system, see Safari) and be able to switch from several parsing modes including loose, strict, ... All in all, you are not going to do that kind of reliable parser with a regexp (contrary to what some people try to show without short-sighted teasers).

My 0.5 cent



Thursday, May 20, 2004 7:13 AM by Justin Rogers

# re: How hard/easy is an HTML parser using the BasicLex/BasicParser design?

I think Darren's parser is well on the way to being fairly efficient and resilient to real world HTML. I agree that the SGML parser, even with the loose DTD, can be rather rigid.

I think the end result of Darren's work will be some relatively powerful HTML tools. I already have some ideas for what he is working on, as well as some small tools I'm planning myself.

Creating order out of chaos is how I refer to this process. If I create just slightly enough order to make my life easier, then a job well done.
Thursday, May 20, 2004 7:32 AM by TrackBack

# re: Internal and External Iterators

Thursday, May 20, 2004 10:33 AM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

ahh. Ok, thanks. Guess I will stick with the Bind parser for now. Which would be fine if you can tell it what vars are valid. Not sure on this point. Do you have to "bake" in the grammer, or can you supply some array structure to the parser to parse out the language that is valid and know when something is not valid (i.e. invalid syntax, invalid keyword, etc.)
Thursday, May 20, 2004 11:12 AM by TrackBack

# Short on time but here's some important parsing stuff...

Thursday, May 20, 2004 11:12 AM by TrackBack

# Short on time but here's some important parsing stuff...

Thursday, May 20, 2004 11:12 AM by TrackBack

# Short on time but here's some important parsing stuff...

Thursday, May 20, 2004 11:13 AM by TrackBack

# Short on time but here's some important parsing stuff...

Friday, May 21, 2004 8:26 AM by Joel B.

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

I know you have been busy, I was just wondering if you had time to look into this matter.
Joel
Saturday, May 22, 2004 1:18 AM by David Olsen

# re: Why is photon mapping popular in gaming right now, and will it stay?

Wow. I thought hardly anyone even knew about photon mapping, let alone used it in games already. Anyway, I'll tell you what I thought of to improve it's realtime performance for lights that don't move. Just shoot the photons in an ordered fassion of some sort so that when something moves you dont have to update the entire photon map, just get the light source to re-trace the photons which hit it last frame and might hit it this frame.
Saturday, May 22, 2004 6:55 AM by TrackBack

# Running your expressions asynchronously and making them cancellable.

Saturday, May 22, 2004 5:34 PM by David Levine

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

An interesting class. I might extend this so that the actual operation was called via a generic interface so that the actual work/cancellable operation can be anything.

I whipped up a little test based on this and found a couple of things you might consider.

First, in the ThreadPool_WaitCallback method you should either add a generic catch block or restructure it so that the call to result.Complete(); is in the finally block. The reason is that if the call to perform the work operation (in your example, the Match(...) call) ever throws an exception other then ThreadAbort, the call to result.Complete will never execute and you will instead have an unhandled exception. This is likely to result in some higher level code hanging.

The other thing is that you might use Thread.Interrupt rather then Thread.Abort to signal the worker thread to interrupt its processing. For one thing, your callback might not have sufficient security privileges to call ResetAbort, and for another, using Abort has undesirable side effects that would argue against using it in a generic manner - unless you know that the code you are interrupting can gracefully handle an abort I'd use some other technique.



Saturday, May 22, 2004 6:15 PM by Justin Rogers

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

I have developed a generic version of the above class, but it doesn't help people trying to do their operations today. Nor does it solve very many problems.

Generic methods can't accept variable numbers of parameters, so the BeginInvoke/EndInvoke method signatures constantly change.

Since this is for regular expressions, I get more performance having the layout abovce. However, you can add an exception local, and move Complete into the finally block, as well as setting the caught exception (if anything but a ThreadAbort) on the async result. When the user calls EndInvoke you re-throw the exception.

I tested the abort code with and without ResetAbort and it doesn't thrash the system either way.

How does Interrupt buy you anything? Interrupt won't cancel a running thread, only Abort does that. Interrupt is capable of signalling a thread that is in a wait state, not a running state. As for the side-effects, I'm well aware of them. With another 50-100 lines of code we could remove those side effects, and make this completely unsuitable for use on a web server.

The two choices are app-domains or using something closer to a native thread. Both approaches make the code iffy in a server environment while making it much more stable and robust in a client environment.
Saturday, May 22, 2004 11:57 PM by TrackBack

# Generic, Cancellable, Asynchronous operations? Yeah, I'll blog about that.

Sunday, May 23, 2004 7:26 AM by David Levine

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

My original thought was that the class was nicely layed out and a generic version of it would be useful to have; my comments mainly followed from that starting point. By targeting the class to a single type the design is simplified and the comments do not necessarily apply. e.g. if Regex.Match will never throw an exception then there's no need to move the call to Complete.

Using Interrupt has pros and cons. It will interrupt the target thread when it makes a call that puts it into an alertable wait state so its effects are more predictable, but the downside is that if the target thread does not make such a call it will have no effect on it. Again, this may not be useful with Match.

I've had mixed results using abort; if the thread has wandered off into unmanaged code it wont be delivered until it returns to managed code. I've had better results when unloading appdomains - I could unload an appdomain even when I could not abort a specific thread in it.

I don't quite see how using a native thread would work. Would it use the RegEx class, and would you use native signalling methods to interrupt it?

Sunday, May 23, 2004 5:00 PM by Justin Rogers

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

Native threading API's can be used to ensure that a given thread gets shut-down. This is the not nice way to go about things and basically has the layout of:

Cancel() -> Abort() -> Oops, we still haven't aborted after some time limit? -> TerminateThread.

The goal is to ensure things get cancelled. The Win32 threading API's provide a fast easy way to cancel the operation, while at the same time introducing a slight chance of data corruption. Again, the concept is that in a server environment, you don't want to be loading a large number of AppDomain's. You could share AppDomain's but then bringing one down would mean cancelling some other code that was running in the domain as well.
Monday, May 24, 2004 5:57 AM by Dave Hallett

# re: Debugging eventable code, one stupid mistake and how to fix it.

This is an ugly hack, but IIRC, you can just call

this.slideshowTimer.Tick -= new EventHandler(this.Form_SlideShowAdvance

before you call

this.slideshowTimer.Tick += new EventHandler(this.Form_SlideShowAdvance

and if the event is not hooked up, nothing will happen. Not very high-performing code, I'm sure :o) But I think it works.
Monday, May 24, 2004 6:05 AM by Justin Rogers

# re: Debugging eventable code, one stupid mistake and how to fix it.

Definitely a hack. I wound up writing my eventing manager and it works quite well. I tend to use it more often than not in any sort of asynchronous programming scenario with eventing, or when hooking up many hundreds of events in notification scenarios.

You see, the delegate managers tend to work extremely well at ensuring events are hooked in a specific way, and I like that. Take a basic RTC scenario, where you have a group of chatters. Someone owns the group or controls it, while others are members. If the group is now an object with a single broadcast event for sending events, then you could hook that event any number of times through varying code-paths (possibly by mistake), with an event manager I can ensure that it only gets hooked once. I can also allow the hooks to be dependent upon moderation by the group owner. There are so many protections added by the EventManager.
Monday, May 24, 2004 7:49 AM by David Levine

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

I agree, it definitely is not a nice way to go about shutting down a thread :-) I experimented with TerminateThread back in runtime v1.0 and decided to not use it in .NET.

One issue was getting the native system Win32 thread handle - there's no exposed mechanism for getting it. I've seen an undocumented method that uses hardocded offsets into the thread class. I came up with an alternative method that worked well enough for my experiment - get the list of all the system threads in my process before creating my worker thread, create the worker thread, get a new list of system threads and look for the new thread and use its handle. If I could not find it or if there were more then one new thread, I would terminate the worker thread and repeat - eventually I would find the handle.

After that I was able to terminate the underlying system thread, but the problem was that the part of the runtime that controlled threads and execution state has no idea that I had yanked the rug out from underneath it. It still thought the managed thread object was alive and well even though I had terminated the system thread that it was mapped to. Perhaps there's a way to tell the runtime that the thread had died but I didn't find it.

Also, TerminateThread does not clean up after a thread so that the thread's Win32 stack is still allocated (1Meg of virtual address space), owned objects (mutexes, etc.) are still owned, so deadlocks can result, resources may leak, data can get corrupted, etc. I've used TerminateThread in control engines but only as an absolute last resort.

Future (present???) versions of the runtime are supposed to support models built on execution units other then threads, such as fibers. I haven't worked with fibers so I don't know how well they map into using the native API to control them from the managed world.

Even though creating/unloading an appdomain is a somewhat heavy operation it does not have all those issues. From what I've observed it appears to be fairly aggressive at unloading an appdomain even when a thread with stack in the doomed appdomain has become "stuck" and unresponsive. If the task was a very long running operation it might be worthwhile to give it its own appdomain, but as you point out there are other issues to consider on a server - all designs involve tradeoffs.


Monday, May 24, 2004 10:54 AM by TrackBack

# XPathDocument, can you dig it? Yep, and you can edit it too.

[ Via Justin Rogers ]XPathDocument, can you dig it? Yep, and you can edit it too....
Monday, May 24, 2004 1:50 PM by William Stacey

# re: XPathDocument, can you dig it? Yep, and you can edit it too.

You may find this link interesting.

http://www.devx.com/dotnet/Article/17829/1954?pf=true
Monday, May 24, 2004 3:19 PM by Justin Rogers

# re: XPathDocument, can you dig it? Yep, and you can edit it too.

The DevX article is fairly well written. However, the use of helper methods for operating on the tree is disappointing. Currently the interoperation of XmlDocument->TreeNode is tenuous at best. You can store XmlNode references on a TreeNode, which becomes very powerful, but it doesn't appear to be anywhere near the performance of the XPathDocument.

I will note that the TreeView may get a data-binding syntax before Whidbey gets released. Might already have it, just I haven't managed to get it working yet.
Monday, May 24, 2004 9:43 PM by TrackBack

# Take Outs 24 May 2004

Take Outs 24 May 2004
Tuesday, May 25, 2004 3:17 AM by Dave Hallett

# re: Debugging eventable code, one stupid mistake and how to fix it.

You make a good point: the hack is indeed no good at all for such situations. Thanks for the response!
Tuesday, May 25, 2004 5:59 AM by Matt

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Very interesting article, Thanx !
Tuesday, May 25, 2004 8:31 AM by Darren Neimke

# re: BasicParse with State/Transition graphing, a much more robust parser.

Thanks Justin... A quick tip: I found it useful to expose the this.mStreamOffset from the TokenStream class. I exposed it as a readonly property named "Position".
Tuesday, May 25, 2004 12:57 PM by Justin Rogers

# re: BasicParse with State/Transition graphing, a much more robust parser.

I could see that. Another useful feature is to not eat \n and have the TokenStream count line numbers and line character positions. I'll look at adding this meta-data into the classes.
Tuesday, May 25, 2004 3:02 PM by David P

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

This is a stupid function. it doesnt actually remove the full split string, only the first char.

Example:
teststring: "data1_-_data2_-_data3"
split: "_-_"

returns:
data1
-_data2
-_data3

Rather than:
data1
data2
data3

this just seems... lame. needs to clean up the garbage. This is otherwise a great approach to split on string.

if you have suggestions for a tweak, please let me know. I was toying with the code, but have not followed the logic properly/completely yet.

David P.
roachslayer@hotmail.com
Tuesday, May 25, 2004 3:21 PM by Justin Rogers

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

Hey, no reason to call it garbage man. Here you go:

private static string[] SplitByString(string testString, string split) {
int offset = 0;
int index = 0;
int[] offsets = new int[testString.Length + 1];

while(index < testString.Length) {
int indexOf = testString.IndexOf(split, index);
if ( indexOf != -1 ) {
offsets[offset++] = indexOf;
index = (indexOf+split.Length);
} else {
index = testString.Length;
}
}

string[] final = new string[offset+1];
if ( offset == 1 ) {
final[0] = testString;
} else {
offset--;

final[0] = testString.Substring(0, offsets[0]);
for(int i = 0; i < offset; i++) {
final[i+1] = testString.Substring(offsets[i]+split.Length, offsets[i+1]-offsets[i]-split.Length);
}
final[offset+1] = testString.Substring(offsets[offset]+split.Length);
}

return final;
}
Friday, May 28, 2004 1:09 PM by Cepheus

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Thanks for an excellent article covering some very much needed information for me. I've been working on a multithreaded network application and I've been stressing over how to deal with the UI, and the methods you examined here have solved it for me. :) Thanks a bunch!
Tuesday, June 01, 2004 3:05 PM by zjames

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

This works really well. Thank you.
Tuesday, June 01, 2004 3:22 PM by Justin Rogers

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

I never linked this in. However, the stand-alone ImageFast library is much more robust than the above code. It allows for operation in scenarios other than Windows Forms based.

http://weblogs.asp.net/justin_rogers/articles/131704.aspx

Tuesday, June 01, 2004 6:22 PM by TrackBack

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

Tuesday, June 01, 2004 6:41 PM by William Stacey

# re: Bringing an end to the posts on decompositions of numeric ranges into validating regular expressions.

Just curious if you checked out any of these lexers and what your thought about them?

Grammatica
http://www.nongnu.org/grammatica/index.html
ANTLR
http://www.antlr.org/grammar/list
csLex
http://www.cybercom.net/~zbrad/DotNet/Lex/Lex.htm
Tuesday, June 01, 2004 7:04 PM by Justin Rogers

# re: Bringing an end to the posts on decompositions of numeric ranges into validating regular expressions.

I've looked at pretty much all of the lexical analyzers out there. I really only have one problem with them in general, and that is the initial complexity involved. They assume that you can easily define your grammar in some abstract format, and most people, until they've actually written their own lexical analzyer won't fully understand that abstract format or how to find issues with it.

I guess I'd say that creating your own lexer/parser is an exercise that gives you access to knowledge you'll need later when you implement whatever your lexer/parser was creating. In your case you were trying to read bind-logs. The lexical analysis file for bind logs is actually fairly complex with lots of optionals and this/thats, but just looking at the stream of values it appears fairly simple. It really is, but you have to go through the exercise of processing that data before you gain an understanding of it.
Wednesday, June 02, 2004 6:03 AM by nani

# re: Awesome Windows Forms message pump trick...

nani
Thursday, June 03, 2004 8:11 PM by Tandem_Guru

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

I think this is THE most detailed description regarding Winform pitfalls that most likely confuse a lot of newbies like me.

Thanks for the clarification!

How about another paper regarding UIThread and other background thread so that it could really complete the whole topic?


Thanks again!
Thursday, June 03, 2004 10:52 PM by William Stacey

# re: Bringing an end to the posts on decompositions of numeric ranges into validating regular expressions.

I agree fully. I looked at them, but did not want to speed the hours needed to figure them out. I probably will at some point, but was looking for something I could quickly wrap my arms around and go. Guess that is not possible and just need to roll the sleeves up or use xml for now. Cheers!
Friday, June 04, 2004 9:55 AM by TrackBack

# Faster than the speed of light (that's pretty quick you know!)

Friday, June 04, 2004 10:20 AM by TrackBack

# Faster than the speed of light (that's pretty quick you know!)

Saturday, June 05, 2004 7:24 AM by Ole Lytjohan

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Nice article, but you should really make some people read through it first before posting it. It's filled with errors :/

It's not a big thing, the article by itself is nice, but it would just have been much better, if those issues where fixed.
Saturday, June 05, 2004 7:40 AM by Justin Rogers

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Would be nice if you pointed out the errors so they could be fixed. Up to now I've had many people read through it and not point out any errors.

If you are speaking of spelling mistakes or grammar issues then I won't even bother, but true errors in function are definitely something I can post addendums for.
Saturday, June 05, 2004 7:42 AM by Ole Lytjohan

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Previous post of mine was in error. No errors.
Read some of it wrong, just shows :)
Saturday, June 05, 2004 7:42 AM by Justin Rogers

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Tandem: What exactly are you talking about with the extension paper you propose? Are you thinking of an additional use case scenario that perhaps I've missed or didn't fully explain so that you could apply it to your problem?
Saturday, June 05, 2004 8:18 AM by Justin Rogers

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Ole: Hey, no worries. I've definitely posted mistakes before and been called for it, so I'm always curious what I'm going to get in return after a posting. Thanks for reading it so closely.
Sunday, June 06, 2004 11:01 AM by TrackBack

# re: Harry Potter and the 142 Minutes of My Life I'll Never Get Back

Sunday, June 06, 2004 5:21 PM by TrackBack

# Take Outs for 6 June 2004

Take Outs for 6 June 2004
Sunday, June 06, 2004 5:22 PM by TrackBack

# Take Outs for 6 June 2004

Take Outs for 6 June 2004
Monday, June 07, 2004 9:27 AM by ej2

# re: New little series I'm going to call, Causality4. An examination of causality in terms for specific groups of people.

Very interesting. Have you seen the movie 12 Monkeys? It deals a little with causality.
Monday, June 07, 2004 12:21 PM by Jerry Dennany

# re: I often get asked how one would ship a single assembly app that is in multiple assemblies...

I did something sorta like this a while back.

http://weblogs.asp.net/jdennany/archive/2003/09/24/29035.aspx

Monday, June 07, 2004 12:52 PM by zjames

# re: ImageFast: A stand-alone library for quickly loading GDI+ images without validation.

Thank you thank you thanks! Works great. Original wasn't working on Win2000 at home, then it DID work, WAS working at work on win2000, but today it wasn't... weird. This fix got it working again. Hopefully for all time. Thanks!
Zac James
Monday, June 07, 2004 9:36 PM by TrackBack

# Dynamically running an Assembly in a separate process (with your tail between your legs)

Monday, June 07, 2004 10:30 PM by TrackBack

# re: Dynamically running an Assembly in a separate process (with your tail between your legs)

Tuesday, June 08, 2004 4:35 AM by TrackBack

# re: Draft posts... what should they display for 'date posted'?

Wednesday, June 09, 2004 12:16 AM by soft

# re: Joel's Lightweight Code Gen spells SUWEET for small scripting languages in games.

good!
Wednesday, June 09, 2004 9:01 AM by Fredrik Normén NSQUARED2

# re: What are your thoughts on the new ArraySegment structure in Whidbey?

I think it could be useful, it will reduce the number of argument we need to pass to a method and I don’t need to remember the offset and the count. It’s already located in my array segment. For example if we want to pass an array and its offset and count we need to write something like this:

int[] myArrayOfStuff = new int[10];
FillSegment(myArrayOdStuff, 5,5);


public void FillSegment(Int[] myArray, int offset, int count)
{
…
myStream.Read(myArrayOfStuff, offset, count);
…
}

With the ArraySegment<T> this could be written:

int[] myArrayOfStuff = new int[10];
ArraySegment<Int> mySegment = new ArraySegment<Int>(myArrayOfStuff, 5,5);
FillSegment(myArrayOdStuff);

public void FillSegment(ArraySegment<T> myArraySegment)
{
…
myStream.Read(myArraySegment.Array, myArraySegment.Offset, myArraySegment.Count);
…
}
Wednesday, June 09, 2004 10:00 AM by kvr

# re: A conversion from integer to long form english... I could write that ;-)

Shouldn't Fourty be spelled as Forty?
Wednesday, June 09, 2004 10:37 AM by Justin Rogers

# re: What are your thoughts on the new ArraySegment structure in Whidbey?

I agree, however, the later in this case is actually more code to write, and can appear to be less clear.

I would also point that the possiblity of error that exists when using an array segment, by mistyping and using the array's properties is quite high.

And while I understand why they made the object immutable, you will also wind up creating a new segment every time you call your Fill command. With integers you treat the offsets like pointers and increment things. With Segments, you are forced to allocate a new one every time you want to move around your buffer. This creates object allocations where before there were none.

Not saying this is a bad thing, just pointing it out. Glad to see someone is making use of the segment.
Wednesday, June 09, 2004 10:56 AM by Spectrum

# re: A long MUD-Dev thread on intelligence of NPC's, and some extremely faulty arguments...

10 PRINT "DISAGREE IM A DUMB PROGRAM"
Wednesday, June 09, 2004 12:14 PM by TrackBack

# System.Net.NetworkInformation.Ping

After sorting out a problem with some ICMP "ping" code last week, it was nice to see that there is a Ping class in Whidbey, as described in Justin Rogers' post TraceRoute Whidbey using System.Net.NetworkInformation....
Wednesday, June 09, 2004 2:25 PM by TrackBack

# A conversion from integer to long form english... I could write that ;-)

Wednesday, June 09, 2004 5:59 PM by AT

# re: A conversion from integer to long form english... I could write that ;-)

How about to convert them in both ways ?
English -> number ?
Wednesday, June 09, 2004 9:16 PM by Brian Grunkemeyer

# re: Generic, Cancellable, Asynchronous operations? Yeah, I'll blog about that.

This is unfortunately a very flawed attempt at a cancellation infrastructure. The design flaw here is you're relying on aborting threads. Short of calling methods with reliability contracts and/or constrained execution regions, you can't abort threads safely. You have no idea how the Regex code is implemented internally, but it might have an appdomain-wide cache of things that it edits while doing a lookup. If that's the case, there's no guarantee that collection update will succeed if you abort it. You can be corrupting appdomain and possibly process-wide state with this code, so you should follow this up by promptly unloading the current appdomain (and possibly the process, if you can't guarantee the process state won't be corrupted). We may change Thread.Abort to be more "polite" eventually in Whidbey, but you still run the risk of corrupting appdomain & process state, as we may not "harden" all the parts of the Framework you need.

What can you use? There are three techniques you should look at:

1) Cancellation regions & cancellation signals
2) ICancellableAsyncResult
3) Some cooperative polling technique in your code to support cancellation

If you're running on Longhorn, Whidbey's new CancellationRegion & CancellationSignal classes will allow you to cancel any disk-bound IO request (assuming the driver supports cancellation). For async IO requests, ICancellableAsyncResult provides a cancel method that uses some other Longhorn feature to cancel that particular async operation.

Short of that, the best technique you can use is cancellation. One of the changes we want to make for Beta 2 is to is to add a property to CancellationRegion indicating whether a cancellation was requested. You can change your code that is CPU-intensive to poll this property and throw an OperationCanceledException. Unfortunately, it has to be done in this cooperative manner if you want to avoid state corruption that would require you to pessimistically unload the appdomain.

For more info, Chris Brumme's discussions on Reliability may be interesting (I don't know if they go into this much detail though in publically released material yet). Additionally, this same notion of avoiding corruption went into the design of Mark Gabarra's PoVBot (an AC2 game bot), with the introduction of a cooperative sleep (in addition to a normal sleep) in the bot's scripting language. The feature would allow you to interrupt the script at this point with an async handler of some sort (like a timer that fires every minute), do some operation, then recover to some known state. The script and/or the event would have to be written to deal with any state changes to the UI, etc.

(Yes, Mark & I were both surprised that a game bot would have any commonality with our reliability work. Fortunately, once he was working on the bot, we already had the hard conceptual part figured out.)

If you want more information about some of this stuff, look on the BCL web site (http://www.gotdotnet.com/team/clr/bcl/TechArticles/techarticles.aspx) and annoy us if you can't find what you want.
Wednesday, June 09, 2004 9:27 PM by Brian Grunkemeyer

# re: I'm going to do something unfair to make a point: int.TryParse versus DWC.NumberUtilities.

The need for these methods wasn't realized until too late in the ship cycle. We added one TryParse method because it was causing some noticable delays in an important perf scenario, then we got around to doing the rest significantly later.

I don't imagine your TryParseInt32 is culturally correct. That's probably a significant portion of why Int32.TryParse is slower.
Wednesday, June 09, 2004 9:42 PM by Brian Grunkemeyer

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

There are a number of issues here, and I think they're all solved.

First, the stack overflow exception can be avoided by using the CompletedSynchronously property, as Alexei points out. Based on its result, you can tell whether you can recurse or whether you should return and have your caller issue another BeginXxx call. I think you'll find this if you read through enough conceptual docs on our async design pattern, but it is pretty buried.

Secondly, Stream's BeginRead is synchronous like this. Stream's async code paths are implemented in terms of the sync code paths, and the sync code paths are implemented in terms of the async ones. So if you subclass Stream, you only have to implement the sync or the async code paths, and you get the other one for free. Additionally, your users have a consistent programming model.

Thirdly, I've reimplemented Stream's BeginRead & BeginWrite methods in Whidbey to do their work on a threadpool thread instead of their own thread (via async delegates). Your code will probably work if you pick up a more recent build. (I made the change to Whidbey around Feburary or March - you might not see it until Beta 1 or a very recent community drop. I wanted to make this change much earlier in Whidbey, but I was blocked until we added a Semaphore class to fully insulate users from an obscure race I would have otherwise exposed in user code. All the other sync primitives have sometimes-unfortunate threading restrictions, some of which weren't easily discoverable in V1 & V1.1.)

Additionally note that FileStream & NetworkStream do support true async IO using an IO completion port. However, for FileStream, the OS restricts file handles to only one type of IO - either sync or async. So you must explicitly open the file asynchronously if you want fast async IO. (You can do this by passing in FileOptions.Asynchronous in Whidbey, or by passing true to the "useAsync" parameter before that.) If you open it in the wrong mode, it may hurt your perf by up to 10x on pathological cases, and you may end up with the naive implementations from Stream in these cases.

I hope this helps.

Brian Grunkemeyer
CLR Base Class Library Team
Wednesday, June 09, 2004 10:21 PM by Justin Rogers

# re: Generic, Cancellable, Asynchronous operations? Yeah, I'll blog about that.

To provide public commenting:

Contract structures are something I fully agree with. While you could use the above generic cancellation feature for any code, and it would work to some degree, there are code-paths that the remote thread might be in that would cause some form of failure. That is the reason I used my knowledge of the Regex codebase through Rotor to use it in that scenario and primarily as a reliability story for a high capacity web server that is trying to prevent a DOS attack by a user input expression. There are a number of ways to get to this point since even a basic expression can be non-terminating over the properly written input.

All three of your solutions are a hand-shake pattern. I'm not sure that I'll ever be able to fit all of the code I need to cancel into the design paradigm of a hand-shake routine. Some I could, yes, but others are simply going to be impossible.

The reliability store here is that of BlackBox reliability. I can trace every code-path that I'm running and ensure a cancellable IL structure, however, what I can't see behind is the BlackBox of the CLR. Sure there may be some non-cancellable features back there that would cause my code to exit not so gracefully, then mark them and tell me where they are so I can include those in my examination and determine if my reliability story is going to be good enough.

What I would propose for Thread.Abort, if you are making changes, is some awareness in the code of where it can gracefully stop and where it can't. Currently Thread.Abort instantly throws an exception (or actually I think in some cases it does wait a bit), while in the future it could have a timeout of some sort and a return value telling me whether or not the thread truly did abort. If it didn't, an enumeration telling me why would be nice, and I can try the abort a second or third time, again in the interest of a more reliable system.

What I don't want is to riddle my AI code with cancellation checks when I'm already granting it a small subset 5% of the CPU, and said 5% is being shared between 1000 automatons.
Wednesday, June 09, 2004 10:25 PM by Justin Rogers

# re: I'm going to do something unfair to make a point: int.TryParse versus DWC.NumberUtilities.

No, but even the culturally correct version that I wrote is 3x faster. So that wasn't the significant portion at all. The significant portion is the single code-path for parsing all numeric types that does a bunch of extra processing logic in cases where said logic is unnecessary. Having the new TryParse methods around in Whidbey is quite the good thing as I've noted, but I'll still use my own libraries for string and numeric processing since they are quite the bit faster.
Wednesday, June 09, 2004 10:31 PM by Justin Rogers

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

You mean something like what I have below. I've wrapped the concepts of synchronous completion so that I have enough variables to figure out when I need to loop and when the aysnchronous code is doing the work for me. I'll probably blog the below and linkie it to this entry. Also note that I've also implemented a ThreadPool override of the Stream class that provides true asynchronous actions. So I'm guessing we are on the same page.

using System;
using System.IO;

public class JustAStreamWrapper : Stream {
private Stream underlyingStream;

public JustAStreamWrapper(Stream stream) {
this.underlyingStream = stream;
}

public override bool CanRead { get { return true; } }
public override bool CanSeek { get { return false; } }
public override bool CanWrite { get { return false; } }
public override long Length { get { return -1; } }
public override long Position { get { return -1; } set { } }

public override void Flush() { }
public override long Seek(long pos, SeekOrigin origin) { return -1; }
public override void SetLength(long length) { }

public override int Read(byte[] buffer, int offset, int count) {
return underlyingStream.Read(buffer, offset, count);
}

public override void Write(byte[] buffer, int offset, int count) {
}
}

public class SynchronousCompletionHandler {
private bool firstCall;
private Stream underlyingStream;
private byte[] userArray;

public SynchronousCompletionHandler(Stream underlyingStream, byte[] userArray) {
this.underlyingStream = underlyingStream;
this.firstCall = true;
this.userArray = userArray;
}

public Stream Stream { get { return this.underlyingStream; } }
public bool FirstCall { get { bool retVal = this.firstCall; this.firstCall = false; return retVal; }
public byte[] UserArray { get { return this.userArray; } }
}

public class NonAsyncStream {
private static void Main(string[] args) {
try {
DoNoStackOverflow(args[0]);
Console.WriteLine("Past the NSOE method");
DoStackOverflow(args[0]);
} catch { Console.WriteLine("SOE");
} finally { Console.WriteLine("SOE");
}
}

private static void DoNoStackOverflow(string largeFile) {
SynchronousCompletionHandler sch = new SynchronousCompletionHandler(new JustAStreamWrapper(File.OpenRead(largeFile)), new byte[4096]);
sch.Stream.BeginRead(sch.UserArray, 0, sch.UserArray.Length, new AsyncCallback(End_ReadNoOverflow), sch);
}

private static void End_ReadNoOverflow(IAsyncResult ar) {
SynchronousCompletionHandler sch = ar.AsyncState as SynchronousCompletionHandler;
if ( sch == null ) {
return;
}

if ( ar.CompletedSynchronously && !sch.FirstCall ) {
return;
}

while(sch.Stream.EndRead(ar) > 0) {
// ProcessArray(sch.UserArray);
ar = sch.Stream.BeginRead(sch.UserArray, 0, sch.UserArray.Length, new AsyncCallback(End_ReadNoOverflow), sch);

if ( !ar.CompletedSynchronously ) { break; }
}

// Signal final completion
}

private static byte[] foo = new byte[4096]; // Very small so we overflow with smaller files
private static void DoStackOverflow(string largeFile) {
JustAStreamWrapper jasw = new JustAStreamWrapper(File.OpenRead(largeFile));


jasw.BeginRead(foo, 0, foo.Length, new AsyncCallback(End_Read), jasw);
}

private static void End_Read(IAsyncResult ar) {
Stream stream = (Stream) ar.AsyncState;

if ( stream.EndRead(ar) > 0 ) {
Console.WriteLine("One Time Through");
stream.BeginRead(foo, 0, foo.Length, new AsyncCallback(End_Read), stream);
} else {
Console.WriteLine("Less than good");
}
}
}
Wednesday, June 09, 2004 11:13 PM by C# Learner

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

I too find it odd that String.Split doesn't have an overload for splitting on strings.

In any case, I normally use the static method Regex.Split for this.
Wednesday, June 09, 2004 11:35 PM by Justin Rogers

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

Unless the string you are splitting on is rather large the Regex methods are quite a bit slower than what I have above. The above is code very similar to what exists in the base BCL string methods for splitting on characters.

In addition there are some improvements that can be made to the above algorithm, if you are only interested in a portion of the splitting group, for instance:

Return me the first 6 splits only
Return me splits 2 and 8 only

The above prevents unnecessary allocation of intermediate strings, since they only allocate say 6 and 2 strings respectively, rather than an entire group of them. The methods can be programmed to short-circuit if a specfic number of splits aren't available, so they can be used for validation of split data.

In other words, the above is going to be extremely superior to anything that exists in the BCL at current.
Wednesday, June 09, 2004 11:37 PM by Justin Rogers

# re: A conversion from integer to long form english... I could write that ;-)

Converting back would require some math that is extremely lossy. If I were going to go backwards, I would consider the use of a large number library. Since the only publicly available one for .NET isn't that great, and I don't feel like releasing and supporting my own, I'll leave the conversion from English -> number as an exercise for someone else.

Thursday, June 10, 2004 4:17 AM by Wesner Moise

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Using [ThreadStatic] allows you to have thread-safe functions while pre-allocating buffers.
Thursday, June 10, 2004 4:28 AM by Justin Rogers

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

I looked at pre-allocated buffers, and they only solve the problem of small strings. Larger strings that extend past your buffer still require that you allocate a buffer large enough to do some work (again, since you can't do in place manipulations of strings for immutability reasons).

ThreadStatic variables are special, since they are stored in Thread Local Storage or what you call a TLS Slot. Each thread only has a fixed number of these slots, and accesing thread API's can be the death of an algorithm. I'll explain why in a future post when I demonstrate why a certain widely accepted class doesn't scale to this particular problem.
Thursday, June 10, 2004 4:54 AM by AT

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

An question.
Do your code reverse string correctly ?? ;o)
How about supplementary characters in Unicode ??

http://msdn.microsoft.com/library/en-us/intl/unicode_192r.asp
http://www.i18nguy.com/unicode/char-count.html

Thursday, June 10, 2004 5:29 AM by Justin Rogers

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

As for surrogate pairs, I think the only truly performant method of a string reverse could be done by the BCL itself. You see they hold information about a string, bit flags if you will, defining whether or not certain actions need to occur. I'd assume they would have a bit that determines whether or not surrogate pairs actually exist, and if they do, handle that string special case from the others that done.

Anyway, in an attempt to throw out some numbers, a surrogate aware version of the algorithm functions at about 2 seconds per million, while the fully optimized non surrogate aware functions at about .6 seconds. Currently the best algorithm I've received runs over my test data at ~1.4 seconds, but I sent the user a very similar version with better arithmetic I had previously developed that runs at ~1.2 seconds.

Once the strings get large the IsSurrogate calls get really expensive and the algorithm comes out at 35 seconds per million, and 7 seconds for the fully optimized non surrogate aware version.

So in a very long winded manner, YES, I do have an algorithm to compare if a user wants to submit a surrogate aware reversal routine.
Thursday, June 10, 2004 5:34 AM by Justin Rogers

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

I take some of that back. IsSurrogate takes an extremely long time to return a value, and I think there is a faster way to determine if a character is a surrogate. With this faster method the code runs in 22 seconds for long strings, and actually comes in at 1.2 for the smaller strings.
Thursday, June 10, 2004 6:02 AM by TrackBack

# A love little samples like this...

Thursday, June 10, 2004 6:02 AM by TrackBack

# A love little samples like this...

Thursday, June 10, 2004 8:35 AM by Daniel Turini

# That's why I hate "sealed" classes.

I we could derive from String, reversing a string would be a matter of overriding some methods and you could do a 'lazy' string reversion, e.g., str[i] would return base.str[Length - 1 - i].
Ok, some may say that this would be cheating. :)
Thursday, June 10, 2004 10:23 AM by Greg Knierim

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Excellent article...However, not ever using Thread Invokes, I must ask can someone give a good scenario of when I would use this architecture?
Thursday, June 10, 2004 12:05 PM by Sean Lynch

# Converting from english to number

I just used a bunch of case statments.
Thursday, June 10, 2004 12:41 PM by Andy Smith

# re: A conversion from integer to long form english... I could write that ;-)

I did it in 7 less lines of code...
Thursday, June 10, 2004 3:52 PM by Jerry Pisk

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Daniel, what would you return in a ToString() call? That's the point here...
Thursday, June 10, 2004 6:25 PM by Darren Neimke

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Function Method1(ByVal str As String) As String
If str Is String.Empty Then
Return String.Empty
End If

If str.Trim().Length() = 0 Then
Return str
End If

Dim chars As Char() = str.ToCharArray()
Array.Reverse(chars)
Return New String(chars)
End Function


Function Method2(ByVal str As String) As String
If str Is String.Empty Then
Return String.Empty
End If

If str.Trim().Length() = 0 Then
Return str
End If

Dim i As Integer = 0
Dim x As Integer = str.Length - 1
Dim rev As Char()
ReDim rev(str.Length - 1)

Do While i < str.Length
rev(i) = str.Chars(x)
i += 1
x -= 1
Loop

Return New String(rev)
End Function
Thursday, June 10, 2004 6:45 PM by Stuart Hallows

# re: A conversion from integer to long form english... I could write that ;-)

Twenty minutes? Pah! I knocked one out in ten.
BOSH.
Thursday, June 10, 2004 6:50 PM by Cory Smith

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Darren is thinking along the same lines as myself. However, do need to point out that the Trim() method should be removed. Whitespace would be valid in this scenario and would improve the performance since you wouldn't be making a string copy of a potentially huge string just to do the Trim().
Thursday, June 10, 2004 8:25 PM by Justin Rogers

# re: A conversion from integer to long form english... I could write that ;-)

You people should post linkies so we can see your code. Hopefully you followed the article link which also handles numbers up to some astronomical double number, and converts values after the decimal point as well.

Thursday, June 10, 2004 9:34 PM by Justin Rogers

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Final Results:

http://weblogs.asp.net/justin_rogers/archive/2004/06/09/152343.aspx

And yes, I removed all of Darren's extra checks in the final versions. Trim would negatively impact reversing arrays of white space characters and so the algorithm is flawed. However, I like his optimization attempts.
Thursday, June 10, 2004 11:24 PM by Wesner Moise

# re: Performance: Fastest string reversing algorithms... (final results)

Actually, for large buffers, I suspect that you will get better performance by using StringBuilder with a capacity equal to the length of the original string.

But instead of using indexing, try appending blocks of characters with StringBuilder.Append using a char array of some fixed size such as 1024 characters. This eliminates the range-checking overheard associated with using Append, because you are appending several characters at once. At StringBuilder reuses the underlying string.

This uses less memory for large strings and potentially improve performance by invoking fewer garbage collections.

Thursday, June 10, 2004 11:46 PM by Justin Rogers

# re: Performance: Fastest string reversing algorithms... (final results)

Suspecting something and testing is a universal binary pair. Let me know when you've tested it, and that the tests you've run have disproven the very poor StringBuilder (10x worse than the comparitive algorithms in all cases I've tested). Drop us the source code and throw your hat into the competition.

Friday, June 11, 2004 12:02 AM by Justin Rogers

# re: Performance: Fastest string reversing algorithms... (final results)

In the interest of putting to rest any argument between myself and Wesner over the viability of the StringBuiler, I purpose the following algorithm. Note the algorithm uses a character array now to reverse a good deal of characters before trying the append.

Even with the speed boost supplied by our real reversing algorithms demonstrated in the article above, the StringBuilder still takes on the order of 1.1 seconds for shorter strings and 18 seconds for longer strings. I'm still inclined to use the character array reversal algorithms if I truly want to handle long strings.

private static string ReverseUsingStringBuilderCopy(string input) {
StringBuilder sb = new StringBuilder(input.Length);
char[] buffer = new char[Math.Min(input.Length, 1024)];
Console.WriteLine(buffer.Length);
int offset = 0;

for(int i = input.Length - 1; i >= 0; i--) {
buffer[offset++] = input[i];

// We could move our final append
// into this loop, but that would invoke
// two compares per iteration
if ( offset == buffer.Length ) {
Console.WriteLine("Append Buffer in Loop");
sb.Append(buffer, 0, offset);
offset = 0;
}
}
if ( offset > 0 ) {
Console.WriteLine("Appending final buffer");
sb.Append(buffer, 0, offset);
}

return sb.ToString();
}
Friday, June 11, 2004 12:30 AM by Justin Rogers

# re: Performance: Fastest string reversing algorithms... (final results)

However, what is a long string?

Strings less than about 150 characters I'll group into class 1. Strings of about 5000 characters or less I'm going to group into class 2. Anything above is class 3.

A string-builder matched with buffered chunking using class 1 strings gets it's butt kicked by our other algorithms. In fact, anything less than our buffer size, and it doesn't even help if we use string builders, since the allocated buffer kills any chance of using *less* memory. On the upper end of class 2 strngs the builder method starts to get better again, though it is still getting it's butt kicked. As we start into our class 3 strings, the string builder can actually match the other algorithms because of memory limits and it's reliance on a small work buffer. I used a string of 1024*50 characters in order to demonstrate a class 3 string. I like the results, and I think the string builder might be an adequate scenario for reversing strings when your input data is larger than your work buffer. I do propose a better construct for doing this operation than what I proposed before. This one adds a single division step to get rid of the comparison check each time through.

private static string ReverseUsingStringBuilderCopy(string input) {
StringBuilder sb = new StringBuilder(input.Length);
char[] buffer = new char[Math.Min(input.Length, 1024)];

int fullBuffers = input.Length / 1024;
int i = input.Length - 1;
int offset = 0;

for(int fullBuffer = 0; fullBuffer < fullBuffers; fullBuffer++) {
for(offset = 0; offset < buffer.Length; offset++) {
buffer[offset] = input[i--];
}
sb.Append(buffer, 0, buffer.Length);
}

for(offset = 0; i >= 0; i--, offset++) {
buffer[offset] = input[i];
}
sb.Append(buffer, 0, offset);

return sb.ToString();
}
Friday, June 11, 2004 12:34 AM by TrackBack

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Friday, June 11, 2004 3:19 AM by secretGeek

# re: Blog Implementations, commenting, and why everything isn't quite as good as it should be...

really useful systems always get used so much that their usage outgrows their usefullness.

thanks for the entry -- made me stop n think!

cheers
lb
Friday, June 11, 2004 6:13 AM by TrackBack

# re: Productivity -- but to what extreme?

Friday, June 11, 2004 8:29 AM by Ford

# TestCase

EnglishFromNumber(Int64.MinValue)

I would like to say: Math.Abs is evil.
Friday, June 11, 2004 10:09 AM by Justin Rogers

# re: A conversion from integer to long form english... I could write that ;-)

Yes, the version that uses Double should fix some of that. Follow the article link and if you have the same problem. I could possibly see a round-off error for Int64 even when using double as the base number type.
Friday, June 11, 2004 12:03 PM by Ron

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Sadly, I'm a bit late to participate. I'm way behind on my blog reading. :( My first thought was something very close to Daniels. I took both of his methods (sans optimizations and written in C#) to compare them against my version.

static string RevStr(string input)
{
char[] output = new char[input.Length];
int i = 0;
int x = input.Length - 1;

while (i <= x)
{
output[i] = input[x];
output[x] = input[i];
x--;
i++;
}

return new string(output);
}

On the small strings, RevStr always beat Method1 by a few tenths of a second and usually beat Method2, by a few hundredths of a second. In the cases there Method2 did beat RevStr, it was by a hundredth of a second or less.

On larger strings, RevStr consistently beat Method1 and Method2. The larger the string grew, the smaller the difference between RevStr and Method1. Conversely, the difference between RevStr and Method2 increased.

I'd be interested in seeing other comparisons.
Friday, June 11, 2004 12:36 PM by Haacked

# re: Blog Implementations, commenting, and why everything isn't quite as good as it should be...

Hey there, We've been discussing how to add comment tracking to RSS Bandit without adding too much network overhead.

Currently in RSS Bandit, any comments you make are put in the "Sent Items" folder. One thought is to have us check for new comments for any item in the sent items.

As for a voting system, I talked about that in these posts:
http://haacked.com/archive/2004/04/24/356.aspx
http://haacked.com/archive/2004/04/24/359.aspx

There's also AmphetaRate which might be a neat feature to add to aggregators.

Great ideas you have!
Friday, June 11, 2004 3:49 PM by Kevin Dente

# re: Performance: Fastest string reversing algorithms... (final results)

Interesting. However, has anyone ever actually needed to reverse a string in a real application? :P
Friday, June 11, 2004 10:51 PM by Justin Rogers

# re: Performance: Fastest string reversing algorithms... (final results)

String reversal is often used to determine palindromic identity, reformat portions of a string for different UI displays, perform basic cryptographic cyphers, and is a great root of examination to understand basic performance characteristics about a language or platform. The string reversal process in .NET is probably one of the most unique forms of string reversal, compared to other languages that allow in place string manipulation.
Saturday, June 12, 2004 12:40 PM by TrackBack

# How do you store your RSS? A look at XmlDocument, aggregation, sorting, and XPathDocument

Saturday, June 12, 2004 2:23 PM by Frans Bouma

# re: How do you store your RSS? A look at XmlDocument, aggregation, sorting, and XPathDocument

I wrote an rssfeed creator page today, and here's what I did:

- the .aspx page contains a text/xml contenttype directive and contains a repeater control.
- I load the data in the code behind and fill the repeated through databinding.
- I set the cache parameters to 7 days.
- At the places where the data shown in the rss feed (it's a forum system) is changed, I reset a flag in a hashtable in the application object
- I defined a Validate event handler in the .aspx page, which checks if the feed has been changed (by checking the flags in the hashtable). Based on that, it validates/invalidates the cached data.

Seems to me the way to go, or is this not that great?
Saturday, June 12, 2004 3:13 PM by Justin Rogers

# re: How do you store your RSS? A look at XmlDocument, aggregation, sorting, and XPathDocument

Glad to get some response. Well, Frans, your solution is a source solution. As in, you are the source of the feed. My proposal is primarily from a consumer perspective. You see, on the source side, RSS has minimal impact. You just render your XML out and you are done. You do get hit for bandwidth though, and that is where the change modification headers come into play. What I would recommend is the following:

1. Create a web request to your feed. Upon return, save the LastModified property.
2. Make a second call to the same feed and set the IfLastModified property of the request to the date you previously saved. If you get a 304 exception, then you are good, if not, then you need to figure a better way to set your headers.

That is only the first step, however. Going a step further and implementing partial feeds based on the IfLastModified flag would also be a great step in the right direction. I know of no implementations of partial feeding. I am thinking of writing a demo application this weekend to demonstrate what I'm talking about though.


Saturday, June 12, 2004 5:08 PM by Austin Ehlers

# Another alternative

How about unsafe code:

unsafe static void ReverseString(string source)
{
int y=source.Length-1;
int mid=source.Length/2;
fixed(char* sfixed=source)
{
char* s=sfixed;
char t;
for(int x=0; x<mid; x++, y--)
{
t=s[x];
s[x]=s[y];
s[y]=t;
} }
}
Results with 1million iterations:
Method 20 200 2000
ReversePointer 0.453125 2.546875 23.171875
ReverseCharArrayCopy 3.71875 12.09375 95.78125
ReverseCharArrayCopyAndReverse 4.59375 11.484375 81.4375
ReverseUsingStringBuilderInline 8.828125 67.546875 658.34375
ReverseUsingStringBuilderCopy 2.875 13.578125 117.015625
Saturday, June 12, 2004 5:21 PM by Justin Rogers

# re: Performance: Fastest string reversing algorithms... (final results)

Well, for one, that breaks just about every single rule of string immutability and can really cause the .NET run-time to heave. You should read up on Brad Abrams blog and check out various comments from Brian Grunkmeyer.

Functionally, what you've posted works. However it breaks the .NET run-time at the lowest levels. Even allocating a new string full of nothing but space, and then copying the reversed string into it is breaking the rules.
Saturday, June 12, 2004 6:23 PM by Austin Ehlers

# re: Performance: Fastest string reversing algorithms... (final results)

Unsafe code is for those who need fast access, and know what their code will do. None of the "special bits" used to designate a string with non-ASCII characters are contained in the range of
fixed(char* c=str)
c[0] to c[str.Length-1]

The length of the string is an Int32 stored just before c[0]. The non-ASCII info is stored in the top-two bits on the int. (0xC0000000)
Saturday, June 12, 2004 6:54 PM by Justin Rogers

# re: Performance: Fastest string reversing algorithms... (final results)

One of the tenets of the first competition was that thread safety was a must. Your method, for obvious reasons is not thread safe because you overwrite the string in place. If, however, you copied your string, then everything would be fine, so I'll take your algorithm and append a string output = string.Copy(input);, and then work over the copied string. That'll prove it to be thread safe.

Another thing to note is that since we are making a proper copy of the string first, and we are not changing the contents of the string, only the ordering, we can hopefully disregard negative side effects of using the NLS bits...

I'll make a posting later with the methods devised, there are faster versions of the algorithm than what you posted when using unsafe code.
Saturday, June 12, 2004 7:34 PM by Austin Ehlers

# re: Performance: Fastest string reversing algorithms... (final results)

Ok, added string.Copy in the for-loop. Also, I actually tested 10million, not 1. Here's the new results:

C:\>f 20 1000000
ReversePointer method: 00:00:00.1250000
ReverseCharArrayCopy method: 00:00:00.3437500
ReverseCharArrayCopyAndReverse method: 00:00:00.4843750
ReverseUsingStringBuilderInline method: 00:00:00.8906250
ReverseUsingStringBuilderCopy method: 00:00:00.3125000

C:\>f 200 1000000
ReversePointer method: 00:00:00.5468750
ReverseCharArrayCopy method: 00:00:01.1562500
ReverseCharArrayCopyAndReverse method: 00:00:01.1875000
ReverseUsingStringBuilderInline method: 00:00:07.0156250
ReverseUsingStringBuilderCopy method: 00:00:01.5000000

C:\>f 2000 1000000
ReversePointer method: 00:00:04.7656250
ReverseCharArrayCopy method: 00:00:09.5000000
ReverseCharArrayCopyAndReverse method: 00:00:08.4843750
ReverseUsingStringBuilderInline method: 00:01:09.4687500
ReverseUsingStringBuilderCopy method: 00:00:12.6875000

I also didn't know this was a contest (I followed the link here from a NG post)
Saturday, June 12, 2004 7:39 PM by Justin Rogers

# re: Performance: Fastest string reversing algorithms... (final results)

Here you go, I gave you proper attribution and refactored the algorithms out. The unsafe code is clearly faster, as one would expect. I made some notes that you might be interested in as well. In all honesty it would be possible to achieve the same speeds without unsafe code if we were allowed the ability to create a volatile string for some short period of time and party on the character array using Array.Reverse which if you follow the rotor source is identical to our unsafe based swap methods.

http://weblogs.asp.net/justin_rogers/archive/2004/06/12/154375.aspx
Saturday, June 12, 2004 10:39 PM by TrackBack

# re: Performance: Fastest string reversing algorithms... (final results)

Saturday, June 12, 2004 11:42 PM by sean grimaldi

# re: I applaud the NetworkInformation namespace, however, where is my IsNetworkAvailable property?

Hey I'm no expert here, really, but IsNetworkAvailable would't help much. All it would answer is was the network available a moment ago when the method ran. If the network was there so what? What you really want to know "will the network be there when I send my expensive network call so I don't have to handle errors if it isn't." Of course, when it comes to network programming I am usually wrong, so don't get too exited and just let me know what you think. I'll probably learn something from you.
Saturday, June 12, 2004 11:50 PM by Justin Rogers

# re: I applaud the NetworkInformation namespace, however, where is my IsNetworkAvailable property?

Sean, the use of something like IsNetworkAvailable would be for the Mort programmer that is looking for an easy way to determine if they should be running in online/offline mode. Irregardless of how you program your network stack and how often you check the connection, it can alwasy die in the middle of your expensive network call anyway. Right now my point is that NetworkChange still forces you to enumerate the network adapters and determine the state of the network to determine if you are still actually on the network. A mixture of IsNetworkAvailable (that does the work of finding out if the network is there), and making sure the event doesn't have any permissions on it, so anybody can hook it would be ideal. That way Mort can use the property, and Elvis can use a mixture of the property and the event. Einstein programmers are likely to write their own code anyway to determine the network status (aka, I'd probably ping the local gateway myself) and switch between local endpoints if multiple adapters are available. However, when I write code to throw out to users, and demonstrate how easy it is to switch their programs between online/offline mode, the property would be hella handy.
Sunday, June 13, 2004 1:54 AM by Duncan Mackenzie

# re: I applaud the NetworkInformation namespace, however, where is my IsNetworkAvailable property?

I haven't tried this code on the May drop... but this is from the March Community preview;

If My.Computer.Network.IsConnected Then
If My.Computer.Network.Ping("www.duncanmackenzie.net") Then
Debug.WriteLine("Site Available")

My.Computer.Network.DownloadFile( _
"http://www.duncanmackenzie.net/Articles/", _
My.Computer.FileSystem.SpecialFolders.MyDocuments, _
ShowProgress:=True)
End If
End If

It seems like My.Computer.Network.IsConnected is at least part of what you are looking for...
Sunday, June 13, 2004 1:01 PM by denny

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

Hmmm.... some very good points in there...

Sunday, June 13, 2004 1:06 PM by Justin Rogers

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

Yes, truly a question of what you are gaining and at who's expense. I clearly think this new paradigm is a shift towards fraud protection at the expense of every user that has to fill out a form. So much for Gator and my automatic form filler.
Sunday, June 13, 2004 4:00 PM by David

# Capturing panel events

Hi.
I'm really impressed with your work.
But I can't figure out how each panel can control the various events. For exemple, your license panel should be able to control if the license has been accepted or not, and change the nextPanel accordingly.
Another problem is how each panel can control the form it belongs to. For exemple, to change the "Wizard Step Title" text.

Thanks. David
Sunday, June 13, 2004 6:02 PM by Pavel Lebedinsky

# re: Commenting on ChrisAn's reliability posting because he BlogX'ed himself into a no comment corner

I'm not sure if hardening the framework against OOM and other similar conditions really makes sense.

With a lot of work it would probably be possible to harden the CLR and libraries so that it's theoretically possible to write apps that can recover from OOM, or do some cleanup before terminating. But even then you'd have to spend lots of time and effort testing each "hardened" application to make sure running out of memory doesn't cause corruption in any of the components.

For most server applications it's easier and more reliable to simply recycle the process. This can even be done before the process actually runs out of memory (the IIS/COM+ worker process model).

Client apps like Word is where it gets more interesting. If I paste a huge Visio drawing into a Word document and Word runs out of memory, should this result in process termination? Probably not (however one could imagine a model where UI and actual processing are implemented in separate processes so the UI process cannot easily run out of memory and the background worker process can be recycled without the user even noticing it).

However if the data in the document was really important (like some financial data) then I would take no chances and restart the process then load the last known good version of the data.

What is needed in my opinion is some kind of configurable policy that describes how an application wants to handle critical failures (OOM being just one example, others could include unexpected SEH and managed exceptions, stack overflows etc). The default should be to kill the process and submit a Watson crash report, but individual apps should be able to override it.
Sunday, June 13, 2004 7:34 PM by Justin Rogers

# re: Adding a design time dialog and creating a VS project sample

David, you'd have to look at Article 6 that covers the metadata API and container. I haven't found the time to push out Article 7, which actually implements the metadata API into the wizard itself and allows for the features of controlling Next/Previous enable/disable, as well as changing the Wizard Title etc...
Sunday, June 13, 2004 7:56 PM by Justin Rogers

# re: Commenting on ChrisAn's reliability posting because he BlogX'ed himself into a no comment corner

Recycling, in and of itself is a possible source of corruption. You have to make sure and properly save out state and do many other things when this happens. There are currently hundreds of ASP .NET applications that don't properly save out state during an application shutdown and recycle. Users performing multi-page actions all get their chance to lose data in this scenario.

If, as you say, it were some financial data, who can say that hte LKG is the best version of the data to load? What if the LKG misses approximately 2-3 seconds of important information that I typed in right before I loaded the Visio document? Or should the app save before doing any paste operation? That sounds costly. I'd venture that hardening against OOMs is crucial, and allowing me the ability to save out the version of the document that I have in memory containing my 2-3 seconds of important data is better than loading an LKG. Hell, recovering that large amount of memory possibly and contuing the process of running Word may be even more important.

Some processes just can't be terminated when something goes wrong. You can't just terminate someone's game client in the middle of their game, thats crap, but games tend to gracefully handle OOMs quite a bit. They have a true reliability story and they provide fallback functions for cleaning memory, such as refreshing and reloading the entire graphics set currently being used with lower resolution versions to conserve memory.

I'm not advocating every application take care of this process, and as I noted, this would be a special mode, a form of reliability mode. Most of the CLR could benefit from the creation of the special mode and be more hardended to OOM's but only with special consideration could you set up the ability to handle an OOM and recover or save your state.

As for spending a bunch of time testing my hardened application, that is a matter of opinion. In many cases the CLR itself will better recover when an OOM strikes, meaning I get that much hardening for free and they do the testing. My app being hardened against OOM's and other critical failures is nothing more than a feature of my app. I'd spend just as much time working this feature as I would any other feature. Which app would you buy of the following two, "Excel that recycles if your Workbook gets too big", "Excel that gracefully recovers from a large Workbook and offers recommendations for solutions, one of which is to shut-down and reopen the workbook."

Sunday, June 13, 2004 10:13 PM by I have to disagree on your second and third points

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

I have written a custom CAPTCHA (the generic term for the type of human-detection code you abhor) for my browser-based game because after all there's not much point in the game if it becomes an exercise in who runs his script the longest. So for some purposes there's a choice between trying to foil scripts and shuting down your site entirely... Anyway, doing this since 2001 I have found the following:

- humans are remarkably good at pattern recognition. Most people miss a couple while they are starting out, but very seldom after they get the hang of it. (The penalty for missing in my game is a suspension of one hour so it's not the end of the world, and you can pay a game-currency fee to avoid even that.)
- anyone with good written English skills can read them. A significant portion of my user base does not speak English as a first language. (Actually, the people who have the most trouble are American school children; I use a cursive font to make things a bit more difficult for the would-be cheater, and apparently cursive is becoming something of a lost art in today's education system. So I link a cursive tutorial.)

And no, my captcha doesn't rely on adding noise to the image for the simple reason that, as you have shown, it only takes a few minutes to write code to strip that sort of stuff out. I've included the url if you're curious.

P.S. warning: The variable 'delta' is assigned but its value is never used
Sunday, June 13, 2004 10:15 PM by Jonathan

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

wow, I flubbed that pretty nicely. url is nicely hyperlinked with my "name" in the above post.
Sunday, June 13, 2004 10:46 PM by Justin Rogers

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

Where exactly do you implement your pattern system? I just messed around the site for about half an hour and didn't see one. I think using your limiting system is probably more than enough to stop the average bot.
Monday, June 14, 2004 12:21 AM by JFo

# re: Is it just me or does the TableLayoutPanel not really help all that much?

Justin -

You've hit the nail on the head. This scenario is one of the ones that's going to be much easier using the old Dock and Anchor standbys we've come to know from previous releases. Imagine though, that you've databound all your labels to some fields in a database - and the labels would have to grow and shrink depending on the widths of the strings... or you're shipping this app in another language, and all the strings are different widths - suddenly the autosized column becomes much more useful.

Regarding Dock, you can also use the Anchor property to align and stretch the Text boxes - anchoring left,right should stretch it to fill the cell. (That still doesnt help turn multiline to true if that is what you really wanted though).

Regarding Resize, some of this can be done today with the control - if you play with the row and column styles collections, you can change the SizeType to be Absolute, Percent or AutoSize.

I hope by the time Whidbey has shipped you'll find this control easier to figure out and a little more consistent between runtime/design time (WYSIWYG).

These suggestions are all great!

JFo
Monday, June 14, 2004 10:36 AM by Jonathan

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

yeah, it starts serving them to new players after 6h? I forget the cutoff. But here's a page with some examples:

http://www.carnageblender.com/challenge/help.tcl
Monday, June 14, 2004 7:58 PM by Justin Rogers

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

Okay, so I've played the game a bit more and gone through a number of the challenges. I would dare to take the position that the idea of noone hacking this yet is absurd. You clearly used a fixed font set when creating these words, so with a glyph set of only 26, a dictionary file of only 3 and 4 letter words, and the ability to make annotations within the dictionary, the bot could be trained to do the scanning in less than 20 seconds per page, and if you ever repeat the same item (which I swear I've seen at least once so far, I've actually played quite a bit I guess ;-) then a very simple CRC to word lookup could be devised to answer questions on known samples in under a second.

About 8 years ago I did work for a small phone company branch (the phone company wasn't small, just the branch). The position of the company was to sell various services in previously untapped markets, so they tended to sell to third world countries. Now, being a small branch, they often got tasked with doing input jobs based on polls and collections they had taken. Generally the documents were printed on a set of very specific printers, each of which had specific glyphying problems. You start to see a problem in that the software based OCR's of 8 years go simply couldn't handle this type of data. Hell, it was hard enough to put the data in by hand, since you often times couldn't read the document without scanning and enhancing it anyway.

Long story short the process was very tedious. However, not creating a general solution like the OCR software was doing, I created a specific solution that identified the printer the document came off of by looking for tell-tale glyph abnormalities. Once this was done I could easily load the appropriate subset of matching glyphs for processing the document in question in much faster time than the OCR software and with many less errors. What had previously been a full day job of correcting OCR errors at the average of 25 per document, became a 30 minute job of correcting the 1 or 2 errors present in 1 of every 10 documents. Humans may be great at general OCR, but we sure can't touch a computer when it knows exactly how to expect the incoming data stream to look.
Monday, June 14, 2004 8:40 PM by Justin Rogers

# re: A quick note on security and anti-spam tactics that take advantage of human pattern matching abilities...

Quick correction, the number of glyphs is closer to 30 something, a couple of the letters have some strange deviations (the e for one). Randomizing the skew translation would stop a uniform glyph approach, but would make the words harder to read for human. I still like the concept of overlapping two words since the algorithm tends to have a hard time differentiating which pixels belong to which word. Still, it doesn't take long to converge onto a matching glyph with the current system.
Tuesday, June 15, 2004 6:47 AM by Ashish

# re: Do syntax parsing and code highlighting require language parsers or something more?

I have something to ask about C# code pasring, however which may not be exactly related to this article.
Has Microsoft provided any method to parse the source code files and return a parse tree in the form of related CodeCompileUnit object. There is a method called CreateParser of CodeDomProvider. This is virtual method. CSharpCodeProvider and VBCodeProvider inherits CodeDomProvider but do not provide implementation for this method.
The return type of CreateParser is System.CodeDom.Compiler.ICodeParser. Which again has an abstract method called Parse(StreamReader) and which returns the object of type CodeCompileUnit.

CodeCompileUnit provides CodeDOM graph which represents Source codes in a tree structure. However, it seems that Microsoft has not provided any method with actual implementation of parsing C# or VB.Net source codes and represent the given source codes in the form of CodeCompileUnit Object.

Will you please spend some time for me & help me in this regard ?
Please email me at meeashish@yahoo.com

Thanking you in anticiaption of early reply

Ashish
Wednesday, June 16, 2004 5:51 AM by 耳机

# re: Generic predicates are pretty powerful, but the FindAll implementation doesn't show it.

yes
Wednesday, June 16, 2004 5:52 AM by green raisin

# re: Performance: Different methods for testing string input for numeric values...

i like
Wednesday, June 16, 2004 5:53 AM by mini e-bike

# re: Performance: Different methods for testing string input for numeric values...

yes
Wednesday, June 16, 2004 6:15 AM by foot scooter

# re: Performance: Different methods for testing string input for numeric values...

thank you
Wednesday, June 16, 2004 6:16 AM by electric motorcycle

# re: Performance: Different methods for testing string input for numeric values...

good
Wednesday, June 16, 2004 12:56 PM by nospamplease75@yahoo.com (Haacked)

# RE: Multiplying numbers in the Middle Ages was considered difficult. The solution to this may surprise you.

This is the technique that the mathemagic guy uses for mentally squaring numbers quickly.
Thursday, June 17, 2004 4:51 PM by Kurt Summers

# re: Fast absolute, and statistical line counting algorithms for use with progress notification

Hello. Can you send me this code in VB.NET?

Very helpful!

kurt_summers@hotmail.com

Thanks!!
Thursday, June 17, 2004 6:47 PM by patrick

# Phone number to Words code

Can I use this code if I like?
Thursday, June 17, 2004 10:29 PM by Justin Rogers

# re: Trying my hand at the old Phone number to Words teaser project!

Sure.
Wednesday, June 23, 2004 7:01 AM by Grey

# re: WinForms Wizard Series Article 6, a quick look at implementing a meta-property storage class.

When is article 7 coming up ? ... and the Gaming sample ;-)
Wednesday, June 23, 2004 10:52 AM by Madan

# re: Follow-up on Screen Capturing in Whidbey, I'm a fool who doesn't do all of his homework...

Is this a c# code, i don't find copyfromscreen method in graphics class. How to do that let me know.
Wednesday, June 23, 2004 5:59 PM by TrackBack

# Background Processing

Wednesday, June 23, 2004 6:15 PM by TrackBack

# Interesting Post on Math and the Middle Ages

Monday, July 05, 2004 3:01 PM by Phil

# bringing that post back to life

Hi everyone,

(first of all, excuse my English, I'm not a native speaker)

Seems like I'm a bit late on this topic but since I'm working on a anti-bot image generator at the moment, I thought I might get a few more informations here...

I quickly wrote a PHP-script that generates an image (http://www.blutch.net/checkimage/index.php), and even though its quite simply construction, I feel like it's not really easier to OCR'ize than Microsoft Passport's one, for instance (yet I feel like the characters are easier to read for a human). It merely uses random rotations, two different blurred fonts and a few random lines (whose utility hasn't convinced me yet).

You seem to know quite a lot on the topic, so you might want to give a few more recommendations on what I could do to improve the proofness (if that word exists) of the image.

Thanks in advance (if anyone ever reads this),
PS

(answer by mail in addition to posts here would be appreciated: blogs [ a t ] lar ampe·com (you know of course what to to with the whitespaces)).
Tuesday, July 06, 2004 5:07 AM by David Larsson

# re: What are your thoughts on the new ArraySegment structure in Whidbey?

The ArraySegment<T> struct would be MUCH more useful if it also implemented IList<T>, ICollection<T> and IEnumerable<T>. Then it could be used as an argument to APIs that take generic lists as input (which we will hopefully see more of instead of forcing callers to use plain arrays just for type safety). You would also be able to enumerate over a segment with foreach().
I like the idea of hiding the underlying array for the reasons you mention.
For completeness, there should be an even more general ListSegment<T> that wraps an IList<T> instead of a T[]. Of course, there are problems if the underlying list is modified, but that's scout's honour, right?
Tuesday, July 06, 2004 12:23 PM by Steve

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

in the .net the WndProc has the type:

void WndProc(ref Message m)

While in MFC it has the type:

LRESULT WindowProc(UINT message,WPARAM wParam,LPARAM lParam );

Where did the LRESULT go... has anybody a clue?
Thursday, July 08, 2004 7:19 AM by Ben

# re: Does anyone find the FTP support in Whidbey counter-intuitive?

I haven't had a chance to try out the new Ftp classes yet, so I can't really comment. But have you filed suggestions/bugs regarding the above issues?

The site is here:
http://lab.msdn.microsoft.com/productfeedback/

It seems to be the best place to voice your opinions if you want something to be reviewed and possibly changed/improved.
Thursday, July 08, 2004 10:30 AM by Mike Swaim

# re: Does anyone find the FTP support in Whidbey counter-intuitive?

You could try Indy. It started out as a Delphi library, but there's a .net version out as well.

http://www.indyproject.org/
(And you should probably still complain to MS.)
Thursday, July 08, 2004 7:17 PM by Darren Neimke

# re: Adding some much needed dock sizing for System.Windows.Forms.Label... (Whidbey, but probably works on V1.x)

You spelled Label wrong :P

Very nice... thanks for the tip!
Thursday, July 08, 2004 10:07 PM by Justin Rogers

# re: Adding some much needed dock sizing for System.Windows.Forms.Label... (Whidbey, but probably works on V1.x)

All of the intellisense features in the world can't stop my intellistupid names!
Friday, July 09, 2004 12:09 AM by Darren Neimke

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Greg, a common example might be if you wanted to update some UI off the back of the Timers.Timer elapsed event because the timer is running off on a different thread you will need to Invoke back onto the main thread before touching the UI.
Friday, July 09, 2004 12:12 AM by Darren Neimke

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Justin,

Great article!

The one thing that I didn't quite get was in section 4 when you state:

****************************
If we didn't make this check and instead just started a WaitOne on the handle, we'd hang indefinitely.
****************************

Why? Are you saying that, under that circumstance you'd be guaranteed to hang indefinitely? Don't quite get that I'm sorry.
Friday, July 09, 2004 6:49 AM by Darren Neimke

# re: Adding some much needed dock sizing for System.Windows.Forms.Label... (Whidbey, but probably works on V1.x)

BTW... yes it does work in V1.x ;-)
Friday, July 09, 2004 12:16 PM by DavidL

# re: Watch out for the VS 2005 Debugger, it lies sometimes!

Is the debugger set to break on 1st chance exceptions? It might show the LOC where the exception is occurring.

I haven't run the Whidbey or Longhorn bits but I've been told that unhandled exceptions will be handled differently then the v1.1 runtime so that the app will be taken down when one occurs - do you know if ignoring the network async socket exceptions will still be ok? I've seen similar exceptions on blocking socket calls.

Sunday, July 11, 2004 3:47 AM by Robert Temple

# re: If you could pick the reason why GDI should be .NET accessible, what would it be?

Its gone!

They have taken it away in the Whidbey beta 1.
The namespace was changed to System.Internal.Gdi and all the public classes changed to internal.

I need the GDI stuff because I'm using a TextBox control as a temporary editor for a line of text in a custom control. Its similar to the TreeView, when you start a label edit.

Because the text in the TextBox is still drawn using GDI, I need to draw the string below using GDI, otherwise the appearance of the line of text changes when the user is switched into edit mode.
Sunday, July 11, 2004 5:48 AM by Greycells

# re: How writing software that can do anything turns into winning the lottery ;-)

You are just too cool man !
You are what every programmer would like to be in 10 years from now
Sunday, July 11, 2004 8:47 PM by TrackBack

# A reader of my blog points out that System.Drawing.GDI is frigin gone in the betas...

Monday, July 12, 2004 3:10 AM by Jonathan de Halleux

# re: Performance and Memory: StreamWriter/StreamReader/FileStream...

You should provide some benchmark if you talk about performance :)

You can use my benmarch framework NPerf to do that easily ( http://www.codeproject.com/gen/design/nperf.asp )

Cheers,
Jonathan
Monday, July 12, 2004 4:32 AM by Justin Rogers

# re: Performance and Memory: StreamWriter/StreamReader/FileStream...

Hey, no bullshit advertising ;-) Eck, just kidding. The above are general guidelines. Demonstrating performance on disk bound IO is going to be different depending on many circumstances.

Generally I make a real habit of providing performance benchmarks, but when the cold start time of a Windows Forms application can range between 5 the 25 seconds on the same machine (all disk bound IO related), then you can see where local benchmarks don't really mean much.
Monday, July 12, 2004 6:07 AM by Chris Bell

# re: Throwing out some WinForms candy to all the kiddies: Singleton Forms...

Great, I've been looking for something like this for a while. A couple of things, doesn't the KeepAlive method need a call to Hide()? And what is the LooseSingleton method for? Seems like it creates non-singleton instances of the form, which seems a bit odd...
Monday, July 12, 2004 6:37 AM by Justin Rogers

# re: Throwing out some WinForms candy to all the kiddies: Singleton Forms...

Woops, added the Hide, I was being rushed to a dinner when I was typing that in. I didn't even compile the source to make sure it worked.

As for the LooseSingleton, there is a reason for that. Often times you need multiple instances of the same form, but you still want them to keep their resources allocated. You might have a common dialog for instance, that keeps some state based on different code paths. The LooseSingleton allows you to grab a form and attach it to a field local and use the field local as the singleton, while still giving the ability to have multiple field locals each as it's own singleton.

Perhaps a Hashtable of names instances with an indexer would better demonstrate this process?
Monday, July 12, 2004 8:41 AM by Chris Bell

# re: Throwing out some WinForms candy to all the kiddies: Singleton Forms...

Thanks for the explanation, makes sense now!
Monday, July 12, 2004 10:16 AM by TrackBack

# Whidbey FTP Support Short - Directory List Format Support

Tuesday, July 13, 2004 3:04 PM by Frans Bouma

# re: Confused Programmers Question Code... (C# 2.0 and the nullable syntax)

Add this to the confusion:

int? foo;

// datareader dr is initialized and looped through
foo = dr["Foo"].Value; // can be NULL (DbNull.Value)

this doesn't work at the moment. However this is (IMHO) one of the most useful areas for nullable types.
Tuesday, July 13, 2004 3:20 PM by nospamplease75@yahoo.com (Haacked)

# RE: Confused Programmers Question Code... (C# 2.0 and the nullable syntax)

It actually sorta makes more sense if you spend a lot of time reading regular expressions for fun.

? = 0 or 1.

Tuesday, July 13, 2004 4:59 PM by Per Hultqvist

# re: Confused Programmers Question Code... (C# 2.0 and the nullable syntax)

Since the letter u is used for unsigned, my proposal is to use the letter n as an prefix (n for nullable), i.e. :

nint = nullable int
nlong = nullable long
...

I just realized when typing that we would then need the following too:

nuint = nullable unsigned int
nulong = nullable unsigned long
...

Now I'm not so sure if it is a good idea anymore...oh well...time to sleep anyway.../Per
Tuesday, July 13, 2004 11:10 PM by Justin Rogers

# re: Confused Programmers Question Code... (C# 2.0 and the nullable syntax)

To Haacked: Yeah, use something from regular expressions because that isn't already ranked as the most confusing programming construct in history ;-) j/k, I think it is a good point.

To Per: The problem with prefixes is that they'll work for some types, but not others. For instance, every integral type is a reserved word, as is every unsigned integral type. It would introduce many more reserved words to add the n to these already reserved keywords.

You also couldn't differentiate between two value types, Perhaps struct Between {} and struct nBetween {}, trying to be a bit too clever there, but it demonstrates the point.

I would say having a nullable keyword is nice, however, that is nearly as verbose as the Nullable<> format already, and so wouldn't work very well.

My guess is that these nullable types are considered to be so extremely important that they warrant a shortcut in the form of the type post-fix nullable operator. So far, I've made good use of them in my code, but still not enough to warrant a new language prefix. I'm not sure making it easy is the right thing here, since many users are inclined to use any features that are easy, even if they aren't the right feature for the job.

IMHO, using the question mark in this circumstance is going to be perfect, since most programmers will be asking themselves "Do I even need this damn thing?", "What in the hell am I doing here?", and possible "Why would C# make it so easy if it were wrong?".
Tuesday, July 13, 2004 11:25 PM by nospamplease75@yahoo.com (haacked)

# RE: Confused Programmers Question Code... (C# 2.0 and the nullable syntax)

Oh C'mon. It's no more confusing than PERL syntax. oh wait, that is confusing. ;)
Wednesday, July 14, 2004 1:39 AM by James

# re: I spent far too much time making the Terrarium server Hack-Proof...

Schneier's Law:

"Any person can invent a security system so clever that she or he can't think of how to break it."

:)

Wednesday, July 14, 2004 2:50 AM by Justin Rogers

# re: I spent far too much time making the Terrarium server Hack-Proof...

That is an interesting statement. Maybe there is some hope that some other users will find flaws in the consitency model that I didn't forsee.

One thing to note is that after two years my own skills have increased immensely over what they were when the code was originally written. In fact, the code didn't even look like mine as I was going through it the first couple of times. I used tests against the model that I didn't have available when I first made it. Even to these new tests it stands up. I'll be very interested to see an actual hack come through.

That said, machines are power, and a user with a large subnet of machines could easily exceed the numbers I'm capable of pushing here at home. There are other issues such as worker process roll-over that could expedite changing and logging with ones new Guid before the 12 hour time limit. Another issue is the ability to change the default settings through config that might open up some users to consistency hacks if they run their own servers.
Wednesday, July 14, 2004 4:34 AM by Pattern Guru

# Programmers aren't confused between syntax and semantics, are they...?

So for me, using a question mark (ASCII 63) in code is something different than writing a question mark (that wriggly thing with the dot under it) on a piece of paper. Or typing it into a Word document or web log comment box, for that matter.

If a programmer isn't confortable with abstractions and semantical stuff like this, how much of programmer are you really...?
Wednesday, July 14, 2004 4:36 AM by Pattern Guru

# Programmers *are* confused about natural language... ;-)

Ehh... "comfortable" and "of a programmer", of course...
Wednesday, July 14, 2004 5:03 AM by Peter

# re: Darn, I'm officially doing two webcasts now ;-) I guess it won't be that bad...

I am looking forward to your Poker webcast. Btw, when is the book comming out ?
Wednesday, July 14, 2004 5:32 AM by Justin Rogers

# re: Confused Programmers Question Code... (C# 2.0 and the nullable syntax)

I'm not really speaking of the comfortability with the abstractions, I'm more pointing out the very real scope of human behavior with respect to symbols.

Hearing running water and wanting to pee doesn't make you less of a man, just because a man should be able to hold his liquids.

Nor does having a pre-programmed feeling towards a symbol appearning in code have any bearing on how good of a programmer you are.

For instance, if you saw a ;) as a code construct or a ;-) even, you might for an instance smile, even if you don't want to. The same point is in using things like question marks, which will cause most people to double take, at least the first few times.
Wednesday, July 14, 2004 5:36 AM by Justin Rogers

# re: Darn, I'm officially doing two webcasts now ;-) I guess it won't be that bad...

Book is due very soon. Still in the review process and finishing up a couple of chapters. Battling with various absent .NET 2.0 features and countless *cough..bugs..cough* makes creating extremely refined applications fairly difficult.

Glad to hear you are interested in the Poker WebCast. I only think I'm going to make it an hour on that particular run and I'm reaching a bit in some places in the slide deck to make up for lost time. The problem is the lack of support during the web-cast for really showing off any Managed DirectX, and so I'm making a simplified control layout game. Another issue of course is not being able to set up an appropriate multi-user framework for showing off the multiplayer functionality. That means living with the AI.
Wednesday, July 14, 2004 6:17 AM by Peter

# re: Darn, I'm officially doing two webcasts now ;-) I guess it won't be that bad...

Too bad (I hope there will be download to try it at home with DirectX).
Wednesday, July 14, 2004 10:23 AM by TrackBack

# Peter's Blog » Creating an Instant Messaging Application

Peter's Blog &raquo; Creating an Instant Messaging Application
Wednesday, July 14, 2004 6:31 PM by Rich Miller

# re: C# 2005 Express WebCast 1 of 2 is now complete!

Your to hard on yourself, it went great.
Thanks
Thursday, July 15, 2004 1:37 AM by Michael

# re: C# 2005 Express WebCast 1 of 2 is now complete!

Agreed! Thought the webcast was great! (I have never been here before, I'm here cause I saw the presentation.)

In fact the one thing was that you really were to hard on yourself about was formatting code, PLEASE FORMAT THE CODE (heh) I found it unreadable the way C# Express was generating it and as you formatted it, I could easily process what was what and it made it a lot more useful to me cause I could understand what was being written (When your seeing this low-framerate desktop and your hiting tab it looks like to code just appears so it was hard to "get") so again thats why the formatting helped a lot!

All in all great job!! Wish I could attend the poker one but I have the IRL meeting at the same time :-( But it gets recorded so thats cool.

P.S. (Last comment) I also liked that you ran into a couple of compile snags way to often things are so pre-packaged that I think I get less out of it, I mean were going to run into the exact same things when we try and write it :-)
Thursday, July 15, 2004 8:13 AM by Peter

# re: C# 2005 Express WebCast 1 of 2 is now complete!

I agree with Rich. The webcast was great and I hope the Poker webcast will go even better.
Monday, July 19, 2004 8:48 AM by Micael Baerens

# re: Solitaire .NET will be parking at www.Games4dotnet.com, the first actual game to get released...

Damn - too many users connected..
Monday, July 19, 2004 9:38 AM by Peter

# re: Solitaire .NET will be parking at www.Games4dotnet.com, the first actual game to get released...

I would definitely change the background :) - but anyway it's the content that matters not the appearance.
Monday, July 19, 2004 1:33 PM by Jason Olson

# re: Solitaire .NET will be parking at www.Games4dotnet.com, the first actual game to get released...

Yeah, I think as long as you have good content (which I have no doubt you will), I could care less about the look of the site. I've been waiting a *long* time for you to bring the site up (I think ever since you first mentioned it). I can't wait....
Monday, July 19, 2004 1:55 PM by jaybaz [MS]

# re: I'm liking C# Express more every day. Personal insights from a notepad warrior...

Hey Justin,

I'm from the team that makes the C# expansions feature, so I was interested when I read "I've deleted all of the default expansions from my snips list and added my own".

I'm curious what snippets you added. Perhaps you have some suggestions of a few that we should ship with Whidbey?

BTW, thanks for all the great feedback. Please keep it coming.
Monday, July 19, 2004 2:10 PM by Jeff

# re: Solitaire .NET will be parking at www.Games4dotnet.com, the first actual game to get released...

Good idea, but you do realize that you're stomping all over a certain shitty TV network's trademark by using "G4," right?
Monday, July 19, 2004 5:06 PM by Justin Rogers

# re: Solitaire .NET will be parking at www.Games4dotnet.com, the first actual game to get released...

Too many users connected? Don't tell me I have to go drop the site back onto a more stable box. If I must, I must I guess.
---

I'm assuming you mean the background for the entire site and not the black or the content area which is a kind of steel blue. The background for the site is a steelblue and black imprint. It was originally white and black, but simply didn't go well with the site. I'll definitely look into fixing the background.
---

As for the G4 trademark, I would recommend checking out the trademark offices and noting that there are a few thousand G4 trademarks. The trademark is on the image, not on the letters or order of combination. My G4 appears to be original for now.

If anyone would like to help with the branding of the site please let me know. You can submit sketches, finished images, or whatever, and I'll see if they fit better as a branding logo. Here are the branding features behind the site:

1. It is a .NET Game Programming Site. All content will discuss either abstract game programming or use the .NET Framework or Managed DirectX. No exceptions at this point. All art will be discussed as it was finished or produced for working within a .NET Game. That means lots of GDI+ enhancement tips.
2. The current branding allows for shortenings "G4", "G4.net", "Games4".
3. The current branding allows for interesting layouts "4 within a G", "Games in a circle around a 4", "Games 4 in a circle around .net".

I'm trying to stray away from usage of things like C# and VB .NET in the branding portion, though I'm not beyond doing it. A solid logo independent of the reference graphics would be nice as well, something that can go on games, splash screens, integrated as an icon, etc...
Monday, July 19, 2004 5:32 PM by Justin Rogers

# re: I'm liking C# Express more every day. Personal insights from a notepad warrior...

Ah, the expansions list is based around common programming features from a code perspective, but isn't geared towards the features that a user winds up programming. Some common expansions I added have been:

1. Default IO Loops - About ten or so IO loops that I use very frequently when working network and file based IO.
2. N-Tier deep looping constructs - I added about 15 looping constructs so far, but three of these are just 1, 2, and 3 level deep looping. The 1 was already done of course. Other for loops include the shortcut loop, the not null while, the value type while, etc...
3. Various Surroundings - I often place debug code, assertions, or test loops around code and so a series just for this purpose was added. Again, most of these are to my style so if you head back through my numerous code postings you'll find some of my test work and realize the repetitive nature involved.
4. Common control initializations - Toss in some basic layouts. I joked with Dan about UI refactoring, however, it wasn't all that much of a joke. I have some Windows Forms based snips that help working with the designer generated file.

I notice there is a lot of power in the snips and expansions format. If you want to contact me privately I'd love to get more information on the functions available. A spec or something on the feature set maybe? Currently I'm just poking in the dark with the possible power.
Tuesday, July 20, 2004 3:54 AM by Paul Bartlett

# re: BlogShares has changed their graphing, but is still using PHP...

Tip: add a link to BlogShares with your own ID as referrer and earn chips, e.g. http://blogshares.com/?referer=16545
Tuesday, July 20, 2004 4:11 AM by Justin Rogers

# re: BlogShares has changed their graphing, but is still using PHP...

Hehe, Thanks Paul, I'll have to remember that in the future. I'm just a casual player of the game though, so I doubt I'll post any more about the site.

Unless of course it converts over to ASP .NET, which I highly doubt.
Wednesday, July 21, 2004 1:16 AM by Chyld M.

# book details

As a software engineer myself, I would agree that my most productive coding occurs late...really late.

I have some questions about your new book:
When will this book hit the shelves? And can it be preordered on Amazon.com?

Thanks.
Friday, July 23, 2004 3:34 AM by Asheesh

# re: Throwing out some WinForms candy to all the kiddies: Singleton Forms...

Nice article but one question.
Shouldn't we make the constructor private, so as to prevent others from instantiating an object of this class? We anyways are creating a singleton object in the current method.

Please let me know if I'm missing something.

Thanks,
Asheesh
Friday, July 23, 2004 11:48 AM by Justin Rogers

# re: Throwing out some WinForms candy to all the kiddies: Singleton Forms...

You can't make the constructor private, else you can't make the form designable. You can make the constructor private at the end if you wish.

This is what I call a loose singleton pattern because you control the singleton nature. Really the main purpose of the singleton property and method is to control form closure so that the form's are reusable.
Monday, July 26, 2004 2:38 AM by Peter

# re: Soooo much to do sooo little time. Some goodies while you wait for the real stuff...

Justin, those 2 links aren't working.
Monday, July 26, 2004 4:05 AM by Justin Rogers

# re: Soooo much to do sooo little time. Some goodies while you wait for the real stuff...

Alrighty, appears all of the spaces were messing things up or perhaps the # in the C#... Anyway, I've gone and fixed them up to shorter names. They should work now. Thanks Peter.
Monday, July 26, 2004 7:55 AM by Darren Neimke

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

Here's a pattern and an input string to test against:

pattern: a([^b]+|.)*c$
input: accccccccccccccc.

To crank up the time it takes for the Match to complete just add more padding into the middle of the string.

BONUS POINTS : for the first person to correctly reply with how many operations are performed against the above regex with an NFA engine!
Monday, July 26, 2004 12:20 PM by TrackBack

# Running matches Asynchronously for a bit of protection

Monday, July 26, 2004 2:56 PM by TrackBack

# S dot One heeft het over .NET » Optimalisatie van StreamWriter/StreamReader/FileStream

S dot One heeft het over .NET &raquo; Optimalisatie van StreamWriter/StreamReader/FileStream
Tuesday, July 27, 2004 1:22 AM by Asheesh

# re: Throwing out some WinForms candy to all the kiddies: Singleton Forms...

Thanks for the explanation! Makes sense now.
Wednesday, July 28, 2004 11:02 AM by Rudi

# re: Watch out for "Power Toys", sometimes you get more than you bargained for.

I guess the reason why they have put in the long hours to get the animations to work is because if you try your sample, it works.

If you add a button, it still works, if you add a Label, I get this exception:
Unhandled Exception: System.ArgumentException: hdc
at System.Windows.Forms.DibGraphicsBufferManager.CreateCompatibleDIB(IntPtr hdc, IntPtr hpal, Int32 ulWidth, Int32 ulHeight, IntPtr& ppvBits)
at System.Windows.Forms.DibGraphicsBufferManager.CreateBuffer(IntPtr src, Int32 offsetX, Int32 offsetY, Int32 width, Int32 height)
at System.Windows.Forms.DibGraphicsBufferManager.AllocBuffer(Graphics targetGraphics, IntPtr targetDC, Rectangle targetBounds)
at System.Windows.Forms.DibGraphicsBufferManager.AllocBuffer(IntPtr target, Rectangle targetBounds)
at System.Windows.Forms.Control.WmPaint(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.Label.WndProc(Message& m)
at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.SendMessage(HandleRef hWnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at System.Windows.Forms.Control.SendMessage(Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.Label.WndProc(Message& m)
at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.CallWindowProc(IntPtr wndProc, IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at System.Windows.Forms.NativeWindow.DefWndProc(Message& m)
at System.Windows.Forms.Control.DefWndProc(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.Label.WndProc(Message& m)
at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.CallWindowProc(IntPtr wndProc, IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at System.Windows.Forms.NativeWindow.DefWndProc(Message& m)
at System.Windows.Forms.Control.DefWndProc(Message& m)
at System.Windows.Forms.Form.DefWndProc(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
at System.Windows.Forms.ContainerControl.WndProc(Message& m)
at System.Windows.Forms.Form.WndProc(Message& m)
at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at AnimateWindow.Toast.AnimateWindow(IntPtr hwnd, UInt32 dwTime, UInt32 dwFlags)
at AnimateWindow.Toast.Show() in d:\my documents\visual studio projects\animatewindow\animatewindow\toast.cs:line 72
at AnimateWindow.Form1.button1_Click(Object sender, EventArgs e) in d:\my documents\visual studio projects\animatewindow\animatewindow\form1.cs:line 87
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.SendMessage(HandleRef hWnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at System.Windows.Forms.Control.SendMessage(Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.Control.ReflectMessageInternal(IntPtr hWnd, Message& m)
at System.Windows.Forms.Control.WmCommand(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
at System.Windows.Forms.ContainerControl.WndProc(Message& m)
at System.Windows.Forms.Form.WndProc(Message& m)
at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.CallWindowProc(IntPtr wndProc, IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam)
at System.Windows.Forms.NativeWindow.DefWndProc(Message& m)
at System.Windows.Forms.Control.DefWndProc(Message& m)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
The program '[2284] AnimateWindow.exe' has exited with code 0 (0x0).
at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.ComponentManager.System.Windows.Forms.UnsafeNativeMethods+IMsoComponentManager.FPushMessageLoop(Int32 dwComponentID, Int32 reason, Int32 pvLoopData)
at System.Windows.Forms.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
at System.Windows.Forms.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.Run(Form mainForm)

And I guess they did not have the option of fixing that...
Saturday, July 31, 2004 2:49 PM by Joel Moore

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

I glad I found this. Although much of the article went over my head, I learned enough from it to fix a bug that's been hassling me for some time now (showing a hidden window from a worker thread). Now I realize I'm going to have to buckle down and learn a lot more about multithreading in .NET before using it like I am. After I've done some learnin' I'll come back and read this again to see if more of it soaks in.

Thanks for saving my hide in the meantime.
Monday, August 02, 2004 6:11 PM by TrackBack

# Allegiance

Tuesday, August 03, 2004 7:09 AM by TrackBack

# Allegiance

Wednesday, August 04, 2004 1:37 AM by Mike Marinich

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Justin,
Thanks for the most detailed info on the topic I could find up to date (before running ILDASM)

I'd like to get some clarification on how the messages that result from calling Invoke/BeginInvoke synch up with user generated UI messages like mouse clicks or key strokes.

In the #6 you state that BeginInvoke results in a message post to the window pump. In this case any mouse click should get posted before or after my message and it’s processed without interfering with my delegate. I'm not certain whether the Invoke also ends up with a message in the window queue.

My UI is updated from multiple threads alone with the user actively interacting with it. The way to make sure the thread generated updates are sequential with the windows event (mouse/key) processing is to ensure the thread calls are posted to the same queue where windows messages are posted to.

If you have any suggestions, any other links I can gather more info from, I'd greatly appreciate it.

Thanks

My email is mike@crediware.com
Wednesday, August 04, 2004 4:02 AM by Justin Rogers

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Mike, I recommend reading section 7 very thoroughly. All BeginInvoke calls are processed when the very first message gets through the pump. This means they are all batched and processed at the same time. What can this mean for Mouse events?

Well, say you send a BeginInvoke (PostMessage), and the user clicks the mouse, and then you call another BeginInvoke.

When the UI thread kicks in, it will process the very first message and execute BOTH delegates, and then process the mouse. If you expected your delegate to occur after the mouse event then you'll need to come up with a better way to do invokes that links each delegate to the message that was sent to start it.
Wednesday, August 04, 2004 11:27 PM by Darren Neimke

# re: Darren's "Killer Reason for LookAround" posting has been haunting me

Nice article Justin! Some useful breadcrumbs for me to follow here :)
Thursday, August 05, 2004 10:57 AM by Nick Codignotto

# re: A slightly better WinForms wizard, and slightly more work.

Hi!

I took your code and all I did was change all instances of "Panel" to "Control". I had to add a parameterless ctor() to my designer-friendly Control-derived class, but now I can design "panels" (controls, really) in the designer and create a wizard with ease.

I even created a WizardPanelBase so my Wizard panels (controls, really... I have to come up with better names), look like this using partial classes in C# 2.0:

public partial class WelcomePanel : WizardPanelBase
{
public WelcomePanel() : base()
{
InitializeComponent();
}
}
Thursday, August 05, 2004 11:08 AM by Jeff Lewis

# re: Xml in the WebBrowser? What about easy RSS in the WebBrowser in Whidbey?

I borrowed this from somebody a while ago:
http://consultutah.com/rsspretty.xslt

It makes rss feeds pretty like:
http://consultutah.com/rss.aspx


Thursday, August 05, 2004 3:51 PM by Michael Ash

# re: Darren's "Killer Reason for LookAround" posting has been haunting me

I replied to Darren's post so rather than repeat myself I'll just refer you there but I mentioned a few reasons why lookaheads seemed to be needed, or at least the better choice for some regexes I wrote.
Friday, August 06, 2004 6:55 AM by Omer van Kloeten

# re: The problem with APIs like System.IO is that people ask questions like "How do I copy a StreamReader?"

I'm almost sure that the StreamReader's Dispose method closes the file itself, so you don't have to call the .Close method...
Friday, August 06, 2004 7:53 AM by Justin Rogers

# re: The problem with APIs like System.IO is that people ask questions like "How do I copy a StreamReader?"

You are correct in that you don't have to based on the language semantics provided by the using statement. However, in the interest of explicit rather than implicit programming, it never hurts to call the Close method.
Monday, August 09, 2004 10:35 AM by TrackBack

# Running matches Asynchronously for a bit of protection

Tuesday, August 10, 2004 4:56 PM by Wang-Cheung

# re: FunHi.com, a dating service, a new game, a bit over the top?

I think Funhi is a bit over the top on...... itself. The Asian Pride thing couldn't be more pale and worn out. It's so pathetic how every single asian girl in the world is a wannabee super model. In the end, from what I've seen (as a professional photographer) the majority of all these so called Asian "bombshells", or better backyard hoes just become porn actresses anyways. And even in that route they're not concidered at the professional level.
Wednesday, August 11, 2004 4:41 PM by William Stacey

# re: I'm liking C# Express more every day. Personal insights from a notepad warrior...

If you get a chance. Maybe you post some on using DataConnectors and Object bindings in Forms and Master/Detail, etc. Cheers!
--
wjs, mvp
Thursday, August 12, 2004 9:41 AM by John Kemp

# re: Math: A bi-weekly journal describing the use of math in solving programming problems

I'm planning on offering a 1 or 2 day workshop to middle-schoolers at my church who are interested in learning game programming. Do you have any suggestings for what language would be best? Most (all?) of these kids won't have any idea about C++/DirectX. I was thinking about going into basic game theory (I enjoyed your blackjack web casts), basic programming, and maybe a simple 2d scroller with simple collision detection. Any thoughts?
Thanks,
John
Thursday, August 12, 2004 10:20 AM by Justin Rogers

# re: Math: A bi-weekly journal describing the use of math in solving programming problems

Couple of thoughts.

1. If you are located in Washington State then I'd be more than happy to help you out by delivering some of the G4 content and see how the kids like it.

2. With middle-schoolers focus on games they are already familiar with and show them how they work. Hard-core mathematics might be too much for them to understand, but there are many relational concepts and hidden rules you can point out that they might not know yet.

3. Remember these kids are going to be more familiar with console games than computer games. Take advantage of that and build your samples around well-known mainstream games. Use icons like Mario, Pokemon and Tony Hawk, etc...

4. If you plan on using a game, make sure you have something playable and short that they can see. Don't expect them to know about the game already. I made this mistake when sampling a few games and only realized later that the target audience had never even played those games.

5. If you do make a game as complex as a 2D scroller, don't make the game itself the project. Instead spend the extra time to write the code for the 2D scroller, and give them some way of changing the levels and working with collision detection at the design level rather than the programming level.

The point here would be showing them how to construct worlds out of the blocks, letting them see the blocks and how things interact with them, and how the block doesn't have to represent the precise shape of the object being displayed.

By changing the parameters for the rectangles and/or collision spheres they can change how the game is played and interact with principles of consistency. Set up small challenges for them like creating a set of steps or something more ingenious like creating the entrance to a hidden zone by making use of the bounding regions in a creative way.

In the end, you only have 1 or maybe 2 days to push all of the content. Plan for 3 days worth of content and then judge interest. If the kids are happy with game theory and basic games then there is no reason to start into the process of programming. The programming tends to be an open ended tool, often more destructive than good until the user is adept with it.
Thursday, August 12, 2004 11:01 AM by John Kemp

# re: Math: A bi-weekly journal describing the use of math in solving programming problems

Thanks for the feedback. I really appreciate it. I'm actually in St. Louis, MO so it may be a bit far for you to come. I like what you said about staying more high level. I tend to get so excited about this stuff I would most likely try to do 4 semesters of material in 2 hours. Anyway, great insight, especially since you've presented to kids before, and keep up the good work. I really enjoy reading your blog.
Thursday, August 12, 2004 1:09 PM by TrackBack

# UI Update from threads, interesting Blog.

Sunday, August 15, 2004 7:05 PM by Darren Neimke

# re: Math Installment #1: Image layout logic

Nice article Justin... thanks for that!
Monday, August 16, 2004 4:17 AM by Peter Jausovec

# re: Math Installment #1: Image layout logic

Great post. I am looking forward to the next one.
Monday, August 16, 2004 6:37 AM by TrackBack

# No magic about ApplicationContext

Friday, August 20, 2004 6:54 PM by Darren Neimke

# re: Math Installment #2: I needed some circular oriented text

Justin, this is really useful stuff. I'm not that fussed about the code as I'd rather write it myself for the practice.

One thing that would be nice is a screenshot of your result.
Saturday, August 21, 2004 12:17 AM by Justin Rogers

# re: Math Installment #2: I needed some circular oriented text

Awesome, thanks a bunch for the feedback. I added a screen capture of the final rendering. The only unfortunate side effect is the number of extra space that GDI+ places around the measured text. It appears to put an extra pixel on the left and right. Maybe there is some work to be done in the layout logic that I haven't seen yet.
Saturday, August 21, 2004 2:47 PM by Doug_H

# re: Math: A bi-weekly journal describing the use of math in solving programming problems

Enjoy following your blog (almost day by day). Have learned much! Hope you can keep posting.

My dilemma is in choosing a starting language for my (braniac) 17 year old son. He's a serious gamer and can, when he's interested, digest extremely complex technical stuff quickly. However, he's also pretty lazy and easily bored. I think maybe that if he got a taste of programming (esp. games) he'd be hooked.

I prefer C++ and so would like to teach him programming using that, coupled with his interest in gaming, as the "carrot dangling in front of the rabbit". I have voluminous C++ tools and resources, especially 2D and 3D graphics and gaming development - plus I'm comfortable in that area. The problem is that using that route requires a pretty substantial period where he wouldn't be seeing many stimulating rewards for the efforts that he'd be required to make. I've been checking out Blitz Basic (3D game development language) as a possible alternative learning environment but was wondering if you or anyone had any insights into how to help easily bored but talented kids in learning programming these days. If I can figure a way to do this successfully with my son I'd like to offer my skills to the community and maybe help some other bright but "problem" kids catch the programming bug.
Saturday, August 21, 2004 4:01 PM by Justin Rogers

# re: Math: A bi-weekly journal describing the use of math in solving programming problems

Something is wrong with the comment posting tool here and I can only post at about 700 characters at a time. Doug, contact me through my contact form, and I'll drop you the text I was going to post as a comment through email.
Saturday, August 21, 2004 8:51 PM by Stephen H.

# re: Math Installment #3: A quick look at wavy text

Excellent postings. Very informative and enlightening.
Saturday, August 21, 2004 9:45 PM by Hart

# re: Math Installment #3: A quick look at wavy text

Wow! an actual blogger with content. Great stuff. I would love to be able to download some source code.
Monday, August 23, 2004 6:50 AM by Justin Rogers

# re: Math Quickie: Relationship between arc distance and linear distance.

Mentioned proving that further interpolation would return a more accurate result. Since the interpolated sample is smaller than the actual arc distance, we just have to prove that by interpolating over more parts the result is larger and therefore closer and more accurate to the arc distance.

First, take your two points and find the hypoteneuse (hyp). The hyp has a starting and ending point that lies on the curve. Now, keep these two points the same. How many points do you need for a triangle? Well, you need 3, so pick one point ANYWHERE. Doesn't matter where. Now, the length of two legs of a triangle is guaranteed to be larger than the third. The only exception is when you haven't defined a triangle, but in fact have defined a point on the original line. In that case the distance of the two legs is equal to the length of the hypoteneuse.

Using this basic geometric rule of a triangle we can define any new point along the curve to create two lines that will always be longer or equal to the original. Longer in our case means more accurate.

Tuesday, August 24, 2004 8:21 AM by TrackBack

# Math Installments...

...by Justin Rogers. For your browsing convenience, here are links to the first four installments: Math Installment #4: Bounding Regions Math Installment #3: A quick look at wavy text Math Installment #2: I needed some circular oriented text Math Installment...
Tuesday, August 24, 2004 8:21 AM by TrackBack

# Math Installments...

...by Justin Rogers. For your browsing convenience, here are links to the first four installments: Math Installment #4: Bounding Regions Math Installment #3: A quick look at wavy text Math Installment #2: I needed some circular oriented text Math Installment...
Tuesday, August 24, 2004 8:21 AM by TrackBack

# Math Installments...

...by Justin Rogers. For your browsing convenience, here are links to the first four installments: Math Installment #4: Bounding Regions Math Installment #3: A quick look at wavy text Math Installment #2: I needed some circular oriented text Math Installment...
Tuesday, August 24, 2004 11:33 AM by Wallym

# re: That was one hell of a Men's Track 400m event.

Ouch. Sounds like golf. I had a handicap index of 10.67 this summer. It was the lowest I have ever gotten it. The problem is that to get it to that level, the amount of training both physically (@ the gym) and hitting golfs balls was taking away from work and family time. Due to work considerations, I have dropped golf for the moment. To get the handicap much lower, I figure that several weeks of five day golf practice sessions of 45 minutes was needed. figure in 1 hour at the gym 6 times per week and there just was not enough time in the day for this amount of practice and work (real work and a programming book). Ouch, where is the instance success pill.

Wally
Tuesday, August 24, 2004 1:56 PM by Bruce Johnson

# re: That was one hell of a Men's Track 400m event.

I was always impressed with similar calculations related to marathon runners. The world record (which is below 2:05) works out to just a tad over 4:45 per mile. Consider that only 50 years ago, Bannister broke the 4:00 mark for the mile and that even now the world record is only slightly below 3:45. This means that the runners are stringing 26+ consecutive miles together, each of which is run faster than the fastest mile I ever did. Just more statistics that keep me in my place :)
Tuesday, August 24, 2004 2:23 PM by Justin Rogers

# re: That was one hell of a Men's Track 400m event.

I can relate to those calculations as well. Being from a smaller school, we didn't have a year round track program, so cross~country had to fill in the middle. That means I spent a lot of time in distance races.

Best mile was a 4:38, and that wasn't in competition. Best 2 mile was 10:57 meaning I barely put together two 5:30's. 3.1 I normally ran in the 18's, but got a couple of 17's. That is where things get strange because an 18 flat is around a 5:45 mile, very close to my best two mile time.

Now, my best long distance running came in the road miles. I find it easy to zone out and don't do well at all in competition. I often beat most of my teammates in practice, but often came in 2nd or 3rd in competition. I'm better at running against myself than others. That said, I had a 12 mile run to get home that I ran anywhere from 1:10 to 1:30. That is the closest I came to running a marathon (actually very close to a half marathon) and I was only putting together 6's.

This draws close to the point of the human endurance. It turns out that there is some limit where the body becomes stressed and fatigues quickly, but under which you are capable of extremely long periods of exertion. I was a 5 minute miler for the most part and I was capable of stringing together endless 6 minutes miles. It was just slow enough that it didn't kick in my burn rate.

Oddly enough my 400m while I was a miler was around 58, making 1:15's fairly easy to turnover one after the other.

Olympians crunch these numbers to the breaking point. A fast 100m time is in the 10's, and so they string 11's together for the 400m. A fast 400m is probably around 50 and so the miler's are stringing together 55's.

Turns out everyone has some potential for speed (raw power, muscle mass) and then some potential for endurance (muscle efficiency). Drawing a balance between the two becomes the crux of performance in a specific event. It also turns out to be a measure that confines athletes to specific events. The faster I got at the 400m and 800m the worse I got at my mile. Oh, I could still bust the 5's but I completely reversed my progress into the 4's. As I got faster I gained leg mass and slowly lost endurance. My capacity for raw speed was much better, but I just couldn't maintain.

Amazing how all of that disappears when you get a job working with computers ;-)
Wednesday, August 25, 2004 9:16 PM by Darren Neimke

# re: Human Categorization, Activation, Familiarity and Learning

I read this quote this morning and, as it is somewhat related to what you have written I thought that I'd add it here...

---------------------------------------------

Quote from Ellen Ullman's book, "Close to the Machine":

"Human thinking can skip over a great deal, leap over small misunderstandings, can contain ifs and buts in untroubled corners of the mind. But the machine has no corners. Despite all the attempts to see the computer as a brain, the machine has no foreground or background. It can be programmed to behave as if it were working with uncertainty, but -- underneath, at the code, at the circuits -- it cannot simultaneously do something and withhold for later something that remains unknown. In the painstaking working out of the specification, line by code line, the programmer confronts an awful, inevitable truth: The ways of human and machine understanding are disjunct."
Wednesday, August 25, 2004 10:15 PM by Justin Rogers

# re: Human Categorization, Activation, Familiarity and Learning

This is a fair approximation of the current understanding of the brain. However, we are getting closer and closer to examining the circuitry of the brain as it were and when we do we may find many more parallels there that show a level of programming as of yet unseen.

While at this juncture we can't perceive the circuitry of the brain we are quickly approaching a point in time where the circuitry of the computer is becoming sufficiently complex that it may soon become incomprehensible. In fact it will have to at some point become incomprehensible in order to solve the problems of great complexity that are only in the realm of human intelligence to solve today.
Friday, August 27, 2004 10:46 AM by Mike Swaim

# re: Spatially oriented puzzles, density, and perceived complexity...

But, did you like the game?
Friday, August 27, 2004 11:24 AM by Justin Rogers

# re: Spatially oriented puzzles, density, and perceived complexity...

To briefly describe my own personality, I enjoy complex challenges and don't require instant rewards. I consider games that are relatively complex, but that have short term playability (aka they give instant rewards) not quite the game for me.

That said, I played all 120 cards in a short period of time. Does that mean I liked the game? Not really, since I was mainly playing in order to obtain a challenge from the deck. There are some warrants to the game, and I'm continuously trying to improve my perceptions by more closely examining how it is played and possibly how to develop more challenging puzzle cards.

I might find it very interesting and even fun to watch others play the game (for short periods of time yet again because anything longer would be irritating if they proved to be poor problem solvers). Some multiplayer rules might prove to be fun where players take turns. Add some dice and you can take up to the rolled number of turns. You can start to see where this would provide some challenge and strategy.
Friday, August 27, 2004 1:41 PM by Christian Romney

# re: Call to Action: I need a long term mentally challenging puzzle of some sort...

Try chess, the combinations are infinite. :)
Friday, August 27, 2004 2:02 PM by Justin Rogers

# re: Call to Action: I need a long term mentally challenging puzzle of some sort...

Actually the combinations in chess are finite. Chess is a game of competition against an opponent and isn't intrinsically a puzzle that one can solve. The puzzles that do exist for chess, such as the reverse position solutions, are quite interesting since you get to control both sides of the board, but are also very memory based.
Friday, August 27, 2004 3:32 PM by Omer van Kloeten

# re: Call to Action: I need a long term mentally challenging puzzle of some sort...

Try the ilovebees [1] puzzle. It's a pretty weird one, but sure is a hard puzzle.

[1] http://www.ilovebees.com
Friday, August 27, 2004 6:28 PM by Jeff

# re: The minimum ability to navigate a 3D world isn't playability!

I guess I have to see the show now. The first Xbox Oddworld was easily one of the best platformers ever. Yeah, it's the typical platform formula, but the details made it fun, right down to making Abe fart.

Heh heh. What were we talking about? Dick and fart jokes never stop being funny.
Friday, August 27, 2004 6:48 PM by Justin Rogers

# re: The minimum ability to navigate a 3D world isn't playability!

Hey, I won't take away from the original XBox Oddworld. I've always been an advocate of that particular series because they have traditionally had very unique playability. The program details the next version which gets away from the puzzle solving underpowered hero genre and adds some first person shooter features. They've changed the type of game they are delivering from their very unique and successful approach to that of the mainstream successful approach. We'll see how it turns out.

The program isolated three games, Halo 2, Crimson Skies and the next Oddworld. We can't even talk about two of the games because they aren't out. But we can safely assume they pulled out the heavy hitters here. The top 1% of the crop. With the top 1% we are guaranteed that they'll be good (for the most part).

Even with the prior success of Oddworld, if you start to supplant great gameplay with scripted sequences you are going to lose some of your player base. Tales of Symphonia, one of the greatest games I've played in a while, made about 80% of the dialogue completely optional, however, I still rue losing a battle that has a long introductory sequence. Boy, I haven't seen THAT 10 times already now (they have a Hard mode that more than doubles the challenge of an enemy and it tends to get you killed quite a bit if you leave it on).

Saturday, August 28, 2004 7:07 AM by Craig

# re: Math Installment #2: I needed some circular oriented text

Nice piece of code, have you had any thoughts on how to draw the text so that it is centered?

For example if you had a smaller piece of text at the top it would appear to be symmetrical.
Saturday, August 28, 2004 11:37 AM by Justin Rogers

# re: Math Installment #2: I needed some circular oriented text

I take it what you are thinking is drawing the text around a particular angle. You can easily measure the entire string.

stringWidth = size(string).x

Then you need to offset the angle by converting half the distance into an angle.

angularChange = stringWidth / Carcd
angle = angle +/- angularChange / 2

Now you can use angle to start the text and it will be centered on the start angle. If you drew some text at say 90 degress and 270 they would now appear opposite one another and symmetrical.
Saturday, August 28, 2004 3:13 PM by TrackBack

# Math Quickie: Centering

Tuesday, August 31, 2004 7:34 PM by Scott Mitchell

# re: The minimum ability to navigate a 3D world isn't playability!

FPS games, like Halo, are low on my enjoyability level, since I usually only play them until I beat them. (Of course, with multiplayer, I'll keep playing that, but never the single player again.)

My favorite games are sports games or strategy games, because of the high replayability factor. The game that has probably gotten the most of my time has been Civ II. I play a game called C-Evo every now and then that mimics CivII, but with many improvements. It's free and open-source, if you're interested:
http://c-evo.org/
Tuesday, August 31, 2004 11:13 PM by Justin Rogers

# re: The minimum ability to navigate a 3D world isn't playability!

I'll definitely have a look at the source, thanks for pointing this game out.

The only thing I might point out is that while Halo does have a low replay factor in the presence of other games, there currently aren't any games that rival Halo to fill in the void. Oddly enough, every party I have, ends with me kicking everyone's ass at Halo. I'm also very surprised by the number of teenagers that are still playing the game.

Now, all of the replay value is in the multiplayer and fortunately for Halo there hasn't been any competition.

I'd say that strategy and sports games suffer the same downfall in that you need viable competition. Many players find that competition agianst the AI, but others aren't challenged at that level and lose interest in the game altogether. I would say that effort placed into the AI to make it complex is definitely a playability feature though, and since strategy and sports tend to spend the most time on AI they tend to have some of the highest playability factors. A game like Unreal Tournament, with it's bots could have trumped Halo, but of course, the bots turned out to be pushovers.
Thursday, September 02, 2004 4:32 PM by Addy Santo

# re: Physics Quickie: Consistency of Measure

None of your posts from the last few months are even remotely .Net related. Do us all a favor and set the Posts->Advanced Options->Display on Homepage to 'NO' for future posts of this type... Thanks!
Thursday, September 02, 2004 6:59 PM by Justin Rogers

# re: Physics Quickie: Consistency of Measure

Not only is your statement highly inflammatory, but it is desperately wrong. Looking at my 20 posts for the month of August, I see that about half of them are .NET related... I'd definitely say that isn't bad.

If you don't like the math, then don't read my posts. I'm a game developer. I develop games. It takes a lot of math and physics to do that and so that is what I'm going to blog about.
Friday, September 03, 2004 4:28 AM by Barry Kelly

# re: Physics Quickie: Consistency of Measure

> Can anyone draw a relationship between the
> distance formula of acceleration and the
> geometric formulas of a triangle?

If you graph velocity (Y-axis) against time (X-axis), acceleration looks like an upward slope, constant velocity looks like a flat line, and deceleration looks like a downward slope.

The funny thing about a graph drawn in this way is that the area under the curve (i.e. the line indicating velocity at that moment) is equal to the distance travelled (or, if you consider area under the X-axis to be "negative area", then the total area is equal to the displacement.)

The reason the area under the curve is equal to the distance travelled has to do with calculus. The area under any curve is the integral of the equation of the curve. When you consider the relationship between velocity and displacement, it's obvious that the rate of change of displacement is velocity. Mathematically speaking:

s: displacement
v: velocity

ds/dt = v

Since the differential of the displacement with respect to time is the velocity, the integral of the velocity is going to be the reverse - the displacement.

Considering your equation d = 0.5at^2 (although when I learned that in school, we called it 's = ut + 1/2 at^2'), you're starting from the speed of zero (i.e. you have no 'ut' term since initial velocity, u, is zero).

This means that if you draw a simple constant acceleration from 0, it looks like a triangle, and the total distance travelled is the area under the triangle. The time taken is t (measured along the X-axis) and the final velocity is v (but is equal to a*t since the acceleration is constant). This means the total area under the triangle is 1/2 a*t*t, or, in other words, d = 0.5at^2.
Friday, September 03, 2004 4:46 AM by Justin Rogers

# re: Physics Quickie: Consistency of Measure

Very thorough post. To draw direct correlations the area of a triangle is:

b = base
h = height
area = 0.5bh

b = t
h = a*t
area = 0.5at^2

By using basic substitution we go from the classical equation for area of a triangle of 0.5bh to the equation of distance by acceleration of 0.5at^2.

Glad to see someone took the time to solve the bonus problem. Thanks Barry!
Friday, September 03, 2004 11:28 AM by brianbec

# re: Physics Quickie: Consistency of Measure

It would be very interesting to create "dimension-sensitive" math libraries for physics that would use types and coersions to make units errors impossible. In pseudocode, instead of writing "42.6", for instance, you'd have to write something like "Cm(42.6)", and, if a function expected feet, such as

Seconds timeRequired (Feet distance, Feet/Second velocity)
{
if (velocity != Feet(0.0)/Second))
return distance / velocity;
}

and you called it like this

timeRequired (Cm(42.6), Meters(60.2)/Second);

the appropriate transformations would be applied.

The tricky part, using C# or C++ types, would be getting the compound units like Feet/Second right in a generic way. I would know how to do something like this in Scheme or Haskell, which latter has a very elaborate and interesting type system. Nice inspiriration. http://weblogs.asp.net/brianbec
Friday, September 03, 2004 1:47 PM by Paul

# re: NumericUpDown in Windows Forms and a small acceleration hack...

I had a quick question on why the number calculated uses return (int) (0.5*acceleration*(tCurrent*tCurrent)); instead of just calculating the returned number as a velocity and using acceleration * tCurrent?

Thanks
Friday, September 03, 2004 4:51 PM by Justin Rogers

# re: NumericUpDown in Windows Forms and a small acceleration hack...

Ah a great question.

Acceleration is how fast the velocity is changing. If you run the sample program, acceleration is responsible for the delta betweens scores being different.

Velocity is how fast the score is changing at some point in time. The score always changes for the positive, but at some point we want it to start changing more slowly again. This terms holds our current speed, while acceleration holds how we change that speed.

Distance is how much of the score we've traversed. The area under the curve. If you wanted a similar function that came up with the same results in terms of score values, at each time point, you'd have to do a different type of work.
----

As an example, take a football player at the 0 yard line. Now, you tell him to run for the 100 yard line. As he first starts out he is building speed. The rate at which he builds speed is his acceleration, while his speed at any point in time is velocity. As he gets to the 30 or 40 yard line he is probably at maximum velocity, but for sake of our argument, let's say he continues to accelerate to the 66 yard line (2/3rds).

He starts to get tired at the 66 yard line and begins to slow down. His positive acceleration, turns negative and he carries his velocity forward (let' say it reached 8y/s), but now his velocity starts to decrease as well. He ran a bit too fast and he is spent. He begins decelerating at nearly twice the rate he accelerated. He finally hits the 100 yard line and collapses in a pile of pads. What values are usable?

a0 = 2y/s^2, v0 = 0y/s, d0 = 0
a1 = -4y/s^2, v1 = 8y/s, d1 = 66y
a2 = 0y/s^2, v2 = 0y/s, d2 = 100y

Both acceleration and velocity are too important in controlling how the yardage changes with respect to time. Only distance (sometimes referred to as displacement) can effectively measure the actual yardage covered during the event.
Friday, September 03, 2004 5:02 PM by Justin Rogers

# re: Physics Quickie: Consistency of Measure

Interesting way to write your velocity I have to say. Meters is given the property of being a dimensional unit, while seconds is given the property of being dimensionless... I think a rational approach might be the very verbose:

Term(Coeffficient, Units[])

timeRequired( new Term(42.6, new Units[] { Cm }), new Term(60.2, new Units[] { Meters, Neg1Seconds })

Of course you start to be very verbose in this system... The units themselves carry the unit type, and the unit power. Division is units with negative power. Examining the units would have to be more functionally driven as well.

distance.EnsureUnits( Feet );
velocity.EnsureUnits( new Unit[] { Feet, Neg1Seconds });

Seems like a dead ringer for a dynamic language though. If you make any progress on making this work in C#, let me know. I'm very curious about the performance characteristics and would love to look at some optimization schemes.
Friday, September 03, 2004 5:25 PM by brianbec

# re: Physics Quickie: Consistency of Measure

looks like Andrew Kennedy of Microsoft Research is a specialist in this field. I'm looking into it ...

http://research.microsoft.com/~akenn/units/

Friday, September 03, 2004 6:30 PM by Jennie

# re: Using simple and blind rules, what is the best tic-tac-toe player you can make?

Justin, you know I can beat you at tic tac toe... :)
Saturday, September 04, 2004 1:39 AM by Justin Rogers

# re: Using simple and blind rules, what is the best tic-tac-toe player you can make?

Tic-Tac-Toe is a game of losing, not of winning. You can't beat someone at tic-tac-toe, they beat themselves. In fact the set of rules {Win, Block, Highest} will either win or tie any game where the AI is able to go first.

If the AI is the second player, then it will lose in two circumstances... You can fix one circumstance by swapping the rule Highest out for the rule Lowest after the first turn. There is still one game this doesn't fix.
Saturday, September 04, 2004 1:59 AM by Justin Rogers

# re: Using simple and blind rules, what is the best tic-tac-toe player you can make?

For a more complete opponent we can emphasize summation of the opponents moves and come up with some more rules.

Parity - Returns whether the sum of the opponents moves is even or odd treating 0 as returning a random parity (for first moves ;-)

ReverseParity - The reverse of the opponents pairty. Same random return for 0.

Highest(Parity) - Returns the highest rated board spot with a parity (with respect to the magic square) that matches the past in parity.

Win,
Block,
Highest(ReverseParity)

For the two failing games, selecting pieces based on the players move selections (summation with respect to the magic square) is enough to always win or tie...

Parity = Sum of Opponents Tiles % 2
ReverseParity = Sum Of Opponents Tiles + 1 % 2

If you don't want special treatment of parity then you can make the move dependent on the global turn.

Win,
Block,
{
gt=1: Highest(rand%2),
gt>1: Highest(ReverseParity)
}

gt stands for global turn or the turn with respect to all players. This is different from another variable lt, which stands for local turn.
Sunday, September 05, 2004 7:38 AM by Denny Figuerres

# re: Performance: Don't rely on the C# compiler to optimize your math...

much like some different code I am tuning...

in my case it's a webservice that does a an sql transaction.

I can call a ws func() that does a sql open conn, exec and close in less than 1 second.

avg is about 00.035 or 1/3 of one second.

but the code for the sql trans call is having poor runtime perf. it's based on code that worked as a direct sql in the app so I am looking for some induced gc from adding strings.

and a possible chnage to an array that was an arrqay of strings that was sorted.

may chnage how it gets done as It may be better as an array of Byte or short int.

just cause the old version worked in one case when you chnage the code you need to test for new problems....
Sunday, September 05, 2004 10:32 AM by TrackBack

# RegexLib Testing Tool - The new Options Panel

Sunday, September 05, 2004 10:35 AM by TrackBack

# RegexLib Testing Tool - The new Options Panel

Monday, September 06, 2004 3:57 PM by George V. Reilly

# re: Physics Quickie: Consistency of Measure

There's a bunch of work going on in the C++ Boost world to handle dimensional analysis: see http://www.artima.com/cppsource/metafunctions.html for an introduction.
Tuesday, September 07, 2004 10:12 AM by TrackBack

# RegexLib Testing Tool - The new Details Grid

Tuesday, September 07, 2004 10:14 AM by TrackBack

# RegexLib Testing Tool - The new Details Grid

Tuesday, September 07, 2004 8:09 PM by TrackBack

# Now THIS is blogging...

Tuesday, September 07, 2004 8:09 PM by TrackBack

# Now THIS is blogging...

Wednesday, September 08, 2004 12:17 PM by TrackBack

# Custom Attributes in Whidbey

Wednesday, September 08, 2004 12:20 PM by TrackBack

# Custom Attributes in Whidbey

Thursday, September 09, 2004 7:11 AM by TrackBack

# Custom Attributes in Whidbey

Monday, September 13, 2004 6:35 PM by DarthPedro

# re: Math Installment #1: Image layout logic

This is definitely a cool series of posts.
Monday, September 13, 2004 9:25 PM by Paul

# re: Math Quickie: Introduction to combination formulas

Hey Justin,

I was hoping you could clarify a little math for me I was wondering why we use C(n + m - 1,m) in the suit combinations for 5 cards instead of using C(n,m)?

Thank you
Tuesday, September 14, 2004 12:39 AM by Justin Rogers

# re: Math Quickie: Introduction to combination formulas

C(n, m) means "Pick 5 things, but never pick a single thing more than once. Once you've picked your thing, the order doesn't matter. The identity if your combination is given by the 5 identities of the parts and does not consider their location"...

As we examine suits, you could only pick each suit one time given our old equation because it doesn't allow repetition. The C(n + m - 1, m) is a simple extension of the equation that allows you to select already picked items.

One sanity check would be, is m < n... Well, if not we've got a problem because we wind up with bigger numbers on the bottom than the top of the fraction. Remember that the original C(n,m) equation is built in such a way that as we make each selection it is removed from the sample set and so the successive values after each selection get smaller and smaller. If we use the same concepts for m > n, we'd run out of things to pick from.

Why is the new equation the right equation? Well, we have 4 suits and we are going to pick 5. "4^5" would be how many ordered selections existed... 1024. We've shown this is 10 bits before, (2 bits to represent 0 through 3)*5 = 10 bits. Now it would only be natural to throw the ordering out the window, as it is unimportant if we are just packing the suits in any order. By this extension we know we need and unordered principle, but also one that allows reptition, aka, not the original C(n,m)...

Hopefully all of that taken together holds the jewel that you were looking for. If it doesn't, then try examining some expansions using small numbers, especially expansions of successively larger problems n = 5, m = 2 is given. Look at the resulting set of expansion data for the *what is added* and that will give you a more spatially oriented way of solving the problem. Try computing the ordered selections n^m, then subtracting the C(n+m-1,m) result. Where did the extras go? In some of our equations we could show that all of the extras vanished because we divide by the arrangements. See if you can find something analogous for yourself. Might help explain the equations a bit better.
Tuesday, September 14, 2004 8:40 AM by Jonathan de Halleux

# re: Testing: Pair-wise, upper and lower boundings

I would be interrested to see a concrete implementation of that algo and compare it with the other approach from TestFu. Are you planning to release any code ?
Tuesday, September 14, 2004 10:05 AM by Justin Rogers

# re: Testing: Pair-wise, upper and lower boundings

As I continue through examining the algorithms bit by by, I'll start to build up a code base... I already have code that generates optimized pair-wise combinations for any number of features that contain 2 options (aka, a trivial case).

I do plan on working through more complex cases, especially in the interest of finding boundings, which is of more interest to me than the generating functions. If you see my later post I've skirted around the concept of solution domains as well. I think there is definitely some value in optimizing existing test coverage, meaning I'll also cover them in more depth.
Wednesday, September 15, 2004 4:40 AM by Jonathan de Halleux

# re: Binary features seem to be very easy to test with solid mathematical limits

I've stumbled across this survey on combinatorial testing ( http://www.isse.gmu.edu/techrep/2004/04_05.pdf ), definitely worth a look.
Wednesday, September 15, 2004 7:50 AM by Steve

# re: Performance: Don't rely on the C# compiler to optimize your math...

Where can I get more information about optimizing mathematical function?
Wednesday, September 15, 2004 6:56 PM by Justin Rogers

# re: Performance: Don't rely on the C# compiler to optimize your math...

Steve: Great question, and really no answer. You have to check the small blogs and various resources tediously and slowly build up a list of gems that can help you make your math that much faster.

Math optimizations come in a number of different formats, so you might miss some of them. For instance, the above is a compiler optimization or lock thereof. To us, division and multiplication mean the same thing, to the computer it is something drastically different.

Then there are ways to throw out various portions of the work. Something like n!/(n-1)! actually means n divided by 1 or n. The lower expansion cancels out all but one term in the upper expansion.

And finally there are preallocation otpmizations. Here you move math out of work loops and do all of the computations before hand. sin/cos look-up tables were brilliant at this back in the 90's in many 3D DOS games. I also posted a discrete fourier transform algorithm in the physics section that originally had 10 or 15 mathematical operations in various levels of the loop structure and I was able to remove most of them entirely, remove them to a parent loop so they were evaluated less frequently, or move them outside of the loop so they are only calculated once.

Asking questions never hurts. If you have specific math that you think might make a good posting, let me know and I'll talk about it. Contact me through the blog contact link in the upper left. I find it hard to continuously come up with content that people like.
Thursday, September 16, 2004 4:02 AM by Steve

# re: Performance: Don't rely on the C# compiler to optimize your math...

Thanks for your offer. I'll take a rain check on it. I'll get in depth of genetic algorithms, neural net algorithms, etc. Much place for math optimization.
I am currently working with GDI+ which don't let much place for significant optimization I think. In most cases the limits are the internal GDI+ methods.
Getting back to the division/multiplication example: I checked it out. Using 32-Bit-Int results a speed ratio of 2:1. Using a long value results 1:1. The speed enhancement is gone. It might be easy to understand if I whould know the cpu works. :-)
Thursday, September 16, 2004 9:00 PM by Justin Rogers

# re: Performance: Don't rely on the C# compiler to optimize your math...

GDI+: There are plenty places for performant math algorithms when working with GDI+. Check the Math series that I've posted over the month of August/September and you'll see some great uses of math in GDI+ in terms of custom layout logic and path drawn text.

Mult vs Div: Since I didn't post a full testing scenario there are a number of issues that could cause a false or different reading. The JIT often optimizes based on context and at times even optimizes out unused calculations.
Thursday, September 16, 2004 11:48 PM by Jonathan Ellis

# 8 hours? Damn.

Maybe I'll just play through KotOR again instead. :-|
Friday, September 17, 2004 1:07 AM by Larry Osterman

# re: Gaming: Fable, must or bust?

Since when has AC's marriage been restricted to heterosexual marriage? There are no such restrictions.
Friday, September 17, 2004 1:17 AM by Justin Rogers

# re: Gaming: Fable, must or bust?

Well, then that must be a change to their initial rule set then. Since marriage was something that was run by the admins, handing out special jewels and doing the fireworks initially, they were limiting it to marriage between an in game male and in game female character. Externally it didn't matter. If they've changed their rule-set since then I apologize for not keeping up with it.

Note this is in regards to the original AC prior to the AC2 release. Up until I left AC2 there wasn't currently a marriage facility in that game and I'm not sure if they added one.
Friday, September 17, 2004 1:42 AM by Larry Osterman

# re: Gaming: Fable, must or bust?

Ah - marriage isn't done by admins any more - it's a series of quests now.
Friday, September 17, 2004 2:53 PM by Jeremy Brayton

# re: Gaming: Fable, must or bust?

The makers of Black and White did this? You know they took an incredibly long amount of time to produce that game as well. Funny thing is I think B&W was actually cool but only with the creature isle expansion pack. Throw those 2 into one game and it would have been a keeper.

I was looking forward to Fable too but it looks like I'll join Jonathan and play KotOR again. That was 30 hours of gameplay just to go through the story. I rushed through it too, I could have probably extended it to 40 or so if I really wanted to.

That is a little sad though. 4 years of development for 8 hours of gameplay. Not worth $50 to me.
Tuesday, September 21, 2004 10:26 AM by Roger PW

# re: Gaming: Fable, must or bust?

This post pretty much how I felt after playing.

I played this game through in about 11 hours and was really “into it” until it abruptly ended, leaving me with nothing but shock, disappointment and a desire to play something else. The abrupt ending really spoiled the game for me because I figured I would be able to continue even after completing the main missions, but I had to revert my character back to an old save in order to keep playing. Once I did that (and knew how the story ended) I lost interest in continuing the side quest so I tried to play the evil route. Since I’m not very good a playing a games through twice, this quickly ended and I moved on to another game. Also, the game was extremely easy; I never died.

While I was playing I thought the interaction with the NPCs, the side quests, the demon doors, buying houses and changing physical appearance were all great features but nothing really every became of any of it because the game ended so quickly.

FYI: If you do decide to play this game, you’ll get to a point in the main quest where you are asked if you want to continue because from that point on you’ll have to focus on the main quest and stop all active side quests. If you say yes here, your game is pretty much over so if you don’t want to end the game say no and keep playing.
Tuesday, September 21, 2004 9:35 PM by TrackBack

# re: Quand Microsoft tente de nous remettre dans le droit chemin

Wednesday, September 22, 2004 11:17 AM by Jerry Pisk

# re: Objects with dense events, but sparse usage can benefit from custom event storage.

Is it really worth it? even if you have 2,500 extra fields that would translate to some 10K of memory to store the handler references. I use a lot more than that for temporary buffers, saving 10K of RAM doesn't seem like a big deal.
Wednesday, September 22, 2004 7:11 PM by Justin Rogers

# re: Objects with dense events, but sparse usage can benefit from custom event storage.

Temporary memory buffers are reclaimed. This is permanently allocated memory attached to a long running object. There are some differences there in that the 10k for the temporary buffers will be constantly re-used and reclaimed.

What you get from custom storage is faster object instantiation, smaller object layouts on the heap, etc... If there is a way to examine this in terms of performance, I think most of the bottleneck would be at the processor level, but I'll see if there are any out of the box tests that might make a case.
Wednesday, September 22, 2004 9:35 PM by Jason Olson

# re: Some security considerations for systems with events.

Great post! Quick question: did this knowledge come from mostly experience or, for us book geeks, is there a good resource out there for us to learn this kind of stuff?
Wednesday, September 22, 2004 9:50 PM by Justin Rogers

# re: Some security considerations for systems with events.

Haven't seen any books on this sort of thing. If there are any books, I'd be happy to hear about them though.

If you are thinking of just eventing and scripting in general you might try the Game Programming Gems series. There isn't any focus on the way .NET handles events though, which is where my own investigation and experience comes into play with respect to my articles.
Thursday, September 23, 2004 10:35 AM by TrackBack

# Update on the Windows Forms Delayed Handle Creation Bug

I got an email about a week from a fellow called Jeff Berkowitz who had experienced the same problem with the delayed window handle creation that I posted on here. Jeff's post of 9 Aug 2003 describes his discovery of the problem, and in his email he references this incredible...
Thursday, September 23, 2004 10:37 AM by TrackBack

# Update on the Windows Forms Delayed Handle Creation Bug

Thursday, September 23, 2004 7:09 PM by TrackBack

# Visions of Viruses to Come?

Thursday, September 23, 2004 8:57 PM by TrackBack

# Visions of Viruses to Come?

Friday, September 24, 2004 11:58 AM by Barry Kelly

# re: Creating custom collections for better performance and filling in BCL holes.

You reckon you should use Buffer.BlockCopy instead of Array.Copy for copying arrays of primitive types.

On the other hand, I wouldn't use your insertion mechanism at all - it's got O(n^2) complexity. Skip lists are easier to program than binary trees and give probabilisticly similar results.

-- Barry Kelly
Friday, September 24, 2004 12:01 PM by Jeffrey Sax

# re: Performance: A quick-fix for the Pow functions. Acceptable approximations for a magnitude of speed.

If you *really* need speed, it may be better to use separate methods for the smallest exponents. I don't think these methods are small enough to be inlined by the JIT compiler.

A couple of aesthetic comments:

1. You really don't need the long exponent in LongPow: 2^Int32.MaxValue has over 600 million digits, and won't fit in a long anyway. :o)

2. Why not overload rather than use different method names?
Friday, September 24, 2004 12:51 PM by nospamplease75@yahoo.com (Haacked)

# RE: Natural Language & AI: Dissecting the process of reading.

The algorithm you mentioned doesn't munge words with three letters or less. This means we get a LOT of context words (such as prepositions) to help make sense of the rest of it.

I think it's a combination of pattern recognition and context recognition. The first and last letters matter the most, but the letters in between still have to be the correct letters. You can't replace those arbitrarily.

Likewise, some words do slow reading down. Those are the longer ones without "tall" letters to help guide the eye.
Friday, September 24, 2004 5:13 PM by Justin Rogers

# re: Natural Language & AI: Dissecting the process of reading.

Well, this is a nationally renowned study. The qualifications for the algorithm were set by them, so I'm sure they are aware of 1, 2, 3, and sometimes even 4 letter symmetries.

My main concern was whether or not 2 letters and a dictionary of words you are very familiar with would be enough to reconstruct a message. I'll have a game ready that tests the theory using some algorithms. I'm going to start light with basic matching, but then add some context matching code if and only if it is required.
Friday, September 24, 2004 5:24 PM by Justin Rogers

# re: Performance: A quick-fix for the Pow functions. Acceptable approximations for a magnitude of speed.

1. Yep, that was just a typo.
2. So you know what you are getting. Using overloads is awesome, but to avoid explicit casting to get the right method I've included the various names in place. You may also choose to return a different data type for overflow. OIntPow would return a long, OSinglePow would return a double.
Friday, September 24, 2004 5:48 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

You might think that Buffer.BlockCopy would buy you something. Turns out, it can either save or lose you fractions of seconds on any of the given tests. You tend to get 5% faster results on the smaller lists and then the error changes +/- 5% as the lists begin to get larger. That makes Array.Copy pretty efficient.

I'm not sure anyone was talking about a binary tree. I surely wasn't. I was talking about a balanced, sorted array, which is completely different. The only removal of complexity in the balanced sorted array is shifting half of the array rather than the full array.

If you have a faster algorithm, lets see the code. This definitely fills a whole somewhere since the functionality isn't in the BCL by default, it maintains only a single array for data storage making it behave like other collections classes, and it is much faster than something like the SortedList which replicates data when you want a value sorted collection. It is also easy to make thread-safe with regards to atomic insertion, while still allowing reads in between writes.
Saturday, September 25, 2004 1:18 PM by Barry Kelly

# re: Creating custom collections for better performance and filling in BCL holes.

Your "balanced sorted array" buys you no complexity improvement. It's only different by a constant factor. It's still got O(n^2) complexity for n iterated insertions. It isn't worth its implementation effort.

To get O(n log n) complexity, you need to use some kind of balanced binary tree (e.g. red/black tree or AVL tree) or something similar. I mentioned balanced binary trees simply because they exhibit the fastest possible complexity bound - namely, O(n log n) - for iterated insertions of *arbitrary* data.

(For limited universe data like integers, there are obviously O(1) insertion algorithms which translate into O(n) when iterated like you are doing in your test).

I mentioned skip lists because they are easier to program than balanced binary trees but have probabilistically similar average complexity. Check out the obvious - http://google.com/search?=skip+list - for details.

Certainly, .NET needs something like C++'s set<> and friends or Java's TreeSet / TreeMap. C#'s collection library is pretty dire. Things will improve a bit with C# 2.0 and generics, but more collections with a wider range of implementation mechanics is needed.

-- Barry Kelly
Saturday, September 25, 2004 1:20 PM by Barry Kelly

# re: Creating custom collections for better performance and filling in BCL holes.

Should obviously be http://google.com/search?q=skip+list .
Monday, September 27, 2004 2:46 PM by Drayk

# re: Gaming: Fable, must or bust?

In response to Roger, you CAN play through after the end of the game. You CANNOT skip the credits, is the only thing. Many people keep hitting the great yellow Y button and skipping the credits and if you do so you have to fight the final boss all over again.

Otherwise, I am mixed about this game. I think the hype really catapulted everyone's expectations for this to be THE RPG of all time, and that's not what we received.

Good points first:

Expressions. People have discussed this at length already so I won't beat this one to death.

Tattoos/Hairstyles/Other Appearance Stuff. This was pretty neat, to mix and match the hair the beard the tattoos and what have you and have it affect how people react to you. Definitely a feature I'd like to see in other games.

Skill/Levelling System. I've actually seen a lot of people complain about this, and I don't quite understand why. I found this to be a user-friendly and overall enjoyable system. To me it makes sense that if you use a bow most often, you should gain more skill with the bow than with a sword. Also that you can use general experience to go toward purchasing ANY skill, is a great feature as well.

Other things I 'sorta' liked, weapons and the clothing but I felt there could've been a little more variety. Yes there's a good amount but probably 1/3 of the available outfits are 'dark' and 'bright' versions of the same thing. Same graphic, different colors, maybe an alignment alteration. To me that's just rather lazy.

Bad points:

Too short. I suppose I was comparing it to Morrowind, where you can put 100 hours into the game and still not even be done. Major flaw. This could've been countered by an immense amount of sidequests and other secrets, perhaps a crafting system so you can open your OWN shop and not just buy one after you've, ah, put it up for sale so to speak.

Alignment made no difference. Yes it affects how you look, but that is IT. It does nothing to alter the story or the way people perceive you (that's based on your Attractiveness/Scariness ratings) and to me that was a massive let down.

Lack of open-endedness. This is probably my biggest complaint, and again I compare it to Morrowind (which is most likely an unfair comparison, but hey). That is one of the MAIN features we were all expecting, open-ended gameplay. The story is linear, and at some points cliche and predictable. Enjoyable and entertaining also, yes, but still.

Weapon and Armor variety. Like I said above. The outfits are neat, the fact that they impact the way people perceive you as neat, but three versions of the SAME outfit with nothing but a color change and some modifier changes is stupid. Weapon-wise it was the same weapon, just different grades (iron, steel, obsidian, master, legendary) although the legendary weapons are neat-looking so those are exempt from the complaint.

Summarized: The game itself I enjoy. I like it. But I don't, heart-thumpingly, absolutely and completely love it like I expected to.

That's all I can really think of at the moment. I won't suggest or knock the game, because honestly I think everyone's going to have a different opinion. I don't regret buying Fable, and I don't hate the game. I just wish we would've all got more of what we were promised.
Monday, September 27, 2004 5:10 PM by Scott Mitchell

# re: Creating custom collections for better performance and filling in BCL holes.

I don't see why you are using an array here. If you want a sorted list, why not use a linked list? This would remove any need of copying data over to maintain sort order - you just plunker down the sucker in the list where he belongs.

Of course, a sorted array (be it implemented like you have it, or as a linked list) takes linear time for inserts since you must potentially scan the entire list to find where to insert the item. As Barry points out, self-balanced BSTs or skip lists would provide log n insertion time, since at worst log n comparisons would have to be made to find the spot to insert the item (rather than n).

I have an article on self-balancing BSTs and skip lists available here:
http://msdn.microsoft.com/vcsharp/default.aspx?pull=/library/en-us/dv_vstechart/html/datastructures_guide4.asp

(I also have a version for 2.0 in the works...)
Monday, September 27, 2004 5:48 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

A number of issues... We are storing integers, and no matter what you do, adding a linked list feature to storing integers is going to double the size of the data storage.

We are doing a binary-search, so finding our insertion point is log n + 1, and we copy at most n/2 elements. I'm not exactly sure I see this as bad performance by any means.

We also get all of the benefits of arrays and linear access to our data. What is the 50th element, array[50]... What is the 5000th element array[5000]... I think if you do the testing for various situations that you'll find not only is the balanced array insert very fast, but in many cases it is as fast or faster than using your other structures. It will also be half the size of those other structures.
Monday, September 27, 2004 6:34 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

I'm first to admit this is a naive test. But now matter how much you talk about collections and performance, if you ship something with poor performance, it doesn't make a great impact. While your SkipList implementation was not optimized to be fast I'm sure (though talking about perf you probably should have made it fast) and it wasn't optimized for storing integers, but rather made generic to work with IComparable (not that any implementation of my sorted array would be this slow)... Here you go:

Current Count: 10
Sorted Array: 00:00:02.5536720
Balanced Sorted Array: 00:00:02.5937296
Array Sort: 00:00:01.3920016
Skip List: 00:00:21.1904704

21 seconds? Nearly 1 magnitude slower at 10...

Over larger lists, much larger, over 10000 elements, the skip lists start to perform. I use that term loosely, because they still only perform as well as the array copy.

Thinking about the global performance of an algorithm given billions of elements may work perfectly well for some. However, most of the time you are only sorting 10-100 elements, and often times you have 100's of those collections sitting around that need to be sorted.
Tuesday, September 28, 2004 9:40 AM by PatternGuru

# re: Creating custom collections for better performance and filling in BCL holes.

If you want to use an array as basic stored and need something thats asymptotically better than O(n^2) then why not try a heap? Basically that's a binary tree squashed into an array using shl/shr operations in an indexing scheme. Transforming an arbitrary array to a heap takes O(nlgn) and inserting takes O(lgn). Getting the sorted list out of it can be more difficult, but can be done in O(nlgn) with peeking the highest/lowest value in O(1) and removing it in O(lgn), making it great for priority queues. It all depends on how you are going to use that sorted list and what you want to be fast in what cases.

Sounds like a job for http://www.amazon.com/exec/obidos/tg/detail/-/0262032937 (or something similar) to me...

I do agree that the default .NET collection support is somewhat lacking right now.
Tuesday, September 28, 2004 10:36 AM by Chris Chambers

# re: Creating custom collections for better performance and filling in BCL holes.

Well, I guess I'll throw my hat in the ring here.. I don't even remember what I was looking for when I found this page several hours ago.. but here's what I've come up with. It's sort of an indexing-sorting-array-tree-with-one-level-of-bucket-type-nodes.

Inserting is mediochre up until around 1000 entries, after which, it leaves the SortedArray implementation that started this whole thing in the dust. Lists over 5000 are particularly nice.

It's also fairly good about managing it's "slack." I'm sure the SortedArray could be better, and that performance doesn't depend on this, but when it grows, it doubles the size of the list unconditionally. That's a pretty bold assumption to make.

Accessing by index is about half as fast as SortedArray. It's hard to compete with flat array indexes.

Searching (IndexOf, for example) is, as a side product of the design, absolutely terrific at all set sizes. It can process 40,000 IndexOf requests on a collection of 40,000 values (random index requests, so hit or miss) in just about 0.28 sec on an athlon xp 3000+. For comparison, it takes the SortedArrayList a little over 9 seconds to do the same operation.

Memory footprint is typically the {size of data} + 3% or so. Sometimes better than Sorted List, sometimes worse. Depends on if you hit a number of elements that SortedList really likes. (10, 20, 40, 80, etc) Of course, I'm pretty sure SortedList could allocate smaller chunks without a huge performance hit, so that would solve that problem.

I guess that about does it. I've really enjoyed reading this discussion. Made me think outside the box, and that's never a bad thing.

If anyone is interested in the source for Rube(Goldberg)SortArray, and also the program I used to benchmark it, I can zip the code and post it somewhere. I'd be delighted to see what others think of the (slightly wacky) idea and also to get second opinions for benchmarks.
Tuesday, September 28, 2004 11:02 AM by Mike Swaim

# Winforms vs ASP.Net

I think that ASP.Net is so popular because it's much better old ASP. WinForms hasn't caught on for a variety of reasons. There's the large deployment package (.net), the pain of conversion of old VB code, the slow start up speed vs. native code. It's often slower than native code, and the UI's particularly slower than native code. And from a Delphi perspective, you have fewer controls that aren't as useful.
Plus, you've got Avalon coming along in a couple of years that'll obsolete WinForms anyway. Why jump to a temporary solution?
Tuesday, September 28, 2004 12:12 PM by Gabe Halsmer

# Oh really...


Interesting take on code security. However, you're describing a very unique situation. Terrarium is a game played by people with more time then I have. Defeating a dominant animal requires understanding its behavior. Either through watching it, experimentation, or as some people have opted for, reverse engineering its code.


:: Why do you think ASP .NET is so
:: popular and Windows Forms is still
:: riding coat tails?

Ease of deployment. Seriously, you don't think code protection is why people choose ASP.NET. How many window form apps do you think are reverse engineered?

Tuesday, September 28, 2004 3:17 PM by Justin Rogers

# re: Implementing AI wars where code is the primary asset. (focus on Terrarium)

Professional component developers have their number 1 gripe at code being too easy to disassemble. They feel their intellectual property is constantly being examined and taken away from them. In a server environment, they feel that have slighly more control, especially if they own the site. In a Windows Forms application, you own nothing once it is on the end users machine. I happen to know for a fact, that many developers, including many large game houses have chosen not to use C# because of their perception that code is an asset and placing it on the end users machine is much riskier they feel than their current development platforms.

The concept that Windows Forms will be obsolete in a couple of years is stretching it, especially considering the Windows Forms/Avalon interop. Windows Forms will be around for quite some time, probably another 5 years. After 5 years it will succumb to future technologies, just like every other technology that hits the market, the lifetime is limited by some future advancement. People program DirectX every single day, and every 2 years it gets obsoleted for a brand new version.
Tuesday, September 28, 2004 3:28 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

You should definitely post it so we can take a look. Test results arrived at under different conditions often provide bad heuristics for judging performance. A great example would be running on an Athlon versus an Intel.

I'll toss your code into the mix, and I'll upgrade it to generics and Whidbey if appropriate, since I also want to see if a platform upgrade is going to give us any more performance in the non array based scenarios.
Tuesday, September 28, 2004 3:36 PM by Justin Rogers

# re: Implementing AI wars where code is the primary asset. (focus on Terrarium)

Code Security Note: What is the first thing people did when ATI came out with their control panel? I think I saw quite a few blogs with the IL and people ripping off their controls. I chose the route of visually examining their controls and making something similar, but most people just wripped the IL.

AI: I also need to point out that many people that put time in the Terrarium are also professional developers by day, hobbyists at times, and students. While you might feel they have more *time* than you, Terrarium is a game and a competition. Understanding their explicit behavior rather than programming a resilient creature is a break in the paradigm and circumvents all competition. I really wish we had fixed that.

Question: If there were an IntellectualProperty attribute you could place on an assembly, that upon opening in ILDasm or Reflector, you'd get a message telling you the source was intellectually protected. Would you do the right thing and have the app close down without actually looking at the code? This is a social engineering approach to prevent arbitrary glancing at code you know someone cares about.
Tuesday, September 28, 2004 3:38 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

PS > Were you doing performance against SortedList or the SortedArray I posted above? You mix terminology a few times. I never even bothered throwing SortedList into the mix because it was far too slow.
Tuesday, September 28, 2004 5:20 PM by Chris Chambers

# re: Creating custom collections for better performance and filling in BCL holes.

Oh, I see I *was* mixing terminology. Up too late, followed by up too early, I guess. I meant the SortedArray posted above in all cases. SortedList is a different beast altogether.

I could have gone generics and IComparable<T> with this, but I wasn't sure from the discussion if 1.x features only were more appropriate. Shouldn't be too hard to convert, though, if you want to give it a go.

Here's a link to the project. It's Whidbey beta 1, but I don't think there is anything in RubeSortArray that would keep it from compiling under the 1.x framework. Not sure about the little test program. It's just a little console app. Be warned, though, Whidbey steals the Console output (and input) and just throws an exception for Console.ReadLine. But this "feature" can be disabled.. And the link is:

http://www.bluehungarians.com/RubesArray.zip

URL recognition? Hope so.

And now, a few qualifications:

1. Pardon the badly written and possibly unclear code.

2. 1000 items is the breakover point. Below that threshold, only searching is faster. Above it, things start to get good.

and 3. Who knows if this will be any good at sorting objects other than int. It should be fine, but I just don't know.

There ya go. I'm really interested to see if it's any good in your benchmark. I have a tendency to test for what I'm testing for, if you know what I mean.. and I may have picked testing code that favors my object in some way.

Cheers
Tuesday, September 28, 2004 5:30 PM by Chris Chambers

# re: Creating custom collections for better performance and filling in BCL holes.

ps. One more qualification. Performance is, of course, better (more representative?) in release mode. Or with the optimized flag, anyway. I figure it's only fair since I'm up against the framework's Array.BinarySearch and Array.IndexOf.. Doesn't make a huge difference, but it does make some.
Tuesday, September 28, 2004 7:03 PM by Chris Chambers

# re: Creating custom collections for better performance and filling in BCL holes.

Ok, one more post. I decided to go ahead and try it with generics. Here's that version of the project.

http://www.bluehungarians.com/GenericRubesArray.zip

As far as the testing goes, I just replaced all the integer-only versions with the generic<int> class. Performance seems about the same in release mode. Debug mode took a little bit of a hit on indexed access.

Should work with any IComparable<T>. I love generics. I only wish they'd hurry up and release .net 2.0 so I could use them in more than just proof-of-concept projects. -sigh-
Tuesday, September 28, 2004 10:50 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

I'm probably not going to get around to performance analyzing things this evening, but I'll try and get something up and running tomorrow. Would you mind if I cleaned the code and included it in a larger code dump that demonstrates all of the algorithms? I'll drop it as an article, and link it in a short posting examining the heuristics of all of the methods discussed so far. That'll give us a good base for any future discussions.
Tuesday, September 28, 2004 11:02 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

PS > Your code adds that extra level of abstraction from 1 sorted array to many sorted arrays in order to reduce the number of operations during copies and searches, with the additional overhead of managing lists. It'll be interesting to see the threshold that this implementation passes the BalancedSortedArray, because I've basically added the same heuristic by maintaining *two* conceptually separate arrays, with one being on each side of some mid-point in a single dimension array.

An array BST with shifting might be similar in approach. As one user has suggested this, and I happen to have an array BST implementation, I'll try and clean that up for inclusion as well.
Wednesday, September 29, 2004 12:38 AM by Chris Chambers

# re: Creating custom collections for better performance and filling in BCL holes.

The BalancedSortedArray idea is interesting. I had considered that each "node" should maintain an array that could grow either at the head or the tail.. I was going to do it as one array with the data centered and space at the top and bottom (sounds like what you were describing).

I also considered an "optimization" for small sets (less than 1000, in this case). For no-splitting of the initial node until it becomes "profitable" to do so, speed wise. However, I haven't been able to imagine a good heuristic for what is a profitable number at which to begin splitting the list. I guess I could just tell it not to split until it has collected 1000 items, but magic numbers are... well, bad. I'd much prefer a more robust way to decide it's time to split the first node. On the bright side, this would give insertion times that approximate the SortedArray class's times up until 1000 items (since it's basically the same procedure), and then after that, it could switch to splitting nodes and indexing.

Feel free to include my code in your article. I may tinker with it for a while, if you wouldn't mind an update.

Anyone have recommendations for a good (and free) code profiler for the 2.0 framework? I had a look at nprof.. and it's ok, but I just wondered if anyone had a particular favorite.
Wednesday, September 29, 2004 12:40 AM by Chris Chambers

# re: Creating custom collections for better performance and filling in BCL holes.

ps I meant to say, "Feel free to include my code and do any cleaning up you'd like to do."
Wednesday, September 29, 2004 3:18 AM by Chris Chambers

# re: Creating custom collections for better performance and filling in BCL holes.

Doh. Earlier, I just tacked on IndexOf to your SortedArray for my benchmark, but I used Array.IndexOf instead of Array.BinarySearch in those tests. Needless to say, after making the change, SortedArray is now the faster at searching any size list. Silly me.

That leaves only two real benefits to the RubeSortArray:

1. Significantly better insert times for large lists
2. Possibly more efficient array allocation

And the downside:

1. Worse insert times for small lists
2. Worse searching times for all lists
3. Indexed access is about half the speed of SortedArray

So, after all that, maybe the RubeSortArray may be a lot of implementation for just a little benefit. But hey, if you need to store an often-changing list of 50,000 sorted objects, but don't want all the overhead of linked lists, it might just be for you.

-- Oh yeah, I tried adding the balanced array idea into the nodes, but the speed increase was marginal while the amount of slack doubled.
Wednesday, September 29, 2004 10:48 AM by Steve

# re: Performance: Don't rely on the C# compiler to optimize your math...

Mhhh, I got a hint to change my VisualStudio configuration from "debug mode" to "release mode". The result is very interesting. Division is now 2 times faster than multiplication. From now on I always let the compiler optimizing my code. :-)
Wednesday, September 29, 2004 11:58 AM by Justin Rogers

# re: Performance: Don't rely on the C# compiler to optimize your math...

I'd be very skeptical of results where division is actually 2 times faster. If you are using literals in division, then it is possible the JIT is optimizing base 2 divisors into shift operations. Ideally it would do the same thing for multiplication.

There are other circumstances as well, including the intermediate result modes. It is actually possible to run benchmarks where the FPU is scheduled allowing other instructions to run concurrently, kind of like a freebie. If you aren't storing the result of the div/mul in your test, or you aren't using it, then the JIT might just run your branch logic (looping construct) while the division is running and be ready for the next division by the time the instruction has completed. Even worse, if you don't use the result of the division, you may not be doing any work at all, since the JIT can remove code that doesn't appear to be doing any work. My recommendations are:

a) store the result in a variable scoped outside of the loop.
b) use the variable in a method call, such as Console.WriteLine so the JIT knows the value is going to be used.
c) never compare using absolute timings, but instead relative timings

You already appear to be doing c. The point here is that the JIT seems really unstable. I run performance tests all day long and I my results are constantly fluctuating, even with small changes to the source. Often times growing a method by a few instructions will drastically change the performance (drastic being +/- 5% or more).
Wednesday, September 29, 2004 2:19 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

The amount of slack would definitely double, and the balanced array really doesn't start helping until you get over 500 to 1000 elements in the array. That means each of your RubesNodes would have to be at least that large for there to be a difference.

Thankfully, your algorithm also benefits from the BinarySearch replacement I wrote, which halves the time taken by SortedArray for smaller arrays (getting increasingly less crucial for larger array lengths).

I like the array allocation for larger arrays and the insert times do get a bit better. The algorithm is a bit more complicated and definitely specialized for very large arrays. I was thinking of adapting each of the algorithms for doing topN..bottomN as well, but realized these just set the SortedArray up to be performant since it works best at small array sizes.
Wednesday, September 29, 2004 3:55 PM by Steve

# re: Performance: Don't rely on the C# compiler to optimize your math...

Ok, I will do some more tests tomorrow and consider your recommendations. Thanks for your help, I have less experiences testing performance of source code. I will sent you the source code to analyse it, if I'll get clear results.
Thursday, September 30, 2004 12:29 PM by Scott Mitchell

# re: Creating custom collections for better performance and filling in BCL holes.

Justin, you gave the following numbers:

Current Count: 10
Sorted Array: 00:00:02.5536720
Balanced Sorted Array: 00:00:02.5937296
Array Sort: 00:00:01.3920016
Skip List: 00:00:21.1904704

Care to explain what your test was doing? Was it just creating the structure and populating it with 10 elements? Or is it creating the structure and then searching the structure for various elements repeatedly? How about, adding a couple elements, searching, adding some more elements, searching, etc., and doing that a few thousand times?

They say there are three types of lies: lies, damned lies, and statistics. I could come up with a test where the SkipList beats the pants off of the array. My point is, the SkipList - or any data structure for that matter - has a time and a place. I'm not deriding your data structure, as it is obviously the best one when you only have a handful of elements. But to assume that it is universally sufficient is a mistake.
Thursday, September 30, 2004 5:24 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

Sorted Array, especially with the rewritten binary search is much faster than SkipList in the all of the scenarios you've pointed out.

Remember that searching for an element (IndexOf) is a BinarySearch. Remember that getting an element at an offset is a direct look-up (because we are using an array). The only time the SkipList could be faster is during inserts, because it isn't reallocating or moving array elements.

However, the overhead and management of the skip lists, and the fact that you used off the shelf components, such as CollectionBase for your NodeList all impact how fast it is. I'm sure you could take a few hours and come up with a better implementation. However, you'd be nearly 3 times the size for many of the numeric data types.

Does it have it's place? Of course. In the .NET world it appears to begin operating well at very high insert levels around 15,000+. At low insert levels the overhead of setting it up just doesn't make it worthwhile, as you can see in the tests. Adding/Searching doesn't make any difference because the array has ALWAYS been faster at those two things.

With that in mind, and assuming a performance increase through fixing the SkipList, we might be able to move the threshold from 15k to 5k (I won't say might, I'll actually do this and post it at some point). At that point I'd probably take some of Chris Chambers code (or probably before) and use that instead. The Slack on his is less than a SkipList and keeping the arrays small for repositioning would keep them competitive.
Thursday, September 30, 2004 5:54 PM by Scott Mitchell

# re: Creating custom collections for better performance and filling in BCL holes.

****************
Sorted Array, especially with the rewritten binary search is much faster than SkipList in the all of the scenarios you've pointed out.
****************

I apologize for not being clearer, I wasn't pointing out situations in which I thought the SkipList would outperform, merely taking some guesses at what your test case involved (which I am still in the dark on).

In any case, it seems like we've reached some common ground: different data structures each have their time and place. Basically, a data structure is designed to do well at particular tasks. For example, an array offers great access time, but sucks for searching. So you improved that with the SortedArray, so it could use BinarySearch, but the overhead now becomes inserting, since you have to shuffle around blocks of memory to do so. The SkipList (and other balanced BSTs), as you noted, improves on the searching from a native array, and insertions on a SortedArray, but has the space overhead and non-constant time access that you mentioned.

Bottom line is that there is no Holy Grail, no "ultimate data structure." You use what's best suited for your conditions. My initial point was that copying and recopying this memory seemed an inefficient approach. What I should have prefaced that with was, *assuming you are going to be working with large amount of data*. With a few hundred or few thousand records, it's no biggie.
Thursday, September 30, 2004 5:57 PM by Scott Mitchell

# re: Performance: Linked Arrays's now and later

Justin, in your "Performance"-related entries, I couldn't find anywhere where you spell out your perf. testing methodologies you are using. (I know this entry doesn't have hard numbers, but earlier ones do). Did you post about this earlier (i.e., what a test case uses, how you do your timing, the testing computer's benchmarks, etc.) that I missed?

Thanks.
Thursday, September 30, 2004 6:51 PM by Justin Rogers

# re: Performance: Linked Arrays's now and later

Unfortunately, perf testing methodology on each of the entries I write would be quite long, so I try to focus on pushing or teaching a single item in each entry. In general my perf setup involves pre-allocating the testing structure so it doesn't affect the underlying algorithm times, testing all of the boundaries and extremes conditions outside of the initial performance structure to ensure consistency in the algorithms (sometimes this is absolutely exhaustive taking many hours), followed by any loop testings...

I've tested using high performance timers and the standard date-time timers... While the high perf timers work well they are nearly identical to the date-time timers once you've worked over a sufficient number of iterations.

My timings are not based on physical timing of the algorithm, but relative timings with respect to one another. To get these in a GC independent and accurate manner I run and re-run the tests for quite some time, averaging together results, and switching the order the algorithms are run in.

I often use a number of IL or source level changes that I've identified as affecting the JITer as well, and then make sure all of these different JIT time results are consistent.

I have posted on my benchmarking machine. It is a P4 2.8 ghz processor with a gig of RAM. This is definitely an optimized and fast machine, so I also test on a number of additional machines ranging from a PIII 500 with 128, PIII 800 with 256, P4 1.4 with 1 gig. A pretty good smattering of Intel machines I'd say.

I can more specifically detail these into a posting if people are more interested in how I test.
Thursday, September 30, 2004 7:42 PM by Justin Rogers

# re: Creating custom collections for better performance and filling in BCL holes.

To note, my only major consideration was finding the cut-off, where it would be obviously profitable to make the switch from the array heuristics to the skip-list or binary tree. There is definitely a cut-off, and I'll probably talk about that at some point, but in order to do that I need the fastest, most highly optimized versions of those structures available.
Friday, October 01, 2004 7:41 AM by Ramon Smits

# re: How is that RemoveAll method on the generic List actually implemented?

Your version seems to use boundschecking.. And in your example it is very obvious that Array.Copy will generate a performance hit thus it would be very strange if the framework team would decide to use this implementation. As you say yourself "a poor RemoveAll" implementation..

A much better implementation is to use the internal array and shift elements. After that just create a new array with the size of the element count that matched the predicate.

So think about a fast easy to implement design and implement it and THEN analyse the performance. If that implementation is much slower, than you can say "I get all warm and fuzzy when the native methods work so well."

Now your remark is just information that just is not correct and of no interest for anybody.

Just my 2 cents..
Friday, October 01, 2004 7:59 AM by TrackBack

# Predicate samples for other methods on List

Friday, October 01, 2004 9:44 AM by Dan Golick

# re: Performance: Whidbey generic Queue didn't get hit with the same performance stick as the generic Stack

Well a few minutes with reflector convinced me that generic queue and generic stack use
T[] _array;
No linked-array.
I think the linked array is a great way to avoid re-allocating and copying the array as the size of the stack or queue grows.

I think you performance tests are dominated by this array copying.

If you have an application where you are tuning for performance and you know the upper bound on the size of the collection the best thing you can do is set the capacity (perhaps in the constructor).

If you re-run your performance tests with the capacity set in advance you'll see some remarkable improvements.

I'm surprised you are seeing such poor performance in the generic queue relative to the stack this may have to do with your testing methodology. Of course the queue has to do more. It must maintain a tail and a head and manage wrap-around. Nonetheless I think you have some problem such as reallocating the queue each time and not the stack.
Friday, October 01, 2004 10:01 AM by Dan Golick

# re: How is that RemoveAll method on the generic List actually implemented?

The generic version moves non-matching items one at a time so that it coallesces all remaining matching items at the end of the array and then clears them:

public int RemoveAll(Predicate<T> match)
{
if (match == null)
{
ThrowHelper.ThrowArgumentNullException(ExceptionArgument.match);
}
int num1 = 0;
while ((num1 < this._size) && !match(this._items[num1]))
{
num1++;
}
if (num1 >= this._size)
{
return 0;
}
int num2 = num1+1;

while(num2 < this.size)
{
while ((num2 < this._size) && match(this._items[num2]))
{
num2++;
}
if (num2 < this._size)
{
this._items[num1++] = this._items[num2++];
}
}

Array.Clear(this._items, num1, this._size - num1);
int num3 = this._size - num1;
this._size = num1;
this._version++;
return num3;
}
Friday, October 01, 2004 2:57 PM by Wallym

# re: We tend to use MOD parity everywhere. Alternating list styles, working with collections...

Nice. I had never thought about it that way. I had merely thought about it as a modulus operation.

Wally
Friday, October 01, 2004 5:46 PM by Justin Rogers

# re: Performance: Whidbey generic Queue didn't get hit with the same performance stick as the generic Stack

Array copying is definitely a performance hit. That really doesn't account for why the Stack comes so darn close to the LinkedArray implementation in terms of performance. Currently it is marked on my list of things that don't make sense. I'll run this down some other time.

Setting a capacity is definitely intuitive, but you often have an idea of your upper bound, but on the average your actual usage is much smaller. There are two metrics here, memory vs speed. You'd like an algorithm that reduces your memory usage and maximizes your speed. I generally like to use lower bounds when setting my capacity, especially with something like the linked array queue. Why? Well, with the linked array, whatever size we set that growth factor to in the beginning, we'll continue to add more segments of that size as we go along. In my tests I tend to use 10 as the growth factor, so that queues of length 100 have 10 segments, 1000 have 100 segments. That is a lot of segments. If I knew my lower bound was in the range of 100 and my upper bound in the thousdands, it would be more beneficial to set that initial growth factor to something around 100 or 200. I minimize the use of space and maximize the performance.

Well, I think at this point we are assuming that we both have the same versions of Whidbey. That may not be the case and could lead to some nastily confusing results. I feel pretty secure in why the Queue is soo much slower, they are simply doing far more array work than should be necessary. If the Reflector tool is correct, they are also making the mistake of *clearing* the array as elements are removed by setting them to T.default. I can't see where this solves any problems, especially since T.default can be an actual valid program value (0 or null).
Friday, October 01, 2004 5:57 PM by Justin Rogers

# re: How is that RemoveAll method on the generic List actually implemented?

Ramon: My information provides an upper bound, as mentioned a naive implementation, and as such is definitely useful in analyzing the algorithm at hand. It shows that in this case they didn't use the slow Array.Copy, however, they use it consistently throughout the rest of the Framework in situations where they have more performant options. It is clear to me, from studying their code, that they might make this mistake. We proved they didn't make the mistake and we are all the better for it.

Dan: Hey, you even fixed up the Reflector code a bit ;-) Not many opportunities for speed improvements there. Removing the Clear would be nice for sure. You can remove 5 or 6 ops from the code with just a basic comb-through. Thanks for your input.
Friday, October 01, 2004 8:23 PM by Dan Golick

# re: How is that RemoveAll method on the generic List actually implemented?

I had to fix the reflector code because I couldn't stand the gotos!

My experience is that the clear is pretty damn fast as it eventually becomes a fast
rep stosl
where the whole fill is done in 1 (many cycle) machine instruction.
Friday, October 01, 2004 8:28 PM by Justin Rogers

# re: How is that RemoveAll method on the generic List actually implemented?

Well, I'm certainly happy you fixed it up a bit. After comparing the exact RemoveAll algorithm, we really are able to remove some extra operations, even if we leave in the Array.Clear...

I agree that both memory copy and clearing operations tend to be extremely fast and generally we don't have to worry about them too much. My only concern is needed versus unneeded. They do similar *clearing* in other functions like Pop and Dequeue. This changes the paradigm of:

return data[current--];
to
int value = data[current];
data[current--] = T.default;
return value;

The need for this clearing is insidious, because you can't honestly do it in one place and not do it in all others. You can see that it has much more impact when working on items a single item at a time. Almost like running a C++ program in debug mode and having it clear all of your memory for you ;-)
Saturday, October 02, 2004 10:39 PM by dudu

# re: Some security considerations for systems with events.

Great post!
Sunday, October 03, 2004 7:58 AM by Dan Golick

# re: Performance: Refactoring generic List.RemoveAll... Speed lost in implementation and Array.Clear...

I guess the risk we take with removing the Array.Clear is that some object won't get garbage collected because it is now still reachable.

I like the improvement in the while loop.
Do you think we could do better?
data[hoffset++] = data[toffset++];
seems awfully inefficient if there are multiple match failures in a row. If you modify the loop to find the start and count of the items we are keeping you could use Array.Copy (which is very efficient) to blast a block at one time.
Sunday, October 03, 2004 4:29 PM by Justin Rogers

# re: Performance: Refactoring generic List.RemoveAll... Speed lost in implementation and Array.Clear...

Ah, for reference type collections the clear makes good sense then, while value type collections it tends to make no sense whatsoever.

It would be an extra if check to add functionality for that (damn), which would cost you one extra operation on a reference type (making it a bit slower) and give you extra performance on value types (making them slightly more than a bit faster). Definite trade-off. Not to mention you'd have to add some type checks, probably during construction, to make the differentiation.

Array.Copy has a lot of extra overhead for making calls to it. There would probably be specific sets, where you are trying to remove ranges, that might make sense. If your data is so ordered, it would make much more sense to use an alternate means, such as RemoveRange, based on IndexOf and LastIndexOf expressions where you use your Predicate.

If you want to paste in an algorithm that uses Array.Copy instead, go ahead. I'll drop it into the test harness. Give me some data that is specifically stacked in it's favor as well.
Sunday, October 03, 2004 6:27 PM by AndrewSeven

# re: Optional parameter overloads in C# and cascading calls...

I was recently working on a navigation controller for web projects. It handles building a url string and redirect to a url.

I started with a version that had many variations on the methods. There were two variations on every method for http/https urls.
Its implementation was spread across various methods, with all kinds of cascading calls.

It was hard to make sense out of, and when the moment came to alter it's behavior, I would have had to override many methods as they were all virtual.

I started a new class with the same methods, but I put all the implementation into a single method that I called from every override. I used "Implementation" as a postfix for the name of the "real" method.
The overloads stopped being virtual, and all the behavior can be changed by overriding the protected "BuildUrlImplementation".

I've been trying to put stuff in the xml comments to indicate what kind of call each overload will actualy make.



Monday, October 04, 2004 4:23 AM by TrackBack

# Great article on WinForm UI thread invokes

Monday, October 04, 2004 9:34 PM by DarthPedro

# re: A comical presentation of modern day RPG development from a gamer's perspective...

This is hysterical.
Tuesday, October 05, 2004 9:24 PM by Brian Grunkemeyer

# re: Reliability, CER, and new programming challenges and decisions...

ConstrainedCopy does not copy the array, but instead it disables some of the tricky games that Array.Copy does (and is thus constrained). Most people may not know this, but Array.Copy does a surprising amount of crazy operations. It will downcast for you, box & unbox value types, widen primitives from one type to another, and it used to copy into and out of arrays of Variants (when we still defined Variant as a public type, before V1 shipped). I believe it even handles copying from a Remoting proxy to a MarshalByRefObject to the real object itself.

ConstrainedCopy gets rid of the junk. I needed ConstrainedCopy to simplify writing a hardened ArrayList implementation that would survive async exceptions like thread aborts, out of memory exceptions, and perhaps stack overflow exceptions. What I wanted from ConstrainedCopy was what the C++ camp has called a "strong exception guarantee" - either the entire operation happens, or we throw an exception and no other observable state changes happened. Hence, the WillNotCorruptState value. ConstrainedCopy will only do the operations that the CLR can prove will work based on the types of the arrays themselves, instead of on the values within the array (ie, no downcasting) and no operations that may fail if we happen to run out of memory half-way through (ie, no boxing). We also got rid of primitive widening to keep things simple. Note that the implementation of this method is provided by the CLR itself (by a mechanism we call FCALL, allowing us to write "manually managed" code in the CLR itself), and is not subject to thread aborts. Also, the reliability contract allows us to eagerly prepare the method if we're called from within a constrained execution region, hoisting all the CLR-induced failure points to some point before we run this code. Therefore, we can assume that if we call this method and it doesn't throw an exception due to invalid arguments or array type mismatches, then the copy operation happened in full. (While this may strictly not be true in some pathological cases where the OS cannot page in your memory from a swap file due to a disk error or some other catastrophic kernel-level corruption, we believe those are so truly pathological that they aren't interesting.)

As the name suggests, it really is a constrained version of Copy, and constrained in terms of what functionality it will support. However, in return for these constraints, it does allow you to make some stronger assumptions when using it to build reliable code. I may try writing a blog entry on this at some point - check the BCL blog in a week or so for details.

Brian Grunkemeyer
MS CLR Base Class Library team
Wednesday, October 06, 2004 12:02 AM by Justin Rogers

# re: Reliability, CER, and new programming challenges and decisions...

Any, as of yet, public documents we can take a look at? Following the Ximian implementation of Mono, they've been checking in reliability level code changes. Apparently they have strict rules against using Rotor or the MS produced libraries, leaving me to believe they are working based on some documents.

Maybe something similar to the C# 2.0 draft specs?
http://www.dina.dk/~sestoft/ecma/CSharpStandard-June2004Draft.pdf
Wednesday, October 06, 2004 7:07 AM by Alexander Neumann

# re: Type identity for data is important. How do we refactor collections in the face of Whidbey?

If ArrayList is to List<T> then CollectionBase is to Collection<T>.
Wednesday, October 06, 2004 5:13 PM by Justin Rogers

# re: Type identity for data is important. How do we refactor collections in the face of Whidbey?

Collection<T> bears some resemblance, but the code pattern has changed quite a bit from CollectionBase. They took a hands off approach where much of the work previously being done is now offloaded to an implementation level. I'm not sure there is very much parity between Collection<T> and CollectionBase.
Wednesday, October 06, 2004 8:01 PM by Duncan Godwin

# re: Reliability, CER, and new programming challenges and decisions...

This presentation contains a few notes about upcoming reliability features in the CLR:
http://research.microsoft.com/collaboration/university/europe/Events/AcademicDays/Portugal/2004/Don%20Syme.ppt
Thursday, October 07, 2004 10:49 AM by Dan Golick

# re: A great article on the value of a good programmer...

Hey Justin,

I also love making my code highly performant. But I tend to profile and count on the profiler to tell me where to spend my time. It's different if you are writing libraries. When implementing a library you have to think about the use cases. Collection classes of course need more care because by their nature they will be looping over items many times.

I notice in a previous post that you were factoring a loop into two loops one for a null comparison and one for non-null. I often find myself doing this in critical code. Today I came upon a case where I had a loop being called many millions of times that had many tests inside of it. Now I'm toying with the idea of using code emit to create the code for the loop so I can create a version without the tests taylored for the job I'm doing.

What do you think? Is this a brilliant idea or a terrible one? The down-side is that debug may be harder but the performance should be terrific.
Thursday, October 07, 2004 10:50 AM by Sahil Malik

# re: Performance inspiration found in List.Contains() method plus some additional functionality we can add.

Awesome .. and since I was reading this blog thru an aggregator, I had no clue of your background. I might not be too off in assuming that you work for microsoft, probably writing parts of the .NET framework.

One thing that these RSS feeds should have is a little 50 word author profile that is easily visible with every post. Like in your case "Senior Software Design Engineer and Consultant" -- didn't show up in my aggregator (saucereader).

Anyway, I just wish there was this tool around to help me filter my content better.
Thursday, October 07, 2004 10:52 AM by Sahil Malik

# re: Performance inspiration found in List.Contains() method plus some additional functionality we can add.

Incidentally, I am so thrilled about generics; I posted a message on my blog last night about what generics imply - which is terribly understated I feel. People with a good C++ background would probably understand that generics let you do what you couldn't do without them ... unlike most other technologies that are simply a better way of doing something you could already do. (CTE's in Yukon for example)
Thursday, October 07, 2004 12:17 PM by Haacked

# re: A great article on the value of a good programmer...

Hi Justin, Mythical Man Month by Fred Brooks is one reference for those claims.

I believe it. As a senior developer manager/architect, I'm currently trying to recruit four new developers. You'd think it would be easy given all the rhetoric about unemployed programmers, but the disparity in quality between any two candidates are very high. I wrote a couple of quick posts about my current experience.

http://haacked.com/archive/2004/10/01/1294.aspx http://haacked.com/archive/2004/10/06/1308.aspx
Thursday, October 07, 2004 1:36 PM by Barry Kelly

# re: A great article on the value of a good programmer...

I reckon the major difference is not in performance of code produced but in actual productivity. The two key abilities, to understand the problem and create a solution which fits into the existing code environment in a non-breaking fashion, are scarce.

Trying to focus on performance as the first thing almost always leads to mistakes and overwrought code which needs to be rewritten when the requirements are more clear - which they always are after every iteration, at any level.

-- Barry Kelly
Thursday, October 07, 2004 2:08 PM by Barry Kelly

# re: A great article on the value of a good programmer...

Dan Golick: dynamic code generation can be a huge win if the compiled code is then static and reusable and the cost of producing it can be amortized over all the times it is used.

A project I'm working on had an inner loop which used Activator.CreateInstance() to construct objects which were loaded up from an assembly at run-time. Throughput doubled when I did 4 things:

1) Created an interface with a method which had the same signature as the constructor call.

2) Dynamically generated a class which implemented the interface and called the constructor which Activator would have called via reflection.

3) Constructed a static instance (using Activator, ironically) of the generated class.

4) Replaced the single call to Activator.CreateInstance() with a call to the appropriate method of the appropriate generated class's instance.

The reason throughput doubled was that (in a sample small test run) 10 seconds of 20 seconds was taken up inside the Activator.CreateInstance() method. The construction code fell off the radar after the optimization.

-- Barry Kelly
Thursday, October 07, 2004 2:32 PM by Justin Rogers

# re: Performance inspiration found in List.Contains() method plus some additional functionality we can add.

I've often worked with MS on various projects. I always keep a resume link in the News section of my blog in the upper left, again, something that is missed if you read through an aggregator (quite a shame). I wish my author would aggregate with my title. I do my work through my own company, DigiTec Web Consultants, LLC. and I'm always looking for new and interesting contracts. Most people tend to not realize that.
Thursday, October 07, 2004 2:58 PM by Justin Rogers

# re: A great article on the value of a good programmer...

Dan: Can conditional compilation work in this case? I definitely try to refactor and remove the excess code and this can often be a way to put the checks in and remove them later.

Haacked: Always looking for contracts if you need a surgeon for some code. As you can see from the blog I'm fairly focused on what I like to do, but I have a broad range of talents, and a fairly decent resume (see the News section). I also have doc format available if you want to hit that contact button at the top of the page. I had someone WRITE the doc version, so I think they caught all of my spelling errors (though I have added a line or two).

Barry 1: I like to make the following comment whenever I'm asked how long it takes me to do an in-depth performance analysis. "The more you work something, the better, faster, and more efficient you become at it". This is a very general concept, but it applies to everything that one might do. There is the basic factor of intelligence that you can layer on top of this for the *figuring things out* part, but once we all breach the 168 mark and get into hard to measure ranges you can toss that out as well. It takes me the same amount of time to write performant code as it does for me to write normal code. That is because in my head the algorithm comes out quickly (pardon the odd joke) without thought of refactoring really. Often times I do go back later just to double check, but I don't think I sacrifice any productivity for it. I often think managers don't realize you can get a secure, performant, productive developer in a single package any more. Putting all three of those things together probably meets at least the three dev requirement. I figured those three things were the reason I could put the senior in my title.

Barry 2: For your interface problem, the binding in Activator.CreateInstance can be pretty nasty. There are other options though, including finding the constructor you need through reflection and invoking it directly while caching the ConstructorInfo. A static factory method (a method that returns an instance) is a second option. Not knowing your scenario, you may have already found your best option.
Friday, October 08, 2004 12:41 AM by TrackBack

# re: I was not aware of that...IsNumeric in C#

Friday, October 08, 2004 3:36 AM by James

# re: We've all recently fallen prey to the weblogs.asp.net changes to increase throughput on the server.

Is there an easy way to have webcontrols without IDs? When nested deep inside a number of other controls they can get very long.
Friday, October 08, 2004 2:25 PM by Justin Rogers

# re: We've all recently fallen prey to the weblogs.asp.net changes to increase throughput on the server.

Yep, customize your HtmlTextWriter to ignore that specific attribute. You'll see that in HtmlControl.RenderAttributes there is an explicit WriteAttribute for ("id", ClientID) giving you an idea of what you'll have to look for in WriteAttribute.

To be more selective you can override RenderAttributes on specific control types. HtmlAnchor is giving .Text a problem, so they could selectively create a better version that removes the id field.
Friday, October 08, 2004 2:39 PM by Scott Mitchell

# RE: We've all recently fallen prey to the weblogs.asp.net changes to increase throughput on the server.

Some of those IDs are needed because they are referenced in the CSS. But many of them (those of the 'HyperlinkX' flavor) can be scratched. The other suggestions are great, but I wonder at what point taking the time to make these small improvements outweighs the actual cost of just having them there... (i.e., the view state is, what, 100 bytes?)
Friday, October 08, 2004 2:52 PM by Justin Rogers

# re: We've all recently fallen prey to the weblogs.asp.net changes to increase throughput on the server.

That was odd, had to unmoderate that comment for some reason...

In response, there has been a huge amount of work already done to compress the page sizes down. Most explicitly in terms of real compression which results in a much smaller page size. If all of THAT work has already been done, but you are worried about compressing a 2k CSS file as well (they enabled compression for the CSS file as noted by ScottW) then I think removing 100 bytes of viewstate and 2k worth of id's would be a great improvement (especially since the main page isn't cached on the user's machine for nearly as long as the CSS file is).

1. Removing ID's is a 3 line code change.
2. Getting rid of viewstate is a 1 liner
3. Smaller class names is a couple of minute grep/replace.
4. Stripping tabs is a 5 line code change.
5. Compactifying links is another very cheap change as well.

Half a days work (maybe, probably less) and you've just cut your page sizes by another 10%? I think the existing changes that were made took far longer than this and some of them resulted in the same or smaller pay-offs. The only super win space change you get is your compression. All the rest are trade-offs (compression is a computational trade-off, but gzip is one of the fastest compression algorithms).

Saturday, October 09, 2004 1:10 AM by TrackBack

# re: Can we please add support for String overloads wherever we see char[] in the framework?

Saturday, October 09, 2004 11:50 PM by Larry Fuqua

# re: Doing a comprehensive review on the "Beginning .NET Game Programming in VB .NET", need your feedback.

I purchased the C# version within the last month. I had been wanting to try my hand at hobbyist game programming for a while so it looked like a good book to get started with C#.

After going through the first few sections on GDI+ stuff, I was very pleased with how fast it was to pick up. Now, to put this in perspective, I am a software engineer (although not a C# expert) so although there were a few errors in code in the book, I was able to correct the problem and move on. One thing I was particularly pleased with was the fact that when I downloaded the sample code from APress, I had no significant problems building and running them. I have had some bad experiences in the past with not being able to build the sample code for a book.

As I started into the DirectX 3D stuff I was a little skeptical. I had heard bad things about DirectX in the past. However, the next sections went so well that I didn't complete the final chapters but immediately began working on my own project of a 3D RPG style game written using C# and Managed DirectX.

A couple of the things that gave me problems:
- camera/perspective. This really nees to be explained better, its such a critical element. This book seemed to recognize this fact, since a fair amount of space was devoted to this topic in the book. However, some very key things were not explained. I would suggest authors try finding someone who is new to 3D programming and have them read the proposed text for this section. Once done, let them then ask questions where they were confused, then ask them what were the key hurdles in comprehending how it worked. I've read several different authors attempt to explain this topic, but it always seems that the author knows the subject too well to put themselves into the shoes of a beginner.
- point out that a lot of the Managed DirectX classes are in fact structs. For a beginner at 3D programming, such as myself, this was an issue. I was confused by certain assignments because I was assuming the object was a class instance when in fact it was a struct. Veteran 3D programmers would probably not have this problem.
- difference due to Summer 2004 release. Since I just recently bought this book I was working against the newest DirectX SDK, so of course some things had changed (took me a while to find out why I didn't have a DirectX wizard). How can you plan for this?

All-in-all I was pleased with the book. I still go back to it for little tidbits. I do think that anytime someone puts out a 'Beginning' book they should put out a 'Intermediate/Advanced' book at the same time. Anyone like myself will be immediately looking for the next step up as soon as they are done with the first. If you wait a year, well I'll have already found what I need in the meantime so you miss out on the extra sale. I would have immediately grabbed an 'Intermediate/Advanced' version of this series.
Tuesday, October 12, 2004 11:08 AM by David Weller

# re: Doing a comprehensive review on the "Beginning .NET Game Programming in VB .NET", need your feedback.

Great feedback, Larry! (yes, I'm watching this thread :-). I will be posting an update to the C# code shortly, as well as the VB code (the book make it to the bookstores before I had posted the code to Apress).

Your feedback about wanting an Advanced/Intermediate book is interesting. I haven't had many people ask for something like that, so I'm wondering if other reviewers felt the same way...
Tuesday, October 12, 2004 3:57 PM by Justin Rogers

# re: Doing a comprehensive review on the "Beginning .NET Game Programming in VB .NET", need your feedback.

Yeah, I was hoping for a few more revievers myself as well. Can't run a testimonials portion of a review without at least 2!

I'll have some feedback for you as well. Nothing major, since I think this book has done a great job of hitting beginner issues, the format of compiled/completed games allows future growth by the reader, etc... Some things are stylistic, some things are just questions, and in many areas I'll just be expanding on your references section.
Tuesday, October 12, 2004 6:24 PM by TrackBack

# I'm gone for four days and look what happens...

Tuesday, October 12, 2004 10:11 PM by Sally_Fish

# re: Gaming: Fable, must or bust?

ok Fable rocked any one who thinks otherwise can go eat a dick.
Wednesday, October 13, 2004 7:12 AM by TrackBack

# Justin Rogers with another great blog post series...

Wednesday, October 13, 2004 8:40 AM by dotnet developer

# re: Game Development: Book resources for users interested in more in-depth world generation discussion.

Publish those c# samples for "Real-Time 3D Terrain Engines Using C++ and DirectX 9 (Game Development Series)" and I'll buy the book for sure!
Wednesday, October 13, 2004 8:49 AM by Justin Rogers

# re: Game Development: Book resources for users interested in more in-depth world generation discussion.

Haha. I see how it works ;-) Yeah, I figured that might be the case. The audience would widen with a version in Managed DirectX and C#... I'll send off an email sometime tomorrow and see if I can't track down Mr Snook or his tech editor. I'll also see if they want to put some Managed DirectX based Terrain code in the next version of Game Programming Gems. I had mentioned something a while back and they got excited, but I think they locked down the articles for the next version of the book already.
Friday, October 15, 2004 10:42 AM by Raymond Chen

# re: Quickly vaidating chess moves, a mixture of performance and ingenuity.

Hm, the above code says that A1-G1 and A1-H2 are legal knight moves.
Friday, October 15, 2004 11:07 AM by Ron

# re: The recent effects of voice chat on gaming. Unbalancing, breaking down borders, improving teamplay, you decide!

Admittedly, my last experience with this is a couple years old. I am generally a role-player, but I kept it strictly in-game. When using voice chat, I used my normal voice and way of talking. I found the capability greatly helped with raids, but hardly ever used it for normal grouping.

In terms of privacy, it's less invasive than an actual phone call. Generally, I already felt comfortable with the people I was chatting with. It didn't seem that big a deal to be concerned with what they might think of my voice.
Friday, October 15, 2004 2:13 PM by Kenneth Russo

# re: The recent effects of voice chat on gaming. Unbalancing, breaking down borders, improving teamplay, you decide!

I don't think there's much question that it's all of those things. Back in the days of Ultima Online a few co-workers and I would sometimes stay late at work to play just to get the advantages of fast communication while PvPing. And I can remember playing with a phone attached to my ear.

First used voice-chat, BattleCom, shortly after the release of EQ. Completely and totally changed the gaming experience. Nobody in our group wanted to type anymore. I remember being elected the "spokesperson" to the rest of the gaming world because I typed faster than the others.

It had downsides too. We became quite insular. The server became those of us who used B-Com (us) and those who didn't (everyone else). Grouping with people outside our click was less fun because typing was introduced and we generally felt that typing reduced the fun. Poor outsiders who would group with us would invariably ask shortly after joining why everyone was so quiet. We'd try to keep them involved, but the general consensus was it was too much effort.

The male roleplaying a female thing, heh. We outed so many.... Our saying was "Everquest: the place where men are men and so are the women." For the record a number of our regulars (primarily males, but we had a few female regulars too) played female characters exclusively. Must have been a Wood Elf thing (female Wood Elves were drawn to be easy on the eyes).

While there's no doubt that performance is enhanced (e.g. PvP), the biggest draw for us was the enhanced social experience.


Friday, October 15, 2004 5:24 PM by Justin Rogers

# re: Quickly vaidating chess moves, a mixture of performance and ingenuity.

Ah, good find there. Appears rank checks are going to be required then. I had actually reduced them out, though apparently that was a big mistake, in order to reduce instructions. Shows you what happens when you don't write all of the tests you need for a particular method.

if ( dloc/8 == cloc/8 ) { return false; }
switch(...) {
...
}
Friday, October 15, 2004 8:23 PM by Raymond Chen

# re: Quickly vaidating chess moves, a mixture of performance and ingenuity.

That still lets A1-H2 slip through...
Friday, October 15, 2004 9:45 PM by Justin Rogers

# re: Quickly vaidating chess moves, a mixture of performance and ingenuity.

Raymond is definitely kicking my ass here. First time I've worked through algorithms for chess that weren't based on a 2D board layout. Normally, you can simply take the difference of clocX...dlocX and clocY...dlocY and compare the results to make sure they return a 1 and a 2.

Since adding more checks to the switch would cause extra overhead that simply isn't needed, we can instead refactor a new algorithm that works based on rank and file within our linear array space.

switch(Math.Abs(dloc/8 - cloc/8)) {
case 1:
if ( Math.Abs(cloc%8 - dloc%8) != 2 ) {
return false;
}
break;
case 2:
if ( Math.Abs(cloc%8 - dloc%8) != 1 ) {
return false;
}
break;
default:
return false;
}

Integer divisions can be removed by shifting if need be. Though it doesn't provide much extra. We can also us bit masking to avoid integer modulus (& 8 is the same as % 8). Probably not needed.
Monday, October 18, 2004 2:12 PM by Paul

# re: Probability selection using a chance tree. Solutions for standard and loaded dice.

Hey Justin,

Had a quick question on the Find probability method I was wondering why do you return the current if chance is less than prior.
if ( chance < prior ) {
return current;
}

Thanks
Monday, October 18, 2004 6:55 PM by Justin Rogers

# re: Probability selection using a chance tree. Solutions for standard and loaded dice.

Great question... I assume that the random number generator is going to range from 0 to GrowthWeight-1 of the entire tree. This will give me the values {0, 1, 2, 3, 4, 5}... Since in the dice example, the CurrentWeight is 1 for the first die and the value in the PRNG is 0, we have to find a way to accurately return the item. In this case I do the following:

prior += current.CurrentWeight;

Since prior was initially 0, then now it is 1. Comparing 0 < prior yields a result of true, which is what we are looking for. Now say we had passed in 1 from the PRNG, now we have to test the left side:

if ( chance < (prior + Left.GrowthWeight ) )...

At this point GrowthWeight for the Left side should be 3 (1 for the first node, 3 for the Left, and 2 for the Right if our tree is laid out appropriately). Since 2 < 4, we now traverse the left side. But we don't know which item on the left side is the closest match, so we start our entire process again recursively, using the previous value of prior (root node's current)

The second time through we'll add prior += current.CurrentWeight = 1 + 1 = 2. We do the test again, 1 < 2, which is true and so we return our node.

We could have set up the PRNG to return numbers in a range from 1 to n... 0 is a kind of special case in doing 0 to n-1... because we don't have to do as much math.
Tuesday, October 19, 2004 12:13 PM by Paul

# re: Probability selection using a chance tree. Solutions for standard and loaded dice.

Thanks for the quick response. I have another question on FindNode. At this point we are searching for a chance, so in the test driver dice program if we pass in 2 from the PRNG we are searching the tree for a dice roll that has a chance of 2 of showing up? A chance of 2 which based on the dice numbers we filled the tree with will always be a roll of 4?

Thanks
Tuesday, October 19, 2004 3:03 PM by Justin Rogers

# re: Probability selection using a chance tree. Solutions for standard and loaded dice.

The tree controls all of the probability selection and return. When we initially set up our probability tree, because we do so in a linear manner, certain assumptions can be made. Such as number N always returns value M. These assumptions are unsafe and are unimportant. If we wanted to rebalance the tree at any time we could (self-rebalancing), in which case N may return some new number that is not M.

To elaborate the dice sample, the PRNG is generating numbers from 0 through 5 (6 discrete values). These values correspond to chance indexes into the probability tree. Take the following 6-sided dice where the sides only contain the values 1 and 2:

{1, 1, 1, 2, 2, 2}... The number of possibilities is 6, the number of chances for either of the values is 3. Numbers from the PRNG index the entire set of possibilities and the tree maps that index, using the chance weightings, to a value of either 1 or 2.

Hopefully that is more descriptive.
Friday, October 22, 2004 10:08 AM by TrackBack

# re: Performance Comparison of Java/.NET

Sunday, October 24, 2004 1:47 AM by Jerry Pisk

# re: Architecting your own cache. Speed, efficiency, memory consumption, AND it has to actually work?

Searching a b-tree hashtable is also a simple binary search, especially if you keep your tree balanced. But inserts and deletes are a lot more efficient in a tree than in an array, especially once your cache grows to more than few items. But I'm going to read your previous posts, maybe there's something I'm not seeing here.
Sunday, October 24, 2004 2:04 AM by Justin Rogers

# re: Architecting your own cache. Speed, efficiency, memory consumption, AND it has to actually work?

A few elements is actually more like a few thousand elements when it comes to running under the CLR. Array copying is very fast and efficient (insert/delete) and all of the collections even moving forward to Whidbey are going to continue backing with array's rather than use more specialized data structures.

Hashtable and other dictionaries are also array backed and I'm pretty sure that continues moving forward.

In fact there are only a very few small locations where anything but array's are used. Namely some code in the Windows Forms area that I've pointed out previously.
Sunday, October 24, 2004 4:14 PM by haacked@yahoo.com (haacked)

# RE: I sure could use a million dollars, what about you? Transitioning from Riemann's to P vs NP...

I have an elegant proof of this that I wrote in the margins of my notebook... Oh. The dog tore it up.
Sunday, October 24, 2004 8:42 PM by Justin Rogers

# re: I sure could use a million dollars, what about you? Transitioning from Riemann's to P vs NP...

Whether your comment is in wit or jest is something left to the engaging mind. However, the implied allusion to Fermat in regards to the proof of his "last" theorem is somewhat ingenious in this context.

To note, Fermat's Enigma, a very small and highly achievable book for those of mathematical or non-mathematical mind alike, covers the history, if not the actual work, very well.
Sunday, October 24, 2004 9:00 PM by Peter Waldschmidt

# re: I sure could use a million dollars, what about you? Transitioning from Riemann's to P vs NP...

Too late.

http://news.zdnet.com/2100-3513_22-5229702.html?tag=zdfd.newsfeed
Sunday, October 24, 2004 9:39 PM by Justin Rogers

# re: I sure could use a million dollars, what about you? Transitioning from Riemann's to P vs NP...

De Brange has also demonstrated a previous attempt at the proof that wasn't successful. His current attempts may yet prove fruitful though. I'm not aware if he has of yet published his proof in an actual mathematical journal, nor have I seen very many individuals line up to examine/protect the proof as valid.

If you have more of a link than a public news site that would be great. His full paper is available online and several portions of the proof are less than accessible to my own inspection. Which is actually one of the reasons I started to switch off the problem, giving me a chance to study in some of the areas that I'm unfamiliar with in relation to this latest *proof*.
Sunday, October 24, 2004 10:16 PM by frebes

# re: Are neural networks data or code? Game developers always have an interesting twist...

Great info.
Monday, October 25, 2004 3:43 AM by TrackBack

# Code Only: Bidirectional roman numeral parsing.

Monday, October 25, 2004 8:17 AM by Michael Weinhardt

# re: Turning bitboards from potential moves into legal moves, pawn moves, and conditional rules.

Hey Justin,

I'm building a chess game myself, primarily for the challenge since I'm no game designer. So thanks for your chess and game discussions.

Having no experience in game design, and wanting to build a chess game for WinForms Whidbey, would there be any books you recommend, or a single, uber-book that I *must* read before building a game, or even a site that goes hard core into the details?

Cheers!
Monday, October 25, 2004 11:59 AM by Justin Rogers

# re: Turning bitboards from potential moves into legal moves, pawn moves, and conditional rules.

I'll have a book out pretty soon that covers application development under Whidbey. If you really want to target Whidbey then you'll need a book that covers the new features since they are indispensable in quickly creating a solid game or application.

I'm a strong proponent of genre programming, so I don't have any books to immediately recommend for your consumption. Genre programming is where you find a resource specifically targeted at the game or type of game you are trying to create. While I've reviewed several books on .NET Game Programming, I don't feel that any of them have really covered the board game genre.

In retrospect board game programming is one of the less exciting types of programming in book terms. The graphics and UI engines are generally very simplistic, while the game rules and data structures are highly specialized to the game itself. Books targeting this genre have to be ready to cover some complex topics in depth to obtain a readership and at the same time are targeted only at people that want to program those particular games using the most advanced techniques.

I am an open book when it comes to the site. If you have specifics that you are interested in let me know. There is also another individual that I'm very excited to be working with that will soon release an online chess engine, and that may also be a very good place to start as well.
Monday, October 25, 2004 12:55 PM by Mike Lorengo

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

A better solution would be to create an array of string [30] lookup = { "0", "I", "II" ... };
Then do a match on the input and return the index. Quick and easy :)
Monday, October 25, 2004 12:59 PM by Justin Rogers

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

Show me!

There is a partial test harness in the code, and I can upload the performance harness and boundary test harness to make sure you catch all of the invalid input.

Remember, on my blog you can't make a statement without backing it up with proof.
Monday, October 25, 2004 1:12 PM by Justin Rogers

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

My goodness, you were speaking more to the design of the original question. I would postulate, in your defense, that they'd accept such an algorithm, and then ask you to write a more general parser. This isn't just based on assumption, but rather the level of detail that is required during any sort of development oriented interview at Microsoft.

Sorry to seem so rude in the comment. Now that I know more of what you are speaking towards, yes, it would be possible to do an IndexOf over the array, and return the index. Just be forewarned and prepared when they ask you to rewrite the parser to accept any value from 1 to Int32.MaxValue and make sure you bring lots of dry erase markers for that huge array.
Monday, October 25, 2004 1:30 PM by Mike Lorengo

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

You'll notice the smiley. If we were doing agile programming, we would take the simplest approach that would work. The point of my post is to understand the problem. You stated yourself the solution only required a max of 30. I'm sure at some point the memory requirements and speed will favor a parser over a lookup. I'm not prepared to back that up however here is a "classic" post for you to read...
http://www.differentpla.net/node/view/58

Keep up the good work Justin, I enjoy your entries.
Monday, October 25, 2004 2:03 PM by Justin Rogers

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

That article was actually fairly enjoyable. Oddly enough I'm not sure I could code using that technique, but then I also yell at people for build fixing (build fixing is where you quickly write code, compile to get bugs, then fix). At least the test-driven technique isn't quite compile fixing. Thinking back I did write an entire page or two full of invalid inputs that I wanted to ensure the function got rid of, so maybe I do more of that than I think?

Looking at that post our integer to roman numeral routine could be more table driven if we chose. In fact, I'll add a table driver right along-side the existing routine and upload in a bit.

Now, the reason int2Rom is easy to write as a table driven approach is that there is no ambiguity in the handling of an integer. Aka, the interger is in the right format, no matter how you slice it.

With the reverse rom2Int the input string can be completely bogus and not represent a roman numeral at all. Even worse, it can be a malformed roman numeral. A table driven approach would work to a point, and then require that you bolt on validation for the basic things like "VV" which is invalid or "DD" which is invalid.
Monday, October 25, 2004 2:31 PM by Derick Bailey

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

I wonder if it would be possible to do roman numeral validation using a regular expression? And then, could you use a single replace parameter in the expression, to convert the numeral to an integer?
Monday, October 25, 2004 2:48 PM by Justin Rogers

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

Short answer is No! I don't believe so. This falls into a class of problems that I've been thinking about for a while in terms of how powerful regular expressions could be. So let's talk a bit about replacements...

1. The standard replacement is to take a Match region and replace it with Groups out of that match.
2. Replacements don't allow mappings in the replacement string. An example would be a conditional capture that if captured, replace with 1, etc... You can map 1 thing (aka replace a Match with some fixed string).
3. You can only construct captures from material in the string. You can't create new material at any point...

Longer answer is... Yes/Maybe! It would be complex, but you can add material to the string before you do a replacement. By appending the digits 0 through 9 as the first ten characters, they can now be used to construct captures.

So what kinds of captures do we have? Well, We use conditionals that look for a pattern in the string, and then create a new capture group that is either empty or filled with a number... By concatenating all of these items in the replacement string, you can possibly retrieve a number back.

The maximum number achievable will be fixed to how you build expression. Remember that thousands can walk on forever, and it isn't possible to *count* them and then use that information to more properly add digits... Actually, I take that back, you kind of CAN do that too...

I might put some work into this. I imagine the place to use it would be in client-side script, however, we could just as easily convert our parsers to Javascript...
Monday, October 25, 2004 4:08 PM by Justin Rogers

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

Just to give an idea of what in the hell I'm yammering about above:

((?<two>II)|(?<one>I))(?(two)(?=.*(?<ones>2))|(?(one)(?=.*(?<ones>1))|(?=.*(?<ones>0))))012

Notice that we match 012 at the end... The input string needs to be of the form II012 or I012 for the above example to work... So what we assume is that we'll be appending 012 to the end of any roman numeral BEFORE doing the replacement.

We'll in essence take 9 sub-patterns to represent the ones, 9 sub-patterns for the tens, 9 sub-patterns for the hundreds... This is a first chance at this after only thinking a few minutes, so if you spot a short-cut that I'm missing please point it out.

Being a tool driven world we live in, I now know the intricacies of the expression, enough such that I can dynamically generate the full pattern with just a small amount of code... Yay!
Monday, October 25, 2004 4:50 PM by Tom McLeod

# re: random number filtering devices

I haven't played a lot of RPG games on paper or computer, but your opening discussion of why filters are necessary, and then the allowance of doubles, made me think of another non-rpg game that has its own sort of 'filter' on multple values. Monopoly's standard rules allow you to roll again on doubles with two 6-sided dice, but they penalize you for rolling doubles three times in a row by sending you to jail. Harmless addition to add fun to the game, or statistically calculated to create balanced gameplay? Interesting.
Monday, October 25, 2004 11:10 PM by Justin Rogers

# re: Create random number filtering devices using the VectorChanceTree

I can't speak to the minds of the original designers of Monopoly, but I believe this would be a gameplay issue. There are several sites on the web that examine the statistics of obtaining any of the properties on the board, given your current location on the board. Something like preventing more than 3 valid roles disallows things like obtaining the highest ranked properties on the first turn.

12 + 12 + 11 = 35, a short step away from the highest priced properties.

The odds there are already pretty low even without the rule. A major feature is actually the filtering of initial property buying. The odds of a user rolling doubles is 1 in 6, so there is a 1 in 6 chance that you can buy two properties in a round. 1 in 36 that you can buy three. There are also odds that in those roles there is a chance you won't be able to buy properties, etc...

At the end of the day, Monopoly isn't a game of real balance though. Most of the game-play is actually social and involves various forms of trading.
Tuesday, October 26, 2004 2:28 PM by Paul

# re: Create random number filtering devices using the VectorChanceTree

Hey Justin,

Is the only caveat then on the triples only filter being that we modify the probability of picking doubles to be 1 in 11 instead of 1 in 6?
Tuesday, October 26, 2004 4:59 PM by Justin Rogers

# re: Create random number filtering devices using the VectorChanceTree

With the current filter, that is the only apparent caveat, yes. My primary focus was to demonstrate a complex filtering behavior with only basic structures.

A dedicated series filter could instead be used that throws out repeated results of a sufficient length and requeries the tree. Implemented as part of the tree this would still be transparent.

Re-arranging the probability tree is an additional option that is based on the contents of the swap filter queue. Items placed in the front of the queue, can be moved to the back. If both items in the swap filter are identical, we can run a range less the value of the item. In this way we never have to actually remove items from the tree.... I'll implement this and demonstrate it shortly as an alternative.
Tuesday, October 26, 2004 6:18 PM by Haacked

# re: I sure could use a million dollars, what about you? Transitioning from Riemann's to P vs NP...

It was an attempt at wit. I was once an aspiring mathematician (Majored in it). Even studied abroad in Hungary under a program for US students started by Pal Erdos. He died a week before we were to meet him. :( He was the greatest living mathematician at the time and it was quite sad for mathematicians everywhere that he died. Of course I was very disheartened because I was very much looking forward to meeting him.
Wednesday, October 27, 2004 4:10 PM by Paul

# re: Creating a fixed impression ad rotation device from the VectorChanceTree

This removal seems similar to operations on a heap am I correct in thinking this is a similar process?
Thursday, October 28, 2004 11:09 AM by KC

# re: Turning bitboards from potential moves into legal moves, pawn moves, and conditional rules.

I have got a pretty advanced chess program written in C# 2.0. Move generation uses rotated bitboards ( http://www.cis.uab.edu/hyatt/bitmaps.html ). I have not posted the code yet on my blog but if anyone is interested in the source, then send me an email at this address, kcdike@cs.umb.edu. It may give you some ideas. Still have to write an evaluation function and implement move ordering.
Monday, November 01, 2004 5:59 AM by TrackBack

# 在多线程中如何调用Winform

Ping Back来自：blog.csdn.net
Tuesday, November 02, 2004 9:21 AM by TrackBack

# Blogging Portals - useability and finding stuff

Wednesday, November 03, 2004 8:15 AM by Andrey Skvortsov

# re: Resizing a Form has always been a pain in the rectum...

It would be more interesting if you talk about layout practice from web development perspective(css particularly) only-most flexible engine at the moment IMHO
Wednesday, November 03, 2004 2:22 PM by Jason Olson

# re: Doing a comprehensive review on the "Beginning .NET Game Programming in VB .NET", need your feedback.

Posted my review in a blog entry:

http://geekswithblogs.net/jolson/archive/2004/11/03/14319.aspx

Wednesday, November 03, 2004 3:00 PM by Justin Rogers

# re: Resizing a Form has always been a pain in the rectum...

When creating agile or adaptive layouts, CSS is probably one of the worst engines I can think of. The ability to interpret the spec to make things work is the sign of a spec that wasn't finished. This results in different browsers supporting varying levels of compliance. I do realize most browsers work well, while IE simply has its own set of rules. There is also the problem of malformed HTML which comprises a larger precentage of the Internet than well-formed HTML.

What would make CSS better? Well, IE has taken steps in the right direction. For a real-layout you need a lyout ordering. You need to be able to lay a particular element with respect to the previous elements using a rich expression syntax that allows previous results to come into play.

The majority of CSS layout is a mixture of fixed size layout in conjunction with percentage based layout. This is the *solution* they've come up with and it results in some interesting effects at times. However, take the following problem and provide a working solution in CSS/scripting:

1. Create a splitter control with 2 panels.
2. Make the splitter draggable.

Not so hard.

1. Create a 3 panel splitter control.
2. Make the splitters draggable.
3. Make sure the top level control fills all available space and that it's panes fill the internals of the control without any overflow.

That is a bit more difficult. A robust layout engine can handle these problems, but CSS requires complex code to be able to provide a workable solution, and that solution is only marginally useful.
Saturday, November 06, 2004 7:18 AM by Wilco Bauwer

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

When you say you heard about another book, you probably mean John Gough's "Compiling for the .NET Common Language Runtime" (http://www.amazon.com/exec/obidos/tg/detail/-/0130622966/ref=wl_it_dp/102-0123623-1192973?%5Fencoding=UTF8&coliid=I146RZZ188X2DN&v=glance&colid=2RH4PEYCKQA3L).
Saturday, November 06, 2004 9:58 PM by TrackBack

# Justin discusses

Monday, November 08, 2004 8:55 PM by Wow

# re: Solving big business problems in our little toolbox application. A use case for Project Distributor.

Finding Reflector in Google is quite easy -
Just type "Lutz" or "Reflector" :)

How is your site different from GotDotNet workspaces? What problem does it solve for the tools authors? Are there really that many good tools out there that you need a peer network?
Tuesday, November 09, 2004 6:34 AM by Edward G. Nilges

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

I am sorry you didn't like the book...but...

You based the review on a superficial scan of the book, so superficial that you don't even realize that there was no CD. Instead, the book provides a link to the Apress download site, which you have not followed.

All you saw was "pretty pictures". Well thank you. But I'm afraid your review is worthless since you did not download the code, a requirement for any serious user of the book.

You decry my choice...to mention yacc and lexx and regex, but develop the compiler and parser using hand code, so as to teach how and why yacc, lexx and regex are important, and also that they can be used as specification languages and not coding languages.

Having written a yacc-type program, I could have included Build Your Own .Net Yacc, but this would demand too-high a level of technical sophistication from the entry-level reader. The book is a Getting Started for those without the considerable technical background that is needed for typical compiler tomes.

Your superficial examination told you the compiler was incomplete and you accuse me of leaving its completion as exercises for the reader. This very serious charge of professional dishonesty is untrue. The compiler as downloaded (remember, no CD, download, get it?) runs all examples in the book.

Missing are functions and subroutines. Again, they would form a complete chapter in themselves and involve explanation of stack frames. Again, this is a Getting Started and tyro's prologemena to the Dragon Book and as such functions and subroutines would be too advanced...especially given the focus on compiling business rules (did you see them?)

The niche my book occupies is the chasm between practical books on generating code for .Net and academic theory, and too my knowledge this niche is not bridged.

In fine, I find it more than uncharitable to "review" a book after a superficial perusal at Border's. I find it a serious breach of professionalism. The recent re-election of Bush may have comforted people in many fields who base book reviews, and wars, on superficial input, but many other people are tiring of this gesture.

I request that you delete this review or post an apology. I would not make this request had you in good faith even just looked at the code but you, like our President, seem to have mistaken mental images for good old Reality.

Finally, your reference to fiscal compensation manages to exhibit what seems to be a corrupted process of reviewing and to show how cheap you can be bought.

Regards


Edward G. Nilges
Tuesday, November 09, 2004 8:43 AM by Justin Rogers

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

To hit some of the fine points of your commenting. I did go back and perform more than a superficial scan of the text and also looked at all of the source code.

Books are a teaching in written form of a given subject. Your book is a teaching of some given subject with nearly no amount of valuable information in the book itself, and assert that the source code is a requirement for understanding the subject matter and gaining worth from your explanation. Let's take another book that is somewhat similar.

The Shared Source CLI book published under O'Reilly shows an excellent example of teaching even though there is a bunch of code behind the scenes. The entire Rotor code-base backs the book, and yet it manages, in only a few hundred pages to dig into and explain this source in great detail (including large amounts of code and explanation in the book and even a few pretty pictures). If you had truly wanted to bridge the gap, you might have done the same. What you have failed to realize is that if you wanted pretty pictures for the less technical users, then you'll also realize that they won't be able to easily process your source code, which goes completely unexplained. I don't care how good the comments are, less technical users don't learn in code.

In essence you've alienated the only audience your book supposedly extends to.

Your political references just adds more of a bullshit call to your comments at all. Your opinion of the political system, or mine for that matter, has nothing to do with the review.

So not only will I not delete the review. I'll assert, after spending another hour on your book and examining a drop of the source code, that even if you had solved world hunger in the code, it doesn't make the book any better.

My reference to fiscal compensation is a joke. Just like many of the comments on my blogs are jokes. Just like the book we are talking about is a joke. You get it? Jokes!
Tuesday, November 09, 2004 9:33 AM by Justin Rogers

# re: Solving big business problems in our little toolbox application. A use case for Project Distributor.

Yep, it is, if you know Lutz and/or Reflector. Let's say you don't know any of those. You don't even know you need a tool yet, but you have a friend. Your friend has a bunch of bookmarks to tool links they use and you are able to peruse their tool links to learn about new tools. It is the same principle behind blogs, where you find things by watching the blogs of users that are adept at finding things or that post useful things.

Let me hit each of your questions in turn and you can let me know if you are still interested.

How are we different from Workspaces? Workspaces is owned by Microsoft and as such you can't install and maintain your own workspace servers. I know more about this than might be apparent at first, since I helped write portions of workspaces. Beyond the locality issues and ownership, workspaces is a source control depot. We aren't looking to be a source control system, but rather a place to host binaries and/or source code for small tools. We also have some features for releasing control of projects and opening projects to the community for further development when you run out of time to support them yourself.

Problem being solved for tool authors? I think here we need to expand our definitions. We don't care about tool authors necessarily. We care about everyone that has a little tool they want to post and share, not just the big tool development guys. With that in mind, we want tools to be able to be posted in seconds, which is now possible. You can also publish source in seconds, snippets, or whatever you want really, because we are just a zip repository for the time being.

As for major tool authors, we are working with them and getting feedback about what they want. Turns out they want feedback management systems, faq support, and the ability to publish short articles and sample code. That is easy enough, and we are working on all of those items.

Major tool authors also want to host their own tools locally, and have the ability to rebrand the UI so that their tools show up in a UI that looks like their site. That is completely possible with the peering modules. You can host your own site, but have that tool aggregated in tens, hundreds, or maybe even thousands of other sites.

Are there that man good tools out there? Let's say we started just by distributing the ASP .NET and Windows Forms .NET Control Galleries... We imported all of the controls into our system (or they implemented our system) and then 50 or 60 community sites across the web peer together and they all show these tools in a UI familiar to the users that frequent those sites. Yes, we do have more than enough good tools for a peer network to work.

These aren't just tools either. They can be partial code, psuedo code, code snippets, code ports of other tools, or whatever else may be useful. I have over 17 million lines of code on my local machine that I've personally written. Making that repository of code available in some way (probably a code file or two at a time) I'd hope would be a great tool for .NET developers.
Tuesday, November 09, 2004 10:31 AM by Jack

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

Keep writing what you think.. You read a book and then told anyone who cares what you think about it..... Sounds reasonable to me
Tuesday, November 09, 2004 11:38 AM by Dave Kekish

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

Just wanted to say thanks for a honest review. I had been thinking of picking the book up. Won't be doing that now.

I've wasted enough money on books only to be disappointed.
Tuesday, November 09, 2004 2:01 PM by Eric W. Bachtal

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

It's tough to put a finger on Mr. Nilges. He's certainly a strange one. A quick look around the net shows him to be a strident supporter of his own book and his way of thinking. First there was his own review on Amazon, then this thread, which includes references to the FBI, conspiracy theories, sabbaticals, and dysfunctional America (he also takes to task the bad reviews on Amazon):

http://groups.google.com/groups?selm=f5dda427.0406060353.4c7e37ea%40posting.google.com

I haven't read his book, but have seen reviews ranging from terrible (most) to great (some). There seems to be nothing in between. For instance, here's Bill Ryan's glowing review (he likens it to Dante's Inferno):

http://msmvps.com/williamryan/archive/2004/09/15/13576.aspx

I appreciate you posting your review, and can't fathom the nerve of an author suggesting you remove it and apologize. It's just incomprehensible. I suppose that behavior alone will be sufficient to prevent me buying a copy.
Tuesday, November 09, 2004 3:02 PM by Darren Neimke

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

I must say that I rarely if ever even crack open the CD's that come with books.
Tuesday, November 09, 2004 10:29 PM by Andrew

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

Mr. Nilges, you ought to come out of your ivory tower and get a dose of that good ol' reality that you tout sometime.

And what in the world does the President of the United States have to do with a book on computer technology? A professional would keep this discussion on topic.

Thank you.
Wednesday, November 10, 2004 9:57 PM by Bill

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

I saw the Trackback and followed it over here. It's a shame things got so heated because I've gotten to know Edward recently and I think he's a great guy.

Let me explain a little why I wrote the review as I did. First and foremost, I was being honest. I wasn't looking for a book on Compiler theory per se, but I wanted to learn how to write my own language in .NET in hopes of learning more about it. Me and two of my co-workers wrote a product called .NET Refactor (www.knowdotnet.com) and it was a chore. If I'd have had this book back then, it would have been much easier to write .NET REfactor and the next version has been redone using a lot of what was in Edward's book.

I have read a LOT of .NET books and continue to - In addition I read a good amount of CS stuff but for the time being, my focus has been on .NET. Edward's book has a lot of sidebar stuff that IMHO, just begs you to start thinking about other issues. I'm not saying it was the case with this book, but I think it's entirely possible to have a book stimulate the thought process in areas that aren't within the domain of the subject matter and still qualify as a great book. This book got me thinking about a lot of different areas and caused me to rethink a lot of stuff that I previously took for granted. Personally, I think the quotes that he uses in there justify the book's price but I can certainly understand how people would disagree with me. I lent the book to two other people and for what it's worth, they shared my feelings about it.

I admire your honesty in the fact that if you don't like something you aren't afraid to say so. If I don't like a book, I simply don't write about it. You'll notice for instance that although I've reviewed well over 40 .NET books in miscellaneous venues, that I don't have one review on an OReilly book. The reason for this is stated above - but in the same vien, I understand that a Lot of people really like their stuff. One time I was somewhat critical of a .NET book (by a Publisher that I'm quite fond of) and the response was met with hostility that was an order of magnitude than anything Edward wrote. As a matter of fact, the person took to personal insults which if anything, only reinforced my comments. I basically wrote that while the book was technically correct, the author came off like a condescending jerk and didn't back up the attitude with content.
Anyway, it's just a personal thing but I respect the fact you write about both sides.

I never knew Edward before I wrote the review but since then I've spoken with him many times. If you read through my blog - you'll see that we've jousted many times over a whole variety of issues. Edward is a passionate about many things and I think that's reflected in his comments. If you read the link above on google - you'll see that that one got pretty heated. To that end though, I do know that there was a deliberate effort by a few people to bad mouth his book because of some old grudges. I think that's probably largely responsible for why he responded rather strongly to the negative criticism.

A while ago when I was in Redmond, I had the honor of hanging out with Mike Hernandez of Database Design for Mere Mortals fame. I'm probably Mike's biggest fan and he's got to be one of the coolest, most down to earth people I've ever met. So after a few beers we were discussing the response to his book - which is almost 100% Overwhelmingly positive. But he told me that it took a while to get used to writing books b/c he took some of the criticism's personally. One comment titled "Database design for Mental Midgets" was particularly harsh and bothered him - not in the sense of anger but just that someone didn't like something he put so much into. Another pretty famous author told him to get used to it, it's part of the game. He said when he came to that realization, then it got a lot easier. You 'know' that you should not take things personally, but when something takes as much time as a book does, it's hard to do for many folks, particularly the first time or two.
Thursday, November 11, 2004 5:49 AM by Edward G. Nilges

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

Hmm, you "performed more than a superficial scan, and you looked at the source code".

But you didn't read the book, did you, much less use the performance based approach I recommend.

This is obvious, for in fact the book explains the source code. For example, the recursive descent algorithm is explained in detail in Ch 8, with a sidebar on "Cain's Amulet" to explain how a state is passed among recursive parser procedures.

I think what happened is you were offended, not so much by the political liberalism, as by my tone which fails to be authoritarian.

As to the relevance of politics. I have consistently found that political conservatives have difficulty in learning to code, since coding involves a necessary deconstruction of social structures: their analysis.

For example, in debates about the reliability of high-tech election systems, conservatives make the claim that because the computer systems are, well gawrsh, computers, they will be by definition accurate.

However, we know that programmers who work for Deibold and similar firms, some with criminal records, have full access as we do to the systems at all times, merely as a condition for doing their jobs. We also know that Deibold's CEO promised Bush to deliver Ohio, and we know of several absurd election results that show that at no time were "business rules" (totalVote >= voteForBush) applied.

Of course, since I do indeed work and have worked in the "real world", at large corporations including Baxter, Household Finance, etc., I have worked side by side with political conservatives competent at their jobs.

But in your case, your clear bias caused you to post a review based on a book you haven't been sent or purchased, without reading the book.

I don't know where you get the idea that computer technology has nothing to do with politics. In my chapter 9 on a humanistic application of business rules to small scale credit, I demonstrate a way to make capitalism work for the small borrower by using modifiable and documentable business rules, in ASCII text form, as opposed to a binary set or a single number.

I have looked over the rest of your site, and clearly we will have to "agree to disagree" about politics. For one thing, to describe the Rehnquist court as not activist is just silly: they appointed the President to his first term.

But I will have to ask you to withdraw comments based on a book you haven't read, much less worked through.

Thursday, November 11, 2004 7:05 AM by Edward G. Nilges

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

Hi, Bill.

I would not have replied to a negative review written by someone who'd at a minimum read the book. This is Book Reviewing 101. You don't pretend to have an opinion until you have finished the book.

So it's not a matter of me being a passionate, sensitive, kind of guy, although I am.

To me, the judgement "not everyone can write a book" sounds like one of those terribly final, terribly "reified" (cast in concrete) form of Human Resources explanations used to justify what usually turns out, on investigation, somebody else's failure to do their homework.

The language has flavor, all its own. It can never be analyzed or subject to any kind of deconstruction because it is post facto and essentially, people "sign off" on it just to get the agonizing process, of encountering a normalized deviance (a deviance accepted as OK for social peace), over.

Personally, I have been told over the course of my 55 years, sometimes by the same person, gee, Ed, you kin write and you oughta write a book, and gee, Ed, you can't write worth jack. The message seems to be that what's most useful are my speling and gramer skilz and that I should seek a career as a ghostwriter or flack rather than attempting to say what's on me mind.

This may be because our society splits the "suits" and the geeks, and the "suits" tech and cognitive skills atrophy, rendering them absolutely reliant on the geeks to justify what is at times unjustifiable, whether it's France's behavior in the Cote d'Ivoire or America's in Iraq (how's that for Fair and Balanced?)

Meaning the suits need press releases and code but can't produce them. Meantime, many geeks, both left and right, question the entire set-up and ask why it is necessary (on the left) for so many people to work for peanuts or on the right to disallow people free choice.

Another message is social schizophrenia, as evinced on the political scene. Hell, even Bush's supporters know he's a bozo (William F. Buckley is antiwar). At the same time, they remained silent so's Kerry wouldn't get elected and open a can of worms, such as the systematic racism of our society, and the fact that gay men are lonely and would like to marry each other rather than hang out in bars. One is familiar with the Rush Limbaugh listener who professes to "hate" Rush.

All this leads me to believe that there is an element here of killing the messenger, although, to be fair, the reviewer here ain't schizophrenic about me. He dislikes me insofar as he knows me and on further acquaintance may dislike me even more. Boo hoo. I think I got over it already.

I have some "negatives" with my own book, and as I stated in the Amazon self-review, I did the best job in the time available.

There is a profound truth somewhere in that simple statement. The real artist, like Michelangelo, makes an end when he is finished, and the work dictates the time taken. But even in the time of Pope Julius and especially now, creative work, unless you fund it yourself as your own entrepreneur, is universally "factored" into chunks of undifferentiated and unskilled labor, and then budgeted with deadlines.

This is also called "reality", although guys like Marx said, "oh yeah" and "sez who", and CONSERVATIVES from G. K. Chesterton and Hillaire Belloc, to Russell Kirk, have ALSO said "sez who"...as does the average Joe when the car doesn't start, the health insurance runs out, or the Cubs lose.

The Apress constraints were quite loose as compared with a typical Death March development project and in fact I delivered far more code than most computer books deliver, code that can be used commercially to create a business rules or macro facility in a shrink wrap product. I'm not going to get a dime from that, of course. Boo, again, hoo.

Nonetheless, Apress has bills to pay and May 2004 was cast in concrete. It can be said that the deadlines kept the book simple and accessible because while this reviewer disdains it (because he's written, ahem, a calc.exe) many other programmers may find it useful.

Since I've coded a compiler generator and pieces of a regex, I would have liked very much to include chapters on both. The problems are that (1) two new chapters would have blown the planned page budget and (2) understanding both products require an acquaintance with set theory, which would demand I add ANOTHER chapter on sets and logic!

Furthermore, my tech editor, Dan Appleman, would have redlined the material because he was editing from the standpoint of the average, good programmer.

Dan is a programming Democrat. He seems from my personal acquaintance to be a younger version of Peter Neumann, the (politically rather liberal) moderator of comp.risks, although I do not know Dan's politics. In computing terms, it appears that Dan's love of explaining things is like mine because, I think, both of us know you don't "know" until you pass it on.

And as a programming Democrat, Dan is a firm believer in the Rights of Computer Languages to liberty, equality and fraternity, especially in .Net.

When I was working last year in Silicon Valley, I had a C# job and was interested in writing a C# version of my compiler, but Dan pitched a fit, both because I was late, and, I think, he wanted to show the world that high quality, compiler code could be written in grubby old VB.

And when I reflected on the struggles of students of color at DeVry to get jobs having learned primarily VB, I was in sympathy with Dan.

Programming skill in fact has very little to do with choice of languages, a choice often dictated by the boss, and the idea that it does is a confusion between the stars and the telescope (as in the aphorism of Dijkstra I quote).

I regard the foregrounding of language choice over knowledge of the applied mathematics as a crypto racist and classist venture.

As to politics. It's been said that the "default" politics of computer people is libertarianism, "let us alone", coupled with dislike of unions and welfare and support for free trade. The only view I share is a support of free trade: as to outsourcing, let it rip: I outsourced myself last year to Fiji and this year to China.

But it seems we have trouble admitting that as mere coders we make the System work in what to me is an opaque way, where, for example, people are judged by Fair-Isaacs "single number" credit scoring in place of business rules and have no appeal. Or where no audit trail exists in an election and the programmer who suggests to a boss with a criminal record that we might apply some business rules (totalVotes <= votesFor Bush) is terminated under employment "at-will".

Another thing I'd do differently is sharply cut back on the digressions in the book, and increase the focus more. Basically, I took my lecture style, which the chair of MIS programmes at DeVry admired, and put it on paper.

But on paper, it is at times just a pain.

At one point in the tech edit, I'd written in the Word document "But, I digress". Dan A commented "yeah, you digress".

With all these links in this blog to conservative sites, I realize I am behind enemy lines. But let us not speak falsely now the hour is much too late.

Criticism, especially by people who do due diligence, doesn't bother me. The reason I spoke up in this blog was an apparent failure to meet the elementary responsibility of a book reviewer. And the connection to politics is inescapable, because the Bush "mandate" means that as regards international politics, the dog ate the homework and that's jest fine.
Thursday, November 11, 2004 8:51 AM by Justin Rogers

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

I wouldn't have posted the review if I hadn't fully read your book. As mentioned in comments and in the posting, I spent over an hour reading your book. I don't need to *work-through* your material. I don't need to more *thoroughly* read your material. And I don't need to *consider* the side bars and notes. I've written compilers, done code generation, and am a huge advocate of code emit in .NET. You can read back through the blog and see that.

I even sympathize with simplistic design and enabling users that aren't familiar with the concepts. You can check out my short postings on a number of different parsing systems under .NET and a few about dynamic code generation.

All that considered, I still don't have any higher an opinion of your book. I've given you due diligence, and have spent more than 5 hours now between the actual book and the source code. I won't spend any more, nor do I need to. That is more than enough time to adequately review a book. You see, if an in-depth review takes say 15 hours, then a slightly less comprehensive review can be done in 5 hours. Odds are that in 5 hours, at least one thing would strike you as interesting and/or usable, especially if you are EXTREMELY familiar with the subject matter. I didn't get that from your book. I would have given it 15 hours and done a better review, but the book doesn't warrant it, nor does it have any information that isn't freely available on other sites. Difficult to find in a hurry maybe, but the information is there and in many cases more accessible than it is in your book.

I'm sorry brah, but no matter how hard you call me a conservative (which I'm definitely not), how much you dislike bush, or why you keep ranting about politics in general, none of that will change the review.

As for your small scale credit application, that is the basics of a data driven design. While I can respect writing custom parser technology for any config files that might contain the data, you didn't live up to the real concepts underlying your book. Primarily because data driven designs aren't about code, compiling or any such nonsense, but instead about changing the parameter values to fixed functions, or possibly even interpreted functions. At best you could call that scripting, but certainly not creating your own language and compiling and certainly not in a chapter 9 of a book that is supposed to be about compiling. Anyway, I'm done with this thought process. I have books that I consider more worth my time, IMHO, and I'm doing the community a disservice by not spending my time on those.
Thursday, November 11, 2004 7:18 PM by Anonymous Coward

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

Two expressions come to mind here:

1. The lady protesteth too much.

2. When in hole, stop digging.

Justin's review made me think "perhaps that book isn't for me". Edward's tortured political rants confirmed it.
Thursday, November 11, 2004 8:30 PM by Another anonymous coward

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

Edward, just a quick question... do you wear aluminium foil around your head so that 'they' can't use their satellites to read your mind?
Thursday, November 11, 2004 9:03 PM by Edward G. Nilges

# re: Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

Can't change the fact, "brah", that you posted a negative review after one hour's perusal and skim. Can't change the fact, "brah", that this is the only conduct for which I need an apology and retraction.

The latest review at the Amazon site, currently under Bill Ryan's review, which because of its quality is a Spotlight, is from Bob in SF. I have no problems with this review because my book did not meet Bob's needs, and he bought it and gave it a chance. I am thinking of revising my self-review to praise Bob.

I certainly don't understand your point about the "small scale" credit application. A case can be made that all humane and all doable MIS applications are "small scale"; and, the phallocentric language of programmers who are forced by the logic of their situation to make reference to unknown "needs" at an unknown, and unimplemented, "large scale" may be the problem.

Many implementers have indeed tried to implement data-driven designs as a set of fixed functions, to which the data "points". Consistently they end up with a mess because they don't realize what they're about.

When the user has a requirement demanding If..then..else, a common response in this venture is to keep the selection in code which prevents the user from controlling it, of course. Likewise for iteration.

On a number of projects, I have encountered a mulish resistance to the idea that what is being developed "really is" a computer. Most recently, engaged in cleaning up a C# and VB mess for DNA synthesis, I realized that the problem was that the original developers had developed a "data driven" design, all right...in which the selection and iteration were in the code, and the steps in SQL Server tables.

As a direct result, the client literally could not understand how to use his system. Paradoxically the original developers claimed that to provide him, not with "data" but a special-purpose language, would "confuse" the client...who had a doctorate in genetics, a subject which is far more challenging than programming.

The union card holders, the programmers, who despite their commonly anti-union sentiments, almost unconsciously keep the selection logic and iteration in code.

Chapter 9 is in fact key, for with experience in the early days, I need to show that application programmers are blocked in enterprise systems development by constructs including "oooh, compilers" that were transgressed effectively in the early days.

As to being a hip sort of dude who doesn't support Bush, or maybe voted for him but doesn't support him, or maybe voted for Kerry but was relieved, or what, objectively the gesture of pulling a book off the shelves, and "reviewing" it is the sort of nihilistic gesture that Bush makes because in a strong sense, he and his team believe that they can manufacture (in part, using computers of course) "reality" and don't really have to do their homework which de dog done ate.
Thursday, November 11, 2004 9:42 PM by Mat Noguchi

# re: The obligatory Halo 2 partial review and thumbs up.

Three years, actually.

But it's definitely nice to read good reviews :)

MSN
Thursday, November 11, 2004 9:52 PM by Justin Rogers

# re: The obligatory Halo 2 partial review and thumbs up.

How about a nifty timeline. I know based on experience that nearly every V2 starts well before V1 ships. I'd be very surprised if there weren't portions of a plot, parts of an engine, and many other things that were being produced prior the V1 going gold and ultimately shipping 3 years ago.
Friday, November 12, 2004 1:02 AM by Mat Noguchi

# re: The obligatory Halo 2 partial review and thumbs up.

Heh, we didn't even think about Halo 2 until at least a month after the Xbox shipped.

MSN
Friday, November 12, 2004 3:13 AM by Edward G. Nilges

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

You could read my response as superficially as Justin read my book and still get the essence.

This is that Justin reviewed the book after an hour's perusal and skim, and thought it had a CD, which means he wasn't qualified to review, especially if he is being paid for referrals. If he is being paid for referrals then his conduct constitutes malpractice in his chosen profession.

I hope the above is quite brief enough even for the dyslexic. The dyslexic may skip the rest if it makes their head hurt.

As to the objections based on word count. Part of the problem is that too many programmers are so poleaxed by reification that they actually think they're being smart when instead of addressing a case point by point they make reference to word count, which today can be determined by automatic means.

Logic 101, people: reference to prolixity is at best stylistic, not an argument.

Justin's further "deep" analysis consisted in going to Apress, downloading the code, and then, apparently, noticing that form1 in the qbGUI project has a lotta lines, so he's still in the reified mode of assuming that prolixity, in and of itself, is a Bad Thing (an incoherent stance because initially he felt that prolixity in prose was missing in the book, also, inconsistently, a Bad Thing).

Of course, the project name indicates the goal of the project and form1 is consistently the main form, a standard clear, in the structural sense, by means of being consistently followed. An alternative would be to name the form after the project, of course. Fairly trivial.

"Hey, Harv, are you into TRIVIA?" "I'm talkin' to ya, ain't I?" - Harvey Pekar, American Splendor

The main problem is that book "reviewers", who are apparently semipro based on the references to referral fees, don't know how to maintain critical distance while, at a minimum in this case, reading the book in its entirety, without having to be told to do so by the author...and in the case of a computer book or math book, working through most of the examples!

For illumination, take a look at my reviews on Amazon such as a review of Richard Posner's Public Intellectuals.

I think Posner is a dope and Public Intellectuals confirmed my view, because Posner assumed without argument that Public Intellectuals should be considered jest folks, and held to Wal-Mart style productivity standards, in which they'd produce a certain rate of True Ideas in defense of the Status Quo, rather than bitch and whine in cafes.

However, I read the book attentively from cover to cover, and if Posner had embedded a CD with pictures of Public Intellectuals in compromising positions, I woulda downloaded same. Only then did I feel qualified (as Amazon requires) to review the book.

I sense that today entirely too many people will read a book or even sit through a TV program only based on an initial liking for the author, producers, or viewpoint to be expressed.

Reading critically is a difficult skill, because at one and the same time you have to understand and form objections. Nonetheless, because Justin is a smart guy, I am certain he could have done so while running the code and watching the pretty pictures.

He could then have posted "I am unconvinced by Nilges' approach and the guy is a total bozo, because his compiler doesn't do proper error checking, for which one needs to use a parser generator. Furthermore, the guy needs his head examined on 'business rules' since allowing the credit scorer to CHANGE the rules is asking for her to get kickbacks from the debtor."

These are two valid objections which I believe I can address, but it's not appropriate to do so here. The point is that they are based on critical reading, something discouraged, I think, both by the conditions of many programming jobs and Fox News.

Justin, on the other hand, seems to feel that one can just sort of paw through books at Borders and make reviews based on a superficial examination.

I read a number of math and computer books but rarely review them, simply because I don't feel qualified to review, whether negatively or positively, until I've worked through most of the examples.

But apparently, when some sort of money or referral fees are involved, some people feel no such compunction.

Finally, the reference to tinfoil hats is a common enough meme which encapsulates for the most part the fear of a genuine epistemological isolation which is imposed, in the American gulag of the streets, on any number of people who fail to toe the mark, and who are damaged by the experience.

But this particular meme is dissolved, like a ghost at cock-crow, by one and only one link from the tinhat person to a community...a network. For me, this is any single reader who like Bill found the book useful.

My boss at Princeton assigned me to work with John Nash at a difficult moment in Nash's recovery precisely because the women programmers were frightened by Nash, and I played a small part in linking Nash to a community. So whenever I hear arguments couched in phrases of adolescent teasing, I reflect that they usually say more about the source, as does this self-revealing anonymous coward.

Yeah, maybe the Martians (who are as we know from Mars) are paying American right-wingers to conduct a systematic campaign of harassment against lil ole me for having the nerve to be an ace programmer while retaining left wing views, using X rays from which I need to protect myself.

Or, maybe, Saddam Husayn had WMDs and also flying monkeys in a hidden palace, and maybe the recent election was Fair and Balanced.

Or something. I suggest that if one SYSTEMATICALLY paws through the shelves at Borders while looking for books to nail, or if one counts words before reading, then of course ignorance replaces knowledge, and "everything is possible and nothing is true".

Sorry, fellas. I'm right, and Justin is wrong. I shall check this site once more for a retraction in a few days in order to avoid clutter, but post no more.

My email is spinoza1111@yahoo.com. If ya need to Get Started on a .Net project that involves parsing, and ya didn't take no Compiler Development class in school, give my book a whack. If youse is a teacher with a class of ruffians who blanch at the prospect of using the Dragon book without prep, assign my book (yeah, baby, 30 copies!!)

But if like Justin, you are having a bad hair day and that cute barista at Border's turned ya down, please engage brain before "reviewing".
Friday, November 12, 2004 7:20 AM by Anonymous Coward

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

If at first you don't succeed...

Imagine this scene:

Justin is an expert in the weewaa technology. He spends some time leafing through a book on applying weewaas to dingdongs. He decides it's rubbish and not worth buying. He tells his friends about it in the hope they won't waste their money. He reads lots of books. He's done this before. So have they. Not a controversial scenario.

Cue Monty Python for scene two:

The author of said book walks into the bookstore and tells Justin he is wrong, wrong, wrong about the book, and that it's slanderous that he should tell any of his friends it's not worth buying, because he's not qualified to make any judgement, and because Fox News Is Bad. From there he launches into an escalating series of shrill, long-winded, pretentious, ad-hominem rants, which seem motivated more by an attempt to justify his worth as a human being than to address the quoted flaws in his book. The occasional drop of fashionable philosophical nonsense serves for a while to distract everybody from asking the obvious questions: What the fuck is this psycho doing here? Did he really just say "But let us not speak falsely now the hour is much too late"??? Are we being Punked?

But let me not tarry here, forsooth. Edward: You are behaving exactly as your worst enemy would if he wished to discredit your professional reputation. Think about it.
Sunday, November 14, 2004 9:45 PM by Edward G. Nilges

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

Quick question...have you considered attacking the problem of creating a Roman Numeral parser using Backus Naur Form? Just thinking about the problem in these terms may clean up your code a bit by associating precisely the elements of the RN with "what is to be done".

I say this not to tease you (for I consider our discussion of my book over) but because I have found BNF useful in surprising areas including the parsing of international phone numbers.
Monday, November 15, 2004 2:47 AM by Justin Rogers

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

If you have some ideas in mind for the roman numeral parser, then I urge you to post them with some demonstration code detailing your line of reasoning. I tend to use my knowledge of performance parsing in .NET when I write my algorithms, such as this one, and that oftens removes from my own mind many different alternative representations for the parser.

If you post something I'll gladly compare/contrast it and even add it into the code-only article as an alternative. I've done this with numerous algorithms, including some offered for the quick integer parsing code I detailed several months back.

Don't worry about teasing me, I don't generally take things out of context and I see your suggestion above as exactly what it is, a valuable suggestion. If I get some time to investigate your suggestion, then I might just do that, or if you have some time to more thoroughly implement your suggestion, then all the better.
Monday, November 15, 2004 6:17 AM by Edward G. Nilges

# re: Quick attempt at a validating roman numeral parser... Lots of gotchas.

A quick approximation:

romanNumeral = [ thousands ] [ hundreds ] [ fifty ] [ tens ] [ five ] [ ones ]
thousands = M [ thousands ]
hundreds = C [ hundreds ]
fifty = [ decrementByTen ] L
tens = ( decrementByOne X ) | straightTens
straightTens = X [ straightTens ]
five = [ decrementByOne ] V
ones = I [ ones ]
decrementByTen = X
decrementByOne = I

The grammar is clearly ambiguous, but this shouldn't scare us because the ambiguity is resolved in a known limited context.

For example, a "preprocessor" could resolve all incidences of a I, acting as a decrement operator by appearing in other than the first position. It could then replace the decrement operator and the next symbol by a bogus symbol representing IV (four) in the case of decrementing five by one.

Once ambiguity is resolved, you start to code one procedure for each nonterminal, passing the input RN and an index.

The ambiguities don't have to be resolved by preprocessing and transcribing. Instead, a routine (get next token) looks at I to see if it is followed by other-than-I, and returns a special symbol in place of the input symbol.

I will implement a code solution and post it this weekend.

The above grammar doesn't let you decrement the C or M symbols and needs further work.

Wednesday, November 17, 2004 7:18 AM by TrackBack

# GC on small managed resource holding large unmanaged resources..

GC on small managed resource holding large unmanaged resources..
Sunday, November 21, 2004 9:06 PM by J. Ambrose Little

# re: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

What if Ed's worst enemy is the one posting here, pretending to be Ed? :-D
Tuesday, November 23, 2004 6:11 PM by Matthew Klump

# re: Interested in Artificial Intelligence? What about Wiki's? Well, now you can have both.

I'll comment, My puzzler project goes a little bit beyond mere first year course work. Although not all that impressive to create a three dimentional rubix-cube style hidden word puzzle, and then find all the hidden words with a match-to list with the same algorithm.
Wednesday, November 24, 2004 2:55 PM by Peter Thomas

# re: The obligatory Halo 2 partial review and thumbs up.

I don't care for how powerful the sword is. Physics don't apply when you wield it either. If you get the red "lock on" while you are in mid air, and you pull the trigger you will lunge straight for the target no matter what direction they are. I've jumped from one platform toward another that was too far to make, but a player was at the edge of that platform, on the downward arch of the jump I locked on and lunged back up and across to the platform and get the guy. It allows you to lunge too far too.

I'd also like some smaller maps. We used to have a great time with maps like Wizard but Halo2's smallest map is at least 4 times larger. We play 4 player split screen every day at lunch and our games used to be really fast pace because of the map size of Wizard, Prisoner, etc. Now everything is so large the games a pretty slow now. I hope some smaller maps will be released via XBL.

Good game though, good review too. Thumbs up. These are my only 2 complaints.
Wednesday, November 24, 2004 8:17 PM by TrackBack

# All about windows handles

Wednesday, December 01, 2004 6:41 AM by TrackBack

# Problems with CF2002 and Visual inheritance in a Wizard prototype

Friday, January 07, 2005 6:04 AM by TrackBack

# Handling exceptions in a worker thread

This post explains how to process the details on your windows form of an exception thrown from a worker thread. Basically I raise an event to the client application and passing the Exception object with it. The call of the event is marshalled using the clients invoke method.
Thursday, February 03, 2005 9:25 PM by TrackBack

# Thread Marshalling

Sunday, April 10, 2005 8:03 AM by TrackBack

# re:The obligatory Halo 2 partial review and thumbs up.

^_^,Pretty Good!
Sunday, April 10, 2005 8:03 AM by TrackBack

# re:Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

^_^,Pretty Good!
Sunday, April 10, 2005 8:03 AM by TrackBack

# re:A first stab at BaseN encoding with a focus on general alphabet encoding.

^_^,Pretty Good!
Sunday, April 10, 2005 8:03 AM by TrackBack

# re:Interested in Artificial Intelligence? What about Wiki's? Well, now you can have both.

^_^,Pretty Good!
Sunday, April 10, 2005 8:04 AM by TrackBack

# re:Solving big business problems in our little toolbox application. A use case for Project Distributor.

^_^,Pretty Good!
Sunday, April 10, 2005 8:04 AM by TrackBack

# re:Language parsing and compiler design doesn't have to be hard, but boy this book really sucks!

^_^,Pretty Good!
Sunday, April 10, 2005 8:04 AM by TrackBack

# re:Resizing a Form has always been a pain in the rectum...

^_^,Pretty Good!
Thursday, May 12, 2005 4:30 PM by TrackBack

# Whidbey help for multithreading woes: CheckForIllegalCrossThreadCalls

A somewhat common programming error is to directly call back onto a piece of UI when you’re on the wrong...
Saturday, May 14, 2005 10:26 AM by TrackBack

# Custom CheckDBNull function in Whidbey using Generics.

Custom CheckDBNull function in Whidbey using Generics.
Wednesday, June 15, 2005 6:13 AM by Jason Mauss

# re: I've finally settled into my new position on the Internet Explorer team...

good to see you bloggin' again.

Do you know if IE 7 is going to have ...gotcha. heh

I'm sure you'll get plenty of real questions about IE 7 in the next few days.
Tuesday, June 21, 2005 9:59 AM by Chris Slatt

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

Good to have you back! When do the posts on using Internet Explorer as a Game Engine start? ;)
Wednesday, June 22, 2005 3:57 AM by carter

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

very nice article, this will save a lot of people a lot of headaches :)

i do have one suggestion though, the images in the article are being resized by IE, leaving us with basically unreadable text. could you perhaps include a link to open them at full size in a new window? (i have sufficed with opening the image directly for now)

it would be nice if IE7 could do a "smart resize" on images the same way photoshop or paint shop pro resize images, instead or a pixel resize or bicubic resize or whatever its doing right now!
Wednesday, June 22, 2005 5:01 AM by Jon Boxall

# Required reading for anyone seriously considering AJAX

Great article - could have done with this 5 years ago...
Wednesday, June 22, 2005 5:38 PM by Andy Smith

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

Hey Justin,

Those leaks...
From my understanding from another msdn article (can't seem to find it now), any leaks caused by an element in a closure would be cleaned as soon as the user navigates to a new page. Is that correct?
Wednesday, June 22, 2005 8:27 PM by Justin Rogers

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

Image Sizes:
They should be fixed/getting fixed. They looked really good in Visio when I was making them. If they aren't fixed shortly I'll try and post alternates somewhere in addition.

Andy:
Closures leak across page navigations. The underlying problem is that there is a circular reference between an element in IE and a closure object in JScript. The only way to reclaim this memory is to recycle the process, so these shouldn't be taken lightly in production level script controls.
Thursday, June 23, 2005 9:06 AM by carter

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

the images themselves are fine when viewing them at full size (right-click -> properties on the image and get the direct URL), so you did a good job with making the images; it's just that viewing them in the article isn't that great because IE is scaling the images down due to explicit width/heights.

and since you're on the IE team, can you suggest the "smart" image scaling that i was talking about above? pixel/bicubic resizes look awful.
Thursday, June 23, 2005 9:28 AM by Luca

# re: I've finally settled into my new position on the Internet Explorer team...

good work, I hope IE7 will be the browser I always dreamt. I love the security.
Saturday, July 02, 2005 6:41 PM by Nicolás Castagnet

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

Very good article!!

Take a look at http://jgwebber.blogspot.com/2005/05/drip-ie-leak-detector.html. There is a very useful tool developed by Joel Webber to detect memory leaks.

But, I have another memory leak problem (in IE6) and I don't know how to solve it. It is very simple to reproduce. Create a page with this code:

<html>
<body >
<iframe id="testframe" src="about:blank"></iframe>
</body>
</html>

Load it in the browser and press Refresh several times (or use the function 'Blow Memory' in the tool that I mentioned above). Also happens if you define a loop that changes the page in the frame, for example:

var f = document.getElementById("testframe");
for (var i = 0; i < 10000; i++)
f.src = "about:blank"

I will appreciate a lot if someone could give me a clue to resolve this problem, or confirms me that it is a bug.

Thanks,
Nicolás Castagnet (ncastagnet@dlya.com.uy)
Tuesday, July 05, 2005 8:36 PM by eeevans

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

Hey, did the source for the Terrarium ever get released?
Thursday, July 14, 2005 6:05 PM by Anonymous

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

In the long run, the best way to remove these leaks would be a fully managed (and thus, garbage-collected) web browser implementation. Since it would have other advantages (improved security, more flexibility to implement some of the weirdest ideas out of the W3C), it hope the IE team will investigate this possibility.
Maybe the development model for the IPv6 stack (first a project at MS Research, then integration into the mainstream product) could serve again (hopefully faster)?
Monday, July 18, 2005 7:38 PM by Martin

# re: New posting on MSDN about script leak patterns and how to fix them for anyone that builds dynamic web apps.

Very nice and well written article. Congrats.

But as IE7 is moving towards standards complience, it would be better not to use the long-time-ago depreciated "language" attribut of the script tag.

So <script type="text/javascript"> and not <script language="JScript">

Bye,
Martin
Saturday, June 17, 2006 2:09 PM by Wim

# re: What are your thoughts on the new ArraySegment structure in Whidbey?

I've written a generic DelimitedArray class because of things lacking in ArraySegment structure.

See: http://weblogs.asp.net/wim/archive/2006/06/14/ArraySegment-Structure---what-were-they-thinking_3F00_.aspx

My DelimitedArray class doesn't allow any access to the underlying array once you've instantiated the DelimitedArray.
Friday, July 21, 2006 1:04 AM by Greg Young [MVP]

# Performance : String Reverse

I can never stay away from a performance challenge. While the others focus on the loop let's focus somewhere else!

Thursday, July 27, 2006 6:27 PM by Patrick Steele's .NET Blog

# FTP classes in .NET 2.0

I had a chance to play around with the FTP classes in .NET 2.0 (FtpWebRequest and FtpWebResponse). My

Thursday, August 10, 2006 11:14 AM by Better Living through Software » Blog Archive » Loopy Decisions

# Better Living through Software &raquo; Blog Archive &raquo; Loopy Decisions

PingBack from http://www.netcrucible.com/blog/2004/04/21/loopy-decisions/

Thursday, August 10, 2006 11:14 AM by Better Living through Software » Blog Archive » Loopy Decisions

# Better Living through Software &raquo; Blog Archive &raquo; Loopy Decisions

PingBack from http://www.netcrucible.com/blog/2004/04/21/loopy-decisions-2/

Tuesday, August 15, 2006 9:29 PM by discarded lies - hyperlinkopotamus

# Crazy BDS-suffering author of a (not too good, apparently) book on writing compilers harasses a reviewer

Crazy BDS-suffering author of a (not too good, apparently) book on writing compilers harasses a reviewer

Thursday, October 05, 2006 11:39 AM by Duncan Mackenzie .Net

# A love little samples like this...

Thursday, November 09, 2006 2:09 AM by Community Blogs

# Code Puzzle #1 - Solution

If you haven't read the puzzle, you can read more about it on my previous post. The questions: (Answer

Thursday, January 11, 2007 4:06 PM by Browser Security News » Implied tags in the IE HTML parser and how that can be interesting.

# Browser Security News &raquo; Implied tags in the IE HTML parser and how that can be interesting.

PingBack from http://www.ie7security.net/2007/01/11/implied-tags-in-the-ie-html-parser-and-how-that-can-be-interesting/

Thursday, January 11, 2007 4:06 PM by Browser Security News » I’ve finally settled into my new position on the Internet Explorer team…

# Browser Security News &raquo; I&#8217;ve finally settled into my new position on the Internet Explorer team&#8230;

PingBack from http://www.ie7security.net/2007/01/11/ive-finally-settled-into-my-new-position-on-the-internet-explorer-team/

Sunday, January 21, 2007 6:58 PM by Browser Security News » Implied tags in the IE HTML parser and how that can be interesting.

# Browser Security News &raquo; Implied tags in the IE HTML parser and how that can be interesting.

PingBack from http://www.ie7security.net/2007/01/21/implied-tags-in-the-ie-html-parser-and-how-that-can-be-interesting-2/

Sunday, January 21, 2007 6:58 PM by Browser Security News » Important changes to the BASE element for IE 7

# Browser Security News &raquo; Important changes to the BASE element for IE 7

PingBack from http://www.ie7security.net/2007/01/21/important-changes-to-the-base-element-for-ie-7-2/

Sunday, January 21, 2007 6:58 PM by Browser Security News » I’ve finally settled into my new position on the Internet Explorer team…

# Browser Security News &raquo; I&#8217;ve finally settled into my new position on the Internet Explorer team&#8230;

PingBack from http://www.ie7security.net/2007/01/21/ive-finally-settled-into-my-new-position-on-the-internet-explorer-team-2/

Sunday, January 21, 2007 7:25 PM by Browser Security News » Implied tags in the IE HTML parser and how that can be interesting.

# Browser Security News &raquo; Implied tags in the IE HTML parser and how that can be interesting.

PingBack from http://www.ie7security.net/2007/01/21/implied-tags-in-the-ie-html-parser-and-how-that-can-be-interesting-3/

Sunday, January 21, 2007 7:25 PM by Browser Security News » Important changes to the BASE element for IE 7

# Browser Security News &raquo; Important changes to the BASE element for IE 7

PingBack from http://www.ie7security.net/2007/01/21/important-changes-to-the-base-element-for-ie-7-3/

Sunday, January 21, 2007 7:25 PM by Browser Security News » I’ve finally settled into my new position on the Internet Explorer team…

# Browser Security News &raquo; I&#8217;ve finally settled into my new position on the Internet Explorer team&#8230;

PingBack from http://www.ie7security.net/2007/01/21/ive-finally-settled-into-my-new-position-on-the-internet-explorer-team-3/

Wednesday, May 16, 2007 7:00 PM by Tom

# re: FunHi.com, a dating service, a new game, a bit over the top?

If you are a researcher, looking for a new and innovative dayting web site, you are not looking hard enough, but you can be forgiven, a new revolutionary thing is not easy to find...if you leave a message to me I will lead you to that web site...by the end of the year that web site will be fully activated.

Friday, May 18, 2007 10:17 PM by Georghios

# re: Visualizations help, so I've uploaded a BitBoard viewer for the chess algorithms to Project Distributor

Nice...

Friday, May 18, 2007 11:47 PM by Vassilios

# re: Visualizations help, so I've uploaded a BitBoard viewer for the chess algorithms to Project Distributor

Cool!

Saturday, May 19, 2007 3:53 PM by Ivan

# re: Visualizations help, so I've uploaded a BitBoard viewer for the chess algorithms to Project Distributor

Nice

Sunday, May 20, 2007 12:37 PM by Vasilis

# re: Visualizations help, so I've uploaded a BitBoard viewer for the chess algorithms to Project Distributor

Nice

Monday, May 21, 2007 2:29 PM by Leontios

# re: Visualizations help, so I've uploaded a BitBoard viewer for the chess algorithms to Project Distributor

Cool.

Tuesday, May 22, 2007 10:51 PM by Kristoffer penisfeldt

# re: Improving the CSS 2.1 strict parser for IE 7

Jeg kan ikke få IE7 til at virke som den skal, diverse funktioner som fx zoom virker ikke optimalt.

Helt ærligt, hvad er det her for noget bokse kode?, det kunne envær jo have lavet... helt ærligt MS, kom nu op på de små pusselanger.

Wednesday, May 23, 2007 12:02 AM by Giorgos

# re: Visualizations help, so I've uploaded a BitBoard viewer for the chess algorithms to Project Distributor

Nice...

Wednesday, May 23, 2007 2:36 PM by Brian

# re: Performance: Different methods for testing string input for numeric values...

I would be curious to know what the performance would look like if all input was valid. (i.e. int.Parse was valid at all times)

Wednesday, May 23, 2007 9:51 PM by MaryJames

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

Hi  all

How I can change avatar in this forum?

Friday, May 25, 2007 1:34 PM by Ryan’s Tech Blog » EventHandlerList, key equality, and auto-boxing in C#

# Ryan&#8217;s Tech Blog &raquo; EventHandlerList, key equality, and auto-boxing in C#

Pingback from  Ryan&#8217;s Tech Blog &raquo; EventHandlerList, key equality, and auto-boxing in C#

Monday, June 04, 2007 3:24 AM by Agnel CJ Kurian

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Just curious... Supposimg I need to invoke several methods with no parameters, would it be more efficient to use the old-style SendMessage or PostMessage via PInvoke? I am not discussing safety here only efficiency. Any thoughts?

Tuesday, June 05, 2007 12:04 AM