Published by

Comments

Monday, January 19, 2004 3:58 AM by Frans Bouma

# re: Towards a more functional TreeView

The real problem is under the hood. The .NET treeview is based on the asynchronous message model of win32. This means that when you do something synchronously in .NET, it is executed under the hood by sending win32 messages. This can result in serious issues.

.NET needs a synchronous, native gui library which is synchronous all the way till GDI+. The decision to base the gui glyphs on win32 stuff is perhaps based on teh fact that it would be 'faster' and 'easier to implement', but it also is the basis for most issues with .NET gui controls.
Monday, January 19, 2004 6:59 AM by anon

# re: Towards a more functional TreeView


Many great things about the windows forms treeview? It doesn't even support multiple selection natively. I think at least half of applications need that.


"Think about that for a minute, because the UI implications and different types of views start to become limitless with"

I don't know if this encourages consistent look and feel UIs, and how much this will hurt average people that are now used to the tree ctrl in windows explorer. In the corporate world, any change in the look and feel and UI costs millions in retraining. I would be willing to know how much corporate IS are spending to get people to upgrade to Windows XP, only for the required retraining. Interesting.



Monday, January 19, 2004 12:55 PM by TrackBack

# Basically, how basic can a Wizard in Windows Forms be?

Monday, January 19, 2004 1:20 PM by Raymond Chen

# re: Towards a more functional TreeView

If a single treenode could shared among multiple treeviews, what would it return for the Parent property or IsExpanded or any other property that can be different in the two treeviews.
Monday, January 19, 2004 8:01 PM by Justin Rogers

# re: Towards a more functional TreeView

I realize there are some architectural issues with my original comments, some things that need or should have been thought through more thoroughly. However, the base points shouldn't be lost. For instance:

While a basic TreeNode has properties that make it functional only when located in a single tree, there are things that could have been done to make multiple view synchronization easier. The current TreeNode can't even have an overriden Text property to allow treenodes to notify a container or factory that it's value has changed and that synchronization needs to occur.

I labeled the article, Towards a more functional TreeView for a reason. I think work can be done to make it better, and I've shown in at least one case exactly what can be done. I'll show the same in another case relatively soon, because the concept of a TreeNode factory that allows you to make *connected* copies that automatically synchronize with one another actually works. And it isn't hard to do at all.

And yes, the Parent property and IsExpanded properties between the various views might actually be different, but the Text property and any properties you add that should be synchronized could easily be.
Tuesday, January 20, 2004 4:49 AM by Darren Neimke

# re: Using the Security System for Defensive Coding

Thanks for the heads-up Justin :)
Tuesday, January 20, 2004 4:18 PM by DavidM

# re: Using Weblogs @ ASP .NET and begin friendly to the printer...

There is another problem - which I have flagged a number of times - if I view the weblogs homepage on my 1024 x 768 laptop - the navbar on the left covers half the posts - which is complete pants
Tuesday, January 20, 2004 4:39 PM by David Cumps

# re: Using Weblogs @ ASP .NET and begin friendly to the printer...

I have the same thing with the layout where a navbar is on the right, and it covers half of the text as well (i believe it was on a .Text site as well)
Wednesday, January 21, 2004 2:18 AM by Vance

# re: Using Weblogs @ ASP .NET and begin friendly to the printer...

That's one of the drawbacks to using div tags and css for positioning. Sure it makes it easier to completely change the look just by changing the stylesheet, but it tends to be somewhat quirky. Especially if you shrink the browser window and watch how things start overlapping.
Wednesday, January 21, 2004 8:18 AM by Rogelio Morrell

# re: Plug-in Framework (Part1): Marking Types for Consumption

Great article!
Wednesday, January 21, 2004 8:51 AM by Scott

# re: .Text offers excellent article hosting, but where is the RSS?

Sorry for the confusion. (agg view column)

The whole point of articles is they are not syndicated.

-Scott
Wednesday, January 21, 2004 9:14 AM by TrackBack

# First article in a series on plug-in frameworks in .NET

Wednesday, January 21, 2004 2:03 PM by chadb

# re: Plug-in Framework (Part1): Marking Types for Consumption

That's good stuff...
Friday, January 23, 2004 5:53 AM by Illumineo

# re: Revisiting the .NET Terrarium

Does it mean I'll be able to write my own agent system with the help of a Terrarium SDK, or something alike?

Friday, January 23, 2004 9:07 PM by Justin Rogers

# re: Revisiting the .NET Terrarium

Once the source code is released, you should be able to do some work and get your own agent system running.

You can also watch my series on Plug-Ins since I'll be discussing a generic framework for getting agents to run safely.
Saturday, January 24, 2004 3:22 PM by Chris McGuirk

# re: Game Engine Design Principles even apply to the most basic of games (or applications)...

If you want an example that is just a tad bit bigger than that, you should check out Axiom http://axiomengine.sf.net, which is quickly approaching a full blown .Net graphics/game engine.
Monday, January 26, 2004 9:34 PM by Matt Berther

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

Justin: I'm with you on this. I cant remember how many times I've seen sample code in the MSDN library where the samples don't even compile.

When I start looking for how to do something, I look towards Google rather than the MSDN, because I tend to find a lot better samples on the web.

As a last resort, Ill use an MSDN sample, but only as that. A sample. I wont cut and paste the code into my project as is.

It's really too bad, however, it has helped me to write better API documentation. :=)
Monday, January 26, 2004 9:34 PM by Edgar Sánchez

# They definetively shouldn't

Many programmers just copy and paste Microsoft samples, if the examples do something silly as using the sa SQL Server user, the programmer will feel authorized to do the same. So please go the "best practices" way.
Tuesday, January 27, 2004 3:00 AM by Scott

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

Yeah, why should Wrox books have a monopoly on errata. I think MSDN should include MORE bad code.

Now that the MVP's are allowed to write KB articles willy nilly it's just going to get worse.

Tuesday, January 27, 2004 4:28 AM by Pavel Lebedinsky

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

> 2. Not checking for file existence before creating the FileStreams.

This is actually the right thing to do. FileStream is documented to throw FileNotFoundException if the file does not exist, and this is exactly what a function like FileCompare should do anyway.

You could call File.Exists() but what would you do if it returns false? Throw FileNotFoundException? Then what about the case where the file exists but you do not have access to it? Or if the file is on a remote share and there's a network problem?

File.Exists is rarely useful in library code like this FileCompare function. It lumps all possible failures into a single bool value and it is prone to race conditions.
Tuesday, January 27, 2004 4:29 AM by Simon

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

The KB above is a prime example, the code they show is to improve the use of getting XML from an ADO stream. However they use a classic perf issue of concatenating strings.
This code can be improved greatly using an array, but they don't want to change the KB.

So don't do as they say or as they do!
Tuesday, January 27, 2004 9:05 AM by Paul Menefee

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

I've got to defend MS here, a bit. The point of the KB it demonstrate a particular code example. We harass them all the time about being verbose and for once they cut to the chase. And just tell us what we need to know. If I'm looking for a quick solution to a particular problem I don't need to see it wrapped in a bunch of production ready code. You know, and after giving this a little more thought perhaps that's their point. If they handed programmers half-baked code that forces the programmer to take ownership and wield the code ore attuned to their application.
Tuesday, January 27, 2004 9:20 AM by Derick Bailey

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

I agree with Paul Menefee.

When I'm looking up code examples in a KB article or on another website, I don't want to see it all wrapped up in best practices... I want to know how it works and why. I don't care that they missed catching a filestream exception for a null string - I know to take care of that.

If every peice of sample code was obfuscated in production ready best practices code, noone would ever learn anything because all they would have to do is cut & paste from a hundred examples and press "play" in their IDE.

I think this is more a question of properly training developers. If your coworkers or employees are dumb enough to cut & paste sample code and expect it to be production ready, maybe you owe it to them to spend some time educating them on best practices and the difference between samples / production code.
Wednesday, January 28, 2004 7:42 AM by Raymond Chen

# re: Towards a more functional TreeView

It's a question of pay-for-play. Do you want to complicate the base treeview with these features (that probably few people will actually use)? You can do a good chunk of this yourself - split your treenode into two objects, a treenode and a treenodedata. create multiple treenodes that share the same treenodedata.
Friday, January 30, 2004 5:55 PM by Justin Rogers

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

Everyone focused on my assertion that file checking should exist, but that wasn't the biggest problem with the code. The largest problem is that their code paths left an open file stream if the first file opened but the second file didn't. I was pointing out ALL of the issues with the sample to demonstrate how many they were, and by pointing out that they left open file streams I also pointed out an extreme example. I honestly think you should thoroughly read the entire message before making comments on only part of it.

As it stands I'm just guessing everyone that defended MSDN here agrees that resources like file streams don't really need to be closed explicitly. And the fact that if you called this method twice before the GC kicked in you'd get an access violation on the file in question because it is already open.
Sunday, February 01, 2004 8:11 AM by TrackBack

# [Article] Game Engine Design: A series of abstractions for better engines.

Tuesday, February 03, 2004 5:29 AM by TrackBack

# Multi-Threaded applications and Abort, careful not to kill your statics...

Tuesday, February 03, 2004 8:00 PM by Brian Grunkemeyer

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

Aborting your threads is one of the things that shouldn't be used by the vast majority of applications. As we started into our reliability discussions for SQL Server, we realized that a thread abort can happen basically between any two machine instructions in managed code. This means you can't guarantee you won't leak handles, and you can't guarantee that you won't corrupt static state. You could try writing finally blocks for the second case, but then we had the possibility of a host injecting a rude thread abort on your thread while running the finally, because the finally was taking too long.

(We invented SafeHandle to fix the resource leakage case, which required two additional pieces of infrastructure - critical finalizers and "must run" code, though we've replaced the second with something way better, mostly because must run didn't mean it must run at all. Ask about CER's when our beta is released, or better yet, don't.) I also had some thoughts on the static class constructor issue, but we opted not to fix them in this version due to resource constraints.

In short, if you get an asynchronous exception (ThreadAbortException, OutOfMemoryException, or StackOverflowException) anywhere within an appdomain, our recommendation in Whidbey is unload the entire appdomain. In the absense of some higher level application protocol where the caller and callee have perfect knowledge of one another and perhaps do some synchronization, you absolutely must pessimistically assume your appdomain is no longer usable.

You can read Chris Brumme's blog for some of his thoughts on Thread.Abort, but I have already given you the extraordinarily scary parts of the argument.

http://blogs.gotdotnet.com/cbrumme/PermaLink.aspx/dac5ba4a-f0c8-42bb-a5cf-097efb25d1a9

If you want to screw around with ResetAbort, well, good luck. I won't fix any bugs in the BCL w.r.t. ResetAbort any time soon. For example, if you get a Thread.Abort while removing an item from the middle of an ArrayList in just the right spot, you'll remove the LAST element from the ArrayList. After looking at this problem in great detail, we realized thread abort makes reliable code nearly impossible to write, if not impossible. You need our new Whidbey features to make this problem tractable.

Attempting to build anything reliable on top of Thread.Abort in anything pre-Whidbey (short of using process recycling) is crazy, and in Whidbey it will be difficult, but possible.
Tuesday, February 03, 2004 9:14 PM by Brian Grunkemeyer

# re: Static Constructors and Plug-In Security...

1) System.CompilerServices.RuntimeHelpers.RunClassConstructor(RuntimeTypeHandle). It will guarantee that the class constructor has been called, or it will throw some sort of exception.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemRuntimeCompilerServicesRuntimeHelpersClassRunClassConstructorTopic.asp

2) No. See another comment of mine on your blog about why you don't want to use Thread.Abort.
Tuesday, February 03, 2004 9:14 PM by Justin Rogers

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

So the only reliable way to run arbitrary code that may have need of being terminated before it is done, is to make liberal use of AppDomains? I realize there is a good deal of work going into cross AppDomain marshalling so the speed issues might be easy enough to overcome at this point.

We overcame many of these issues in Terrarium by hacking around a lot of potential problems and ensuring the code didn't have specific constructs. These included static fields, static methods, any static access whatsoever, making sure code has empty constructors, and preventing finally blocks. The concept being that since this was arbitrary code, it wouldn't need access to these advanced features. Unfortunately, there is no method to do this without an IL grep. There are no run-time features or enhanced security features to apply to code being loaded into an AppDomain that says a) this code can't access statics or b) this code can't use finally blocks.

Now that AppDomain's might turn out to be the only method for reliable secure plug-in usage, I'll go down that path.
Tuesday, February 03, 2004 9:18 PM by Brian Grunkemeyer

# re: Finalizers and the nasty stuff plug-ins can do if they want to...

There isn't much you can do to stop this. GC.ReRegisterForFinale does not have any security permissions on it, not even any host protection permission in our Whidbey builds.

One thing you could do is write your own "IL verifier", where you only load code that meets your programming model constraints. IE, disallow all static variables, or reject any classes that provide a finalizer (noting that Object of course has a finalizer - just test this rule on some conforming types).
Tuesday, February 03, 2004 9:30 PM by Brian Grunkemeyer

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

Our reliability story for managed stored procedures in SQL Server is hinged on being able to cleanly recycle an appdomain if any of the threads (or managed threads running on fibers within that AD) go awry. With Whidbey features like SafeHandle and some exceedingly pervasive out of memory cleanup, we can ensure we don't leak OS resources in this case. If it's good enough for SQL Server, it's probably good enough for you.

There are some guidelines for writing reliable managed code in an appdomain-recycling model like this. I'm not sure we can make them public at this point, but keep an eye out for them in our Whidbey beta release. The most notable thing you should know is lock(typeof(Foo)) is bad, for all Foo. Don't do it. An upcoming FxCop release should have some reliability rules as well that point out some of this information.

BTW, SQL will also use some kind of IL verifier to enforce programming model constraints on stored procedures. Also, check out host protection permission. If you're willing to go all-out and host the CLR in-process using our complex hosting API's, there are ways you can add link demands that not even fully trusted code will be able to satisfy. Unfortunately for you, the HostProtectionAttribute can only be applied to methods like Environment.Exit and Monitor.Enter, and is not useful for problems like "does my class have a .cctor?" Hence, a custom IL verifier.
Tuesday, February 03, 2004 9:57 PM by Brian Grunkemeyer

# re: Multi-Threaded applications and Abort, careful not to kill your statics...

BTW, here's Chris's explanation of our reliability story, complete with a disclaimer against being prematurely slash-dotted. Read this if you want a pretty in-depth understanding of where we are on reliability, and my comments above are in essence added material. If you want more, come work on the CLR team.

http://blogs.msdn.com/cbrumme/archive/2003/06/23/51482.aspx
Saturday, February 07, 2004 1:42 AM by TrackBack

# Preamble to the Plug-In Framework series of articles...

Saturday, February 07, 2004 4:53 AM by Sami Vaaraniemi

# re: Plug-in Framework (Part1): Marking Types for Consumption

Good article!

There is another fairly common approach to plugin frameworks that I think deserves to be mentioned. You can use a sort of a catalogue service (or a configuration file) that returns a list of assembly locations and types, for example: "http://company1/plugins/plugin1.dll, Company1.Plugin1". This bypasses the need for marking types because once the loader has this list it already knows which types to load.

Doing this securely is another story though - you would probably want to require all remotely loaded plugins to be signed.
Saturday, February 07, 2004 7:48 AM by Stefán Jökull

# re: Allegiance Source Code vs Quake II, you decide...

I'd definately go for Quake II. It's a lot more popular and is still played heavily. Being able to do some improvements as well would be nice to show how easy it is using C#.

By the way, how have you been converting it. Have you been doing a straight conversion or have you been making it more object oriented then originally?
Saturday, February 07, 2004 9:28 AM by OmegaSupreme

# re: Allegiance Source Code vs Quake II, you decide...

I vote quake because its so cool. If people see a classic fps in c# they're gonna be impressed.
Saturday, February 07, 2004 11:37 AM by Dan Bright

# re: Allegiance Source Code vs Quake II, you decide...

I vote Q2 also. You can never go wrong with an old standard like that.

Saturday, February 07, 2004 1:36 PM by Shannon J Hager

# re: Allegiance Source Code vs Quake II, you decide...

I would say Allegiance for all the reasons above. "Classic"? "Old standard"? That's going to impress someone? I've seen the old standards: tetris, pacman, etc, ported to C#. If you want to show the power of C# then show a great modern game, not an old standard because it is easier. Going the easy route makes me think that C# can't handle a real modern game.
Saturday, February 07, 2004 10:43 PM by Justin Rogers

# re: Allegiance Source Code vs Quake II, you decide...

To answer everyone's questions:

This is not a straight conversion, this is an object oriented rearchitecture of the original game. There are quite a few areas were it makes sense and some where it has appeared pretty stupid to do this, but hopefully the end result will be a better demonstration of .NET's power than a simple port.

I love the fact that it is an FPS. Would be the first fully managed FPS? Quake .NET?

Shannon makes some good points too though. The Allegiance codebase is much larger and more complex than the Q2 codebase. The time to conversion would be a bit longer, but the Allegiance code makes use of more recent techniques that more properly translate into fully object oriented code than does the Q2 code.

The biggest addition to the Q2 code will be an MDX for DX9 display filter. Rather than using the crappy old OGL or Software Rasterizers we'll go straight out to MDX. Should provide a huge frame-rate increase (over 250 FPS)
Sunday, February 08, 2004 7:03 AM by Nicko

# re: Allegiance Source Code vs Quake II, you decide...

Vertigo Software have already ported Quake II to the .net runtime using Managed C++

See http://www.vertigosoftware.com/Quake2.htm

This was linked from the msdn.microsoft.com home page for a while.

Looks like you are porting to C# which is a bit different, but I doubt that if you just port Quake II it will have that much of an impact, so I suggest that you port the Allegiance engine.
Sunday, February 08, 2004 9:11 PM by Justin Rogers

# re: Allegiance Source Code vs Quake II, you decide...

Vertigo software's port of the Quake II engine is kind of a joke. Quake II can be recompiled onto the .NET Framework using managed code within a few hours, I know, because I've done that already.

The upgrades they did are also a joke using overlaid forms.

This is a true port, with emphasis on redesigning the engine to work at maximum potential under the managed run-time. It also emphasizes a Managed DirectX rendering layer, which I pointed out in previous comments. I don't see how a C# application running at twice the framerate of the C++ version of the application is not much of an impact. I also don't see how the first FPS running fully managed using C# isn't much of an impact.
Sunday, February 08, 2004 9:13 PM by Shannon J Hager

# re: Allegiance Source Code vs Quake II, you decide...

Either way, Justin, keep us posted.
Monday, February 09, 2004 1:53 AM by TrackBack

# Plug-In Framework (Part 1), Update covering configuration based enumeration

Monday, February 09, 2004 5:02 AM by stefan demetz

# re: Plug-in Framework (Part1): Marking Types for Consumption

Excellent
Monday, February 09, 2004 9:23 PM by Rick Ross

# re: Extending the security system with new permissions...

>Do CAS permissions have to be located in strongly named assemblies?

Yes. Any security classes used to extend CAS must be configured as a fully trusted assembly.
Tuesday, February 10, 2004 4:34 PM by Brian Grunkemeyer

# re: Preamble to the Plug-In Framework series of articles...

Look at the CLR's host protection permission in Whidbey. You'll see it labeled as "HPA's" often, since you can only apply this permission by using the HostPermissionAttribute. This allows the host to do two things:

1) Disable user code from calling certain API's for programming model reasons
2) Disable even fully trusted code (like System.dll) from calling certain API's for reliability reasons

Unfortunately, a method or property must be explicitly marked with an HPA for this to take effect. So you can't make an arbitrary set of programming model restrictions, but only choose whether to restrict access along some of the axes we've provided (such as self-affecting threading vs. external threading, synchronization, UI, etc.) Hopefully that's somewhat interesting.
Thursday, February 12, 2004 3:34 AM by Stephane Rodriguez

# re: On the new Longhorn SDK, a designer asks, "Does size matter"...


Reindexing MSDN content usually takes several minutes. Are you ready to pay for this every other week?
Thursday, February 12, 2004 5:21 AM by Justin Rogers

# re: On the new Longhorn SDK, a designer asks, "Does size matter"...

I'm not suggesting the re-indexing of MSDN content. I'm suggesting that the very latest index files be present on my machine based on an automatic from-web update straight from Microsoft. Worried about long downloads, then use a diff merge instead of downloading the entire thing each time.

The indexes will only change when new content is added or old content is modified. I don't think the churn will be enough that new index downloads would be an issue.

Don't use the existing system as an example, it isn't nearly the system that would catch my attention for both performance and extensibility features.
Thursday, February 12, 2004 6:06 PM by stefan demetz

# re: Using the Security System for Defensive Coding

http://dotnetjunkies.com/WebLog/stefandemetz/archive/2004/02/06/6748.aspx
Friday, February 13, 2004 10:03 PM by TrackBack

# PermitOnly is excellent for security, but can be a bane to programming..., enter PermissionSet

Friday, February 13, 2004 11:39 PM by Shannon J Hager

# re: Movie recommendations, what in the heck are they thinking...

Do you want to help make a better recommendation system? I've had one rolling around in my head for a few weeks, if you're interested in hearing about it, shoot me an email via the contact form on my blog.
Saturday, February 14, 2004 12:25 AM by Jason Mauss

# re: Movie recommendations, what in the heck are they thinking...

I've "rated" like over 300 movies on NetFlix, thinking it would produce better reccomendations for me and it still doesn't even come close. I've resorted to clicking on the actors names on the movie detail pages to get a list of films for an actor, then scanning those for the actor's latest films.
Saturday, February 14, 2004 3:51 PM by John

# re: Movie recommendations, what in the heck are they thinking...

Certainly it's not ideal. Over time it does start to help. I've noticed that the recommendations come in different places by different criteria, and ultimately you start feeling, "I've seen all this!" but give it a few weeks and more recommendations and suddenly you find listing you never saw before. Or maybe Netflix expanded... I just know it's reasonably effective, though not ideal.
Sunday, February 15, 2004 5:49 AM by TrackBack

# MatchSparse? An implementation detail.

Sunday, February 15, 2004 5:53 AM by TrackBack

# MatchSparse? An implementation detail.

Tuesday, February 17, 2004 4:06 AM by stefan demetz

# re: Needed a Crc32 class for some code, figured I'd share with the rest of you.

thanks
Tuesday, February 17, 2004 6:49 AM by Darren Neimke

# re: BinaryFormatter can hang within modified stream?

Justin... did you find a suitable hack, err, *workaround* :-) for this?
Tuesday, February 17, 2004 10:24 AM by bilbo

# re: Using the FilePlanet.com model with regards to Web Services and availability...

perhaps look at existing web service models for reference. For instance, eBay's web service has different payment subscription levels. Google limits the number of times you can query the service, etc. I'm having the same problems figuring out how I want to offer my web service. Free to the public hurts my pocket book!
Wednesday, February 18, 2004 3:48 AM by Justin Rogers

# re: BinaryFormatter can hang within modified stream?

I never did work through to the root cause, reason being, I don't have access to the code for the BinaryFormatter itself to figure out why changing the length of a type name in the stream would cause such a hang.

However, examining the types involved, it would take a good deal of inspection through IL in order to get to the root of the problem. The BinaryFormatter appears to be one of the more complex pieces of code present in the framework.

Maybe I'll head on over to the Rotor source in the next week and use that to discover the root issue.
Wednesday, February 18, 2004 5:50 AM by Ricky Dhatt

# re: Causal Gamers, Women over 40, (aka PopCap games and MS Gaming Zone ;-)

Your "unhackable" comment caught my eye. My girlfriend and her friends all play The Sims. Now they all use the cheat to get unlimited cash so they can build the home exactly the way they want. This drives me nuts, not enduring the "challenge" of the game; but she's perfectly content with it.
Wednesday, February 18, 2004 7:21 PM by Jason Olson

# re: Allegiance Source Code vs Quake II, you decide...

Hey Justin, I would vote for Quake 2.
Wednesday, February 18, 2004 8:43 PM by Justin Rogers

# re: Allegiance Source Code vs Quake II, you decide...

Yep, I'm still working on the Quake II source code. Most of the process is done. Rewriting the graphics engine right now is the largest chore because of the GL processes that don't translate well into DirectX 9. I'm also taking the time to improve the code-base using modern techniques and batching.

Stay tuned, because this is going to be one hell of a sample.

(latest development: Adding the ability to import HL models ;-)
Thursday, February 19, 2004 1:10 AM by Darren Neimke

# re: BinaryFormatter can hang within modified stream?

Cool, thanks Justin. I'll check back here from time-to-time as I remember to but, if you do find out anything interesting inside the next 6 months (say), and you remember, can you give me a ping?

Cheers :-)
Thursday, February 19, 2004 5:26 AM by Darren Neimke

# re: BinaryFormatter can hang within modified stream?

got it! thanks mate :-)
Thursday, February 19, 2004 5:32 AM by Frans Bouma

# re: Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

Aren't string in .NET Unicoded?
Thursday, February 19, 2004 5:33 AM by Justin Rogers

# re: Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

They appear to be UTF8 encoded as per the default BinaryWriter would do.
Thursday, February 19, 2004 8:12 AM by TrackBack

# Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

Thursday, February 19, 2004 8:31 AM by TrackBack

# Follow up to corrupted binary stream in BinaryFormatter causing OOM exception...

Thursday, February 19, 2004 8:41 AM by Ken Cox [MVP - ASP.NET]

# re: Pointing out the Control Gallery's...

Hi Justin,

Good timing! I've been doing some housekeeping in the Control Gallery this week and thinking that it should be offering new items and categories as RSS feeds. That would be a valuable addition. (There are also some small formatting bugs that need swatting.)

The problem is that we need someone who knows the existing code (hint, hint) and who could implement these few changes. (Okay, I'm lame in C#.)

The best course would be to contact Rob Howard at Microsoft about getting private access to the current code to make the changes.

Thursday, February 19, 2004 9:46 AM by TrackBack

# Final complaints about BinaryFormatter and the persistence format...

Thursday, February 19, 2004 9:46 AM by TrackBack

# Final complaints about BinaryFormatter and the persistence format...

Thursday, February 19, 2004 3:07 PM by Bill

# re: Are KB articles allowed to have poorly written code? (KB: 320348)

<<
Should KB articles that attempt to create non-existent technology do so with the mindset of LCS...
>>

In my opinion, kb articles, msdn docs and all of the sample code need to demonstrate their associated concepts. Production quality code is, in my opinion, not what these things should be expected to provide. They're not supposed to be teaching people how to write code. They aren't code repositories.

<<
This is actually the right thing to do.
>>

I disagree. The "right thing" to do is to validate input.

<<
As it stands I'm just guessing everyone that defended MSDN here agrees that resources like file streams don't really need...
>>

Your logic is flawed.
Friday, February 20, 2004 5:23 AM by Darren Neimke

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

I really don't have much to add in terms of your questions but, it always interests me to see people who are concerned enough to put conditions at the bottom of loop structures like that (i.e. do...while's as opposed to while loops). It's a good reminder that there's more than one way to do stuff but, left unchecked it opens up another possibility for failure.
Friday, February 20, 2004 5:31 AM by Justin Rogers

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

For the most part, this is straight out of the rotor code, except for our updates. I have to say, putting the condition at the end of the loop here is IMO the right thing to do. It makes the code much more elegant than it would otherwise be.

Now, would I have written the loop the same way had I not been looking at the rotor code when writing my method? Maybe, maybe not. It would certainly have taken me some thought to come to this method, because it is not an oft used method for me.
Friday, February 20, 2004 7:00 AM by Darren Neimke

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

So, what does:

( ( _b & 0x80 ) != 0 )

.. test for? As in, what is 0x80?
Friday, February 20, 2004 7:08 AM by Justin Rogers

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

_b & 0x80 tests the highest bit of the byte.

So basically:
0x80 translates to 10000000 binary

The way 7 bit encoded integers work, is this. Read a byte, take the bottom 7 bits and add it to your accumulator, shifting if necessary (by a multiple of 7). Then check the highest bit, or the 8th bit, and see if it is set. If it is set, then we aren't done yet. We need to read another byte from the stream and continue the decoding process.
Friday, February 20, 2004 12:56 PM by gilad g

# re: Awesome Windows Forms message pump trick...

...And with the power of anon. methods in C# Whidbey, there won't be any need for pre-defining a "Method Invoker" :)
Friday, February 20, 2004 1:31 PM by Bill

# re: An algorithm puzzle from a rotor code examination. Weigh in your thoughts...

<<
do...while's as opposed to while loops
>>

In the Write... code a while loop is used because you don't always want to enter the loop. In the Read... code, a do loop is used because you always want to go through the loop at least once. You could re-code each method to use the other construct, but you'd have to add more code to handle special cases.

<<
left unchecked it opens up another possibility for failure.
>>

Not any more than other constructs. Unless I'm missing something about do..while loops.
Saturday, February 21, 2004 10:03 AM by TrackBack

# Answers to the rotor code algorithm puzzle...

Monday, February 23, 2004 10:58 AM by Jerry Dennany

# re: I promised I would talk a bit about Terrarium, so here goes...

Ah, but it isn't boring. Many, many applications make the same mistakes about writing to the current directory, and you help bring focus to this issue.
Monday, February 23, 2004 3:16 PM by Justin Rogers

# re: I promised I would talk a bit about Terrarium, so here goes...

Yes, most applications do make the mistake of using their own directory as a stomping ground for file IO code. It is hard to decide how to implement your file IO restrictions though, and I think this will be the interesting facet of the Terrarium changes. Currently the code changes just do the right thing, however, there are some sticky issues with when we really do need to write files in user selectable locations.

I'm thinking a change to our permissions code to deny File IO at a very early stage in the loading process might be worth looking into. I'll definitely blog about that kind of change because it starts to get interesting.
Tuesday, February 24, 2004 2:34 AM by Thomas Tomiczek

# re: Rotor FUBAR of the day: Why did IPAddress take a public long constructor?

One World:

CLS.

CLS does not allow usigned variables. So UInt (which IPV4 ACTUALLY is does not work). So - in rder to handle all "official" representations of IP in single int form, they had to make the variable an Int64.

You get the same thing with port numbers on sockets, interesting enough, although there they do not get into compatibility mode. he IpEndPoint has an int as port number - interesting in that port numbers run up 65536 and int is 32 bits. But heck, then: CLS does not allow an ushort to be used :-)
Friday, February 27, 2004 8:38 PM by Justin Rogers

# re: Rotor FUBAR of the day: Why did IPAddress take a public long constructor?

Darn, sorry for the long delay in reply, the flu seems to have found me.

This is a perfect response, since it really points out why certain designs were adopted. I'm glad someone had the insight to point this out, since I probably would have overlooked this.

Why? Well, because having used sockets for so long, the sign of a 32-bit integer has never really mattered. Behind the scenes a negative integer is simply mapped to an appropriate IP, and you never find yourself typing the integer format of an IP address anyway.

I think this FUBAR has been reversed and I'll graciously accept that the *right thing* was done.
Sunday, February 29, 2004 4:15 AM by Kartal Guner

# re: My god what was I thinking, using StringCollection over ArrayList?

I assumed the same thing. Good to know this though. I haven't gotten to the point where I am trying to fine tune the performance yet. The StringCollection has a Contains function which is nice though.
Sunday, February 29, 2004 5:36 AM by Justin Rogers

# re: My god what was I thinking, using StringCollection over ArrayList?

Contains is part of IList, and so ArrayList also has a Contains method. It uses an Equals override to determine equality.

if (item.Equals(_items[i]))

For strings, Equals is an internal call. It looks something like the following code which is a basic C++ string comparision function, with the exception of there being a couple of additional items to handle managed specifics.

//
//
// COMPARATORS
//
//
bool WcharCompareHelper (STRINGREF thisStr, STRINGREF valueStr)
{
DWORD *thisChars, *valueChars;
int thisLength, valueLength;

//Get all of our required data.
RefInterpretGetStringValuesDangerousForGC(thisStr, (WCHAR**)&thisChars, &thisLength);
RefInterpretGetStringValuesDangerousForGC(valueStr, (WCHAR**)&valueChars, &valueLength);

//If they're different lengths, they're not an exact match.
if (thisLength!=valueLength) {
return false;
}

// Loop comparing a DWORD (2 WCHARs) at a time.
while ((thisLength -= 2) >= 0)
{
if (*thisChars != *valueChars)
return false;
++thisChars;
++valueChars;
}

// Handle an extra WCHAR.
if (thisLength == -1)
return (*((WCHAR *) thisChars) == *((WCHAR *) valueChars));

return true;
}
Monday, March 01, 2004 2:02 AM by TrackBack

# Game design books focused on non *core* markets, a short resource list...

Tuesday, March 02, 2004 8:27 PM by Addy Santo

# re: Trials and Tribulations of UI design: An examination of the Focus SVT dashboard...

A friend of mine got a panicky call from his wife a while back, saying "I'm in my car and a red icon just came on with a picture of an explosion with a mushroom cloud. I stopped on the side of the road and I'm afraid to turn the engine back on"

A quick phone consultation with the dealership revealed that the icon she was referring to was "low windshield fluid" and the mushroom cloud was really an attempt to show fluid spraying on the windshield (think of a whale-blowing-air caricature)

Friday, March 05, 2004 1:19 AM by TrackBack

# re: New Terrarium UI

Saturday, March 06, 2004 12:55 AM by TrackBack

# String::Join vs StringBuilder::Append vs StringBuilder::Append /w capacity...

Sunday, March 07, 2004 1:15 AM by Richard Tallent

# re: Doing some perf testing, String::Join vs StringBuilder::Append...

This may be a bit unfair to StringBuilder: after all, it is dynamically resizing its internal structure each time you double the number of characters after 16, whereas your Array has already been instatiated, sized, and populated.

Better apples to apples:
1. Use StringBuilder.EnsureCapacity to overcome the advantage Array has by having a preset allocated size.
2. Move the starting time for Array to above the For loop to remove the bias of overhead from the loop and population effort.

Notwithstanding the above, you've made a good point: if you already have an array, String.Join makes a lot more sense than iterating through the array again and copying it to a StringBuilder.
Sunday, March 07, 2004 2:56 AM by Justin Rogers

# re: Doing some perf testing, String::Join vs StringBuilder::Append...

I accounted for the resize in a follow-up post. If you click on the trackback link for the first comment you'll see the follow-up. While there is stil some incorrect information in the follow-up regarding string interning which turns out not to be happening at all, the timing values are spot on and shows String::Join to be only 25% faster than StringBuilder::Append when using capacity planning.
Saturday, March 13, 2004 8:57 AM by Winnifred Alberts

# re: I promised I would talk a bit about Terrarium, so here goes...

Is there any way you can give us eager little coders an approximate date as to when the terrarium source code is going to be released? I am so excited about the prospects of getting my hands on the code! :-)
Saturday, March 13, 2004 8:08 PM by Justin Rogers

# re: I promised I would talk a bit about Terrarium, so here goes...

Eager coders should take this to heart... Soon. And this time we mean it, and are very close.

Since I won't tell you when I can tell you a little more about what. The Terrarium source release is going to be a true experience. Starting with the hopefully flawless and easy to use install process of all Terrarium components down to the refined look and bug fixes to the game play that have been plaguing some developers.

The source code and documentation are going to be indispensable for reference when programming similar applications in the future. In other words, it is worth a small wait and a bit of uncertainty ;-)
Saturday, March 13, 2004 8:14 PM by Koji Ishii

# re: String::Join vs StringBuilder::Append vs StringBuilder::Append /w capacity...

I don't remember how I get this, but if I remember correctly, what string.Join does is:
1. Create a StringBuilder with capacity strings.Length * 16 (this 16 is a fixed number)
2. call sb.Append
This explains why your SBWithCapacity is a bit slower than string.Join, because string.Join doesn't iterate through the array to detect capacity of StringBuilder.
string.Join is faster if average of the length of the strings are less than 16, but I guess yours is faster if longer than that.
Saturday, March 13, 2004 9:08 PM by Justin Rogers

# re: String::Join vs StringBuilder::Append vs StringBuilder::Append /w capacity...

This is incorrect, String::Join as defined inside of the rotor code-base, iterates and creates a precisely sized string. It does not use StringBuilder internally.

The answer is inside of clr\src\vm\comstring.cpp as ConcatenateJoinHelperArray which is called by String::Join to help stitch the string together.

And here is the code and comments that defines precisely that String::Join allocates a perfectly sized memory area for the concatenation of the strings:

//Figure out the total length of the strings in (*value).
elemCount = startIndex + count;
for (numElems=startIndex; numElems<elemCount; numElems++) {
if (((*value)->m_Array[numElems])!=NULL) {
newLength+=((STRINGREF)((*value)->m_Array[numElems]))->GetStringLength();
}
}
numElems=count;

//Add enough room for the joiner.
joinerLength = (*joiner)->GetStringLength();
newLength += (numElems-1) * joinerLength;
Sunday, March 14, 2004 7:42 PM by Darren Neimke

# re: Testing your tests. Ensuring test validity and preventing common mistakes...

Justin, thanks for this post it cleared up a lot of things for me. One thing though - just for accuracy's sake - when you say:

"In Darren's case he thought he was fixing the problem and that the 30 was the real result he was looking for and that the 4000 had to be inherently flawed"

That's not quite right. I blog'ged them because I knew something was wrong somewhere but just didn't really know where/how to look for it. I actually posted a copy off to the VB guys as well just for them to look at.

Another quirk is that I knew that what I was testing probably wasn't functionally == I was just looking for the quickest way to break strings into smaller pieces (for example I'm sure that my Regex.Split tests are not == to String::Split()).

The biggest surprise for me so far is that the String::Split semantics is different to what I had pre-supposed; and you're right, I would have picked that up if I'd checked the outputs.

Thanks again for taking the time to look into this stuff!
Sunday, March 14, 2004 8:29 PM by Justin Rogers

# re: Testing your tests. Ensuring test validity and preventing common mistakes...

Well, even though you weren't thinking that way, you still made a great example ;-) A lot of people do think that way, especially when the initial tests were showing such a huge performance issue with the VB version.

Many developers would have assumed they had the *fix* after the changes you made were implemented because it appears to rationalize the results. I mean, 4000 milliseconds is a long time compared to the results of the other tests you ran.

As for the String::Split semantics, the only way to get something functionally equivalent that splits on a string rather than a character basis, is to code your own (hence the VB guys Split function implementation). However, their method isn't truly optimal, and I think you could write a faster Split() function implementation. I might just try this.
Sunday, March 14, 2004 9:40 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Sunday, March 14, 2004 9:40 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Sunday, March 14, 2004 9:42 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Sunday, March 14, 2004 9:42 PM by TrackBack

# Testing your tests. Ensuring test validity and preventing common mistakes...

Monday, March 15, 2004 12:24 AM by TrackBack

# re: String::Split - unexpected behaviour?

Sunday, March 21, 2004 4:46 AM by Darren Neimke

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

Justin, just for laughs can you show the metrics for a test where you *do* the ToLower() on the input string. In other words:

private static Regex ignoreCase = new Regex("hello".ToLower(new CultureInfo("en-US", false)), RegexOptions.Compiled);

One of the things that interested me while peeking at Rotor source is that, one of the overloads of AddConcatenate takes a third boolean parameter which stops the ToLower() call from happening when doing a replacement - even if using IgnoreCase. I couldn't really work that one out.
Sunday, March 21, 2004 5:35 AM by Frans Bouma

# re: Performance: If vs Switch in what we might call an extended usage scenario?

Actually the compiler optimizes the switch statement: if it can, it will create a hashtable with name-value pairs. The 'name' part is the value used in the code for the case (such as case "Foo"), the value is the value used in hte optimized case (such as case 4). The hashtable is static.

With a few iterations, this is slower, as it will have to build up the hashtable. For more iterations this is very fast. Check a release build compile of a switch case with strings in reflector :)
Sunday, March 21, 2004 5:39 AM by Darren Neimke

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

BTW... good luck digging through the source to see what IL is produced; have you ever seen the stuff which is Emit'ted from the RegexCompiler module ? :-)
Sunday, March 21, 2004 5:59 AM by Justin Rogers

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

I know why the one is faster versus the other now... I'm pretty adept at going through Rotor ;-)

A character class uses a specialized binary search algorithm across a specialized string of characters that exists as a result of RegexCharClass. This becomes a single call within the compiled binary into a helper function.

With an alternation construct, a butt-load of instructions are emitted that trace each path in the alternation and backtrack if necessary to start on the next item. These are called groups and use something called the grouping stack.

At least this is what I'm gathering right now. I'll probably keep doing work on the source, however, running simulations seems to be turning out better numerical results, even if those results aren't being firmly rooted.
Sunday, March 21, 2004 6:31 AM by Justin Rogers

# re: Want faster regular expressions? Maybe you should think about that IgnoreCase option...

Here are the results of doing a ToLower before calling the expression:

C:\Projects\CSharp\Utilities\PoliCheck>WordIgnoreCase.exe
Ignore Case By ToLower Sanity Check
True
End Ignore Case By ToLower Sanity Check

Ignore Case Sanity Check
True
End Ignore Case Sanity Check

Explicit Case Sanity Check
True
End Explicit Case Sanity Check

Character Class Sanity Check
True
End Character Class Sanity Check

Ignore Case By ToLower, 00:00:02.8440896
Ignore Case, 00:00:01.7625344
Explicit Case, 00:00:01.2518000
Character Class, 00:00:01.1716848
Sunday, March 21, 2004 7:13 AM by Justin Rogers

# re: Performance: If vs Switch in what we might call an extended usage scenario?

However, I wasn't using strings in the example, and instead characters. If you blast out the resulting IL there is very little difference between the switch method and the if statement method in terms of IL and no Hashtable is used (I was already using /optimize+). At this point, I have to continue to assume that there is something happening at the JIT level to optimize the things.

However, I will go ahead and play the part of the diligent worker bee. I'll dump the IL and do a sample walk-through based on data and try to count the number of IL instructions that will be encountered by each process. There are some small differences in the switch vs if, just none that would immediately qualify as providing the speed increase.
Sunday, March 21, 2004 10:23 AM by Duncan Godwin

# re: Making the Regex engine choke... Ever wanted to? I'll show you how...

Hi,

I've been reading Advanced .NET recently and it mentions you can use peverify to tell how the IL is invalid, does this give anything of interest?

Sunday, March 21, 2004 6:07 PM by Justin Rogers

# re: Making the Regex engine choke... Ever wanted to? I'll show you how...

Using PEVerify results in what you would say is a valid assembly. The PEVerify tool must be a loose subset of all rules that can be implemented, and at least with the version I'm running there aren't any flags to specify how strict PEVerify should be since there are four levels of IL safety.

I'm not all that surprised that PEVerify doesn't find anything wrong. However, I am surprised to wake up after nearly 12 hours and find the ILAsm compilation still running.
Sunday, March 21, 2004 8:19 PM by TrackBack

# Monolithic patterns can cause exceptions

Sunday, March 21, 2004 8:21 PM by TrackBack

# Monolithic regex patterns can cause InvalidProgramException to occur

Sunday, March 21, 2004 10:24 PM by TrackBack

# Take Outs for 21 March 2004.

Take Outs for 21 March 2004.
Sunday, March 21, 2004 10:43 PM by TrackBack

# The root of an InvalidProgramException and a possible JIT bug?

Monday, March 22, 2004 1:25 PM by Kit George [Microsoft]

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

Justin, I thougt I would respond to some of your queries here.

I did want to confirm that the added pressure affects everything in Gen-Zero, so it is not tied to your instance per-se. The GC simply alters its collection strategy for all Gen-Zero items as a result of the added pressure, it does not keep a close eye on one particular item.

As for your concern about what happens if you forget to remove the pressure: I have to stress, DON'T forget. We do not try to remove the pressure if you have forgotten. Therefore there is a danger that you can leave the GC in a state where its collection strategy has changed and is no longer optimal. You really do have to make sure that if you add <x> pressure, you later remove <x> pressure when the object is finalized. We like to keep the GC instance agnostic which is why we avoid tieing it to specific items.

As a note: FxCop will be a great help here, since we have rules in place to help ensure that if you AddPressure somewhere, that you also remove the pressure in the finalizer for the class.

As for HandleCollector: you're right, in that it is simply a helper class. It was something which we were using internally, and believe made sense to expose, since people were probably writing their own anyway.

I am sure we'll continue to expand on what's available in this area, and we're always interested to hear your ideas/suggestions for improvements. For example, I have taken note of the request to consider adding an 'AvailableForGC' method in a future release.
Monday, March 22, 2004 8:21 PM by Justin Rogers

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

If you might be able to comment on what the usage of AvailableForGC might be if there isn't any way to actually collect the object after making the examination? What can we do with this information if we do get it.

You also mention again, using the finalizer to make sure the memory gets removed. However, are we instead going to see the Dispose pattern being more important here? AKA, does it matter if the managed object is collected if the unmanaged memory has already been released. The only time the finalizer for a bitmap might be called is if the programmer is relaxed in his disposal of the resource. Now, in the Dispose pattern the finalizer does make sure to call the Dispose methods, so maybe that is what you guys are meaning.
Tuesday, March 23, 2004 4:26 AM by TrackBack

# Discussion on GC in Whidbey

Tuesday, March 23, 2004 4:58 AM by Iain

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

It occurs to me that sure, you could use this stuff to leak resources. However, the only real applicability is for stuff that wraps unmanaged resources - so utilising these could leak memory, but you will only have cause to use these if you already have the ability to leak memory.
Tuesday, March 23, 2004 9:30 AM by Doug McClean

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

To me, an AvailableForGC method doesn't make sense because it would be almost as expensive to implement as an actual garbage collection (e.g., you need to do the full marking pass of the garbage collector to figure out what the answer to this method is), so if you are going to see if something is AvailableForGC you might as well just do the GC and be done with it. Am I missing something?
Tuesday, March 23, 2004 4:44 PM by Kit George [Microsoft]

# re: Making the Regex engine choke... Ever wanted to? I'll show you how...

Justin, this is interesting. We'll be taking a look at this to see what the problem is, since this does look problematic.

Thanks!
Kit
Tuesday, March 23, 2004 9:22 PM by Justin Rogers

# re: The new face of the GC in Whidbey... I'm not sure this is a pretty face...

To Iain: Currently you can leak unmanaged resources, you are right. However, this leakage doesn't affect the GC at all, it has no impact.

Now, if you fail to call RemoveMemoryPressure, you are going to permanently (application lifetime I'm assuming) affect the GC and possibly cause more GCs than are necessary.

So rather than leak a single resource now, you are leaking a new pseudo resource. The pseudo resource is the additional memory pressure placed on the GC that is causing it to change it's heuristics and collect more often because it has been told some object consumes more memory than the GC can deduce.

Thinking really hard on this matter, I really think the memory pressure should be assigned an object instance, or at least have that option. If that object instance is ever collected, it is assumed that the memory pressure, if not already released by the object, gets released. I realize FXCop has rules for this type of thing, but for every developer using FXCop, there are 10 that don't.
Wednesday, March 24, 2004 9:45 AM by Matt Hawley

# re: Installer Products and why I'm not sure I like them very much...

I enjoy using NSIS...its more developer friendly because you get to write the script yourself :)
Thursday, March 25, 2004 3:25 AM by Matt Berther

# re: Installer Products and why I'm not sure I like them very much...

Justin: Regarding item 4, DevStudio 9 does support C# installers. These are loosely similar to C# custom actions, except for the fact that you can not control where in the sequence they occur.

Look in the System.Configuration.Installers namespace for how to author them.

When you want to place one in a DevStudio project, select the component that contains the Installer class and select the option about it being a .NET installer. Arguments can be passed to the installer via the command line arguments option in the same grid.
Thursday, March 25, 2004 3:35 AM by Justin Rogers

# re: Installer Products and why I'm not sure I like them very much...

Yes, we actually run a number of installers, however, I wasn't aware this was as powerful as the Visual Studio custom actions. For instance, I'm able to run my Installer class, but not able to get to needed MSI properties.

Also, yes, not being able to control where they occur is yet another issue with them. I've looked at these, they simply aren't powerful enough to take the place of custom actions, and really only have their place if you don't need the installers to actually interact with the installer process. At least that is what I've taken away from the whole experience.
Thursday, March 25, 2004 7:56 AM by Matt Hawley

# re: More InstallShield DevStudio 9 issues...

PS - its NSIS, not NSSI (http://nsis.sf.net)
Thursday, March 25, 2004 8:36 AM by Justin Rogers

# re: More InstallShield DevStudio 9 issues...

Ah, thanks for the link. Looks powerful and the source is available. I like it. I'll have to code something up now so I can package it using this installer.
Friday, March 26, 2004 8:51 AM by Josh Baltzell

# re: Placing vegetarianism on the chopping block...

I was a veggie in HS and a little while after. I started ebcause someone told me I couldn't do it for all day on Thanksgiving. I was a vegetarian for like 2 or 3 years after that.

I didn't eat healthy or anything, I just didn't eat meat. Then I got a girlfriend (my fianceé now) slowly but surely I was eating fish, then chicken and then after a while I was eating all meats again.

So if you really want to get back in to eating meats for some reason (Don't forget that meat meals are generally more expensive on top of worse for you) then maybe just working up from fish is the way to go.
Friday, March 26, 2004 2:21 PM by Christoc

# re: Placing vegetarianism on the chopping block...

Yummmmm Steak-Ums!

Good luck to you, hope you don't get sick for too long due to this.
Friday, March 26, 2004 3:25 PM by Justin Rogers

# re: Placing vegetarianism on the chopping block...

I do eat fish at times. I'm classified as a lacto/pesco/ovo vegetarian. I guess people weren't comfortable with the whole black and white you are or you aren't, so they had to come up with additional classifications.

Most of my protein comes from soy sources and eggs, so I'm not sure what I'd do without the ovo part. The lacto I could probably do without, but I love coffee drinks and soy milk is still extra (darnit). I still like a piece of fish at times, but does one piece of fish a month still take on the pesco? I don't know. I've seen vegans pile through some things not on their recommended eating list at a doctor's requirement.
Friday, March 26, 2004 5:01 PM by Rj

# re: Placing vegetarianism on the chopping block...

Good luck eating meat again. Yours seems to be one of the most rational reasons for not eating meat.

btw, Irregardless is not a word, except through continued missuse. Regardless works just fine.
Friday, March 26, 2004 5:06 PM by Shannon J Hager

# re: Vegetarian rating of 5 types of meat, a follow-up to the original, all too tempting test.

I went a couple years without meat, but whenever anyone asked if I was a vegetarian, I replied "huh? I don't even know where Vegetaria is. I'm American."


Now that I am an omnivore, I am still down with the veggies. I swear I make the best tofu I've ever had. Vegetarians are cool with me, but humanitarians scare me.
Friday, March 26, 2004 6:22 PM by TrackBack

# Vegetarian rating of 5 types of meat, a follow-up to the original, all too tempting test.

Saturday, March 27, 2004 11:51 AM by Brian Desmond

# re: Installer Products and why I'm not sure I like them very much...

I've never used the VS Installer. All of the installer's I've built have been in DevStudio 9 or Developer 8. Probably 30 or 40 MSIs by now.

It's really not *that* bad, just takes some getting used to. I don't use any of the wizard crap. just skip to the treeview with everything in it & find what I want.
Sunday, March 28, 2004 8:50 AM by Justin Rogers

# re: Performance and Insight: For versus Foreach over a strongly typed array... Some things you might not know.

Someone asserted that the JIT might optimize out the entire loop and make it *not happen* because the resulting value isn't being used. I think this is a highly unfounded assertion, but just in case, you'd think we might be able to save the JIT the trouble of optimizing things out by appending the bar variable onto the Console.WriteLine call. Once you do that, the variable bar becomes *used* and so the JIT couldn't possibly optimize it out.

However, I know the JIT wasn't optimizing things out because I was viewing the resulting assembler for each of the above methods and watching it run. I think that asking the JIT to optimize all of the above code out would be a rather tall order, but I'm assuming the metrics could be there for it. As a human compiler I would optimize the code out since I would realize the value of bar was never used and so the iteration was for naught.

I guess the days of using looping constructs to count time away are over. No more: "Please insert an integer between 1 and 10000: " when you start running a game under the GW BASIC interpreter. And I thought I was so slick when I wrote my own custom timing function to get rid of that message for good, so no user ever had to feel the wrath of submarines and torpedos at warp speed on their ultra fast new 386.
Sunday, March 28, 2004 9:00 AM by RossJ

# re: Found the creator of Ping while doing graphics research on Cone Tracing, go figure!


Do you have more information on your photon mapping software? One of our student developers is working on some photon mapping solution for his Masters dissertation and I thought it would be nice if I could help him find more (not that he really needs more) solutions doing the same thing.
Sunday, March 28, 2004 9:10 AM by Justin Rogers

# re: Found the creator of Ping while doing graphics research on Cone Tracing, go figure!

I'll be releasing more information soon. I just sat down and wrote the software over the past couple of days. I'm still deciding if I want certain math libraries to be written in Managed C++, or whether I should leave the entire thing in C#.

I also plugged in my own partioning library, and I'm finding it to be inadequate. I'll probably plug through another 2-3 different types of partioning trees this weekend.
Sunday, March 28, 2004 9:12 AM by RossJ

# re: Found the creator of Ping while doing graphics research on Cone Tracing, go figure!


He did describe in (excrutiating) details how he was doing this and it *sounded* impressive (I'm not oreinted that way) .. I'll just point him at your blog instead :)
Monday, March 29, 2004 1:25 AM by Omar Shahine

# re: Dynamic versus Cached Thumbnailing in GDI+

I have an application which does just this (http://workspaces.gotdotnet.com/jpeg) however, the perofmance of creating the bitmap just sucks (IMHO). Basically I get what you say, 2 images loaded per second. Applications like PhotoShop can do a whole folder of photos (50 or so) in about the same time.
Monday, March 29, 2004 3:22 AM by Justin Rogers

# re: Dynamic versus Cached Thumbnailing in GDI+

Omar, the reason for their abilities is in how they process the images straight off the file system, rather than first loading the bitmap into memory and then converting it down to a thumbnail.

I will say, however, that on my most recent revisions for the Async thumbnailing code, I've been able to process closer to 5 or 6 thumbnails per second rather than the original 2. I attribute this to some issues with my async code that I've since worked out.

While Photoshop may be able to do an entire folder in about the same time you load only a couple, I would instead point you towards an Explorer folder where thumbs.db or thumbnail caching has been turned off. Thumbnailing images in this fashion operates at about the same speed as the GDI+ code.

So I'm not saying we couldn't get better performance using enhanced caching, or better methods for thumbnailing images without loading the entire bitmap, or even a better thumbnailer that operated faster. What I would say, is that programs that use the same methods we are using, run at the same speed, and I find it an acceptable speed.
Monday, March 29, 2004 5:18 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

The sample program had a single naming issue. I fixed this, but only after there was a reported view on the source sample. If that someone has tried to run the code, sorry for the mix-up, the new code in the entry should run as planned.
Monday, March 29, 2004 5:38 AM by David Levine

# re: Performance: Different methods for testing string input for numeric values...

I haven't done any performance testing on this, but you should add double.TryParse() to your test suite - it avoids the problem that int.Parse() has of throwing an exception if it fails.
Monday, March 29, 2004 5:48 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

Adding performance output for everything except the double.TryParse()

IsNumber String 1: False
IsNumber String 2: True
IsDigit String 1: False
IsDigit String 2: True
HandCodeSwitch String 1: False
HandCodeSwitch String 2: True
HandCodeIf String 1: False
HandCodeIf String 2: True
RegexDigit String 1: False
RegexDigit String 2: True
IntParse String 1: False
IntParse String 2: True
IsNumber 00:00:03.2146224
IsDigit 00:00:02.4635424
HandCodeIf 00:00:00.7410656
HandCodeSwitch 00:00:00.7811232
RegexDigit 00:00:13.3391808
IntParse 00:08:32.7372800
Monday, March 29, 2004 5:50 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

Adding code and timing for TryParse on System.Double.

start = DateTime.Now;
for(int i = 0; i < iterations; i++) {
makeSureTheJitDoesNotOptimizeMeOut = double.TryParse(testStrings[i%2], NumberStyles.Number, null, out result);
}
end = DateTime.Now;
Console.WriteLine("DoubleTryParse {0}", end - start, makeSureTheJitDoesNotOptimizeMeOut);


DoubleTryParse 00:00:13.2490512
Monday, March 29, 2004 7:02 AM by TrackBack

# Terrarium: How do we punish creatures that take too much time?

Monday, March 29, 2004 7:10 AM by Justin Rogers

# re: Performance: Different methods for testing string input for numeric values...

Adding code and timing for Microsoft.VisualBasic.Information.IsNumeric.

start = DateTime.Now;
for(int i = 0; i < iterations; i++) {
makeSureTheJitDoesNotOptimizeMeOut = Information.IsNumeric(testStrings[i%2]);
}
end = DateTime.Now;
Console.WriteLine("Information.IsNumeric {0}", end - start, makeSureTheJitDoesNotOptimizeMeOut);

Information.IsNumeric 00:00:17.1746960
Monday, March 29, 2004 5:33 PM by Josh Baltzell

# re: Vegetarian rating of 5 types of meat, a follow-up to the original, all too tempting test.

I made it all the way through over a year of being a vegetarian without ever being able to make anything that actually tasted good out of tofu.

People generally refer to my prior vegetarian self as a hippie. Now granted I may have been slightly hippieish, but being a vegetarian has nothing to do with being a hippie. Thank you for telling people that.
Monday, March 29, 2004 8:35 PM by TrackBack

# Various things around the web today

Wednesday, March 31, 2004 1:19 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

I've always wondered why this wasn't built into the framework... It's such a common task. I guess Rico didn't stop by this team and talk to them about performance issues with exceptions...
Wednesday, March 31, 2004 1:33 AM by Jesse Ezell

# re: Why is photon mapping popular in gaming right now, and will it stay?

Very cool stuff. Have some demos?
Wednesday, March 31, 2004 1:53 AM by Justin Rogers

# re: Why is photon mapping popular in gaming right now, and will it stay?

Soon. I've been researching photon mapping for a while, and all of my cool stuff is theortical and based on numerical analysis. My first true scene should be the club scene with about 30 static colored lights, 10 floor standing stage lights, and a single strobe.

I'm doing all of the work in C# and Managed DirectX, so there is a curve where-in my performance knowledge is having to be augmented. Namely in the random number generators (the default is too slow), and in spatial storage tress (hard to make these fast in .NET).
Wednesday, March 31, 2004 2:33 AM by Alex Campbell

# re: Following up on the benefits of continued use of int.Parse...

Hi Jason,

Thanks for the feedback. As usual, I agree with almost all of what you say. Your alternative approaches are certainly faster. However, I think in some particular cases, try... int.Parse()... catch still has a place.

Your description of my (not even remotely original) approach as "admirable" is hard to judge. Did you mean "admirable" in the same way that I often describe really stupid ideas as "creative", "novel", "unique", "fascinating", "breathtaking" etc? :-)

I also agree with Jesse that it would have been great if the .Net Framework team had included this functionality in the framework.
Wednesday, March 31, 2004 2:47 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

Admirable in that you were willing to defend the method based on it's additional power to differentiate int's from non-ints. None of the original routines that I tested were truly capable of discerning ints and it was good that you pointed that out.

Even my newer techniques have issues in that they aren't globalizable since they aren't based on culture variances (I pointed this out above).

However, even if the BCL team had provided better options for programmers, my method would still be faster. Unless they dynamically compiled, based on the culture info, a string to integer parser, it wouldn't be faster than my implementation that is hard-coded to work only in the US english case. I am currently in the process of discovering how this might be done, since fast string->number conversions are very important for several of my own applications.
Wednesday, March 31, 2004 3:17 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

If you really want a flexable implementation that matches the .NET framework's, you could probably just use reflector to get the source and then remove the exception throwing stuff (or get the mono source or rotor source).

In any case, Whidbey adds a TryParse method, so at least you won't have to wait too long for this functionality.
Wednesday, March 31, 2004 3:23 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

PS: If you don't mind casting in .NET 1.0/1.1, you can use Double.TryParse and convert to an integer...
Wednesday, March 31, 2004 3:28 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

As for a flexible implementation, a good deal of the code for numeric types is implemented as part of the VM or virtual machine. There are ee calls that take place (InternalCall) that actually handle the conversion of the string to a number. These calls happen to be where the exceptions are thrown from, so reflector doesn't really help.

To answer your Double.TryParse comments. Double.TryParse is slow. Very slow in fact. It is much slower than int.Parse assuming all numbers are valid. You can read my previous blog entry that covers the speed of many other parsing methods.

<a href="http://weblogs.asp.net/justin_rogers/archive/2004/03/29/100982.aspx" title="Justin Rogers">Performance: Different methods for testing string input for numeric values...</a>
Wednesday, March 31, 2004 3:30 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

A second assertion would be that a user can input non integer values into Double.TryParse, that could then be, validly cast into an integer. This would also enable loss of data, and a bunch of other *bad* things. I honestly don't think that using Double.TryParse and casting down is a valid way to test a string for integer value.

Wednesday, March 31, 2004 3:41 AM by Andy Smith

# re: Following up on the benefits of continued use of int.Parse...

Double.TryParse takes, as one of the parameters, a NumberStyle. One of those NumberStyles is Integer. using the integer numberstyle, you are guarenteed an int in the value parameter when it returns true.

And as for your perf argument... You've said yourself that your custom methods only work on US English. I'm sorry, but sacrificing localization for perf is not acceptable. "Anybody can write a fast program that does the wrong thing."

And as for the perf gain over Int32.Parse, well, that throws exceptions on bad data, which is not what you want for perf.
Wednesday, March 31, 2004 3:57 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

How often do you use a localized parse? You aren't using it in WebForms applications, since in most cases the locale is locked down to that of the server. You most of the time aren't using it in WinForms applications, and your configuration files/files being parsed are most likely in some set format. Not to mention unless you store the locale in the file, it will be mis-parsed if you send it across a locale.

My argument would be that sacrificing perf for localization isn't acceptable in most cases. Presenting a reverse argument never really accomplishes much though.

I will note that NumberStyles.Integer is not linked to a specific bit length. So you can still improperly decode a 64 bit integer into a 32 bit integer. You can also decode a 32/64 bit integer into a 16 bit integer. I really don't see what this Double.TryParse really buys, except for a bit of added and additional complexity.

Hell, you can decode an infinite length string using Double.TryParse. Check the following program.

using System;
using System.Globalization;

public class HackDoubleTryParse {
private static void Main(string[] args) {
double retVal;
string[] integers = new string[] {
"7290847123984721093749812374092137498217309821098472198347",
"-210943712908347091823409812734982370947213098412093741924"
};

Console.WriteLine(integers[0]);
Console.WriteLine(Double.TryParse(integers[0], NumberStyles.Integer, null, out retVal));
Console.WriteLine((int) retVal);
Console.WriteLine();

Console.WriteLine(integers[1]);
Console.WriteLine(Double.TryParse(integers[1], NumberStyles.Integer, null, out retVal));
Console.WriteLine((int) retVal);
Console.WriteLine();
}
}
Wednesday, March 31, 2004 4:05 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

It appears to accept up to 308 characters before it returns false. So:

new string('5', 308) would succeed, while
new string('5', 309) would fail.

However, characters may not be the order of the day, but instead a huge value. Since:
new string('1', 309) appears to work, but
new string('1', 310) doesn't.

Very strange indeed. I won't bother looking at the Rotor code for this method, since I'm not sure it would help stave off any afronts.

Wednesday, March 31, 2004 4:33 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

I just ran your test with the Double.TryParse code inserted... 6 secs for 10 mill. conversions really ain't that bad for most apps. If you pass a pre-constructed NumberFormatInfo into the method call it completes the loop a bit faster than RegEx on my CPU (especially if the numbers get large, like 8 digits or so). I don't think the extra time is really all that bad, considering that you are getting a more powerful parsing algorithm that deals with culture and other things like exponents and commas (2.2 vs 6.6 for 10 mill ints isn't too bad). For 99.9% of all apps that kind of perf is still in the acceptable range (beats IntParse by 50x on my machine... I had to go down to 100000 just to make it complete in a reasonable amount of time with the IntParse method included).

Adding a few sanity checks after the double conversion could easily mitigate any conversion issues that TryParse might lead to.
Wednesday, March 31, 2004 4:37 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

Explain your sanity checks, and then examine the timing with your sanity checks added. Since those would count as part of your *algorithm* for parsing integers.
Wednesday, March 31, 2004 5:05 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

Passing in integer as the number style should take care of the loss of any decimal data issues at almost no cost. Then, all you have to do is check to make sure the double is in range of Int32.MaxValue and Int32.MinValue.
Wednesday, March 31, 2004 5:12 AM by Jesse Ezell

# re: Following up on the benefits of continued use of int.Parse...

It actually runs faster with the sanity checks added.

6.0 secs vs 6.6
Wednesday, March 31, 2004 5:24 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

You'd actually have to post the code so we can make sure the time testing is all in the proper places, and that the JIT isn't optimizing out your checks. You also need to come up with some way to process the method returning 0 and True, when the value really isn't 0. See my above code sample for cases where the string is clearly not an Int, but the method returns True and 0.

Wednesday, March 31, 2004 5:36 AM by Justin Rogers

# re: Following up on the benefits of continued use of int.Parse...

Just to let you know, if you only pass in Integer as your number format style, then my method actually is doing everything that double.TryParse will do for you. Note that Integer actually maps to AllowLeadingSign, AllowLeadingWhite and AllowTrailingWhite. So go ahead and use that method if you would like, but it is quite a bit slower than mine.

I guess the Rotor code examination really does make the difference here since it points out the lack of additional processing that TryParse is doing, and that my algorithm should work on any culture for the purposes of processing integers. If you'd like I can even add the flags for turning on parsing of the thousands separator.

Wednesday, March 31, 2004 5:37 AM by nfactorial

# re: Wayne's World: People actually ask about the ellipses character?

<Quote>
many games use pre-generated graphics for printing text since it looks *cool*
</Quote>

Actually, games use bitmapped characters because they render extremely quickly (two stripped triangles as opposed to an outline\fill algorithm). Assuming I take your meaning correctly :)

n!
Wednesday, March 31, 2004 5:48 AM by Justin Rogers

# re: Wayne's World: People actually ask about the ellipses character?

Taken slightly out of context, but not such a big deal. I should have clarified.

Most games do appreciate the performance increase of indexing into a single texture filled with pre-gen characters and overlaying that on top of an arbitrary quad.

However, you can offer many of the same performance benefits, by dynamically rendering and caching your alphabet using GDI to blit said letter onto a texture surface somewhere.

With this in mind, recent usages of a bitmapped alphabet overlay are more for coolness, IMHO, than they are for necessity of speed. Even with the recent usage of the pre-canned font class delivered as part of the DirectX SDK, the proliferation of textured alphabets does persist, along with the lack of an ellipses usage even though the pre-canned font class should be capable of rendering any character in the given character set.

I'm not sure about the performance numbers here, but I've had the experience that it is faster to render an entire text buffer to texture using GDI and overlay the texture on a single quad, than it is to build up my display using many small quads per character. It also presents the benefits of the font layout manager for character spacing, line spacing, and other features that don't exist when you are using a textured alphabet.

Wednesday, March 31, 2004 8:48 AM by nfactorial

# re: Wayne's World: People actually ask about the ellipses character?

I'm not sure I'm talking about the same thing now :) I class dynamically rendering and caching an alphabet, using GDI+, to a texture the same thing as 'pre-generated graphics'. Even if this is done when the application is started.

Certainly rendering a single message to a texture (using the above pre-generated font) and rendering that message will be faster, but I'd still consider that the same thing.

Games (that I'm familiar with, at least) do generate their 'generic' fonts using GDI (or GDI+). I suppose you're recommending this is actually done during run-time initialization rather than pre-generated and packaged on the CD\DVD? I originally thought you meant games chose not to render fonts in the same way as windows for aesthetic reasons.

There are also other bitmap fonts that are created specifically for a game (which are all 'dolled up' to look pretty and fit with the game atmosphere). These are 'just to look cool', but then the OS (if you're lucky to be on an OS in the Win32 sense of the word) doesn't provide such fonts so there isn't much choice. Though I find these types of font are on the decline as they're more difficult to localize and read.

Sorry, I didn't mean to hi-jack the posting based on a single line. The ellipsis being a single character was a surprise to me too! :)

n!
Wednesday, March 31, 2004 9:41 AM by Justin Rogers

# re: Wayne's World: People actually ask about the ellipses character?

Nothing better than talking about gaming on a blog that contains good things on gaming.

Yeah, I think we are confusing different meanings since there are so many reasons for the way text is handled in gaming. In general, you can pre-gen a font before shipping and include that on the DVD, you can pre-cache the font at run-time initialization (which is actually a method I prefer because you can render the font for the target resolution), or you can dynamically render all of your fonts.

Recently, I think most games choose to use the DirectX font class (less pre-gen, more dynamic), or a variation there-of. But they still keep the old pre-gen stuff around for other cool features as you suggest. They might be on the decline, or they might just be getting started. Games are pushing the DVD capacity limits as we speak, and a few megs worth of extra jpegs for different alphabets, coupled with the write once, use many code required to display localized resources, really isn't that bad.

I might have to do some performance work on text engines in games. I know, for instance, that Longhorn will have one of the best text compositing engines ever written. And they are saying the new text compositing has very little performance impact on the system. That leads me to believe that current text rendering systems really aren't a viable bottleneck, especially considering the small amount of text that can be fit on the screen at one time. Just doing basic tests, I can fill a form using GDI+ flow and layout in just a few milliseconds. Since in the DirectX world that render would get re-used for a second or two before I had to re-render. And I get the font rendered for the target resolution, rather than having it shrink to unseeable at super high resolutions, and have it fit the DPI of the target device in the case of 300 DPI screens.

I guess the console market still has a good reason to use pre-gens, as would small gaming devices. I'm betting they'll be in use there for a long time as a performance increase. Maybe rendering three dots is less than the cost of an extra glyph. I'm just not sure. I just know the ellipses at this point, is a character that is either often used (insert ellipsis character) or often mis-used (insert three periods). You decide!
Wednesday, March 31, 2004 11:17 AM by Ben Hutchings

# re: Wayne's World: People actually ask about the ellipses character?

"The most readily apparent to those of us that blog, would be to use the ellipses character directly from the HTML view &#0133. Just make sure the character set being used has such a character."

No, this is wrong. Numeric character references always refer to code points in the document character set (for HTML this is ISO 10646/Unicode) and not in the file's character set.
Wednesday, March 31, 2004 12:00 PM by Justin Rogers

# re: Wayne's World: People actually ask about the ellipses character?

Ben, if you come back, can you clarify this? I've found that using &#0133 here in the .Text blogs system while in HTML view allows me to insert an ellipses. Or I can use Character Map to simply copy/paste one into the Design view. I was just trying to provide a method for individuals to insert such a character (which does work for me at least), but you seem to have additional insight. Can you perhaps demonstrate what you mean?

Wednesday, March 31, 2004 6:38 PM by Jason Olson

# re: Why is photon mapping popular in gaming right now, and will it stay?

Keep the info and posts comin' Justin! I look forward to every new post of yours.

Is there a chance that you will release the source code with the demos? That would be super cool. And I think it would help out the Managed DirectX community in general by seeing some good applications of Managed DirectX and C#.

Keep it up :).
Thursday, April 01, 2004 1:46 AM by Justin Rogers

# re: Why is photon mapping popular in gaming right now, and will it stay?

Since it'll probably be the weekend before I get to format another decent photon mapping article, I'll whet your appetite.

I definitely want to release my code with the demos. That is why I'm not releasing right now. I want my own personal examination of each of the methods and algorithms to be rigorous, to make sure I'm not producing an algorithm, that while nice to look at, has issues with physical law.

I definitely agree that we need some nice MDX/C# samples, and that is why I've chosen to use these technologies instead of the more rapid development I'd achieve by simply using C++. While C# generally provides for rapid development, performance heuristics are new territory and take some time to fully explore. Structures that work in the unmanaged world, sometimes don't work anymore in the managed world.

Glad to see I'm making a good impression and gaining good readership. I've been disappointed lately in the game development websites and their lack of content. Too much in the gaming industry is closed doors or theoretical, and there is a large gap between hobbyist and professional with not much opportunity in between. If I can help bridge that gap just a bit, I'll be happy.
Thursday, April 01, 2004 4:06 AM by TrackBack

# re: System.Drawing.Image Performance

Thursday, April 01, 2004 8:36 AM by Paul Nicholls

# re: Wayne's World: People actually ask about the ellipses character?

On the character entity references page for HTML
http://www.w3.org/TR/html4/sgml/entities.html
it has the ellipses as &hellip;
Thursday, April 01, 2004 9:22 AM by Oleg Tkachenko

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

Completely agree with you, Justin!
Thursday, April 01, 2004 9:32 AM by Steve

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

I couldn't agree more... XSLT is very, very powerful. I try and use it all the time.
Thursday, April 01, 2004 9:42 AM by Jeff

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

I agree with you, but I'm just as guilty. I've used XSLT once for something simple, and that's it. It is very powerful, but like regular expressions, I frequently can't relax and concentrate long enough on it to really get the most out of it.
Thursday, April 01, 2004 11:25 AM by Henry P. Erich III

# re: XSLT used to get all the girls, now it is just a washed up junkie ;-)

I love xslt.

XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(URL_MAIN_BLOG);

XslTransform xslt = new XslTransform();
xslt.Load(Server.MapPath(PATH_MAIN_BLOG_STYLE));

StringWriter writer = new StringWriter();
xslt.Transform(xmlDoc, null, writer, null);

mainContent.InnerHtml = writer.ToString();
Thursday, April 01, 2004 12:10 PM by Omar Shahine

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

I'll try this out today and let you know the results.
Thursday, April 01, 2004 3:54 PM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

in your original code, you can get rid of 1 multiplication by doing

value = value * 10 + val

you don't need to explicitly keep track of a multiplier through each loop.
Thursday, April 01, 2004 4:17 PM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

However, you'd have to work through the number from GSD to LSD that way. We are currently working on the assumption that you are working from LSD to GSD. This allows for us to prevent checking overflow logic for every digit counting GSD to LSD.

Here is a sample:

"123". With the current loop we are doing
3*1+0 = 3,
2*10+3 = 23,
1*100+23 = 123

You see, your method assumes that we go in reverse.

"123".
0*10 + 1 = 1
1*10 + 2 = 12
12*10 + 3 = 123

Now, when you get to the higher numbers, namely the maximum limits. You have to check for overflow conditions manually. The reason for this is that checked arithmetic throws exceptions (bad), so we can't use them. Something along the lines of Int32.Max*2+4, would actually wrap around and become a positive number again, and we would incorrectly process it. This means we have to check digits at the upper extremes. Doing this with a GSD algorithm, would mean checking at EACH digit, or at least doing some stranger logic.
Thursday, April 01, 2004 4:27 PM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Note: If you have an algorithm that works GSD and handles overflow conditions without using checked arithmetic let me know and I'll test it out. While I still think the unrolled code currently being used by the constantly updated source sample is going to be faster, I'll definitely put the methods side by side and run some speed tests. I want the fastest algorithm possible, and right now I don't think I can get it any faster without using pointers (unsafe code), and even if I do use pointers I don't think it will be faster, only more compact and elegant.
Thursday, April 01, 2004 8:13 PM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

well, probably won't be faster than unrolled loop, who knows.

const int tenthBound = Int32.MaxValue / 10;
const int digitBound = Int32.MaxValue % 10;

//
// only for Int32
//
public static bool TryParseInt(string s, out int value)
{

value = 0;

int digit;

//
// since we are being silly, this should be less costly
// than making multiple calls to s.Length. maybe?
//
int max = s.Length;

if( max == 0 || max > 10 )
return false;

bool checkEnd = max == 10; // is this max allowable length?

max = checkEnd? 9 : max; // we only loop 9 times if it's max length

for( int i = 0; i < max; i++ )
{

digit = s[i] - '0';

if( digit < 0 || digit > 9 )
return false;

value = value * 10 + digit;

}

if( checkEnd )
{

if( value > tenthBound )
return false; // x10 will result in overflow
else
{

digit = s[9] - '0';

if( digit < 0 || digit > 9 )
return false;

if( value == tenthBound && digit > digitBound )
return false; // x10 + digit will result in overflow
else
value = value * 10 + digit;

}
}

return true;

}

since this is not really heavy on array access, i doubt unsafe would provide much benefits.
Thursday, April 01, 2004 8:15 PM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

sorry, should have used <pre> to format. I hate HTML.
Thursday, April 01, 2004 8:47 PM by TrackBack

# re: Programming Challenge: Phraser

Thursday, April 01, 2004 10:28 PM by TrackBack

# Entries to my programming challenge

Friday, April 02, 2004 12:25 AM by Omar Shahine

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

So, it seems just as fast. Sweet!!!
Friday, April 02, 2004 2:15 AM by TrackBack

# Take Outs for 1 April 2004

Take Outs for 1 April 2004
Friday, April 02, 2004 5:55 AM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

DJ, here is a small test suite. Your's fails quite a few of the tests. Here is your output followed by expected output, followed by some test code. I'm definitely digging your approach, since it is radically different from the methods I've been using, and your code is a bit smaller than my final code.

<pre>
*** Community.TryParseInt32 ***
False
False
False
True
True
False
False
False
False
False
False
False
False
False
False
False

/* expected output
EC1: False
EC1: False
EC1: False
MAX1: True
MAX2: True
MIN1: True
MIN2: True
EC2: False
EC2: False
EC2: False
PC1: True
PC1: True
PC2: True
PC2: True
PC3: True
PC3: True
*/


Console.WriteLine("*** Community.TryParseInt32 ***");
Console.WriteLine(Community.TryParseInt("9999999999", out retValInt32));
Console.WriteLine(Community.TryParseInt("8888888888", out retValInt32));
Console.WriteLine(Community.TryParseInt("7777777777", out retValInt32));

Console.WriteLine(Community.TryParseInt(Int32.MaxValue.ToString(), out retValInt32));
Console.WriteLine(retValInt32 == Int32.MaxValue);
Console.WriteLine(Community.TryParseInt(Int32.MinValue.ToString(), out retValInt32));
Console.WriteLine(retValInt32 == Int32.MinValue);

Console.WriteLine(Community.TryParseInt("-7777777777", out retValInt32));
Console.WriteLine(Community.TryParseInt("-8888888888", out retValInt32));
Console.WriteLine(Community.TryParseInt("-9999999999", out retValInt32));

Console.WriteLine(Community.TryParseInt("-999999999", out retValInt32));
Console.WriteLine(retValInt32 == -999999999);
Console.WriteLine(Community.TryParseInt("-99999999", out retValInt32));
Console.WriteLine(retValInt32 == -99999999);
Console.WriteLine(Community.TryParseInt("-9999999", out retValInt32));
Console.WriteLine(retValInt32 == -9999999);
</pre>
Friday, April 02, 2004 5:59 AM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

It looks like it may fail for all negative numbers, but I also didn't try it for all positive numbers. The real test suite runs as follows. If you need a positive number test suite that is comprehensive I suggest using:

for(int i = 0; i < Int32.MaxValue; i++) {
int retVal = 0;
string tryParse = i.ToString();
if ( Community.TryParseInt(tryParse, out retVal) ) {
if ( i != retVal ) {
Console.WriteLine("Equality failure in TryParseInt: {0}, {1}", i, retVal);
return;
}
} else {
Console.WriteLine("Processing failure {0}", i);
return;
}
}
Friday, April 02, 2004 6:31 AM by TrackBack

# System.Drawing.Image Performance Update

Friday, April 02, 2004 8:13 AM by dj

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

sorry, should have commented "it only parses positive Int32's". I didn't take negatives into consideration.

did you test how my code fair on performance? (on positives of course). it would be an interesting test (in general) to see how JIT optimizes loops vs. manually unrolling a loop.
Friday, April 02, 2004 9:04 AM by Jon Hanna

# A completely different approach

Why use 1 to 10 comparisons per character rather than one comparison on a result from a lookup multistage table (or single-stage table if you want to have a single dereference but not be able to reduce the size of the data).

This would particularly helpful if you came to support characters like the Arabic-Indic digits or the Bengali digits since there are currently 268 characters in the UCS with a non-null Decimal Value property.
Friday, April 02, 2004 2:40 PM by ed

# re: Trying my hand at the old Phone number to Words teaser project!

I came up w/the same solution you did, even using c#. You wrote a couple things better than I did, so I'm not going to submit mine. The solution to the problem was the same though.
Friday, April 02, 2004 4:45 PM by the1

# re: Trying my hand at the old Phone number to Words teaser project!

Justin,

I posted my own solution at http://blogs.msdn.com/the1/archive/2004/04/02/106691.aspx
Saturday, April 03, 2004 5:11 PM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Just to show I'm still alive on this (I have been super busy with some work), I had some algorithms that were loop based and process pos/negs... I've shown them as commented out in the lib code.

As for the multi-stage table lookup, I would say try to examine the slowness of using arrays. They are super slow to say the least.

Since we are only processing up to Int64 using these methods, we could come up with an array based implementation that used pre-multiplied digits. This is completely possible and would probably be a bit faster than any of the methods I'm using right now, if we went to unsafe pointer arithmetic. I'll see if there is a method for doing the same and having it be fast without the pointer arithmetic.

Note: If we create the pre-multiplied table it would be of size 20*10 or 200 elements. That isn't too big is it? We could optimize out 0's and it would only be 20*9 (20 digit locations are needed to process a 64 bit integer, and 9 digits 1 through 9 have to be premultiplied by 1*10^0...1*10^19.

Saturday, April 03, 2004 5:58 PM by temptress

# re: FunHi.com, a dating service, a new game, a bit over the top?

been a member for bout 4 weeks now. honeslty never had more fun, and i hate the web and never had any fun on dating sites or rating sites. not sure what makes it so much fun but it is!
Saturday, April 03, 2004 6:11 PM by Justin Rogers

# re: FunHi.com, a dating service, a new game, a bit over the top?

I'm definitely looking for feedback here, so I'll make some quick observations.

1. I'm interested in why it is so fun for those that do join. I know it is probably hard to explain, but if you can, I'm willing to hear you out in as much depth as you wish to use.

2. Embarassed to admit you are part of the site? I'm impartial, so feel free to contact me privately if you think you have important comments that would help better define exactly what this site is and why it appeals to people.

3. Note, I signed up just today. However, a short disclaimer. I've signed up on nearly every single dating site known to man. Not to use them, but to figure out more about the services they offer. I'm always researching various web ideas looking for that next exciting niche (unfortunately I'm a researcher and I give all my findings away for free, but that is the fun for me).

4. HotOrNot.com was the last site to do something similar and it was definitely *hot* for it's time. Is FunHi the next logical step drawing research from HotOrNot?
Monday, April 05, 2004 6:01 AM by Jon Hanna

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Arrays are slow? Is that a C# thing? In C and C++ we tend to use arrays for this kind of work to get a speed increase (constant arrays in static memory, creating them dynamically would indeed be super slow). In the case of going from a character stream to integers it wouldn't be that much of an increase (if you ignore the non-ASCII digits), but in other cases the difference is immense.

The table doesn't have to be an array, it could be a binary resource, though whether that's efficient or not in C# I couldn't say.
Monday, April 05, 2004 10:43 AM by Justin Rogers

# re: Doing research to make DWC.Algorithms.NumberUtilities extra fast... C# for now, but maybe some IL later?

Well think about it for a second. Array access in C# has to be safe, so they have to perform a bounds check (this can be optimized out in some cases). Then you have to do some memory manipulation to get the indexed location (multiplication of type size or stride by index?). Now compare this to popping a constant on the stack and doing a single multiplication. Anyway, I really loved your ideas. Here are the links where I talk about implementing both your ideas and some unsafe code using arrays. I just can't get anything faster.

http://weblogs.asp.net/justin_rogers/archive/2004/04/03/107177.aspx
http://weblogs.asp.net/justin_rogers/archive/2004/04/03/107200.aspx
Monday, April 05, 2004 12:21 PM by Matt Hawley

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

[Standing Up & Clapping] Very well said, and I completely agree.
Monday, April 05, 2004 12:57 PM by AndrewSeven

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

Why doesn't the enum itself support this?
enumName.Parse(string enumName) or .FromString


Most MS help suffers an amost unavoidable situation : The better you know the answer, the easier it is to find.

If you don't already know the answer, better Google it. Google is only concensus, not truth, so then you need to go back and try to understand some more.

http://weblogs.asp.net/andrewseven/articles/Ramblings.aspx



Monday, April 05, 2004 1:07 PM by Justin Rogers

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

I think the disconnect in the case of finding ways to parse strings to enums is probably that individuals think the enum is a language feature, and not backed by a system feature (System.Enum). I can definitely see that. Having the instance method as you point out would fix the problem since it would appear that the *language feature* had a *code feature* for parsing enumerations in some way.

I guess in the end, they wanted to leave enum semantics on the enum itself to the language (enum as a lightweight constant), and special enum processing outside of the language (System.Enum statics) so it didn't interfere with the basic operation of the language.

I guess enums are a mixed bag of nuts, but this happens all the time. Just today someone asked how to show a Form fullscreen in one of the newsgroups (tell me you can't get that information with a 5 second search), how to get a textual substring (bejeebus, they don't even look in the docs), and my pet peeve the illustrious "Does anyone have a comparision between managed and unmanaged DirectX", because we don't answer that one 50 times a week.
Monday, April 05, 2004 2:14 PM by Panos Theofanopoulos

# re: [Partial Rant] - A sick cyclic redundancy when providing answers to questions, versus questions asked...

> Has it been well answered? Yes it has,
I eval all those answers close to 5 (with 10 as the perfect).

I see nowhere an answer that doesnot use catch (thus very slow) and is case insesitive, something similar to double.TryParse
Monday, April 05, 2004 3:24 PM by Adam Hill

# re: A mention of C# in a mainstream, widely accepted game programming series book?

We should submit a few MDX articles next year :)
Tuesday, April 06, 2004 3:50 PM by Justin Rogers

# re: A mention of C# in a mainstream, widely accepted game programming series book?

They've never invited me into the process of article submission. However, I'd be more than happy to submit articles for both Game Programming Gems and AI Game Programming Wisdom.
Tuesday, April 06, 2004 8:44 PM by Funhi guy

# re: FunHi.com, a dating service, a new game, a bit over the top?

I think what puts the Fun in Funhi is the social networking that becomes very apparent once you start to get hookups and get to join some funhi vip forums. this all happens behind the scenes. there are many ways to communicate. I suppose some people might think they are going to get some gorgeous girl to fall in love with them by buying them a virtual 'diamond ring', however that happens on dating sites that offer no gift exchange. Yes, it is a game, I have inquired about other features, and when i first signed up, it said 'FUNHI BETA' I have been since notified that there are more additional gaming elements coming. maybe the idea of donations are what will drive the R&D for the next phase of this site? Another aspect of what makes it interesting and a time waster, is my curiousity. its almost a voyeuristic approach to seeing the next person that will signup. I have bought credits, which is the funhi currency, and actually returned gifts to people that for no apparent reason to me other than, i interested them in some way, bought me a gift. Afterwards, i noticed my status changed from being a mere "Playa" to a "VIP" wow.. heeh.. They have a moto that says 'Hater Free'. This probably makes people feel much safer knowing someone is actually looking out for thier well being.
Tuesday, April 06, 2004 10:50 PM by TrackBack

# Solutions to the Phraser programming challenge

Tuesday, April 06, 2004 10:50 PM by TrackBack

# Solutions to the Phraser programming challenge

Wednesday, April 07, 2004 10:44 AM by Ron

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

So far, my uses of Enum.Parse are in evaluating the selection of an item in a listbox previously populated with the enum's values via GetNames(). I don't expect a failure as the listbox is limited to just the enum's values.
Wednesday, April 07, 2004 3:31 PM by Panos Theofanopoulos

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

In my case it was the result of a refactoring of some big switches with string values. I decided that i will use enum(s) and then switch on them (after i succesfully parse them).
Wednesday, April 07, 2004 3:37 PM by Justin Rogers

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

C# refactors large switches with strings into internally managed hashtables for quick switch jumps. Did you do the refactoring for readability or for performance? Did you find the performance better on your refactored sample? If you did this for performance reasons, then do you parse strings into enumerations a lot? Basically, would any of the enhancements above be of use to you?

Also, if you could generalize your refactoring into a repro case that would be very awesome. If not, if you could provide numbers for how large the switch statement actually was, so we might examine what is more performance, the privately used hashtable or the overhead of the enum parse and switch.
Wednesday, April 07, 2004 4:42 PM by Panos Theofanopoulos

# re: How do you find yourself using Enum.Parse? Questions in order to promote a better solution...

Too many questions !!!!

1) C# switch is not case insesitive, you have to do a ToUpper on the string before the switch.

2) Both

3) Yes

4) Yes

5) I already use reflection :(
to avoid the catch

6) i refactored switches that had from 10 up to 50 strings, but the factor parses/switches is not always 1/1 if that's what you are asking. the result of the parsing may stored and switched several times in my case.
Friday, April 09, 2004 12:16 AM by Chris Martin

# re: McWhat? McVeggie? Tastes good and has a low carb content. Can't be true, can it?

Now this is something that I'd thought I'd never see. A veggie recommending McDonalds. If I were still a veggie, I'd virtually smack you across the face. :)
Friday, April 09, 2004 12:47 AM by Justin Rogers

# re: McWhat? McVeggie? Tastes good and has a low carb content. Can't be true, can it?

Yves is virtually the best vegetarian food product company in the US. After making vegetarian food products for 60 years now, I think Mc D's did the *right thing* here and is actually selling a worthy product.

I understand the disgust. I tend to be a *puritan* vegetarian, eating very few of the meat-like foods. However, I'm going to make an exception in this case, probably about once a month.

I still need to figure out if the fries are truly Vegan... (check out the three period ellipses ;-)
Friday, April 09, 2004 2:39 AM by TrackBack

# re: Mutable reference types should not be read-only fields

Friday, April 09, 2004 3:39 AM by Shannon J Hager

# re: McWhat? McVeggie? Tastes good and has a low carb content. Can't be true, can it?

I began the path back to eating meat with sushi. It was as pure as you can get with meat. I have come to the conclusion that the measure of quality of food is the measure how how raw you can eat it. If a food can't be eaten raw, I have to say something is wrong with it.

But the major lesson I learned upon growing a love for sushi, which lead to a love for good, rare quality steaks, was that I was able to give up meat because my whole life I had eaten badly cooked pieces of mediocre-to-bad meat. I think I was 24 before I had a great steak. I actually thought that the stuff I grew up on was as good as it gets, boy was I wrong.

I still say that the major cause of vegetarians is bad cooking. I know people that still put steak sauce on steaks. If you have to put sauce on it, you cooked it wrong or it was a piece of meat that isn't really worth eating.
Friday, April 09, 2004 4:24 AM by TrackBack

# .NET Immutability Tip #1: Nothing is immutable.

Friday, April 09, 2004 11:12 AM by Henry Erich III

# re: Rob Rylea on autocompilation of XAML containing code within IE

I see this compilation issue as alot like Web Forms are done now. You have the "HTML" file and the "Code-Behind" file. Of course you must compile the code. But like now, you can always go and change the .aspx "HTML" without recompiling.
Did I just ramble too much?
Also correct me if I am wrong about being able to change the XAML "markup" file after compilation please. Right now that is just my theory because Im at work and cant test it =]
Friday, April 09, 2004 9:33 PM by TrackBack

# A fun lunch with a couple of Rotor fans

Sunday, April 11, 2004 6:28 AM by Frans Bouma

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Hardcoded ordinals are error-prone. You can however cache the ordinals: in the first record, grab the ordinals for each name, and store these in a table (hashtable or other). Every next row, you use the ordinals cached. You can implement even more faster constructs. (However a hashtable is pretty fast).

What's often forgotten though is that dr.IsDBNull() is very very very slow. Better do:
bool isNull = (dr[ordinal]==System.DBNull.Value);

Saves 20% of speed.
Sunday, April 11, 2004 6:37 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

My point for enumeration based ordinals was in production level applications. Once you've finalized a database interface, the error level of using enumerations to identify columns starts to get continuously smaller. Again, the point in case for the ASP .NET Forums system, of a 7% increase was HUGE.

Code spit automation programs take even more of the error metric away because you can use the results of a data reader to spit the relavent code in terms of enumeration for ordinal mapping, mapping of data to strongly typed object properties, and expanding out DBNull checks as you display above.

Honestly, saying it is error prone is like pointing out that the usage of pointers is error prone. While I agree highly that it CAN be, there is definitely a time to user pointers, and certainly a time to use enumeration backed ordinals for the data reader.
Sunday, April 11, 2004 7:10 AM by RichB

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

7% is an OK increase. It's not great though.

Prior to spending the time replacing the code with ordinals, I'd first look at the SQL I was using. It's quite likely I'd get 50% perf gains from tweaking the SQL. I would also look at internal algorithms - I once re-wrote a complex algorithm which reduced accuracy in the algorithm, but changed it from a O(n^2) to an O(n) algorithm, reducing calculation times from minutes down to seconds. Luckily the reduced accuracy wasn't important.

Only after considering all this would I then look at replacing ordinals.
Sunday, April 11, 2004 7:18 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Depends on the context of 7%. In this case the context of 7% was on the fully optimized SQL set used for the ASP .NET Forums system. This was after massive architectural changes to the database, index tuning, upgrading RAID hardware, etc...

The point here is that thousands of calls per second are being wasted on FieldNameLookup, period. The majority of people may not need real performance, others do. For those that do, or simply want to ship a leaner component, then enumeration backed ordinal lookups are simply that much faster and that much leaner. Can't beat a relatively free performance gain.

So no more assumptions here people. No more spending hours looking at your SQL, just spend 10 minutes and make a frigen enum ;-)
Sunday, April 11, 2004 7:40 AM by Plip

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Two words: Magic Numbers.

*Shudder* ;)

I do not want to maintain code that's littered with Ordinals, especially on Data Schema's I'm not currently familiar with.

http://www.programmingresearch.com/solutions/QAJDOCS/Patterns/no-magic-numbers.html

If you *MUST* user Magic Number please please please use Enums like Justin suggests, or use declared variables: -

int FieldNameThatMakesSense = 9;
string moo = DR[FieldNameThatMakesSense].ToString();

I wasn't aware of the performance loss though, thanks for that, I'll certainly be passing this information on to the rest of the Developers on my team.

And absolutely no need to apologise, it was bad code to make my point look prettier ;)
Sunday, April 11, 2004 7:45 AM by Plip

# re: Interesting article listing game genres, including Programming Games, Demos, Utiltities, etc...

What's scary is that there are games without Genre, they're mixes of several Genre's. I saw that when Gladiator: Sword of Vengence was pushed to Sony and Microsoft for sumission whilst I was at Acclaim.

None of us could agree on what Genre it was in.
Sunday, April 11, 2004 7:49 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Yep, magic numbers makes it sound so dirty. Definitely write yourself a tool that runs your query or SPROC and uses the results to create the enumeration. If you create all of your enumerations using an identical process you can even validate over time that people aren't screwing things up by changing column ordering. This is definitely a concept of performance versus resilience. Depending on your comfortability with your own setup, you'll pick one over the other.
Sunday, April 11, 2004 7:54 AM by Justin Rogers

# re: Interesting article listing game genres, including Programming Games, Demos, Utiltities, etc...

Yes, in those cases you have to classify each game or playability element as a separate genre for purposes of classifying the game. When you do that, you start to define a new genre, thus creating a Venn diagram of the intersections of many and all genres.

We had a thought about something like Pac Man. For the time it was a kind of action game, but would probably be considered a puzzle game today. What about Pong? I would classify that as a sports game, but others would classify it differently. The old games are interesting because each new game created it's own temporary genre since the playability, graphics and techniques were so new.

At the end of the day Andy Smith defined a game genre in a manner I find befitting. He noted that a game's genre is defined solely on the skills it takes to play the game. While we left the definition there, I'm thinking it would be nice to define the skills for each genre to see how well this stands up.
Sunday, April 11, 2004 8:14 AM by Plip

# re: Interesting article listing game genres, including Programming Games, Demos, Utiltities, etc...

Yeah it must be hard for people today though, the genre they decide to slot their game in to predetermines their audience.

I'm sorry I missed the chance to chat with you over the subject, sounds like it was a good night! :)
Sunday, April 11, 2004 9:55 AM by Patrick Santry

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

I like the use of enums for managability. Another reason for the use of ordinals was on one project I worked on where I had a search query, and then I wanted to bind an entirely different search to the same UI. Since all of the field names were referenced in the new query I had to write it so it would match up with the original query, for example:
Select thisfield As theOtherFieldName, thatField As thatOtherField, etc...

I had to do a lot of writing in my sproc to make it match up with what the code was expecting. Whereas by using ordinals, it wouldn't be a big deal and save me some time writing.
Sunday, April 11, 2004 10:35 AM by Jim Ross

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Hurray for Plip! Magic numbers, indeed. Interesting that this young guy from across the pond uses the same term I've been using (with equal scorn) for years.

Re: addressing via an enumeration. I always start out that way. In a typical project, once you've coded it with strings, it's going to be a rare thing if you have the time to go back and replace them. Not to mention the additional debugging cycle you introduce. Ugh!. Start out defining your enumeration, and code using it from the start. For me, it's a whole bunch easier to change an enumeration than embedded magic numbers. And when you're done, you're done.
Sunday, April 11, 2004 11:51 AM by Phil Winstanley

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

I still have a concern here that there is a definate contract between the DAL and the Stored Procedures, if a DBA goes in and moves a column they will break your code, most likly causing Conversion Exceptions in your DAL/BAL.

This isn't the case with Named columns, (mind you there is nothing to stop renaming a Column however it's more likly a new Column will be added to a select, if that goes at the beggining everything else is out by one ... I see gremlins on the horizon)
Sunday, April 11, 2004 12:04 PM by TrackBack

# some .NET blog tips

Sunday, April 11, 2004 3:10 PM by Dave Burke

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Justin, Thanks for pointing me in the direction of using Enums for column ordinals. Never thought of it, but will definitely pursue.
Sunday, April 11, 2004 5:15 PM by Nat Luengnaruemitchai

# Nahhhh dr[dataColumn] is faster

When you populate the data into dataset or whatever, you can assign the columns that you want to use in DataColumn variable and use it from there.... It is even faster than dr[intOrdinal]

DataColumn dcUserName = dt.Columns["userName"];

for(... ;... ;...)
{
blablabla....(dt.Rows[i][dcUserName]);
}

this way, you will achieve both performance and compatibility if you change the SQL statement in the future
Sunday, April 11, 2004 5:30 PM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

That would appear exceptionally intelligent if we weren't talking about the DataReader class. The DataReader class doesn't have a Rows collection, nor does it accept DataColumn as a default indexed property overload.

We are talking about some raw performance here, not the case where you are using the higher level APIs.
Sunday, April 11, 2004 10:28 PM by TrackBack

# Developing a Windows Forms Wizard...

Sunday, April 11, 2004 10:28 PM by TrackBack

# Developing a Windows Forms Wizard...

Sunday, April 11, 2004 11:30 PM by Jacob Morgan

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

I know we have beat O/R mappers to death around here, but this discussion really exemplifies their usefulness.

I don't use O/R mappers to avoid writing SQL. I don't use them to avoid writing domain objects.
I don't use them to avoid building database tables.

I use them to avoid writing hundreds of redundant assignment statements that lead to synchronization hell. Any O/R mapper worth a flip will give you ordinal based lookups based on the explicit list of queried fields. Optimizations such as this can counter some of the lost optimization of dynamic SQL.
Monday, April 12, 2004 5:08 AM by TrackBack

# re: IsDBNull() alternative

Monday, April 12, 2004 2:09 PM by Henry Erich III

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Excelent post and argument!

My Opinion:
Code Gen: Numeric Ordinals
Hand Gen: String Ordinal Lookup

My Opinion Also:
Enums or Constants would be a waste of time for readability gain.
Tuesday, April 13, 2004 12:14 AM by Adam Kinney

# re: Curious about the lives of up and coming game developers and their schooling?

No RSS :(
Tuesday, April 13, 2004 12:49 AM by John Winkle

# re: Curious about the lives of up and coming game developers and their schooling?

Yeah, eventually I will have to find somewhere with RSS but for now this will work. Once I get into a good job again I will find a nice hosting solution and get a good website with blog up and running.

John
Tuesday, April 13, 2004 4:41 AM by Justin Rogers

# re: Curious about the lives of up and coming game developers and their schooling?

Darnit John, you just had to pick something that didn't allow RSS feeds eh... I'll work on getting .Text up on Games4dotnet.com so we can just host the gaming stuff there.

Now we just need Tobi on so we can get some graphics done ;-)
Tuesday, April 13, 2004 5:09 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

The actual framework issue behind IsDBNull being as slow as it is.

http://weblogs.asp.net/justin_rogers/archive/2004/04/13/112100.aspx
Tuesday, April 13, 2004 5:27 AM by TrackBack

# IsDBNull

Justin points to Frans Bouma noting the performance hit using IsDBNull() in .NET and a workaround....
Tuesday, April 13, 2004 6:35 AM by Andy Smith

# re: Found a relatively accurate comparison of game world size to real world size. Games are small!

That's some crazy stuff, but I wonder how size is measured...
Do they measure by getting the size of a person in the world and compare that to use in our world? Or do they measure by walking speed in their world compared to average human walking speed in our world? The two different metrics could give vastly different results.
Tuesday, April 13, 2004 6:40 AM by Justin Rogers

# re: Found a relatively accurate comparison of game world size to real world size. Games are small!

Most games have published metrics. AC for example had published metrics as to how large zones were, how many zones, etc... Obviously the games that encompass actual real world arenas can easily be measured by simply mapping their bounds as compared to an actual map.
Tuesday, April 13, 2004 6:55 AM by kvr

# re: General texture mapping resources... Some good math explanations.

Don't forget this one (from Chris Hecker
of Game Developer Managazine)

http://www.d6.com/users/checker/misctech.htm
Tuesday, April 13, 2004 8:10 AM by TrackBack

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Tuesday, April 13, 2004 8:10 AM by TrackBack

# re: IsDBNull() alternative

Tuesday, April 13, 2004 9:50 AM by Brad More

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Might I suggest Convert.IsDBNull? Performs similarly to the alternative mentioned in one of your references and was graciously provided by our friends in Redmond just for this purpose.
Tuesday, April 13, 2004 12:23 PM by Drew Marsh

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Actually, the fastest code I can imagine would be:

reader[ordinal] is DbNull

Because there is only ever one instance of DbNull you don't need to check against Value, you ca just test the type. Plus, when you check against DbNull.Value you end up re-boxing whatever comes out of the reader to pass it to DbNull's Object::Equals override.
Tuesday, April 13, 2004 12:26 PM by Drew Marsh

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Sorry, small correction... you don't have to box because it comes out as Object already. However, I'd bet dimes to dollars that using "is" is faster than calling Equals.
Tuesday, April 13, 2004 2:42 PM by Shane O'Dell

# re: A slightly better WinForms wizard, and slightly more work.

One thing about adding panel navigation instead of a series of forms is that I don't see a good way to use the VS Designer to create the different panels. Any thoughts on this?
Tuesday, April 13, 2004 4:28 PM by Justin Rogers

# re: Since I can't stand unvalidated performance claims, I looked up the reason behind IsDBNull performance

Convert.IsDBNull looks really good. Under the covers the only additional operation it performs is checking for an implementation of IConvertible and using that to possibly determine if the value is Empty. I'm inclined to worry about this extra goo.

The is keyword is backed by a not so fast crawling instruction isinst. isinst has to do a lot of work, so I could never imagine it being faster than the equality methods.

Be careful of small instructions that hide large details like walking down an object's derivation table.
Tuesday, April 13, 2004 4:37 PM by Justin Rogers

# re: A slightly better WinForms wizard, and slightly more work.

I was hoping someone would point this out. Form navigation does have the benefit of being designable, but at a pretty large cost. The size, location, and navigation UI all have to be replicated in some form or another. Of course you could use the reverse theme that each panel support a UI root where the navigation could be tossed into.

The biggest issue with form wizards is that you lose the forms on the back-end if the get closed. You see, it is easy to Close a form. They provide UI for it even. However, it isn't easy to Close a panel and so you are pretty much ensured that your panel with the controls that contain your data will be there.

That brings us back to designability. The only true designability would be to implement our own designer. That is for both Wizard Forms and the Panels. That might be something I get to in the article series. Another option is to write generic layout code that follows the design guidelines from MS for wizards and dialogs. Not even the WinForms designer does this for you, with it's goofy grid system. You'd have to manually type the stuff in or be a diligent drag and drop artist.

I think this layout designer might be just what is needed for the panels. As for the Form, you can design that in VS (I'll pop out a better version for the 4th article) if you'd like.
Tuesday, April 13, 2004 5:40 PM by Justin Rogers

# re: General texture mapping resources... Some good math explanations.

Yeah, that is a set of ancient articles. Good job pulling that link out of the swirling mass that is the web.
Tuesday, April 13, 2004 6:00 PM by Mickey Gousset

# re: Virtual sales now linked to grief playing? This argument always turns into a mud slinging contest.

You hit the nail right on the head with this one! Good Job!
Tuesday, April 13, 2004 9:34 PM by Roger Heim

# re: A slightly better WinForms wizard, and slightly more work.

Wouldn't UserControls give you the best of both worlds? You would get the benefit of designability and avoid the problem with forms being closed.
Tuesday, April 13, 2004 9:49 PM by Justin Rogers

# re: A slightly better WinForms wizard, and slightly more work.

Yep, with the current framework there isn't any reason you can't use Inherited Controls (to inherit from Panel). Not having time to test how easy this would be I chose not to throw it out there.

Since people really want designability, I'll mark that as my next article and talk about how to make this framework truly designable. Perhaps using UserControl's, perhaps using some other feature.

Wednesday, April 14, 2004 12:17 AM by Scott Mitchell

# re: [Rant] Getting rid of tele-marketers permanently, a case by base.

I like to play songs using the assorted tones of the touch-tone phone before hanging up. One time, though, a telemarketer called me back after I hung up, and mashed her hand into the keypad and uttered a few choice words herself. :)
Wednesday, April 14, 2004 2:08 AM by Justin Rogers

# re: [Rant] Getting rid of tele-marketers permanently, a case by base.

This begs a response from me. The legality of the callback in this case is quite questionable. Just the idea that they would be allowed this sort of option is outside my grasp. So much for my call being monitored for quality control ;-)
Thursday, April 15, 2004 3:22 AM by John Winkle

# re: Curious about the lives of up and coming game developers and their schooling?

Ok, got an RSS feed setup through Feedster.com for those interested.

John Winkle
Thursday, April 15, 2004 9:15 PM by TrackBack

# re: The SLAR on System.Array

Friday, April 16, 2004 3:32 AM by Tommy

# re: Performance: Different methods for testing string input for numeric values...

for(int i=text.Length-1; i>=0; i--) is faster than for(int i=0; i<=text.Length; i++) because it has to access the .Length-property only once. And in this case, the direction isn't really important.
Friday, April 16, 2004 6:08 PM by ChessMess

# re: Tips for noobs at E3! Based on a request by a reader.

Sweet! Thanks for the tips this is mundo helpfull!!

Some followup questions...

1. I've reserved a hotel near the airport, any possible problems with that?

2. Is there fee's to park at the convention center?

3. I plan on taking a digital camera, digital camcorder and laptop with me into the exhibits (I don't have workshops)... probably get a backpack to put them in. Any problems with that much stuff? Could I get 'online' with my laptop there to report back news and info?

4. How much swag does one actually get? lol ... do I need to take a duffle bag as well? :)

5. How close can you get to a booth babe before they hit you with pepper spray and start yelling for security?

6. Is it usually warm enough for shorts around that time or pants? What attire do people generally wear (Jeans vs Casual Dress)?

7. Parties... are there any or is everyone way to tired by the end of the day to do nothing more then hit the hotel and crash?

All I can think of at the moment... Thanks for the great info!

ChessMess
Friday, April 16, 2004 9:37 PM by Justin Rogers

# re: Tips for noobs at E3! Based on a request by a reader.

1. Shouldn't be. Make sure you can get a shuttle service to the convention center. A lot of places have them.

2. I don't recall a fee the last time I went, but it doesn't mean there wasn't one. Keep your info ready at all times. This is a trade show (not open to the public) and they will check it.

3. I've never gotten *on-line* at E3. I guess it is my chance to get away from all that stuff. However, as long as you don't mind carrying all of those items around, take em.

4. Depends on your character I'd assume. The first time I went I walked away with a ton of t-shirts, little toys, etc... Last time I went, I walked home with 1 t-shirt. I didn't *make the most* of the exhibitions last time. Dumb me.

5. As close as they let you get man! A lot of times you can get up close pictures, extra swag, and all kinds of bonuses from talking with the booth babes.

6. People wear all kinds of stuff. Most people try to wear the more stylish clothes, especially the exhibitioners. I've seen people wearing themes for their favorite games, suits, or casual. I'd say wear something comfortable. Hell, I went in wearing fuzzy slippers and my pajamas one time.

7. Again, up to your kind of personality. There are lots of impromptu dinners and such. You are definitely going to be in a party town. Make the most of the night-life there too if you want. I generally use the night after the last day and the next day for any hard partying.

I have to say this is kind of unorthodox for me to write about. The concept that people have questions about trade-shows is pretty strange to me. The first time I went was in 99 I believe and I literally just drove down and solo'ed the whole event walking around going to workshops and attending sessions. I got loads of swag the first time I went. Have to say it's been my most successful trip to E3.
Saturday, April 17, 2004 11:55 PM by Andy Smith

# re: Fast absolute, and statistical line counting algorithms for use with progress notification

great post justin. if you are going to add a toc, maybe you should make em linky? :)
Sunday, April 18, 2004 3:53 AM by Justin Rogers

# re: Fast absolute, and statistical line counting algorithms for use with progress notification

I tried doing linkies before, but the system doesn't like me and it renames a bunch of stuff between HTML and design view. Namely the hyperlinking system fully qualifies the path, which is erroneous.
Sunday, April 18, 2004 11:51 AM by TrackBack

# Early Morning News 3

Sunday, April 18, 2004 8:13 PM by TrackBack

# re: Efficiency of iteration over arrays?

Monday, April 19, 2004 11:59 PM by Lavos

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

Interesting stuff. It's late to add this comment to the party, but I will anyway.

Using enumerations is great, but as was illustrated within the first month of my current job, someone at our provider inserted a column into a single resultset that broke a lot of our production applications that had code dependant on ordinals.

We can blame that on poor testing on our part but it still follows that DBAs and upgrades are going to break old applications that had been chugging along since forever if you depend on ordinals, and thus increases maintenance costs.

With that said, ordinals really do seem to help you out with speed :)


My personal preference if the gains were that important, would be to have a singleton class that more or less has fields/properties/whatever that I can reference by name and do runtime checks (via string names or whatever) to populate with the proper ordinals.

(Or, have all your enumerations defined in a seperate assembly that you can "upgrade" independantly to keep pace with the latest changes.)
Tuesday, April 20, 2004 12:57 AM by Justin Rogers

# re: People are still using dr[stringName], when dr[intOrdinal] is soooooo much faster.

It is equally easy to remove a single column from a result set as add one. Removing a single column would break any form of column indexing method, just adding one happens not to.

I'd argue that any competent DBA would create a new procedure before he'd add columns to an existing result set. If he does add columns why would he add it to the beginning of the column set or in the middle rather than append it to the end.

A small investment in user education for a 7-10% performance gain is well worth it. And since the same DBA could just have easily removed a column, I can't see a valuable reason behind not using the oridinals.

Proper versioning of enumerations is important in any case. Having singleton classes and levels of abstraction does nothing but hurt performance. In many cases the extra indirection of using properties is going to hurt perf a bit, as is doing runtime checks for proper ordinals. There are some solutions, but they all involve dynamically generating optimized code using a CodeDOM or Reflection.Emit and I don't think the majority of users are ready for that step (though I use it all of the time for my own stuff).
Tuesday, April 20, 2004 12:58 AM by JollyGreenGiant

# re: Making money in commercial games Part 2: The programmer.

Intersting aspect and so true. Like with anything that requires human creation, humans break it or make it better. I see the potential here for a little profit myself. (after all, I told my wife I was gonna make some money playing this thing 24 hours a day, I better start)

But, as with anything on this earth, man is not created to design permanent things.. but created to destroy himself and all his surroundings. Eq is just a hyper version of what is to come for all mankind. We will build it and destroy it very rapidly..

I would prefer to play eq on an automated bases, checking in with it daily and seeing my work in progress.. Perhaps i could then have time to give love and time to my family, while i still enjoy my sucess in the game.

Plese let me know of any programmers interested in making money selling their works and creations that would make eq soooo much funner for me and my family.
Thursday, April 22, 2004 2:39 AM by tracdalat

# re: Adding a design time dialog and creating a VS project sample

hey, I've follow these articles from the beginning. Thanks a lot, it's very helpful; and great (of course)
Thursday, April 22, 2004 5:11 PM by TrackBack

# Loopy Decisions

<p>In his blog, <span style="font-style: italic;">Better Living Through Software</span>, Joshua Allen looks at three different ways to loop in C# from an optimizing-for-performance point of view:</p><p><code>foreach (int i in foo) {}<br>
<br>
for (int i = 0; i &lt; foo.Length; i++) {}<br>
<br>
int len = foo.Length; for (int i = 0; ...
Saturday, April 24, 2004 10:17 PM by TrackBack

# Take Outs for 24 April 2004

Take Outs for 24 April 2004
Monday, April 26, 2004 8:27 AM by Justin Shen

# re: Code-Only: Winforms Wizard Series Article 5 (Managed C++)

the syntax of managed C++ is being refined, the new syntax looks more elegant.
Monday, April 26, 2004 8:27 AM by Justin Shen

# re: Code-Only: Winforms Wizard Series Article 5 (Managed C++)

the syntax of managed C++ is being refined, the new syntax looks more elegant.
Monday, April 26, 2004 8:36 AM by Ron

# re: For those using the Wizard Framework, do you want a solid framework, or solid articles?

I've primarily used it to help refactor a wizard I'd already written. I was already in the process of abstracting out a lot of the internal workings, but your articles pointed out a few things I was missing.

As for how I was using the articles, I've primarily built them as-is and then took the time to study what each part is doing. I could then add in features I wanted in a consistent manner. For example, I wanted the panels to trigger when the Next button is available (to require user input before proceeding).

I've enjoyed the articles. Thanks for writing the series.
Monday, April 26, 2004 8:37 AM by Justin Rogers

# re: Code-Only: Winforms Wizard Series Article 5 (Managed C++)

Don't know if I'd call it more elegant. I kind of like the new property syntax, but for the managed pointer changes (namely ^), I can't say that I like that at all. I could have made the above code much more *elegant* as well, but I chose for the quick port, and not necessarily the most elegant port.
Monday, April 26, 2004 10:08 PM by TrackBack

# Wizards in .NET with C#

Justin Rogers is the wizard man. He already wrote his sixth article in the series, I've just started with the first one, basically copying and pasting his code into VS.NET, and got a simplistic wizard app to work right away....
Tuesday, April 27, 2004 12:58 AM by Jesse Ezell

# re: I'm going to do something unfair to make a point: int.TryParse versus DWC.NumberUtilities.

Yah, too bad they left them out of the first release... which is really strange, considering that they remembered to put in double.tryparse...
Tuesday, April 27, 2004 10:13 AM by Daniel Jin

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

I've learned that optimization is actually performed to JIT away the bound checks. For example, in the following code

for( int i = 0; i < array.Length; i++ )
{
// do something with array[i]
}

JIT compiler does a pattern recognition on that and JIT away the bound check typically occurs on array[i]. I don't know how ArrayList is handled since there's a bound check on ArrayList indexer in addition to the underlying array. but I've learned now to trust that JITer will (or at least eventually will) do the right thing. as an outsider, you never know what JIT is designed to do, some manual optimization could actually be counter productive and hurt the performance.

about stack, queue, and list. theoratically, they should all be linked list based, not array based. I sure hope the queue is not implemented with an array, otherwise popping off the head is going to be really costly on every call.
Tuesday, April 27, 2004 10:17 AM by Raymond Chen

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

If access to the internal array were made public, then that would require List<T> (for example) to use an array forevermore. Do you really want to lock in the internals of List<T>? Suppose that somebody discovers that linked arrays are faster than a single array. Do you want to prevent them from using that new faster data structure?
Tuesday, April 27, 2004 12:56 PM by TrackBack

# re: The change from Hashtable to Dictionary

Tuesday, April 27, 2004 2:41 PM by Drew Marsh

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

I agree with Raymond and would point out that if your Decompose method returned the actual array, then it would have empty slots. Remember the array is grown dynamically, but it is grown according to an algorithm that will (hopefully) reduce the number of times that it must be grown. If you've added 75 items to the List<T> and it decided that it had to grow a few times, the internal array might actually be 200 items. How would you know where to stop if you got your hands on that array? Length would be 200, so you can't just go by that. If you're storing a reference type, sure you could stop when you find a null, but what if you're storing a value type such as Int32? Would you stop when you find Int32.default? Well what if 0 (which is Int32's default) is a valid value??
Tuesday, April 27, 2004 3:50 PM by Jerry Pisk

# re: Darnit, they break the Hashtable (err Dictionary) then ask us how we think they should fix it...

Why don't you keep using Hashtable if you don't like the templated Dictionary?
Tuesday, April 27, 2004 5:32 PM by Justin Rogers

# re: Darnit, they break the Hashtable (err Dictionary) then ask us how we think they should fix it...

Dictionary is quite a bit faster actually. So I'd much prefer to use it, avoid the boxing and casting costs, and still not have the exception.
Tuesday, April 27, 2004 5:43 PM by Justin Rogers

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

Drew: As mentioned in my post, the esoterics of length are up to you. The intelligent Decompose could have an out parameter that stores your length for you, or you could store the length before you do a Decompose (doing this is more dangerous since the length can change).

Raymond: Very interesting to point out the performance there. Of course a linked array would be exceptionally slower on indexing (one extra level of indirection), on CopyTo, marginally slower on enumeration. So yes, I'd love to lock them into using a single array. The only thing linked arrays does fix is the grow factor. So rather than guesstimate that they might make List<T> faster, go ahead and make a LinkedArrayList<T> that uses the alternate storage method and depending on the user's scenario, they'll make use of the appropriate class. Don't take away my options by limiting me.

Daniel Jin: I guess Joshua Allen was correct. The JIT optimization is for array's because Length can't change, not for collections where Count (not Length) can change behind the scenes). You can't just JIT optimize away all bounds checks my friend, else they wouldn't have put them there in the first place.

As for Stack, it is best implemented using an array, since it grows from index 0 up. As for Queue, it is also best implemented using an array with head and foot pointer indices into the array. They may find it more appropriate to use a linked list for this guy in the future. So maybe, just maybe this one could be taken off the list
Tuesday, April 27, 2004 7:14 PM by Daniel Jin

# re: Since I'm on collections, strongly typed arrays, versus List<T> for very large collections... I want a Decompose method...

> You can't just JIT optimize away all bounds checks my friend

you are correct. I meant to say *sometimes* bound check can be JITted away.

> As for Stack, it is best implemented using an array, since it grows from index 0 up ...

Raymond definitely made a more convincing argument on that.
Tuesday, April 27, 2004 9:56 PM by TrackBack

# Real Time Colorizing - Some initial thoughts

Wednesday, April 28, 2004 12:41 AM by TrackBack

# Take Outs for 27 April 2004

Take Outs for 27 April 2004
Wednesday, April 28, 2004 12:41 AM by TrackBack

# Take Outs for 27 April 2004

Take Outs for 27 April 2004
Wednesday, April 28, 2004 6:27 AM by David Levine

# re: Darnit, they break the Hashtable (err Dictionary) then ask us how we think they should fix it...

I like the idea of being able to choose between a dictionary that throws and one that returns a sentinel value. But I want to be able to set the value to use so that I can use the sentinel value that is most appropriate (even for references). A null value is not always appropriate.

I haven't looked at the new language features yet so I don't know what Nullable<t> does - this may be more appropriate.

Wednesday, April 28, 2004 10:19 AM by Laurent

# re: Allegiance Source Code vs Quake II, you decide...

Why not to publish some pieces of your work already ? I am sure a lot of people would love to give you a hand.
Friday, April 30, 2004 4:13 PM by Chris Stueck

# re: DWC.Algorithms.NumberUtilities

How do I compile this under visual studio.net??I have not built a library yet.

Thanks.
Friday, April 30, 2004 7:49 PM by Justin Rogers

# re: DWC.Algorithms.NumberUtilities

You'd create what I think VS calls a new Class Library project and copy the above code into it's own .CS file and compile. It should just work from there.

If you want to include the source directly in your project (aka not as a library), you can do that also. Just make the above source its own source file and then compile your application as normal.
Saturday, May 01, 2004 12:14 AM by Daniel O'Connell

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Out of curiosity...what are the results you are seeing when you run this with int keys? I'm seeing Nullable win when the key is a string and integral win when the key is an int. It seems a strange result to me.
Saturday, May 01, 2004 12:59 AM by Justin Rogers

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Below are my results given varying input to the Dictionary. Using both String as a key and Int as a key shows that Nullable always wins still.

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.7310512

Start Dictionary<String, Int> Dictionary
Int Dictionary Complete: 00:00:00.9713968, 0

Start Dictionary<String, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.6509360, 0

Start Dictionary<Int, Int> Dictionary
Int Dictionary Complete: 00:00:00.4105904, 0

Start Dictionary<Int, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.2804032, 0

Using TryGetValue
Found: 50
Not Found: 11000001
Saturday, May 01, 2004 2:06 AM by Daniel O'Connell

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Strange, the result is still the opposite on this system. I wonder which is the odd ball.
this is my results for Int keys:

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.0156250

Start Int Dictionary
Int Dictionary Complete: 00:00:00.2343750, 0

Start Nullable<Int> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.3437500, 0

Using TryGetValue
Found: 50
Not Found: 11000001


and my results for string:

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.9062500

Start Int Dictionary
Int Dictionary Complete: 00:00:01.2656250, 0

Start Nullable<Int> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.6250000, 0

Using TryGetValue
Found: 50
Not Found: 11000001


What are you compiling with? I am doing just csc /optimize+, perhaps there is some minor difference in that?

Also, there are some interesting differences there...my speed for int dictionaries outstripes yours for the most part, however my results for string dictionaries shows your results as faster. Could this perhaps be the result of cpu specific JITing?

These results are rather stable here, I've run both multiple times...this is curious indeed.

Saturday, May 01, 2004 3:12 AM by TrackBack

# re: The change from Hashtable to Dictionary

Saturday, May 01, 2004 4:28 AM by Justin Rogers

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

The only thing I can think is that we aren't running the same test code. I've double checked mine just to make sure that I wasn't doing anything stupid. To humor me, please run the code at:

http://weblogs.asp.net/justin_rogers/articles/124403.aspx

If you are still getting the same results after running the test there then let me know. Since you coded up your int keying test separately (as I didn't supply the code for an int key above), there must be something we are doing that is different. Thanks.
Saturday, May 01, 2004 4:48 AM by Justin Rogers

# re: Nullable Versus Integral Data Types (C# 2.0 Test Code)

Running this code a few more times Dictionary<int, int> and Dictionary<int, Nullable<int>> are definitely close in performance. They've actually traded off in some tests putting one or the other at the faster speed depending on code layout within the class file (aka the JIT is having a huge impact, and is very inconsistent, so we'll have to wait for a later release for better numbers)

However, the Dictionary<String, int> simply performs poorly and I don't think there is any good reason for it. There isn't any casting, or boxing, or anything in the IL that would make it intrinsically different from the Nullable version and the Nullable version is actually doing extra work to convert the integer value into a Nullable<int>. The extra indirection through the op_Implicit call should actually make it slower.

When I started this I expected to examine how much worse the Nullable version was, and I was expecting it was only going to be a small marginal amount making the use of Nullable types acceptable. I didn't expect to find Nullable to be faster, nor should it be under any circumstances faster than the integral type.
Saturday, May 01, 2004 7:28 AM by TrackBack

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Saturday, May 01, 2004 3:01 PM by Bill Meyers

# re: Fast absolute, and statistical line counting algorithms for use with progress notification

fuck
Saturday, May 01, 2004 4:54 PM by Daniel O'Connell

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

Well, my code for the int test was basically the same asyours, just with type changes(I took your code, changed stringsForKeys to an int[] and took out i.ToString() for the most part). My results from your posted code, however still show the same thing:

Start Pre-Processing Strings
Pre-Processing Strings Complete: 00:00:00.9375000

Start Dictionary<String, Int> Dictionary
Int Dictionary Complete: 00:00:01.2343750, 0

Start Dictionary<String, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.7812500, 0

Start Dictionary<Int, Int> Dictionary
Int Dictionary Complete: 00:00:00.3437500, 0

Start Dictionary<Int, Nullable<Int>> Dictionary
Nullable<Int> Dictionary Complete: 00:00:00.4062500, 0

Running it several times is showing a fair amount of change on Dictionary<String,Int> sometimes its as much as 1.2 seconds, sometimes as low as .90.

Other tests I've run have shown string, int to take upwards of 12 seconds when using the Add method...its very strange indeed. This must be a manifestation of the JIT right now.

Out of curiosity, what kind of hardware are you running on?

Intel\AMD?

I'm running P4's here...
Saturday, May 01, 2004 7:31 PM by Justin Rogers

# re: Generic Dictionary and the KeyNotFoundException. Examining speed of Nullable types versus integral types...

I ran that on an Intel P4 2.8. I added some comments to the actual test case code that explain my findings after running the tests many times. It does appear that the JITer is very unstable since code reordering, addition of code, changing reference locality (aka, adding a usage of all four of the dictionary's and the string table to AFTER the tests are complete to ensure none of them are GC'ed while the tests are running).
Monday, May 03, 2004 8:38 PM by darrenford

# re: Software Design and Home Depot, who would have thought they went together so well.

Have you ever heard of paper? They sell it at Wal-Mart.
Tuesday, May 04, 2004 2:37 AM by Justin Rogers

# re: Software Design and Home Depot, who would have thought they went together so well.

I went the paper route for a while. I bought 50 notebooks at 10 cents a piece from Target during a going back to school sale or some other similar event. I figured I'd scope each concept in it's own notebook and/or group similar concepts into the same book. Drawing on paper isn't easy. You either use pen (and it shows up well and becomes unchangeable) or you use pencil (which shows up like butt, makes the paper dirty and still doesn't erase very easily).

You could say just throw it out if it isn't good enough and start over. I'd rather not take ten pages to spec something out that I could do on one with the ability to use multiple colors, easily erase and restart portions, the abilities of whiteboard are endless.

You tend to get a small high from working in close proximity with the markers for a long period of time. Some people get headaches, I just feel a bit better and keep on writing ;-)
Tuesday, May 04, 2004 4:56 AM by Joel B.

# re: People are confused by ApplicationContext in Windows Forms, but there really isn't any magic happening.

The message pump started by Application.Run must have a mechanism for telling the OS "hey gimme just these messages -- I don't want those other ones", and a mechanism for dispatching them (i.e. code must have a way to register event handlers with the message pump).

Well, it is unclear to me how both mechanisms are implemented.
More specifically, I would like to understand how things work and why there is an Application.AddMessageFilter() method but no way to register an EventHandler

Anyone who knows, or can suggest a book that explains the mistery, could just drop me an e-mail joeblk50@microsoft.com
Thx
Joel
Tuesday, May 04, 2004 5:02 AM by nfactorial

# re: If you could pick the reason why GDI should be .NET accessible, what would it be?

I haveto say my number one reason for GDI access is speed. GDI+ is suitable for loads of cases and provides really nice output. But the old GDI kicks its butt in speed terms (due to the hardware acceleration).

ie. A recent test app I wrote, I needed to draw a background in the window to make it look like graph paper. All those lines hit GDI+ badly (even though they were simple, horizontal & vertical lines). I wrote a aimple wrapper for GDI and used that for the graph background and the speed increased dramatically.

Of course, this probably wont apply when Longhorn's available. But for now its my number one reason, when you need it to be fast and you're not bothered about the quality the old GDI rocks :)

Though I do agree your reason is perfectly valid to me too :)

n!
Tuesday, May 04, 2004 5:08 AM by Dave L

# re: .NET Immutability Tip #3: Protect your properties AND your methods.

It's an interesting pattern. Q: Why wouldn't you want it to signal callers that the object is immutable when calls are made to change its state?

I suppose you can have different categories of immutable objects, ones where the caller does not care if an operation fails and others where an inability to change its state is a "don't care", but it's difficult for me to come up with use cases where the calling code really doesn't care about the success/failure of the operation. Where would you use this?

For the pattern where the caller does care the immutable object could signal the results by throwing an exception, which is non-performant, but there aren't any good alternatives when setting property values. If performance is an issue then the caller should check the immutable flag prior to the call, but that adds overhead too.

I prefer to avoid directly accessing field1 and the flag just to avoid the perf hit of invoking the property setter before the access. Unless you are in a tight loop invoking it thousands of times per second the overhead is unlikely to be a big perf hit, and the downside is that you now have lots of places that have to be touched if changes are made to the property access. This has the feel of a false saving.


Tuesday, May 04, 2004 5:08 AM by nfactorial

# re: If you could pick the reason why GDI should be .NET accessible, what would it be?

Oh, I may have misread you. I thought you were suggesting there *should* be a System.Drawing.GDI namespace. But seeing one of your later future posts it looks like you're saying there *is* a System.Drawing.GDI namespace :) That's cool.

Sorry, I don't have access to Whidbey yet. But I've now learned to read all your new posts before I start writing replies :)

n!
Tuesday, May 04, 2004 5:17 AM by Franck

# re: Allegiance Source Code vs Quake II, you decide...

I hope you don't really expect to blow the "old crappy OGL" just by using MDX... I'm a DX user usually but your statement is plain stupid. At first you should start by hoping you can come somewhere not too far from OGL performances... when will people stop feeding the $ beast?
Tuesday, May 04, 2004 6:31 AM by Justin Rogers

# re: .NET Immutability Tip #3: Protect your properties AND your methods.

Immutability patterns come in two forms. The first is that you have a single code piece capable of updating some data, and a bunch of others that have access to it. Think about a gaming scenario where I want to share the game state between all of the AI and other code.

That code can't change the state or it would
break the game, and it shouldn't. Nor should it try to change the state.

The second pattern is that a type is mutable for some time where many code points have access to changing it, and then it becomes mutable. In this case once the object becomes immutable your algorithms should stop trying to change the value. Maybe in your case though the fact that they do try to change the value is an exceptional case and something you need to find through debugging. Rather than toss exceptions you might find that you'd rather use asserts instead.

As for knowing an object is immutable, you can always extend a read-only property that returns the immutable flag. Then code can always find out whether or not the object is mutable and you no longer need to rely on the exception.

In general, I think most programmers are to used to owning all of their code. In most cases you are going to share a lot of code, and the mutability is for protection against many sources changing data when they shouldn't. Having an exception in this case could ruin a real-time simulation. After all, once the state is complete, the user shouldn't be changing it anyway.

I'll be covering some extended scenarios soon. MS tends to adopt the private derived encapsulating class (aka ArrayList.ReadOnlyArrayList which derives from ArrayList, but is private and can only be instantiated by calling methods on ArrayList).
Tuesday, May 04, 2004 6:40 AM by Justin Rogers

# re: People are confused by ApplicationContext in Windows Forms, but there really isn't any magic happening.

You need to pull out a book on the Windows message pump. The OS really doesn't have a mechanism for saying you only want certain messages, you build that into your message pump, and pump those you don't want down to DefWndProc.

As for Dispatching, everything is based on windows handles. Controls get messages because messages are sent to the HWND. UnsafeNativeMethods.DispatchMessageW(ref msg); is responsible for this process.

As for filtering, you can always filter messages by adding a message filter on Application.AddMessageFilter. You pass it an object that implements IMessageFilter which contains the PreFilterMessage method. You can process messages here. If you want to add event based notifications, you can stack those on top of PreFilterMessage by simply returning false (that way the message is dispatched as normal), and then firing an asynchronous event massing out the Message structure.
Tuesday, May 04, 2004 6:43 AM by Justin Rogers

# re: If you could pick the reason why GDI should be .NET accessible, what would it be?

Speed is the reason I'm assuming they added it. If you look deeply into V1.1 WinForms code you'll find that at some point they traverse the GDI+ to GDI border in order to render over the screen. GDI+ is great for rendering high detail components, but works even better when you cache the results and store the final result in a native GDI format for updating the display whenver you need to refresh.

My only major issues are that the GDI namespace is lacking a great deal of features, and rather than demonstrating GDI functionality, it is actual mirroring GDI+ functionality in terms of method names and feature set. This is pretty bogus since the two APIs have different problem sets they solve.
Tuesday, May 04, 2004 11:19 AM by Ron

# re: I keep telling everyone WinForms printing gets easier in Whidbey, but that doesn't help them today...

>Maybe it is time to make a larger investment in Crystal Reports?

Or maybe you could check out Active Reports. http://www.datadynamics.com/
Tuesday, May 04, 2004 12:18 PM by Marty

# re: Software Design and Home Depot, who would have thought they went together so well.

My office has a whole wall (14' X 8') filled with the stuff from Home Depot. I am like you there is nothing like being able to draw and draw without having to crumple up paper and restart.

I then use my digital camera to take a picture of what I want to keep before erasing.

Wednesday, May 05, 2004 7:27 AM by Shannon J Hager

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

Looks like another "who wants to be the lowest bidder" site. Guru.com was like that, there was always someone more desperate than you or some kid in highschool that didn't mind working for minimum wage in his evenings.
Wednesday, May 05, 2004 7:40 AM by Justin Rogers

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

I'll find out soon enough. You have to remember the high school kid probably won't get the work done and get kicked off the site early on. Hopefully I'll get an opportunity to do some of the site work and can find out a bit more about it.
Wednesday, May 05, 2004 12:09 PM by Antwerp_Diamond

# re: FunHi.com, a dating service, a new game, a bit over the top?

What are the barriers for a neat marketing ploy, surrounded by competition?
Wednesday, May 05, 2004 4:15 PM by darrenford

# re: Software Design and Home Depot, who would have thought they went together so well.

Are you using the 1/8" hardboard with the smooth white coating that they also sell in 4X8 sheets ?
Wednesday, May 05, 2004 11:19 PM by Justin Rogers

# re: Software Design and Home Depot, who would have thought they went together so well.

It might come in larger sheets at other Home Depot's, since I think I might have seen it before in Bellevue in larger sheets. But yes, 1/8" HB MB.
Thursday, May 06, 2004 2:20 AM by Claus Konrad

# re: Plug-in Framework (Part1): Marking Types for Consumption

Hi Justin

What is happening to the rest of the articles?
We are looking forward to read the remaining 4 chapters of the Plugin-framework articles series...

Thursday, May 06, 2004 8:57 AM by Scott

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

Actually Shannon-- we have a fair bidding system, unlike our competitors, you cannot post a bid below 75% of the average bid rate for a project with 3 bids. So if the project has 3 bids averaging $1000, you cannot submit a bid below 750. We try to keep everything fair unlike the other sites! (I know from personal experience, I hate the other sites because you get undercut at the last minute and lose a 2500 job to someone who bids $500)
Thursday, May 06, 2004 12:12 PM by Shannon J Hager

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

That seems like a good idea. 3 bids at $1000 each, then 1 at $750, the next at $704, the next at $670, the next at $640, eventually it would pretty much bottom out, even moreso if someone actually dared to bid higher than average.

I never had last minute bids cut me off, personally, I just had competition from people with lower cost-of-living (or lower self-worth). The projects always seemed to be looking for lowest bidder and, in my experience, there is always another lower bidder.
Thursday, May 06, 2004 1:29 PM by darrenford

# re: Software Design and Home Depot, who would have thought they went together so well.

When you get to the point of covering your wall, how do you plan to handle the seams? I wish I had asked back when Marty was watching since it sounds like he already has dealt with that.
Thursday, May 06, 2004 2:23 PM by Scott

# re: A new contact bid site for IT professionals. www.CorporateRebel.com. Could it be useful?

Well it's difficult from my position to be strict in bidding and appeal to the bidders, while still appealing to the job-posters that are looking for affordable, quality labor.

Personally I always go with the guy that has the best resume and feedback/stats (if available). I've posted a couple of my own personal jobs on the site and am about to choose 2 winning bids. I look for best resume, not cheapest bid, and I know a lot of job-posters are similar.

Any way you slice it, it's a fight to get work and stay employed (in the freelance world), so you just gotta keep at it and try hard to build relationships. If you do happen to sign up I'd love to hear your feedback on bidding and using the system. It's my customers' feedback that is going to shape the development path for the site.

Thanks,
Scott
Thursday, May 06, 2004 8:29 PM by Justin Rogers

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

Just for laughs. Note the Stream uses Stream.SynchronousAsyncResult for its IAsyncResult implementation. Isn't that funny. Even more funny is it's implementation of CompletedSynchronously:

IL_0000: ldc.i4.1
IL_0001: ret

In other words, it ALWAYS completes synchronously.

Damn, I just thoroughly read the docs (V1.0 docs on this machine, I'll check the V1.1 docs later) and everything in there seems to IMPLY that the methods really are asynchronous. They do state that the asynchronous versions are implemented on top of the Read/Write synchronous versions and that if you replace the Read/Write methods then things will still work for you. That's cool, IF the BeginRead/BeginWrite methods were actually doing asynchronous by loading a ThreadPool item.

They even have recommendations for multiple simultaneous asynchronous reads, and I quote "Multiple simultaneous asynchronous requests render the request completion order uncertain." Based on the fact that the implementation is SYNCHRONOUS, I don't think that will be a problem.
Thursday, May 06, 2004 9:28 PM by Chris

# re: Tips for noobs at E3! Based on a request by a reader.

Is there fee's to park at the convention center?

$10 - $15 depending on where you park.

7. Parties... are there any or is everyone way to tired by the end of the day to do nothing more then hit the hotel and crash?

Plenty... but your not invited. The big parties are usually reserved for celebrities and such.
Friday, May 07, 2004 9:48 AM by Steve

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

That is a real stinker... It sort of defeats the purpose of even using it to begin with.
Saturday, May 08, 2004 1:38 AM by Hmnt(Sayona_Hmnt@hotmail.com)

# re: Performance: If vs Switch in what we might call an extended usage scenario?

http://www.dotnet247.com/247reference/msgs/35/178987.aspx
Saturday, May 08, 2004 2:34 PM by TrackBack

# .NET Links

Well, I have a ton of links built up that I'd like to post, so instead of posting them individually, I'll just throw them all at you at once. Enjoy. :) I'm not sure where I saw it first, but...
Saturday, May 08, 2004 2:34 PM by TrackBack

# .NET Links

Well, I have a ton of links built up that I'd like to post, so instead of posting them individually, I'll just throw them all at you at once. Enjoy. :) I'm not sure where I saw it first, but...
Monday, May 10, 2004 6:21 AM by Joel B.

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

I find intriguing using message filtering to debug applications.
Now, if only there were a simple matrix (i.e. multidim multitype array) where I could just use m.Msg as an index, getting 1) a string with the message name 2) a proper type that I can pass to GetILParam to extract proper LParam.......things would be easier.
Anything to suggest?
How have you implemented message debugging in C#?

Joel
Monday, May 10, 2004 10:15 AM by Justin Rogers

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

Yep, I've done some message debugging using the methods I've detailed above. Mainly for synchronicity issues where I need ordering of messages to determine why my component isn't working the way that I think it should.

Casting between the m.Msg index and a set grouping of stuff isn't easy. For many messages the ID is static, however, a user can re-define or reallocate any value in the user range to do whatever they want.

Roving id's can be created by using RegisterWindowMessage so that each time a program launches there is the possibility of getting a different ID for the same message.

To answer any questions on how I debug things, I quick write the Message structure down to a binary file. I worry about name mapping later when I'm checking the message ordering. For more advanced debugging (delving into the structure), I use my PreFilterMessage handler as the place where I put the breakpoints. Since you don't have code for the Winforms user controls, it isn't easy to set code backed breakpoints inside of their WndProc, so breaking the debugger at your message filter is super handy. Then you can hand cast any of the parms to user defined structures in the debugger to investigate your results.
Monday, May 10, 2004 8:42 PM by Alexei

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

There are two parts to this problem one is your side the other one is on Stream class.

1) The Stream class problem is that it should be issuing a blocking call in BeginRead().

2) You problem is that on every BeginXXX call you should be checking ar.CompletedSynchronously and if so proceed on the same thread. In fact, sync completions would give you perf improvement since an async completion needs to flow execution context onto a new thread.


IAsyncResult ar = BeginDoSomething(... _MyCallback, myState);
if (!ar.CompletedSynchronously)
return;
ProcessSomething(true);

void MyCallback(IAsyncResult ar)
{
if (ar.CompletedSynchronously)
return;
ProcessSomething(false);
}

void ProcessSomething(bool onMainThread)
{
try {
EndDoSomething(ar);
}
catch (Exception e)
{
if (IsNotExpectedException())
throw; // This may and should bring your process down.

if (onMainThread)
throw; // Something on the main thread may want to handle it.

//
// If something on the main thread could handle an exception call it here.
//
HandleException();
return;
}
HandleSuccess();
}
Tuesday, May 11, 2004 6:11 AM by Joel B

# re: I've done a little experimenting and......

First of all, as you all are aware, I'm very inexperienced. This is a sort of disclaimer...

I wrote a simple Windows Forms application, to test how I could use Application.AddMessageFilter();

1) I went through winuser.h and a few other headers. I built a hashtable where the key is the msg number and the value is a string (WM_xxxxx), so I can easily understand what is going on.

2) I implemented a message filter as described, adding each message to a collection holding msg.HWnd plus the message description if a match is found in the hashtable, otherwise just the message number when the value is unknown (to me)

3) Surprise!
Very few messages have a known value. They fall into just 3 categories: mouse events including button clicks, keyboard events, WM_PAINT (only if not an indirect cause of a mouse movement). Then there are a bunch of events with value 0xc0xx (which are supposed to be private and I have no idea about how to investigate them).
Sometimes there is also a message with value 0x118 (this value falls within the system messages range, however I could not find any reference to it anywhere so I have no idea of what that means -- Anyone has a clue?)

4)I then decided to implement a void WndPro() override.
This time my collection gets all the messages that I expected to catch. It behaves pretty much like I was expecting from using Application.AddMessageFilter()


Any idea about why Application.AddMessageFilter() behaves that way? Could it be still useful in any way?
Tuesday, May 11, 2004 11:11 AM by Justin Rogers

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

Joel, I'm going to try and get around to a more broad code-path examination of this today with some sample code, since I think that will help you out quite a bit. Stay tuned!
Friday, May 14, 2004 12:05 PM by Lorenzo Barbieri

# re: Wanting to fix all my past mistakes, here is an updated ImageFast library that doesn't rely on System.Drawing for init...

The link only works in the HTML page, not in the RSS feed, it's missing the http://weblogs.asp.net/justin_rogers part...
Friday, May 14, 2004 3:56 PM by TrackBack

# Wanting to fix all my past mistakes, here is an updated ImageFast library that doesn't rely on System.Drawing for init...

Sunday, May 16, 2004 12:33 PM by Sacha De Vos

# re: WinForms Wizard Series Article 6, a quick look at implementing a meta-property storage class.

I can't wait to see the next article ! Thanks for the very good ideas ... First time I see a working real life example of the model/controller/UI approach.
I had some troubles adapting this wizard so that it uses a modal dialog within an existing application though. Anyways this saved me a whole bunch of work. thanks again...
Monday, May 17, 2004 6:33 AM by Darren Neimke

# re: BasicParse with State/Transition graphing, a much more robust parser.

Very nice Justin - do you have a full code dump of this stuff yet or will you post something when you have it completed?

Also, what language is this parser built for? I presume that it is built for a subset of a C-like language?
Monday, May 17, 2004 8:26 AM by TrackBack

# NullReferenceException = LoadFromFile( ... ) ;

Monday, May 17, 2004 1:54 PM by William Stacey

# re: BasicParse with State/Transition graphing, a much more robust parser.

Very good Justin. I am still studying it but looks like your really moving on it. Cheers!
Monday, May 17, 2004 2:49 PM by Ron

# re: BasicParse with State/Transition graphing, a much more robust parser.

Jason,

A comment on the switch in ConvertTokenToType.

Should the switch on token.TokenData evaluate case " ": (space)? Didn't the StringToTokens method toss the spaces? If you're coding for the possibility someone called StringToTokens without tossing anything, shouldn't you also test for tabs, carriage returns, and line feeds to mark them as white space?
Monday, May 17, 2004 2:58 PM by Justin Rogers

# re: BasicParse with State/Transition graphing, a much more robust parser.

[Darren] I have a full code dump for this. I'll get it up shortly.

[Ron] The Lexer and Parser are two different modules. My switch statement is not more complete because my input to the lexer is always explicit. The easy override in the lexer that just takes a source input is only there as a demonstration of how you might use it. The switch statement for that case would be much different.
Monday, May 17, 2004 3:44 PM by Nat

# re: Using PInvoke with GDI+ just isn't an easy thing to do. System.Drawing.dll must have some *magic* I don't know about.

using System;
using System.Runtime.InteropServices;
using System.Threading;

[StructLayout(LayoutKind.Sequential)]
struct StartupInput
{
// Fields
public int GdiplusVersion;
public IntPtr DebugEventCallback;
public bool SuppressBackgroundThread;
public bool SuppressExternalCodecs;

// Methods
public static StartupInput GetDefault()
{
StartupInput s = new StartupInput();
s.GdiplusVersion = 1;
s.SuppressBackgroundThread = false;
s.SuppressExternalCodecs = false;
return s;
}

}

[StructLayout(LayoutKind.Sequential)]
struct StartupOutput
{
// Fields
public IntPtr hook;
public IntPtr unhook;

}

public class LockImageAndLoad
{
[DllImport("gdiplus.dll", CharSet=CharSet.Unicode)]
public static extern int GdipLoadImageFromFile(string filename, out IntPtr image);

[DllImport("gdiplus.dll", CharSet=CharSet.Unicode, ExactSpelling=true)]
private static extern int GdiplusStartup(out IntPtr token, ref StartupInput input, out StartupOutput output);

[DllImport("gdiplus.dll", CharSet=CharSet.Unicode, ExactSpelling=true)]
private static extern void GdiplusShutdown(IntPtr token);

private static void Main(string[] args)
{
IntPtr ptr1;
StartupOutput output;
StartupInput input = StartupInput.GetDefault();
GdiplusStartup(out ptr1, ref input, out output);
try
{
IntPtr pimg = IntPtr.Zero;
GdipLoadImageFromFile(@"c:\windows\Rhododendron.bmp", out pimg);
}
finally
{
GdiplusShutdown(ptr1);
}

}
}
Monday, May 17, 2004 6:05 PM by Justin Rogers

# re: Using PInvoke with GDI+ just isn't an easy thing to do. System.Drawing.dll must have some *magic* I don't know about.

Yeah, I quickly figured that out and posted the more complete ImageFast library.

http://weblogs.asp.net/justin_rogers/articles/131704.aspx
Monday, May 17, 2004 9:07 PM by TrackBack

# re: Using PInvoke with GDI just isn't an easy thing to do. System.Drawing.dll must have some *magic* I don't know about.

Tuesday, May 18, 2004 5:01 AM by Justin Rogers

# re: Introducing symbol tables with a C# example using 'get' as the point of interest.

To shed some light on the method C# uses to process the get accessor of a property I offer up the following:

1. It appears all of C#'s symbol resolution happens after the parse tree has been created. They do have a name manager in their lexer/parser, but I think it is more to save space and resolve keywords than any other purpose.

2. For the get accessor they use a special entry in this name manager, but when the lexical token for get is returned it is returned as an identifier no matter what the scope. So they don't use a symbol table for conditional processing during the lexer/parser phase.
Tuesday, May 18, 2004 8:24 AM by Darren Neimke

# re: Introducing symbol tables with a C# example using 'get' as the point of interest.

Most of this stuff went whooooshing over my head because I haven't really done anything on symbol tables before - they are a wierd beast!

I was going to ask you how you would add keywords into your BasicParse language. Up until now you have only parsed single char tokens and strings.

I was wondering what would happen to the lexer if you add a keyword such as "var" to your grammar. Would you simply add the chars 'v', 'a', 'r' to your terminal lookup table such that it would now become:

public static Token[] StringToTokens(string tokenString) {
return StringToTokens(tokenString, " \n\r\t{}\"=;.()[],var", " \t\r\n");
}
Tuesday, May 18, 2004 7:38 PM by Darren Neimke

# re: Adding keywords to BasicParse, keeping it short and sweet.

Thanks Justin, that's really cool - I think that I'm beginning to *get* symbol tables now. Just one thing, I think that you have an error in your TransitionParseVar() method. Shouldn't the test:

if ( this.tokenStream.Current.TokenType == TokenType.Id )
...be written as:

if ( this.tokenStream.Current.TokenType == TokenType.Var )

Tuesday, May 18, 2004 9:42 PM by TrackBack

# re: Why can't I use the same variable as an inner loop does?

Tuesday, May 18, 2004 10:14 PM by Justin Rogers

# re: Adding keywords to BasicParse, keeping it short and sweet.

Nope, we've already Eaten that token. Remeber that every time control is passed back to the state loop, we advance by one token. That is why in the Open state I check for Var, and then return ParseState.ParseVar. By the time I enter the ParseVar transition method, I'm already on to the next token, which by all accounts should be Id.
Tuesday, May 18, 2004 10:46 PM by Kieu Huu Chien

# re: More InstallShield DevStudio 9 issues...

Installer
Wednesday, May 19, 2004 11:48 AM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

Very Cool Justin. Is this complete now for the bind language more steps to go? Cheers!
Wednesday, May 19, 2004 5:32 PM by Justin Rogers

# re: Adding keywords to BasicParse, keeping it short and sweet.

William, the bind language has a separate language definition from this language, though they do seem similar. I'll code up a parser for precisely the bind scheme this evening, I've kind of gotten off track a bit in writing about other areas of the compiler that are more interesting to me.

Things that need to be added in order for the bind scheme to work are:

1. TokenStream patching to allow include files (these are really cool and I'll talk about them since it allows processing of include files inline without preprocessing).
2. Changes to the language definition based on the way bind works. Bind has more structure than just a basic key/value system. They allow complex inline types with multiple name/value pairs that appear in a structured manner.

For the full bind configuration compiler, I think that a keyword based language will be important, and the parsing will be much more strict. Arbitrary identifiers won't be allowed, since bind has no use for arbitrary identifiers. On the back-end, I'll map these fixed format types directly to class definitions, so the final compilation format will be a configuration object you can use directly. This compiler will be more an interpreter, since it is going to translate, at run-time, a given configuration into some other more usable format.
Wednesday, May 19, 2004 7:06 PM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

Your the man. Are you a Mensa member? You do more stuff in an hour then I can do in a month. Cheers!
Wednesday, May 19, 2004 7:22 PM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

BTW. If/when you get a generic french brace parser to parse any config with key words (maybe your are there already), would you care it I "wrap" it in a MSH "cmdlet" and put it up at the beta place for Monad shell? Will provide proper credit.

$obj = get-config -file msomefile.txt
echo $obj.Var1
ecjp $obj.Var2
$obj | set-config file mynewfile.txt

Need to think more on property access, etc. Could also gen a Provider for access like:

>cd ./myconfig ## "CD" to Config Provider.
>ls ## get-childitems in container.
Item Value type
---- -------- -------
var1 "string" string
var2 32 int
var3 "one", "two" string[]
...
TIA
Thursday, May 20, 2004 1:11 AM by Justin Rogers

# re: Adding keywords to BasicParse, keeping it short and sweet.

We need to define semantics for a *generic* french brace parser. The reason for this is that the Bind format that started this entire discussion, has items such as the following:

acl "alias for acl list" {
...
}

How does a generic french brace parser perform on the above? Do we create an acl element, then give it an Id, or do we create an acl element then immediately nest another element based on the string (aka, default nesting of identifiers) and put any values into this second nested element.

What about strings of identifiers that exist in bind, such as:

controls {
inet * port 50 allow { address_list } keys { keys_list }
}

In the bind specification the inet command is a single command, and multiple instances of said command can exist within the controls section. Portions are optional.

Now, if I throw out the assertation that a statement terminator must exist my generic parser can turn the above into:

<controls>
<inet Value="*" />
<port Value="50" />
<allow></allow>
<keys></keys>
</controls>

Multiple instances would simply turn into linearly defined items. Another option is to demand statement terminators, and parse all you can. I could turn the above then into:

<controls>
<inet Value="*">
<port Value="50">
<allow></allow>
<keys></keys>
</inet>
</controls>

Things start to get really sticky when you parse based on arbitrary grammars. You start to lose all of the meaning between Item/Value/Type association. That means there can't truly be a 100% generic french brace parser unles you are willing to accept the results in a very non-determinate form.
Thursday, May 20, 2004 6:37 AM by Stephane Rodriguez

# re: How hard/easy is an HTML parser using the BasicLex/BasicParser design?


Me think that you have no real choice if you intend to write a parser as part of the renderer. You need to apply the same rules than IE. That's why dominant positions are such a shame, especially when the software is so much broken and full of internal choices.

Regarding the parsing service itself, although Lovett's SGML parser might be regarded as a reference, I guess it becomes completely useless in the real world, where most of the time you get invalid html. The strength on an html parser should be to have an efficient diagnostics mechanism (with a meaningful report system, see Safari) and be able to switch from several parsing modes including loose, strict, ... All in all, you are not going to do that kind of reliable parser with a regexp (contrary to what some people try to show without short-sighted teasers).

My 0.5 cent



Thursday, May 20, 2004 7:13 AM by Justin Rogers

# re: How hard/easy is an HTML parser using the BasicLex/BasicParser design?

I think Darren's parser is well on the way to being fairly efficient and resilient to real world HTML. I agree that the SGML parser, even with the loose DTD, can be rather rigid.

I think the end result of Darren's work will be some relatively powerful HTML tools. I already have some ideas for what he is working on, as well as some small tools I'm planning myself.

Creating order out of chaos is how I refer to this process. If I create just slightly enough order to make my life easier, then a job well done.
Thursday, May 20, 2004 7:32 AM by TrackBack

# re: Internal and External Iterators

Thursday, May 20, 2004 10:33 AM by William Stacey

# re: Adding keywords to BasicParse, keeping it short and sweet.

ahh. Ok, thanks. Guess I will stick with the Bind parser for now. Which would be fine if you can tell it what vars are valid. Not sure on this point. Do you have to "bake" in the grammer, or can you supply some array structure to the parser to parse out the language that is valid and know when something is not valid (i.e. invalid syntax, invalid keyword, etc.)
Thursday, May 20, 2004 11:12 AM by TrackBack

# Short on time but here's some important parsing stuff...

Thursday, May 20, 2004 11:12 AM by TrackBack

# Short on time but here's some important parsing stuff...

Thursday, May 20, 2004 11:12 AM by TrackBack

# Short on time but here's some important parsing stuff...

Thursday, May 20, 2004 11:13 AM by TrackBack

# Short on time but here's some important parsing stuff...

Friday, May 21, 2004 8:26 AM by Joel B.

# re: Windows Forms message filters. How they are handy, misunderstood, and when are they not useful.

I know you have been busy, I was just wondering if you had time to look into this matter.
Joel
Saturday, May 22, 2004 1:18 AM by David Olsen

# re: Why is photon mapping popular in gaming right now, and will it stay?

Wow. I thought hardly anyone even knew about photon mapping, let alone used it in games already. Anyway, I'll tell you what I thought of to improve it's realtime performance for lights that don't move. Just shoot the photons in an ordered fassion of some sort so that when something moves you dont have to update the entire photon map, just get the light source to re-trace the photons which hit it last frame and might hit it this frame.
Saturday, May 22, 2004 6:55 AM by TrackBack

# Running your expressions asynchronously and making them cancellable.

Saturday, May 22, 2004 5:34 PM by David Levine

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

An interesting class. I might extend this so that the actual operation was called via a generic interface so that the actual work/cancellable operation can be anything.

I whipped up a little test based on this and found a couple of things you might consider.

First, in the ThreadPool_WaitCallback method you should either add a generic catch block or restructure it so that the call to result.Complete(); is in the finally block. The reason is that if the call to perform the work operation (in your example, the Match(...) call) ever throws an exception other then ThreadAbort, the call to result.Complete will never execute and you will instead have an unhandled exception. This is likely to result in some higher level code hanging.

The other thing is that you might use Thread.Interrupt rather then Thread.Abort to signal the worker thread to interrupt its processing. For one thing, your callback might not have sufficient security privileges to call ResetAbort, and for another, using Abort has undesirable side effects that would argue against using it in a generic manner - unless you know that the code you are interrupting can gracefully handle an abort I'd use some other technique.



Saturday, May 22, 2004 6:15 PM by Justin Rogers

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

I have developed a generic version of the above class, but it doesn't help people trying to do their operations today. Nor does it solve very many problems.

Generic methods can't accept variable numbers of parameters, so the BeginInvoke/EndInvoke method signatures constantly change.

Since this is for regular expressions, I get more performance having the layout abovce. However, you can add an exception local, and move Complete into the finally block, as well as setting the caught exception (if anything but a ThreadAbort) on the async result. When the user calls EndInvoke you re-throw the exception.

I tested the abort code with and without ResetAbort and it doesn't thrash the system either way.

How does Interrupt buy you anything? Interrupt won't cancel a running thread, only Abort does that. Interrupt is capable of signalling a thread that is in a wait state, not a running state. As for the side-effects, I'm well aware of them. With another 50-100 lines of code we could remove those side effects, and make this completely unsuitable for use on a web server.

The two choices are app-domains or using something closer to a native thread. Both approaches make the code iffy in a server environment while making it much more stable and robust in a client environment.
Saturday, May 22, 2004 11:57 PM by TrackBack

# Generic, Cancellable, Asynchronous operations? Yeah, I'll blog about that.

Sunday, May 23, 2004 7:26 AM by David Levine

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

My original thought was that the class was nicely layed out and a generic version of it would be useful to have; my comments mainly followed from that starting point. By targeting the class to a single type the design is simplified and the comments do not necessarily apply. e.g. if Regex.Match will never throw an exception then there's no need to move the call to Complete.

Using Interrupt has pros and cons. It will interrupt the target thread when it makes a call that puts it into an alertable wait state so its effects are more predictable, but the downside is that if the target thread does not make such a call it will have no effect on it. Again, this may not be useful with Match.

I've had mixed results using abort; if the thread has wandered off into unmanaged code it wont be delivered until it returns to managed code. I've had better results when unloading appdomains - I could unload an appdomain even when I could not abort a specific thread in it.

I don't quite see how using a native thread would work. Would it use the RegEx class, and would you use native signalling methods to interrupt it?

Sunday, May 23, 2004 5:00 PM by Justin Rogers

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

Native threading API's can be used to ensure that a given thread gets shut-down. This is the not nice way to go about things and basically has the layout of:

Cancel() -> Abort() -> Oops, we still haven't aborted after some time limit? -> TerminateThread.

The goal is to ensure things get cancelled. The Win32 threading API's provide a fast easy way to cancel the operation, while at the same time introducing a slight chance of data corruption. Again, the concept is that in a server environment, you don't want to be loading a large number of AppDomain's. You could share AppDomain's but then bringing one down would mean cancelling some other code that was running in the domain as well.
Monday, May 24, 2004 5:57 AM by Dave Hallett

# re: Debugging eventable code, one stupid mistake and how to fix it.

This is an ugly hack, but IIRC, you can just call

this.slideshowTimer.Tick -= new EventHandler(this.Form_SlideShowAdvance

before you call

this.slideshowTimer.Tick += new EventHandler(this.Form_SlideShowAdvance

and if the event is not hooked up, nothing will happen. Not very high-performing code, I'm sure :o) But I think it works.
Monday, May 24, 2004 6:05 AM by Justin Rogers

# re: Debugging eventable code, one stupid mistake and how to fix it.

Definitely a hack. I wound up writing my eventing manager and it works quite well. I tend to use it more often than not in any sort of asynchronous programming scenario with eventing, or when hooking up many hundreds of events in notification scenarios.

You see, the delegate managers tend to work extremely well at ensuring events are hooked in a specific way, and I like that. Take a basic RTC scenario, where you have a group of chatters. Someone owns the group or controls it, while others are members. If the group is now an object with a single broadcast event for sending events, then you could hook that event any number of times through varying code-paths (possibly by mistake), with an event manager I can ensure that it only gets hooked once. I can also allow the hooks to be dependent upon moderation by the group owner. There are so many protections added by the EventManager.
Monday, May 24, 2004 7:49 AM by David Levine

# re: Asynchronous Regular Expressions using the ThreadPool and a cancellation model.

I agree, it definitely is not a nice way to go about shutting down a thread :-) I experimented with TerminateThread back in runtime v1.0 and decided to not use it in .NET.

One issue was getting the native system Win32 thread handle - there's no exposed mechanism for getting it. I've seen an undocumented method that uses hardocded offsets into the thread class. I came up with an alternative method that worked well enough for my experiment - get the list of all the system threads in my process before creating my worker thread, create the worker thread, get a new list of system threads and look for the new thread and use its handle. If I could not find it or if there were more then one new thread, I would terminate the worker thread and repeat - eventually I would find the handle.

After that I was able to terminate the underlying system thread, but the problem was that the part of the runtime that controlled threads and execution state has no idea that I had yanked the rug out from underneath it. It still thought the managed thread object was alive and well even though I had terminated the system thread that it was mapped to. Perhaps there's a way to tell the runtime that the thread had died but I didn't find it.

Also, TerminateThread does not clean up after a thread so that the thread's Win32 stack is still allocated (1Meg of virtual address space), owned objects (mutexes, etc.) are still owned, so deadlocks can result, resources may leak, data can get corrupted, etc. I've used TerminateThread in control engines but only as an absolute last resort.

Future (present???) versions of the runtime are supposed to support models built on execution units other then threads, such as fibers. I haven't worked with fibers so I don't know how well they map into using the native API to control them from the managed world.

Even though creating/unloading an appdomain is a somewhat heavy operation it does not have all those issues. From what I've observed it appears to be fairly aggressive at unloading an appdomain even when a thread with stack in the doomed appdomain has become "stuck" and unresponsive. If the task was a very long running operation it might be worthwhile to give it its own appdomain, but as you point out there are other issues to consider on a server - all designs involve tradeoffs.


Monday, May 24, 2004 10:54 AM by TrackBack

# XPathDocument, can you dig it? Yep, and you can edit it too.

[ Via Justin Rogers ]XPathDocument, can you dig it? Yep, and you can edit it too....
Monday, May 24, 2004 1:50 PM by William Stacey

# re: XPathDocument, can you dig it? Yep, and you can edit it too.

You may find this link interesting.

http://www.devx.com/dotnet/Article/17829/1954?pf=true
Monday, May 24, 2004 3:19 PM by Justin Rogers

# re: XPathDocument, can you dig it? Yep, and you can edit it too.

The DevX article is fairly well written. However, the use of helper methods for operating on the tree is disappointing. Currently the interoperation of XmlDocument->TreeNode is tenuous at best. You can store XmlNode references on a TreeNode, which becomes very powerful, but it doesn't appear to be anywhere near the performance of the XPathDocument.

I will note that the TreeView may get a data-binding syntax before Whidbey gets released. Might already have it, just I haven't managed to get it working yet.
Monday, May 24, 2004 9:43 PM by TrackBack

# Take Outs 24 May 2004

Take Outs 24 May 2004
Tuesday, May 25, 2004 3:17 AM by Dave Hallett

# re: Debugging eventable code, one stupid mistake and how to fix it.

You make a good point: the hack is indeed no good at all for such situations. Thanks for the response!
Tuesday, May 25, 2004 5:59 AM by Matt

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Very interesting article, Thanx !
Tuesday, May 25, 2004 8:31 AM by Darren Neimke

# re: BasicParse with State/Transition graphing, a much more robust parser.

Thanks Justin... A quick tip: I found it useful to expose the this.mStreamOffset from the TokenStream class. I exposed it as a readonly property named "Position".
Tuesday, May 25, 2004 12:57 PM by Justin Rogers

# re: BasicParse with State/Transition graphing, a much more robust parser.

I could see that. Another useful feature is to not eat \n and have the TokenStream count line numbers and line character positions. I'll look at adding this meta-data into the classes.
Tuesday, May 25, 2004 3:02 PM by David P

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

This is a stupid function. it doesnt actually remove the full split string, only the first char.

Example:
teststring: "data1_-_data2_-_data3"
split: "_-_"

returns:
data1
-_data2
-_data3

Rather than:
data1
data2
data3

this just seems... lame. needs to clean up the garbage. This is otherwise a great approach to split on string.

if you have suggestions for a tweak, please let me know. I was toying with the code, but have not followed the logic properly/completely yet.

David P.
roachslayer@hotmail.com
Tuesday, May 25, 2004 3:21 PM by Justin Rogers

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

Hey, no reason to call it garbage man. Here you go:

private static string[] SplitByString(string testString, string split) {
int offset = 0;
int index = 0;
int[] offsets = new int[testString.Length + 1];

while(index < testString.Length) {
int indexOf = testString.IndexOf(split, index);
if ( indexOf != -1 ) {
offsets[offset++] = indexOf;
index = (indexOf+split.Length);
} else {
index = testString.Length;
}
}

string[] final = new string[offset+1];
if ( offset == 1 ) {
final[0] = testString;
} else {
offset--;

final[0] = testString.Substring(0, offsets[0]);
for(int i = 0; i < offset; i++) {
final[i+1] = testString.Substring(offsets[i]+split.Length, offsets[i+1]-offsets[i]-split.Length);
}
final[offset+1] = testString.Substring(offsets[offset]+split.Length);
}

return final;
}
Friday, May 28, 2004 1:09 PM by Cepheus

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Thanks for an excellent article covering some very much needed information for me. I've been working on a multithreaded network application and I've been stressing over how to deal with the UI, and the methods you examined here have solved it for me. :) Thanks a bunch!
Tuesday, June 01, 2004 3:05 PM by zjames

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

This works really well. Thank you.
Tuesday, June 01, 2004 3:22 PM by Justin Rogers

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

I never linked this in. However, the stand-alone ImageFast library is much more robust than the above code. It allows for operation in scenarios other than Windows Forms based.

http://weblogs.asp.net/justin_rogers/articles/131704.aspx

Tuesday, June 01, 2004 6:22 PM by TrackBack

# re: Fast Image Loading without asking for the hot-fix or waiting for the service pack...

Tuesday, June 01, 2004 6:41 PM by William Stacey

# re: Bringing an end to the posts on decompositions of numeric ranges into validating regular expressions.

Just curious if you checked out any of these lexers and what your thought about them?

Grammatica
http://www.nongnu.org/grammatica/index.html
ANTLR
http://www.antlr.org/grammar/list
csLex
http://www.cybercom.net/~zbrad/DotNet/Lex/Lex.htm
Tuesday, June 01, 2004 7:04 PM by Justin Rogers

# re: Bringing an end to the posts on decompositions of numeric ranges into validating regular expressions.

I've looked at pretty much all of the lexical analyzers out there. I really only have one problem with them in general, and that is the initial complexity involved. They assume that you can easily define your grammar in some abstract format, and most people, until they've actually written their own lexical analzyer won't fully understand that abstract format or how to find issues with it.

I guess I'd say that creating your own lexer/parser is an exercise that gives you access to knowledge you'll need later when you implement whatever your lexer/parser was creating. In your case you were trying to read bind-logs. The lexical analysis file for bind logs is actually fairly complex with lots of optionals and this/thats, but just looking at the stream of values it appears fairly simple. It really is, but you have to go through the exercise of processing that data before you gain an understanding of it.
Wednesday, June 02, 2004 6:03 AM by nani

# re: Awesome Windows Forms message pump trick...

nani
Thursday, June 03, 2004 8:11 PM by Tandem_Guru

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

I think this is THE most detailed description regarding Winform pitfalls that most likely confuse a lot of newbies like me.

Thanks for the clarification!

How about another paper regarding UIThread and other background thread so that it could really complete the whole topic?


Thanks again!
Thursday, June 03, 2004 10:52 PM by William Stacey

# re: Bringing an end to the posts on decompositions of numeric ranges into validating regular expressions.

I agree fully. I looked at them, but did not want to speed the hours needed to figure them out. I probably will at some point, but was looking for something I could quickly wrap my arms around and go. Guess that is not possible and just need to roll the sleeves up or use xml for now. Cheers!
Friday, June 04, 2004 9:55 AM by TrackBack

# Faster than the speed of light (that's pretty quick you know!)

Friday, June 04, 2004 10:20 AM by TrackBack

# Faster than the speed of light (that's pretty quick you know!)

Saturday, June 05, 2004 7:24 AM by Ole Lytjohan

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Nice article, but you should really make some people read through it first before posting it. It's filled with errors :/

It's not a big thing, the article by itself is nice, but it would just have been much better, if those issues where fixed.
Saturday, June 05, 2004 7:40 AM by Justin Rogers

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Would be nice if you pointed out the errors so they could be fixed. Up to now I've had many people read through it and not point out any errors.

If you are speaking of spelling mistakes or grammar issues then I won't even bother, but true errors in function are definitely something I can post addendums for.
Saturday, June 05, 2004 7:42 AM by Ole Lytjohan

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Previous post of mine was in error. No errors.
Read some of it wrong, just shows :)
Saturday, June 05, 2004 7:42 AM by Justin Rogers

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Tandem: What exactly are you talking about with the extension paper you propose? Are you thinking of an additional use case scenario that perhaps I've missed or didn't fully explain so that you could apply it to your problem?
Saturday, June 05, 2004 8:18 AM by Justin Rogers

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Ole: Hey, no worries. I've definitely posted mistakes before and been called for it, so I'm always curious what I'm going to get in return after a posting. Thanks for reading it so closely.
Sunday, June 06, 2004 11:01 AM by TrackBack

# re: Harry Potter and the 142 Minutes of My Life I'll Never Get Back

Sunday, June 06, 2004 5:21 PM by TrackBack

# Take Outs for 6 June 2004

Take Outs for 6 June 2004
Sunday, June 06, 2004 5:22 PM by TrackBack

# Take Outs for 6 June 2004

Take Outs for 6 June 2004
Monday, June 07, 2004 9:27 AM by ej2

# re: New little series I'm going to call, Causality4. An examination of causality in terms for specific groups of people.

Very interesting. Have you seen the movie 12 Monkeys? It deals a little with causality.
Monday, June 07, 2004 12:21 PM by Jerry Dennany

# re: I often get asked how one would ship a single assembly app that is in multiple assemblies...

I did something sorta like this a while back.

http://weblogs.asp.net/jdennany/archive/2003/09/24/29035.aspx

Monday, June 07, 2004 12:52 PM by zjames

# re: ImageFast: A stand-alone library for quickly loading GDI+ images without validation.

Thank you thank you thanks! Works great. Original wasn't working on Win2000 at home, then it DID work, WAS working at work on win2000, but today it wasn't... weird. This fix got it working again. Hopefully for all time. Thanks!
Zac James
Monday, June 07, 2004 9:36 PM by TrackBack

# Dynamically running an Assembly in a separate process (with your tail between your legs)

Monday, June 07, 2004 10:30 PM by TrackBack

# re: Dynamically running an Assembly in a separate process (with your tail between your legs)

Tuesday, June 08, 2004 4:35 AM by TrackBack

# re: Draft posts... what should they display for 'date posted'?

Wednesday, June 09, 2004 12:16 AM by soft

# re: Joel's Lightweight Code Gen spells SUWEET for small scripting languages in games.

good!
Wednesday, June 09, 2004 9:01 AM by Fredrik Normén NSQUARED2

# re: What are your thoughts on the new ArraySegment structure in Whidbey?

I think it could be useful, it will reduce the number of argument we need to pass to a method and I don’t need to remember the offset and the count. It’s already located in my array segment. For example if we want to pass an array and its offset and count we need to write something like this:

int[] myArrayOfStuff = new int[10];
FillSegment(myArrayOdStuff, 5,5);


public void FillSegment(Int[] myArray, int offset, int count)
{
…
myStream.Read(myArrayOfStuff, offset, count);
…
}

With the ArraySegment<T> this could be written:

int[] myArrayOfStuff = new int[10];
ArraySegment<Int> mySegment = new ArraySegment<Int>(myArrayOfStuff, 5,5);
FillSegment(myArrayOdStuff);

public void FillSegment(ArraySegment<T> myArraySegment)
{
…
myStream.Read(myArraySegment.Array, myArraySegment.Offset, myArraySegment.Count);
…
}
Wednesday, June 09, 2004 10:00 AM by kvr

# re: A conversion from integer to long form english... I could write that ;-)

Shouldn't Fourty be spelled as Forty?
Wednesday, June 09, 2004 10:37 AM by Justin Rogers

# re: What are your thoughts on the new ArraySegment structure in Whidbey?

I agree, however, the later in this case is actually more code to write, and can appear to be less clear.

I would also point that the possiblity of error that exists when using an array segment, by mistyping and using the array's properties is quite high.

And while I understand why they made the object immutable, you will also wind up creating a new segment every time you call your Fill command. With integers you treat the offsets like pointers and increment things. With Segments, you are forced to allocate a new one every time you want to move around your buffer. This creates object allocations where before there were none.

Not saying this is a bad thing, just pointing it out. Glad to see someone is making use of the segment.
Wednesday, June 09, 2004 10:56 AM by Spectrum

# re: A long MUD-Dev thread on intelligence of NPC's, and some extremely faulty arguments...

10 PRINT "DISAGREE IM A DUMB PROGRAM"
Wednesday, June 09, 2004 12:14 PM by TrackBack

# System.Net.NetworkInformation.Ping

After sorting out a problem with some ICMP "ping" code last week, it was nice to see that there is a Ping class in Whidbey, as described in Justin Rogers' post TraceRoute Whidbey using System.Net.NetworkInformation....
Wednesday, June 09, 2004 2:25 PM by TrackBack

# A conversion from integer to long form english... I could write that ;-)

Wednesday, June 09, 2004 5:59 PM by AT

# re: A conversion from integer to long form english... I could write that ;-)

How about to convert them in both ways ?
English -> number ?
Wednesday, June 09, 2004 9:16 PM by Brian Grunkemeyer

# re: Generic, Cancellable, Asynchronous operations? Yeah, I'll blog about that.

This is unfortunately a very flawed attempt at a cancellation infrastructure. The design flaw here is you're relying on aborting threads. Short of calling methods with reliability contracts and/or constrained execution regions, you can't abort threads safely. You have no idea how the Regex code is implemented internally, but it might have an appdomain-wide cache of things that it edits while doing a lookup. If that's the case, there's no guarantee that collection update will succeed if you abort it. You can be corrupting appdomain and possibly process-wide state with this code, so you should follow this up by promptly unloading the current appdomain (and possibly the process, if you can't guarantee the process state won't be corrupted). We may change Thread.Abort to be more "polite" eventually in Whidbey, but you still run the risk of corrupting appdomain & process state, as we may not "harden" all the parts of the Framework you need.

What can you use? There are three techniques you should look at:

1) Cancellation regions & cancellation signals
2) ICancellableAsyncResult
3) Some cooperative polling technique in your code to support cancellation

If you're running on Longhorn, Whidbey's new CancellationRegion & CancellationSignal classes will allow you to cancel any disk-bound IO request (assuming the driver supports cancellation). For async IO requests, ICancellableAsyncResult provides a cancel method that uses some other Longhorn feature to cancel that particular async operation.

Short of that, the best technique you can use is cancellation. One of the changes we want to make for Beta 2 is to is to add a property to CancellationRegion indicating whether a cancellation was requested. You can change your code that is CPU-intensive to poll this property and throw an OperationCanceledException. Unfortunately, it has to be done in this cooperative manner if you want to avoid state corruption that would require you to pessimistically unload the appdomain.

For more info, Chris Brumme's discussions on Reliability may be interesting (I don't know if they go into this much detail though in publically released material yet). Additionally, this same notion of avoiding corruption went into the design of Mark Gabarra's PoVBot (an AC2 game bot), with the introduction of a cooperative sleep (in addition to a normal sleep) in the bot's scripting language. The feature would allow you to interrupt the script at this point with an async handler of some sort (like a timer that fires every minute), do some operation, then recover to some known state. The script and/or the event would have to be written to deal with any state changes to the UI, etc.

(Yes, Mark & I were both surprised that a game bot would have any commonality with our reliability work. Fortunately, once he was working on the bot, we already had the hard conceptual part figured out.)

If you want more information about some of this stuff, look on the BCL web site (http://www.gotdotnet.com/team/clr/bcl/TechArticles/techarticles.aspx) and annoy us if you can't find what you want.
Wednesday, June 09, 2004 9:27 PM by Brian Grunkemeyer

# re: I'm going to do something unfair to make a point: int.TryParse versus DWC.NumberUtilities.

The need for these methods wasn't realized until too late in the ship cycle. We added one TryParse method because it was causing some noticable delays in an important perf scenario, then we got around to doing the rest significantly later.

I don't imagine your TryParseInt32 is culturally correct. That's probably a significant portion of why Int32.TryParse is slower.
Wednesday, June 09, 2004 9:42 PM by Brian Grunkemeyer

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

There are a number of issues here, and I think they're all solved.

First, the stack overflow exception can be avoided by using the CompletedSynchronously property, as Alexei points out. Based on its result, you can tell whether you can recurse or whether you should return and have your caller issue another BeginXxx call. I think you'll find this if you read through enough conceptual docs on our async design pattern, but it is pretty buried.

Secondly, Stream's BeginRead is synchronous like this. Stream's async code paths are implemented in terms of the sync code paths, and the sync code paths are implemented in terms of the async ones. So if you subclass Stream, you only have to implement the sync or the async code paths, and you get the other one for free. Additionally, your users have a consistent programming model.

Thirdly, I've reimplemented Stream's BeginRead & BeginWrite methods in Whidbey to do their work on a threadpool thread instead of their own thread (via async delegates). Your code will probably work if you pick up a more recent build. (I made the change to Whidbey around Feburary or March - you might not see it until Beta 1 or a very recent community drop. I wanted to make this change much earlier in Whidbey, but I was blocked until we added a Semaphore class to fully insulate users from an obscure race I would have otherwise exposed in user code. All the other sync primitives have sometimes-unfortunate threading restrictions, some of which weren't easily discoverable in V1 & V1.1.)

Additionally note that FileStream & NetworkStream do support true async IO using an IO completion port. However, for FileStream, the OS restricts file handles to only one type of IO - either sync or async. So you must explicitly open the file asynchronously if you want fast async IO. (You can do this by passing in FileOptions.Asynchronous in Whidbey, or by passing true to the "useAsync" parameter before that.) If you open it in the wrong mode, it may hurt your perf by up to 10x on pathological cases, and you may end up with the naive implementations from Stream in these cases.

I hope this helps.

Brian Grunkemeyer
CLR Base Class Library Team
Wednesday, June 09, 2004 10:21 PM by Justin Rogers

# re: Generic, Cancellable, Asynchronous operations? Yeah, I'll blog about that.

To provide public commenting:

Contract structures are something I fully agree with. While you could use the above generic cancellation feature for any code, and it would work to some degree, there are code-paths that the remote thread might be in that would cause some form of failure. That is the reason I used my knowledge of the Regex codebase through Rotor to use it in that scenario and primarily as a reliability story for a high capacity web server that is trying to prevent a DOS attack by a user input expression. There are a number of ways to get to this point since even a basic expression can be non-terminating over the properly written input.

All three of your solutions are a hand-shake pattern. I'm not sure that I'll ever be able to fit all of the code I need to cancel into the design paradigm of a hand-shake routine. Some I could, yes, but others are simply going to be impossible.

The reliability store here is that of BlackBox reliability. I can trace every code-path that I'm running and ensure a cancellable IL structure, however, what I can't see behind is the BlackBox of the CLR. Sure there may be some non-cancellable features back there that would cause my code to exit not so gracefully, then mark them and tell me where they are so I can include those in my examination and determine if my reliability story is going to be good enough.

What I would propose for Thread.Abort, if you are making changes, is some awareness in the code of where it can gracefully stop and where it can't. Currently Thread.Abort instantly throws an exception (or actually I think in some cases it does wait a bit), while in the future it could have a timeout of some sort and a return value telling me whether or not the thread truly did abort. If it didn't, an enumeration telling me why would be nice, and I can try the abort a second or third time, again in the interest of a more reliable system.

What I don't want is to riddle my AI code with cancellation checks when I'm already granting it a small subset 5% of the CPU, and said 5% is being shared between 1000 automatons.
Wednesday, June 09, 2004 10:25 PM by Justin Rogers

# re: I'm going to do something unfair to make a point: int.TryParse versus DWC.NumberUtilities.

No, but even the culturally correct version that I wrote is 3x faster. So that wasn't the significant portion at all. The significant portion is the single code-path for parsing all numeric types that does a bunch of extra processing logic in cases where said logic is unnecessary. Having the new TryParse methods around in Whidbey is quite the good thing as I've noted, but I'll still use my own libraries for string and numeric processing since they are quite the bit faster.
Wednesday, June 09, 2004 10:31 PM by Justin Rogers

# re: An not so asynchronous mistake anyone can make when using the System.IO.Stream class.

You mean something like what I have below. I've wrapped the concepts of synchronous completion so that I have enough variables to figure out when I need to loop and when the aysnchronous code is doing the work for me. I'll probably blog the below and linkie it to this entry. Also note that I've also implemented a ThreadPool override of the Stream class that provides true asynchronous actions. So I'm guessing we are on the same page.

using System;
using System.IO;

public class JustAStreamWrapper : Stream {
private Stream underlyingStream;

public JustAStreamWrapper(Stream stream) {
this.underlyingStream = stream;
}

public override bool CanRead { get { return true; } }
public override bool CanSeek { get { return false; } }
public override bool CanWrite { get { return false; } }
public override long Length { get { return -1; } }
public override long Position { get { return -1; } set { } }

public override void Flush() { }
public override long Seek(long pos, SeekOrigin origin) { return -1; }
public override void SetLength(long length) { }

public override int Read(byte[] buffer, int offset, int count) {
return underlyingStream.Read(buffer, offset, count);
}

public override void Write(byte[] buffer, int offset, int count) {
}
}

public class SynchronousCompletionHandler {
private bool firstCall;
private Stream underlyingStream;
private byte[] userArray;

public SynchronousCompletionHandler(Stream underlyingStream, byte[] userArray) {
this.underlyingStream = underlyingStream;
this.firstCall = true;
this.userArray = userArray;
}

public Stream Stream { get { return this.underlyingStream; } }
public bool FirstCall { get { bool retVal = this.firstCall; this.firstCall = false; return retVal; }
public byte[] UserArray { get { return this.userArray; } }
}

public class NonAsyncStream {
private static void Main(string[] args) {
try {
DoNoStackOverflow(args[0]);
Console.WriteLine("Past the NSOE method");
DoStackOverflow(args[0]);
} catch { Console.WriteLine("SOE");
} finally { Console.WriteLine("SOE");
}
}

private static void DoNoStackOverflow(string largeFile) {
SynchronousCompletionHandler sch = new SynchronousCompletionHandler(new JustAStreamWrapper(File.OpenRead(largeFile)), new byte[4096]);
sch.Stream.BeginRead(sch.UserArray, 0, sch.UserArray.Length, new AsyncCallback(End_ReadNoOverflow), sch);
}

private static void End_ReadNoOverflow(IAsyncResult ar) {
SynchronousCompletionHandler sch = ar.AsyncState as SynchronousCompletionHandler;
if ( sch == null ) {
return;
}

if ( ar.CompletedSynchronously && !sch.FirstCall ) {
return;
}

while(sch.Stream.EndRead(ar) > 0) {
// ProcessArray(sch.UserArray);
ar = sch.Stream.BeginRead(sch.UserArray, 0, sch.UserArray.Length, new AsyncCallback(End_ReadNoOverflow), sch);

if ( !ar.CompletedSynchronously ) { break; }
}

// Signal final completion
}

private static byte[] foo = new byte[4096]; // Very small so we overflow with smaller files
private static void DoStackOverflow(string largeFile) {
JustAStreamWrapper jasw = new JustAStreamWrapper(File.OpenRead(largeFile));


jasw.BeginRead(foo, 0, foo.Length, new AsyncCallback(End_Read), jasw);
}

private static void End_Read(IAsyncResult ar) {
Stream stream = (Stream) ar.AsyncState;

if ( stream.EndRead(ar) > 0 ) {
Console.WriteLine("One Time Through");
stream.BeginRead(foo, 0, foo.Length, new AsyncCallback(End_Read), stream);
} else {
Console.WriteLine("Less than good");
}
}
}
Wednesday, June 09, 2004 11:13 PM by C# Learner

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

I too find it odd that String.Split doesn't have an overload for splitting on strings.

In any case, I normally use the static method Regex.Split for this.
Wednesday, June 09, 2004 11:35 PM by Justin Rogers

# re: string.Split(char[]), but no string.Split(string)... Hum, we should fix that...

Unless the string you are splitting on is rather large the Regex methods are quite a bit slower than what I have above. The above is code very similar to what exists in the base BCL string methods for splitting on characters.

In addition there are some improvements that can be made to the above algorithm, if you are only interested in a portion of the splitting group, for instance:

Return me the first 6 splits only
Return me splits 2 and 8 only

The above prevents unnecessary allocation of intermediate strings, since they only allocate say 6 and 2 strings respectively, rather than an entire group of them. The methods can be programmed to short-circuit if a specfic number of splits aren't available, so they can be used for validation of split data.

In other words, the above is going to be extremely superior to anything that exists in the BCL at current.
Wednesday, June 09, 2004 11:37 PM by Justin Rogers

# re: A conversion from integer to long form english... I could write that ;-)

Converting back would require some math that is extremely lossy. If I were going to go backwards, I would consider the use of a large number library. Since the only publicly available one for .NET isn't that great, and I don't feel like releasing and supporting my own, I'll leave the conversion from English -> number as an exercise for someone else.

Thursday, June 10, 2004 4:17 AM by Wesner Moise

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Using [ThreadStatic] allows you to have thread-safe functions while pre-allocating buffers.
Thursday, June 10, 2004 4:28 AM by Justin Rogers

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

I looked at pre-allocated buffers, and they only solve the problem of small strings. Larger strings that extend past your buffer still require that you allocate a buffer large enough to do some work (again, since you can't do in place manipulations of strings for immutability reasons).

ThreadStatic variables are special, since they are stored in Thread Local Storage or what you call a TLS Slot. Each thread only has a fixed number of these slots, and accesing thread API's can be the death of an algorithm. I'll explain why in a future post when I demonstrate why a certain widely accepted class doesn't scale to this particular problem.
Thursday, June 10, 2004 4:54 AM by AT

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

An question.
Do your code reverse string correctly ?? ;o)
How about supplementary characters in Unicode ??

http://msdn.microsoft.com/library/en-us/intl/unicode_192r.asp
http://www.i18nguy.com/unicode/char-count.html

Thursday, June 10, 2004 5:29 AM by Justin Rogers

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

As for surrogate pairs, I think the only truly performant method of a string reverse could be done by the BCL itself. You see they hold information about a string, bit flags if you will, defining whether or not certain actions need to occur. I'd assume they would have a bit that determines whether or not surrogate pairs actually exist, and if they do, handle that string special case from the others that done.

Anyway, in an attempt to throw out some numbers, a surrogate aware version of the algorithm functions at about 2 seconds per million, while the fully optimized non surrogate aware functions at about .6 seconds. Currently the best algorithm I've received runs over my test data at ~1.4 seconds, but I sent the user a very similar version with better arithmetic I had previously developed that runs at ~1.2 seconds.

Once the strings get large the IsSurrogate calls get really expensive and the algorithm comes out at 35 seconds per million, and 7 seconds for the fully optimized non surrogate aware version.

So in a very long winded manner, YES, I do have an algorithm to compare if a user wants to submit a surrogate aware reversal routine.
Thursday, June 10, 2004 5:34 AM by Justin Rogers

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

I take some of that back. IsSurrogate takes an extremely long time to return a value, and I think there is a faster way to determine if a character is a surrogate. With this faster method the code runs in 22 seconds for long strings, and actually comes in at 1.2 for the smaller strings.
Thursday, June 10, 2004 6:02 AM by TrackBack

# A love little samples like this...

Thursday, June 10, 2004 6:02 AM by TrackBack

# A love little samples like this...

Thursday, June 10, 2004 8:35 AM by Daniel Turini

# That's why I hate "sealed" classes.

I we could derive from String, reversing a string would be a matter of overriding some methods and you could do a 'lazy' string reversion, e.g., str[i] would return base.str[Length - 1 - i].
Ok, some may say that this would be cheating. :)
Thursday, June 10, 2004 10:23 AM by Greg Knierim

# re: WinForms UI Thread Invokes: An In-Depth Review of Invoke/BeginInvoke/InvokeRequred

Excellent article...However, not ever using Thread Invokes, I must ask can someone give a good scenario of when I would use this architecture?
Thursday, June 10, 2004 12:05 PM by Sean Lynch

# Converting from english to number

I just used a bunch of case statments.
Thursday, June 10, 2004 12:41 PM by Andy Smith

# re: A conversion from integer to long form english... I could write that ;-)

I did it in 7 less lines of code...
Thursday, June 10, 2004 3:52 PM by Jerry Pisk

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Daniel, what would you return in a ToString() call? That's the point here...
Thursday, June 10, 2004 6:25 PM by Darren Neimke

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Function Method1(ByVal str As String) As String
If str Is String.Empty Then
Return String.Empty
End If

If str.Trim().Length() = 0 Then
Return str
End If

Dim chars As Char() = str.ToCharArray()
Array.Reverse(chars)
Return New String(chars)
End Function


Function Method2(ByVal str As String) As String
If str Is String.Empty Then
Return String.Empty
End If

If str.Trim().Length() = 0 Then
Return str
End If

Dim i As Integer = 0
Dim x As Integer = str.Length - 1
Dim rev As Char()
ReDim rev(str.Length - 1)

Do While i < str.Length
rev(i) = str.Chars(x)
i += 1
x -= 1
Loop

Return New String(rev)
End Function
Thursday, June 10, 2004 6:45 PM by Stuart Hallows

# re: A conversion from integer to long form english... I could write that ;-)

Twenty minutes? Pah! I knocked one out in ten.
BOSH.
Thursday, June 10, 2004 6:50 PM by Cory Smith

# re: Performance: Give me the algorithm for reversing a managed string that provides the maximum performance...

Darren is thinking along the same lines as myself. However, do need to point out that the Trim() method should be removed. Whitespace would be valid in this scenario and would improve the performance since you wouldn't be making a string copy of a potentially huge string just to do the Trim().
Thursday, June 10, 2004 8:25 PM by Justin Rogers