December 2005 - Posts - Jon Galloway

December 2005 - Posts

MEME: Top 10 Recent Blogs

Back in September, Phil posted on a MEME that didn't get the attention it deserved - what are your Top 10 Recent Blogs ?

...The criteria is purposely vague and very simple. I want you to list ten blogs you've really enjoyed reading recently. Especially the underrated, undiscovered, recent finds. They don't have to be your favorite blogs of all time, just ones that have caught your notice recently.

Here are the official rules...

  • The Blog must have an rss (or ATOM) feed.

[Source: you've been haacked - MEME: Top 10 Recent Blogs ]

Well, it's taken me a while respond to this since I reed 1750 feeds1 and it's hard to pick favorites. With a huge blogroll, it's really important to keep on top of things or I very quickly end up with 7500 unread posts to slog through, but Phil's post prompted me to pick a top 10 list of feeds I'll read daily no matter how busy I've become.
 
I tend to favor metabloggers who share my feed addiction for my "daily" reading category, and read the others when I can find the time (usually on the third shift). Usually if someone says something important, one of these guys will pick it up. Also, since they're so busy reading about everything that's going on in the .NET development world (and presumably writing some code now and then), they're usually to the point and not likely to ramble about non-news, like how rude people on cell phones can be. Many of these blogs are pretty well known in the .NET development community, but if you're only going to read 10 blogs, these are the ones I recommend (in alphabetical order):
     
  1. Christopher Steen - Link blogger extrordinaire.
  2. Coding Horror (Jeff Atwood) - Excellent writer with a lot to say about how software development ought to be done, fun stuff about technology and games, must have tools and utilities, and regular doses of slick code.
  3. Computer Zen (Scott Hanselman) - Yep. If you don't know who this guy is, you've probably stumbled onto my blog by accident (or you're related to me). Famous for many things, including his Ultimate Developer and Power Users Tool List.
  4. Greg's Cool Thing of the Day - Consistently one of the first to write about new software releases, cool software finds, industry events, etc.
  5. Jason Haley - Power link blogger with regular posts into .NET decompiler technologies. Join the "I Love Jason Haley In A Totally Manly Non-Threatening Way Club" today!2
  6. Larkware (Mike Gunderloy) - Mike's Daily Grind is the blog I read if I can only read one. He's omnicient.
  7. Ode To Code (K. ScottAllen) - Great stuff on ASP.NET and .NET in general. Doesn't post quite as often as the rest of this group, but when he does I want to read it right away. How can you not read a blog that has C# code to display joke messages on HP LaserJet printers?
  8. Sam Gentile (feed) - His "New and Notable" linkblog posts are solid, he's got a lot to say about Test Driven Development, and he's a glutton for punishment with all the latest Windows Vista betas. [links updated]
  9. TheServerSide.NET - "Your Enterprise .NET Community"
  10. you've been HAACKED (Phil Haack) - Phil and I met through our blogs, and now I work with him at Veloc|IT. It's good to read your CTO's blog, even if he weren't as frighteningly smart as Phil. Good stuff on ASP.NET, code from his work on SubText and RSSBandit, and practical info on using patterns and framework design in real life applications.

What are your Top 10 Recent Blogs?

UPDATE: Scoblized on 4/3/06. Yippee! By the way, I'd of course recommend my blog as well...

1 That's partly due to the fact that I subscribe to each of the weblogs.asp.net feeds individually. The main feed only gives you the latest 25 posts, and since I use a desktop aggregator (RSSBandit) that's not running 24/7 I'd miss a lot of posts by only subscribing to the main feed.
T-Shirts on back order

Posted by Jon Galloway | 4 comment(s)
Filed under:

DOS format file (XSHD) for SharpDevelop, IMHO Instant Blogger, etc.

I use IMHO Instant Blogger to edit and post blog articles. One of my favorite features is the code snippet editor. It's built on the SharpDevelop text editor, which has support for language syntaxes which are defined in xml syntax highlight definitions (XSHD's). It's a nice way to show color coded source code on a web page.1

I post enough DOS scripts that I finally wrote a syntax file for the DOS Batch grammar. It's probably not 100% complete, but I grabbed the cammand from the DOS help command, some standard EXE's from c:\windows\system\, etc. It's easy to install for IMHO:

1. Save this text as C:\Program Files\Elite Agency\Imho Instant Blogger\formatters\CodeSnippet\styles\DOS-mode.xshd:

<?xml version="1.0"?>
<!-- 
syntaxdefinition for DOS Batch by Jon Galloway -->

<
SyntaxDefinition name "DOS" extensions ".cmd;.bat">
    
    <
Properties>
        <
Property name="LineComment" value="::"/>
    </
Properties>
    
    <
Digits name "Digits" bold "false" italic "false" color "DarkBlue"/>
    
    <
RuleSets>
        <
RuleSet ignorecase "true">
        
            <
Delimiters>~!%^*()-+=|\#/{}[]:;"'&lt;&gt; , .?</Delimiters>
        
            
<Span name "LineComment" bold "false" italic "false" color "Teal" stopateol "true">
                <
Begin>::</Begin>
            
</Span>
            
            <
Span name "LineComment" bold "false" italic "false" color "Teal" stopateol "true">
                <
Begin>rem</Begin>
            
</Span>
            
            <
Span name "Label" bold "true" italic "true" color "DarkBlue" stopateol "true">
                <
Begin>:</Begin>
            
</Span>
            
            <
Span name "String" bold "false" italic "false" color "Magenta" stopateol "true">
                <
Begin>"</Begin>
                
<End>"</End>
            
</Span>
            
            <
Span name "Char" bold "false" italic "false" color "Magenta" stopateol "true">
                <
Begin>&apos;</Begin>
                <
End>&apos;</End>
            </
Span>
            
            <
Span name "EnvironmentVariable" bold "true" italic "false" color "Blue" stopateol "true">
                <
Begin>%</Begin>
                
<End>%</End>
            
</Span>
            
            <
MarkPrevious bold "true" italic "false" color "MidnightBlue">(</MarkPrevious>
            
            <
KeyWords name "Punctuation" bold "true" italic "false" color "Black">
                <
Key word "?" />
                <
Key word "," />
                <
Key word "." />
                <
Key word ";" />
                <
Key word "(" />
                <
Key word ")" />
                <
Key word "[" />
                <
Key word "]" />
                <
Key word "{" />
                <
Key word "}" />
                <
Key word "+" />
                <
Key word "-" />
                <
Key word "/" />
                <
Key word "*" />
                <
Key word "&lt;" />
                <
Key word "&gt;" />
                <
Key word "^" />
                <
Key word "=" />
                <
Key word "~" />
                <
Key word "!" />
                <
Key word "|" />
                <
Key word "&amp;" />
            </
KeyWords>
            
            <
KeyWords name "Literals" bold="false" italic="false" color="Black">
                <
Key word "false" />
                <
Key word "true" />
            </
KeyWords>
            
            <
KeyWords name "IterationStatements" bold="false" italic="false" color="Navy">
                <
Key word "do" />
                <
Key word "for" />
                <
Key word "while" />
            </
KeyWords>
            
            <
KeyWords name "ManagedIterationStatements" bold="false" italic="false" color="Navy">
                <
Key word "foreach" />
                <
Key word "in" />
            </
KeyWords>
            
            <
KeyWords name "JumpStatements" bold="false" italic="false" color="Navy">
                <
Key word "break" />
                <
Key word "continue" />
                <
Key word "goto" />
                <
Key word "return" />
                <
Key word "call" />
            </
KeyWords>
            
            <
KeyWords name "SelectionStatements" bold="false" italic="false" color="Navy">
                <
Key word "if" />
                <
Key word "else" />
                <
Key word "case" />
                <
Key word "choice" />
            </
KeyWords>
            
            <
KeyWords name "Environment" bold="false" italic="false" color="Navy">
                <
Key word "@" />
            </
KeyWords>

            <
KeyWords name "CommonPrograms" bold="true" italic="false" color="Black">
                <
Key word ".exe" />
                <
Key word "devnev" />
                <
Key word "msbuild" />
                <
Key word "append" />
                <
Key word "at" />
                <
Key word "calc" />
                <
Key word "chkdsk" />
                <
Key word "defrag" />
                <
Key word "cscript" />
                <
Key word "defrag" />
                <
Key word "doskey" />
                <
Key word "ftp" />
                <
Key word "iisreset" />
                <
Key word "ipconfig" />
                <
Key word "makecab" />
                <
Key word "netsh" />
                <
Key word "netstat" />
                <
Key word "notepad" />
                <
Key word "ntbackup" />
                <
Key word "print" />
                <
Key word "ping" />
                <
Key word "regedit" />
                <
Key word "regedt32" />
                <
Key word "regedit" />
                <
Key word "soon" />
                <
Key word "sort" />
                <
Key word "shutdown" />
                <
Key word "rundll32" />
                <
Key word "regsvr32" />
                <
Key word "subst" />
                <
Key word "runonce" />
                <
Key word "telnet" />
                <
Key word "write" />
                <
Key word "xcopy" />
                <
Key word "wscript" />
            </
KeyWords>
            
            <
KeyWords name "BatchCommands" bold="false" italic="false" color="DarkBlue">
                <
Key word "assoc" />
                <
Key word "at" />
                <
Key word "attrib" />
                <
Key word "break" />
                <
Key word "cacls" />
                <
Key word "call" />
                <
Key word "cd" />
                <
Key word "chcp" />
                <
Key word "chdir" />
                <
Key word "chkdsk" />
                <
Key word "chkntfs" />
                <
Key word "cls" />
                <
Key word "cmd" />
                <
Key word "color" />
                <
Key word "comp" />
                <
Key word "compact" />
                <
Key word "convert" />
                <
Key word "copy" />
                <
Key word "date" />
                <
Key word "del" />
                <
Key word "dir" />
                <
Key word "diskcomp" />
                <
Key word "diskcopy" />
                <
Key word "dosKey" />
                <
Key word "echo" />
                <
Key word "endlocal" />
                <
Key word "erase" />
                <
Key word "exit" />
                <
Key word "fc" />
                <
Key word "find" />
                <
Key word "findstr" />
                <
Key word "for" />
                <
Key word "format" />
                <
Key word "ftype" />
                <
Key word "goto" />
                <
Key word "graftabl" />
                <
Key word "help" />
                <
Key word "if" />
                <
Key word "label" />
                <
Key word "md" />
                <
Key word "mkdir" />
                <
Key word "mode" />
                <
Key word "more" />
                <
Key word "move" />
                <
Key word "path" />
                <
Key word "pause" />
                <
Key word "popd" />
                <
Key word "print" />
                <
Key word "prompt" />
                <
Key word "pushd" />
                <
Key word "rd" />
                <
Key word "recover" />
                <
Key word "rem" />
                <
Key word "ren" />
                <
Key word "rename" />
                <
Key word "replace" />
                <
Key word "rmdir" />
                <
Key word "set" />
                <
Key word "setlocal" />
                <
Key word "shift" />
                <
Key word "sort" />
                <
Key word "start" />
                <
Key word "subst" />
                <
Key word "time" />
                <
Key word "title" />
                <
Key word "tree" />
                <
Key word "type" />
                <
Key word "ver" />
                <
Key word "verify" />
                <
Key word "vol" />
                <
Key word "xcopy" />
            </
KeyWords>

        </
RuleSet>
    </
RuleSets>
</
SyntaxDefinition>

2. Add add a reference to it in C:\Program Files\Elite Agency\Imho Instant Blogger\formatters\CodeSnippet\styles\SyntaxModes.xml by adding this line:

<Mode file = "DOS-Mode.xshd" name = "DOS Batch (by Jon Galloway)" extensions = ".bat;.cmd;"/>

I also wrote up a simple Quote XSHD with no keywords in case I want an easy way to drop a quote in a scrollable frame (installation is the left as an exercise for the reader):

<?xml version="1.0"?>
<SyntaxDefinition 
name = "Quote" extensions = ".txt">
    <Digits 
name = "Digits" bold = "false" italic = "false" color = "Black"/>
    <RuleSets>
        <RuleSet>
        <
/RuleSet>
    <
/RuleSets>
<
/SyntaxDefinition>

1 I know there are some nice solutions for doing this in Visual Studio (Jeff Atwood's VS addin, Copy Source as HTML), but I like having this built into the blog editor; plus this formatter does some cool stuff like copy the code into a scrolling frame.  

Posted by Jon Galloway | with no comments
Filed under:

IE7 Standalone Launch Script

As of IE7 Beta 3, this launch script no longer works. I've released a new version which works with newer IE7 versions (including RC1). Read about it here, then download it here


UPDATES
8/31/2006 - Version 1.6 works with IE7 RC1. Grab it here.
7/26/2006 - Version 1.5 works with IE7 Beta 3, but requires a few more files so I've bundled it into a zip file for distribution. Grab it here.
6/29/2006 - This isn't working with IE7 Beta 3 yet. I'll have to test it out more and probably release a new version.
2/8/2006 - Version 1.4 temporarily updates the IE version to work with conditional statements (<!--[if lt IE 7.0]>) based on a recommendation by Thomas Meinike. 3/21/2006 - This build has been tested with the IE7 B2 Preview released on 3/20/2006. It works, in that it allowed me to browse sites in IE7 without messing up IE6 or my default browser association. However, I noticed high CPU usage while browsing and had to close IE7 using the task manager when I was through - it didn't respond to that little red X thing. I believe these are minor inconveniences compared to running a separate virtual machine just to check how a site looks. Obviously, this build of IE7 just came out yesterday so I haven't put this through rigorous testing.
2/2/2006 - Version 1.3 works with IE7 Beta 2 Preview.

The IE6 Cumulative Security Update (Dec 13) broke the IE Standalone Mode that's been around since IE3 and is a big help in designing for forward compatibility. The IE team's response is that the standalone mode is not supported, so do a full install of the IE7 beta if you want to use it. After a short bit of grumbling, I'll tell you how you can still run IE7 in standalone mode despite the security update.

I hope the "by design" solution is a temporary workaround that will be fixed soon, for a lot of reasons:

  • It's going to be really annoying for web developers to have to test on IE6 and IE7 if they can't both run on the same machine
  • It's not the developers' fault
  • Other browsers allow running multiple versions on the same machine
  • IE is making a lot of breaking changes between versions 6 and 7
  • IE's "standard defiance" virtually required (or highly encouraged) non-standard HTML / CSS / Javascript
  • The standalone mode, unsupported or not, has worked for like 10 years now

The symptoms of this recent problem are interesting - in my case, every time I clicked a link in IE it would open in Firefox (my default browser). There are lots of other strange effects other people are seeing, such as blank windows, hanging, and a ringing in the ears. The problem is that IE7 writes a registry key that causes IE6 to shift to an also undocumented "evil" mode, and it's necessary to delete this registry key after running IE7 and before running IE6 with the December 05 cumulative security update.

Okay, on to the solutions...

If you just want to delete the registry key and follow the rules (don't use IE7 at all, take the plunge and use IE7, or use a virtual machine with VirtualPC or VMWare), then you can just copy the following text into notepad, save it as ie7fix.reg, and double-click it. Note: this is not the launch script, it's just a cleanup in case you've used the launch script before and it's causing problems.

Windows Registry Editor Version 5.00

[-HKEY_CLASSES_ROOT\CLSID\{C90250F3-4D7D-4991-9B69-A5C5BC1C2AE6}]

But if you'd like to keep running IE7 in standalone mode, you can use the following DOS batch script. You'll use the batch file to launch IE7, and it will stay active as long as IE7 is running. As soon as you shut IE7 down (that red X button or File / Exit), the batch file will clean up after IE7 so your other browsers will continue to work.

How to use the IE7 Launch Script:

  1. If you already have IE7 set up for standalone mode, skip to step 4.
  2. Download the IE7 Beta Install from MSDN. I know there are versions of this floating around the intarwebs, which may or may not carry the ebola virus.
  3. Open the install file with a decompression program like WinRAR (yep, it's an EXE, but it's a self extracting EXE) and extract them to a folder. Alternatively, you can just run the EXE, make a copy of the files which are extracted in the first step, and cancel the install. I was too chicken to do this on my work computer. There were some other steps to the setup which I've included in the batch file below - if you want to know what they are, read the original info here.
  4. Copy the text below to notepad and save it as IE7.bat in the same folder you're setting up the standalone IE7 copy.
  5. When you want to run IE7, double click IE7.bat - it pops up a DOS window that will hang around as long as IE7 is running so it can clean up after it (by deleting that pesky registry key) when you close IE7. Don't close the DOS window down or your pancreas will implode.
  6. When you're done with IE7, just shut it down. The DOS window will clean up that pesky registry key and delete the standalone files, then disappear. If you shut IE7 down and the DOS window hangs around, just close it and run it again.
Update - If setting up the launch script is more work than you'd like, and you don't mind downloading from a non-Microsoft source, you can easily find zipped IE7 redistributions with the launch script already set up.

@ECHO OFF
TITLE IE7 Launcher 1.4

ECHO 
IE7 STANDALONE LAUNCHER 1.4
ECHO 
Updated for IE7 Beta Preview
ECHO.
ECHO Do not close this window or it will not clean up after itself properly.
ECHO You can pass a URL into this batch filelike this:
ECHO ie7.bat www.microsoft.com
ECHO.
ECHO More info here: http://weblogs.asp.net/jgalloway/archive/2005/12/28/434132.aspx
ECHO.
ECHO When you close IE7this will remove the registry key and shut itself down.
ECHO.
ECHO Setting up IE7 for standalone mode...
PUSHD %~dp0

ECHO Removing IE7 registry key and set the version vector to "7.0000".
%TEMP%.\IE7Fix.reg ECHO REGEDIT4
>>%TEMP%.\IE7Fix.reg ECHO.
>>
%TEMP%.\IE7Fix.reg ECHO [-HKEY_CLASSES_ROOT\CLSID\{C90250F3-4D7D-4991-9B69-A5C5BC1C2AE6}]
>>
%TEMP%.\IE7Fix.reg ECHO [-HKEY_CLASSES_ROOT\Interface\{000214E5-0000-0000-C000-000000000046}]
>>
%TEMP%.\IE7Fix.reg ECHO [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Version Vector]
>>
%TEMP%.\IE7Fix.reg ECHO "IE"="7.0000"
>>%TEMP%.\IE7Fix.reg ECHO.
:: Merge the REG file to delete the IE7 standalone entry
REGEDIT /%TEMP%.\IE7Fix.reg

REN SHLWAPI.DLL SHLWAPI.DLL.BAK
TYPE NUL IEXPLORE.exe.local
ECHO Running IE7...
iexplore.exe "%1"

:: Merge the REG file to delete the IE7 standalone entry
REGEDIT /%TEMP%.\IE7Fix.reg
:: Delete the temporary REG file
DEL %TEMP%.\IE7Fix.reg

ECHO Removing IE7 standalone files...
REN SHLWAPI.DLL.BAK SHLWAPI.DLL
DEL IEXPLORE.exe.local

:: Set the old version vector "6.0000".
%TEMP%.\IE7Fix.reg ECHO REGEDIT4
>>%TEMP%.\IE7Fix.reg ECHO.
>>
%TEMP%.\IE7Fix.reg ECHO [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Version Vector]
>>
%TEMP%.\IE7Fix.reg ECHO "IE"="6.0000"
>>%TEMP%.\IE7Fix.reg ECHO.
REGEDIT /
%TEMP%.\IE7Fix.reg
DEL %TEMP%.\IE7Fix.reg

POPD
ECHO 
Completeclosing...

Please leave any comments here.

Farewell to IE Mac and a checkup on my IE recommendations from June 2004

"A while ago, Microsoft stopped updating IE for Mac, freezing it at version 5. But according to this Microsoft webpage, all support will cease December 31, 2005, and any official distribution with cease January 31, 2006. Also, the webpage suggests 'that Macintosh users migrate to more recent web browsing technologies such as Apple's Safari.'"  Source: Slashdot - Microsoft Ends IE for Mac

I'm glad to see IE5 for Mac being mercy killed. It was a decent browser when it was released, but was condemned to a hideous undead state when development pretty much halted back in 2001. There never was an IE6 for Mac, and Mac IE5 is a pain to support. This may cause some short term problems for Mac users who use (poorly written) sites that are IE specific, but Mac IE5 is not a modern browser and it's time for us to move on. Did we plan to keep using IE5 forever?

Back in June 2004 I wrote about IE Market Share... and Why It Matters. I'll just list the bullet points here; if you want to know more go read it (it's a quick read). Since killing Mac IE was one of my 4 recommendations, it's time for a checkup:

A. IE Usage dropping
B. Why is this happening?
C. Why should Microsoft care?

  1. IE is one of the most frequently used Microsoft applications
  2. It just looks bad for Microsoft to crush the competition and then stop developing
  3. It could precipitate the kind of Open Source badness Microsoft doesn't want to think about (Firefox as the gateway drug to open source / non-MS products)

D. What should happen to IE?

  1. Mac IE 6... or none at all - GRADE: COMPLETE
    Either pick up support for Mac or politely push users off IE onto Safari / Firefox. There's no benefit to having Mac users running a clunky Microsoft browser, it's bad business. Lead or get out of the way. 
    (and from the comments)
     ...Now I know this is crazy talk, but I think just announcing you've stopped developing an application and forgetting about it isn't the best thing, even for Microsoft. You end up with frustrated users who can't do what they want (e.g. WYSIWYG HTML editing with a contentEditable textbox) because the "stupid Microsoft browser doesn't work." So they'll eventually move to a more modern browser, but under less than amicable conditions. I think it's worth considering an official policy from Microsoft that encourages users to upgrade to a newer browser. Hey, we're no longer updating this browser, but there are a few great alternatives that we encourage you to try out. That shows integrity, and why not do it? It's a free application anyway, and you've already gotten all the goodwill out of it you're going to - now get that last bit of goodwill by bowing out gracefully.
    Will be complete January 2006. I'd have liked to have seen it sooner, since it's a pain to support, but I understand that MS has support and end of life policies here.

  2. Standards and Technical Parity with Mozilla browsers - GRADE: NEEDS IMPROVEMENT
    PNG transparency, CSS support, SVG support, etc. Others have spelled it out in greater detail, so I'm not gonna rehash it. I'll summarize it - make web developers happy, because they are the influencers you want on your side. (A personal request - inline images - Mozilla supports it...).
    PNG support, a lot of improvement in CSS. No progress on SVG or Javascript. I don't think IE7 is stacking up well against Firefox 1.5, but it's much better than IE6.

  3. Security - STATUS: SATISFACTORY
    Sure, goes without saying. Needs the same attention Microsoft's been giving all their products on security. I think this has been the only real positive thing here lately, though - security doesn't seem to be as huge issue for IE as it has been in the past. (wherein I invoke the wrath of my largely theoretical user base and they comment flame me to death).
    No, it's not 100% secure, but neither are any of the other browsers. Between IE6 for XP SP2 / 2003 SP1 and IE7, I'm happy with the progress here.

  4. A Roadmap GRADE: INCOMPLETE
    Microsoft's abandoned IE before, so they need to show us this isn't a bouquet and a peck on the cheek. We don't want you back for a weekend, not back for a day (no no no), I said IE I just want you back, and I want you to stay.... We know Longhorn's coming some day, but a lot of folks are going to be on XP or something else for several years to come. Tell them why they sould stay with IE before they feel the need to leave.
    Where is the IE Roadmap? I can't find anything on the IE Home Page. I'm not talking about a feature list for IE7, I'm talking about a published plan for where IE is going in the future. Firefox just released 1.5, but you can already read about what to expect in the Firefox 2.0 release and a decent start on the Firefox 3.0 product requirements. The best I can find for IE is a PowerPoint from WINHEC which is mostly marketingspeak. The .NET and Visual Studio teams have been very open about where the languages and technologies are headed, and IE needs to follow suit. (I'm hoping they've done this and I'm just not aware of it - if so, let me know and I'll update this).

[OT] GMail Mobile is finally here

I've been waiting for this one for a while - GMail finally has a mobile version at http://m.gmail.com. I've occasionally used Pocket Gmail, but Gmail Mobile is a lot smoother. Just in time for vacation, too.

Snip from the product announcement :

Gmail Mobile

Gmail Mobile
Now you can access your Gmail account from your mobile phone or device. Just point your phone's web browser to http://m.gmail.com. Your Gmail account stays synched, whether you access it from the web or the mobile interface. It's easy to use and it's free (but yes, your wireless plan could still charge you).

It also has these cool features:
• Automatically optimizes the interface for the phone you're using
• Opens the attachments you receive in messages, including photos, Microsoft Word documents and .pdf files
• Lets you reply by call to people whose phone numbers are in your Gmail Contacts list

Learn more

 

Posted by Jon Galloway | with no comments
Filed under:

[code] MapPoint - CSS Uploads with Zip and Chunking

The Microsoft MapPoint Web Service allow for uploading location data in XML following the Access 2003 XML format. The Customer Data Service limits uploads to 1MB chunks, but it allows the XML to be zip compressed. That's nice because zipping XML can cut the file size by up to 90%.

I didn't find any good sample code for zipped, chunked uploads from an XML string, despite some concerted googling. The MapPoint SDK sample code fulfils the technical requirements without being at all useful in real life - it shows how to upload a file. If you were going to upload a file, you might as well use the MapPoint CDS website.

In my case, I got a serializable object from a webservice, serialized it to XML, transformed it with XSLT to Access 2003 XML, zipped it, and did a chunked upload to the MapPoint CDS webservice. I'm not going to go through the process of transforming XML - there are plenty of good references for that. I'm goint to assume you've got an XML string in Access 2003 XML format and you're ready to upload it. This code requires a WebReference to the MapPoint CSS webservice. This is .NET 1.1 code so I'm using SharpZipLib for zip compression; if you're running on .NET 2.0 you've got native zip compression so you can eliminate that dependency.

using System;
using System.Configuration;
using System.IO;
using System.Xml;
using System.Xml.Serialization;

using Mappoint;
using MappointCSS;
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
using ICSharpCode.SharpZipLib.Zip;

namespace MappointUploader
{
    
public class UploadWorker
    {
        
private bool WriteLocalFiles;

        
public UploadWorker()
        {
        }

        
public void Process()
        {
            outputConsole("Starting process.");
            outputConsole("Connecting to CRM Webservice.");
            WriteLocalFiles = (ConfigurationSettings.AppSettings["WriteLocalFiles"] == 
bool.TrueString);

            
string uploadXML = string.Empty;
            
            
//TODO: Set uploadXML to an XML string in Access 2003 XML format.
            // In my case, I'm calling a webservice and transforming the output with an in-memory XSL transform.

            //Write file for debugging / production support
            
writeToFile(uploadXML, @"XML\TransformOutput.xml");  
            
            
//Load string into byte array and compress using "upload.xml" as the document name inside the zip
            
byte[] uploadBytes = System.Text.Encoding.UTF8.GetBytes(uploadXML);
            outputConsole("Compressing XML (" + uploadBytes.Length + " bytes uncompressed).");
            uploadBytes = compress(uploadBytes,"upload.xml");
            outputConsole("Completed compressing XML (" + uploadBytes.Length + " bytes compressed).");

            
//Connect to Mappoint Customer Data Service for upload
            
UploadSpecification uploadSpec = new UploadSpecification();
            System.Net.NetworkCredential credentials = 
new System.Net.NetworkCredential( 
                ConfigurationSettings.AppSettings["CSSUser"],
                ConfigurationSettings.AppSettings["CSSPass"], "PARTNERS");

            CustomerDataService cds = 
new CustomerDataService();
            cds.Credentials = credentials;
            cds.PreAuthenticate = 
true;

            UploadSpecification spec = 
new UploadSpecification();
            spec.DataSourceName = ConfigurationSettings.AppSettings["Mappoint.DataSourceName"];
            spec.EntityTypeName = ConfigurationSettings.AppSettings["Mappoint.EntityTypeName"];
            
if(ConfigurationSettings.AppSettings["Environment"] == "Service")
            {
                spec.Environment = LocationDataEnvironment.Service;
            }
            
else
            
{
                spec.Environment = LocationDataEnvironment.Staging;
            }
            spec.MaximumGeocodingLevel = GeocodingLevel.Street;
            spec.RejectAmbiguousGeocodes = 
false;

            
string jobID = cds.StartUpload(spec);
            outputConsole("Connecting to Mappoint CDS (" + spec.Environment + " environment, Job ID " + jobID + ").");

            writeToFile(uploadBytes, @"XML\TransformOutput.xml.zip");

            
int byteCount = uploadBytes.Length;
            
long bytesUploaded = 0;
            
int chunkSize = 1000000;
            
byte[] chunk;

            
while(bytesUploaded < byteCount)
            {
                
if(byteCount-bytesUploaded > chunkSize)
                {
                    chunk = 
new byte[chunkSize];
                }
                
else
                
{
                    chunk = 
new byte[byteCount-bytesUploaded];
                }
                
                Array.Copy(uploadBytes, bytesUploaded, chunk, 0, chunk.Length);
                bytesUploaded = cds.UploadData(jobID, chunk, bytesUploaded);
                outputConsole("Uploaded " + bytesUploaded + " bytes.");
            }

            cds.FinishUpload(jobID, bytesUploaded);
            outputConsole("Upload complete.");
        }

        
private void writeToFile(string input, string filename)
        {
            
byte[] buffer = System.Text.UTF8Encoding.UTF8.GetBytes(input);
            writeToFile(buffer, filename);
        }

        
private void writeToFile(byte[] buffer, string filename)
        {
            
if(WriteLocalFiles)
            {
                
using (FileStream fs = new FileStream(filename,FileMode.Create))
                {
                    fs.Write(buffer,0,buffer.Length);
                }
            }
        }

        
private byte[] compress(byte[] buffer, string entryFileName)
        {
            MemoryStream memory = 
new MemoryStream();
            ZipOutputStream stream = 
new ZipOutputStream(memory);
            stream.IsStreamOwner = 
false;
            stream.SetLevel(5);
            ZipEntry entry = 
new ZipEntry(entryFileName);
            entry.DateTime    = DateTime.Now;
            entry.Size        = buffer.Length;
            ICSharpCode.SharpZipLib.Checksums.Crc32 crc = 
new ICSharpCode.SharpZipLib.Checksums.Crc32();
            crc.Reset();
            crc.Update(buffer);
            entry.Crc = crc.Value;

            stream.PutNextEntry(entry);
            stream.Write(buffer, 0, buffer.Length);
            stream.Finish();
            stream.Close();
            
return memory.ToArray();
        }

        
private void outputConsole(string message)
        {
            
string output = string.Format"{0} : {1}", DateTime.Now.ToLongTimeString(), message);
            
using (StreamWriter sw = File.AppendText("RunStatus.log"))
            {
                sw.WriteLine(output);
            }
            Console.WriteLine(output);
        }
    }
}

Now, about that Access 2003 XML data format. It's finicky, since it includes XSD which describes the data format of all columns, and CDS rejects the upload if it doesn't match. One trick is to model it in a simple Access table and export as XML (with XSD), then use that in your XSL template or XML source. Get something really simple working and add a field at a time if necessary.

Remember that MapPoint requires a few fields in your upload data. - EntityID, Latitude, and Longitude. The EntityID must be a unique integer. Since my datasource used a GUID identifier, I manufactured a unique int ID in the XSLT:

...
<EntityID>
    <xsl:number 
value="position()" format="1" />
<
/EntityID>
...

The Latitude and Longitude are required double (SQL float) columns; if a value is present then MapPoint will not geocode the position and will use the provided Lat / Long values. Even if you're not planning on overriding geocoding, though, these columns must be present. Robert McGovern's MapPoint article on DevEx provides a valid sample XML file with EntityID, Latitude, and Longitude that you can buld on, and there's more info on MSDN - Mappoint Data Source Format.

Posted by Jon Galloway | 1 comment(s)
Filed under:

DOS batch script to restore a SQL 2000 database and grant permission to ASPNET account

I've found the most reliable method of  moving SQL Server databases between servers is a simple backup / restore. The restore is a bit of a pain in the neck, though - it takes some unnecessary clicky clicky in the SQL Server Enterprise Manager (ugh).

Here's a simple DOS batch script that does away with most of the repetitive work. I put a copy of this in a \Project\Database directory, fill in the DBNAME variable, and from then on the restore is as simple as copying the SQL Backup file over and running the batch file. The granting permission to the ASPNET user thing is important if you're using trusted connections, which is a durn good idea. Barry Dorrans has some great info on that here.  

::SET VARIABLES
set DBNAME=NAMEOFDATABASEBEINGRESTORED
set DBDIRECTORY=C:\Program Files\Microsoft SQL Server\MSSQL\Data

TITLE Restoring 
%DBNAMEDatabase

::PUT DATABASE IN SINGLE USER MODE TO ALLOW RESTORE
osql -E -d master -Q "alter database %DBNAME% set single_user with rollback immediate"

::RESTORE DATABASE
osql --d master -Q "restore database %DBNAMEfrom disk='%~dp0\%DBNAME%.bak' WITH MOVE '%DBNAME%_Data' TO '%DBDIRECTORY%\%DBNAME%_Data.MDF' MOVE '%DBNAME%_Log' TO '%DBDIRECTORY%\%DBNAME%_Log.LDF'"

::GRANT PERMISSION TO ASPNET USER
osql --%DBNAME% -Q "sp_grantdbaccess '%COMPUTERNAME%\ASPNET'"
osql 
--%DBNAME% -Q "sp_addrolemember 'db_owner''%COMPUTERNAME%\ASPNET'"

::RESTORE TO MULTI USER
osql -E -d master -Q "alter database %DBNAME% set multi_user"

pause

Usage notes:

  1. Replace DBNAME variable with the actual database name.
  2. restore database osql command should all be on one line; it's just formatted as is to look pretty. (fixed)
  3. This assumes you're using some defaults - for instance, you're saving your SQL database backup as DBNAME.bak, your database file names are DBNAME_Data.MDF and DBNAME_Log.LDF, and your DBDIRECTORY is the default directory. If this isn't the case, it's pretty easy to change the script to suit your bizarre tastes.
  4. This is built to be run in the same directory as the backup file. If that's not the case, replace the %~dp0 silliness with the path to the backup file.
  5. You're running this on a server that can stand a SQL Server restart. I use this on my development server, so that's just fine. (fixed, see update note below)

UPDATE: Original version restarted SQL Server before performing the restore. Based on Sean's comment, I've changed this to use single user mode instead.

Posted by Jon Galloway | 1 comment(s)
Filed under: ,

The software market doesn't reward security, it just punishes perceived insecurity

Random thought: The market hasn't rewarded Microsoft's recent security initiatives (W2K3, XPSP2 etc.). Microsoft pumped a ton of development (read money) into their recent security efforts, which dramatically reduced their attack surfaces. That's a good long term move, but I don't think it paid off in immediate sales. I can say from my personal experience that administrators talk about the improved security they're getting when they install W2K3 for other reasons, but no one moves to W2K3 to get secure.

The market doesn't reward security. It may punish insecurity, but doesn't seem to reward secure OS's or software with increased sales. Despite the fact that W2K3 is much more secure than W2K, it doesn't look like the investment directly paid off in sales revenue.

Posted by Jon Galloway | with no comments
Filed under:

[tip] localhost vs. (local) in SQL Server connection strings

Sample code with SQL Server connection strings often use localhost and (local) interchangeably. They're different.

Server=(local);Database=DotNetNuke;Trusted_Connection=True
Uses named pipes

Server=localhost;Database=DotNetNuke;Trusted_Connection=True
Uses a TCP port negotiated on port 1434 udp, which defaults to 1433

There are many differences between TCP and Named Pipe connection, but if you're on localhost you're mostly concerned with simple access.

  • The default ASPNET account generally has an easier time with TCP, since the  ASPNET user doesn't have access to named pipes by default (http://support.microsoft.com/Default.aspx?id=315159).
  • If you're using impersonation, Named Pipes is usually simpler. If you're using impersonation with no username specified, you're running under the IIS Authenticating user. This defaults to IUSR_MACHINENAME if you're allowing annonymous access, which generally has access to the IPC$ share required for named pipe communications.  

As mentioned in a comment on Peter Van Ooijen's blog a while ago, the easiest setup is to avoid either local or localhost and just use the machine name (e.g.  Server=COMPY386;Database=DotNetNuke;Trusted_Connection=True). Note that this will use TCP/IP rather than named pipes. This isn't always practical in a group development situation where each developer is running a SQL Server instance since each machine will have a unique name.

Info on troubleshooting SQL Server 2000 connectivity here.
Also check out this great info on ASPNET connectivity to SQL Server by RupW in the GDN message boards.

Posted by Jon Galloway | 19 comment(s)
Filed under: ,
More Posts