George V. Reilly's Technical Blog

Building a static website

2013-01-20T08:11:00Z

I've spent time over the last three weeks working on a new website for the Northwest C++ Users' Group. I blogged about the NWCPP website refresh over there. In brief, I moved the website from an instance of the Joomla Content Management System at Just Host to a static website generated by Pelican and hosted at Github Pages, and I'm happy with the results.

SerializationException: the constructor was not found

2011-04-22T22:39:00Z

I just finished another post at the Cozi Tech Blog, SerializationException: the constructor was not found

Security 101 for Developers

2011-04-22T22:31:00Z

The Cozi Tech Blog needed some love, so I wrote a post a couple of weeks ago on Security 101 for Developers.

Generating UUIDs in JavaScript

2010-05-04T22:15:00Z

Another Cozi Tech Blog post: Generating UUIDs in JavaScript

SQLAlchemy Sharding

2010-04-04T07:21:00Z

I wrote up some lessons that I learned about SQLAlchemy Sharding at the Cozi Tech Blog.

Engineering Jobs at Cozi

2010-03-27T06:03:00Z

Cozi is hiring. We have positions in Web Development, Software Engineering, and System Engineering at our headquarters in Seattle.

Full details at the Careers Page.

StackOverflow DevDays Seattle 2009

2009-10-27T07:17:00Z

I spent last Wednesday at Benaroya Hall, attending the Seattle edition of StackOverflow's traveling DevDays conference. It was well worth $99.

Joel Spolsky, owner of FogCreek Software and co-founder of StackOverflow, opened the conference with a keynote about the dichotomy of power and simplicity. People are happier when not overwhelmed with choices. Many of the choices that software forces users to make are essentially meaningless to the users. However, even though people want simplicity, they also want features and different people use different features. Powerful software sells more copies.

He argues that developers and designers should put in the extra work to make good choices on behalf of the users: don't make users feel bad about themselves. Undo is better than a confirmation dialog. You are not in charge of what your users do.

Scott Hanselman spoke about ASP.NET MVC. We're moving away from ASP.NET to Python, but if we were to use ASP.NET again, MVC would be a compelling feature. His presentation was entertaining, if gimmicky.

Rory Blyth introduced iPhone development, in a tone of snarky ambivalence. He mentioned the Stockholm Syndrome. He stressed that Apple's Design "Guidelines" are effectively laws: violate them and you won't make it into the App Store. Looks like there's a lot of tedious messing around to hook things up in Objective-C. At the very end, he briefly demoed MonoTouch, which seemeed a little less tedious.

Cody Lindley introduced jQuery. I've done a lot of work with jQuery, but I still learned a few things. He worked through five facets of jQuery: Find something, do something; Create something, do something; Chaining; Implicit iteration; and jQuery parameters. He has an ebook at jqueryenlightenment.com, which I just picked up.

Daniel Rocha of Nokia talked about the cross-platform Qt (/cute/) toolkit, which runs on Windows, Mac, and Linux. More importantly from Nokia's point of view, it runs on their smartphones. Nokia has changed the licensing of Qt—once very expensive for closed-source apps, it's now free for apps that don't modify the Qt source. Qt is for C++, but there are bindings for other languages, such as Python.

Joel Spolsky came back and treated us to a half-hour demonstration of Fogbugz 7, Evidence-Based Scheduling, and Kiln, their new hosted Mercurial repository. Not terribly interesting to me, but the conference was only $99.

Ted Leung gave us a rather dry Hacker's Introduction to Python from slides rendered unreadable by a poor choice of colors. I've done a lot of Python, so I didn't learn much new. pip is an easy_install replacement that uninstalls; zc.buildout assembles apps from multiple parts; bpython is a fancy REPL.

Dan Sanderson talked about Google App Engine and demoed building apps with Java and with Python. Looked pretty cool and straightforward. We probably won't go that route, since we're pushing data to Amazon's S3, so EC2 makes more sense for us.

Finally, Steve Seitz from the University of Washington gave a cool talk on Modeling the World from Internet Photos. Some of this technology ended up in Photosynth. See Building Rome in a Day for some demos.

Review: Pragmatic Version Control Using Git

2009-10-19T05:44:00Z

Title: Pragmatic Version Control Using Git
Author: Travis Swicegood
Rating: $stars(4)
Publisher: Pragmatic Bookshelf
Copyright: 2008
ISBN: 1934356158
Pages: 179
Keywords: computers
Reading period: 10–18 October, 2009

As part of my personal conversion to Git, I read Swicegood's Git book. It's a decent introduction to Git and you learn how to do all the basic tasks as well as some more advanced topics. The examples are clear and well-paced.

I would have liked to see more about collaboration and workflow in a DVCS world, perhaps a few case studies: how is Git used in the Linux kernel development process; how a small, distributed team uses Git and GitHub; how a collocated team migrates from more traditional tools.

The book avoids discussing the lower levels of the Git object model, which is a reasonable choice for a pragmatic guide.

Recommended.

Third-Party Cookies

2009-10-17T07:15:00Z

Over the last few weeks, I built a PHP application that overlays Approve 71 banners on profile pictures. The actual application is hosted in an iframe and lives on a server in a different domain, eq.dm, than the main server at approvereferendum71.org.

This works fine in most browsers. Then we started getting reports that it wasn't working in IE8 on Win7 RC1. The iframe content was blank.

Poking around, I found the problem with the Fiddler proxy. The landing page on eq.dm was supposed to stick some information into the PHP session, then redirect to a second page at the same site. The second page was in an endless loop, redirecting to itself. In Fiddler, I saw a different PHPSESSID cookie on each response, and no cookie in the requests.

After reading IE 8 only has access to session cookies, I told IE8 to Accept All Cookies and the iframe content appeared. That fixed it for me, but we could hardly ask people to lower their security sessions.

I created a P3P file for the second domain, using the IBM P3P Policy Editor. (KB 323752 has more background on P3P and third-party cookies.)

IE now worked at its default security level. Problem solved! Or so I thought.

A day later, we got reports of similar problems with Safari 4 on Mac OS X.

I sniffed the traffic with Wireshark. Same problem: the “third-party“ cookie wasn't being accepted by Safari.

Unfortunately, Setting cross-domain cookies in Safari indicated that there was no reasonable workaround.

We overcame the issue up playing some DNS games, which was only possible because we control both servers. The second server is now also acting as a subdomain of the first, at dev.approvereferendum71.org. We used ini_set("session.cookie_domain",".approvereferendum71.org") to scope the iframe cookies. I've tried it in a variety of Windows, Mac, and Linux browsers, and it works in all of them.

Gitting Along

2009-10-15T15:24:00Z

In the last few weeks, I've switched over to Git for most of my version-control needs, at home and at work, after putting it on the long finger for months.

We continue to use Subversion at work, but I've recently followed Pavel and Eric's lead in using git-svn. I work locally on my own private branches and git svn dcommit and git svn rebase occasionally. I'm primarily on Windows at work, but I have a Linux box and a Mac Mini too, while at home, I have a MacBook, a Linux netbook, and a Vista desktop. I'm using msysGit, occasionally supplemented by TortoiseGit and QGit. Pavel's on a Mac and Eric's mostly on Ubuntu, so git adoption was easy for them.

When I first tried git-svn under msysGit about a year ago, it didn't work worth a damn. Git-svn works fine now, but it's slow compared to the *nix implementation. The developers say that's due to the fork() emulation of the MSys/Cygwin layer. The rest of msysGit is much faster.

For my home needs, I've had private Subversion repositories at DevjaVu.com and OpenSvn.csie.org. DevjaVu has gone out of business and OpenSvn has been unavailable too often for my liking. It was time to find some new hosting.

I've experimented with private Git repositories at GitHub and ProjectLocker. GitHub is very nice, but charges for private repositories. ProjectLocker provides free private repositories, but is comparatively clunky.

ProjectLocker lets you set up a fresh repository on their server. They tell you how to clone from that, which is great for a new repository. But they don't tell you how to hook it up to an existing local repository. Since I had some difficulty in figuring it out, here's the recipe:

git remote add origin git-foobar@freeN.projectlocker.com:foobar.git
git pull origin master
... merge, local edits and commits ...
git push origin master

I found Git, Xcode and ProjectLocker and Cygwin, SSH and ProjectLocker useful in figuring this out.

Launching 32-bit applications from batchfiles on Win64

2009-09-12T06:16:00Z

I've been running the 64-bit version of Windows 7 RC since June. It's been quite painless on the whole.

One wrinkle that I ran into was with some batchfiles which launch applications in %ProgramFiles% (normally C:\Program Files). Due to the magic WOW64 redirector, 32-bit applications are actually installed into %ProgramFiles(x86)%—normally C:\Program Files (x86)—instead of %ProgramFiles%. This is transparent to the 32-bit applications, which think they're running in %ProgramFiles% (C:\Program Files).

However, the cmd.exe shell is 64-bit (unless you make a special effort to run the 32-bit cmd.exe in SysWOW64), so batchfiles see the 64-bit %ProgramFiles% which contains 64-bit applications.

Hence, a batchfile that launches an installed 32-bit application on Win64 must use %ProgramFiles(x86)%, not %ProgramFiles%.

It sounds trivial to have a batchfile detect which flavor of %ProgramFiles% it should use, but the parentheses in the environment variable name make it tricky to parse. On earlier versions of Win64, the environment variable was called %ProgramFilesx86%. Presumably they added the strange parentheses into the variable name because the directory name always included them.

Here's a tiny batchfile that will launch the 32-bit DiffMerge correctly on both Win64 and Win32 platforms.

@setlocal
@set _pf=%ProgramFiles%
@if not "[%ProgramFiles(x86)%]"=="[]" set _pf=%ProgramFiles(x86)%
@start "" /b "%_pf%\SourceGear\DiffMerge\DiffMerge.exe" %*

I long ago found that the safest way to test environment variables whose values may include spaces, is to surround them with both double quotes and square brackets.

Recovering photos from a corrupted card

2009-09-05T07:45:00Z

I had about 60 apparently corrupted photos on a CompactFlash card the other evening. It might have been due to Lightroom going berserk, but it was more likely from my pulling the card reader out of the computer without ejecting it first.

The photos wouldn't show up under Mac, Linux, or Windows. I tried to chkdsk the card under Windows, which complained about a “raw” disk. That led me to ZAR, the Zero Assumption Recovery toolkit. The evaluation copy retrieved the photos very nicely. Whew!

XKCD's Tech Support Cheat Sheet

2009-08-25T06:57:00Z

To my tech supportees:

XKCD's Tech Support Cheat Sheet nails it! This is essentially what I'm doing when I sit down at your computer and dig you out of your latest hole.

If only you'd make some more intelligent guesses for yourself, you might be able to solve more of your problems.

But it's not quite that simple.

You probably don't understand very much about what you're doing. I have enormous depth and breadth of experience which informs my investigation. I am—no false modesty here—a master of debugging. The extra context helps me hone in on a solution more often than not.

Still, do give the flowchart a shot. Being able to solve your own problems is very empowering.

Moving Photos in Lightroom

2009-08-18T05:42:00Z

When using Lightroom before, I was never able to figure out how to move photos from one folder to another. You'd think that you could just click on a photo and drag it. I just spent twenty minutes figuring out what I've been doing wrong. After you've selected multiple photos, click on the photo thumbnail and not the surrounding gray frame, and then you can drag the photos to the target folder.

I had become accustomed to clicking on the frames to multi-select photos, so naturally I assumed that was also how you dragged a set of photos. But clicking on a frame of a selected photo merely deselects all the other photos.

64-bit Windows 7

2009-07-11T06:12:00Z

I mentioned three weeks ago that I had just repaved my work dev box and installed the 64-bit version of the Windows 7 RC. Nine or ten years after I first ported parts of IIS to Win64, I am finally running my main desktop on 64-bit Windows. With one exception, it's been painless. Programs have just worked, devices have just worked. There are relatively few native x64 applications, but for the most part it doesn't matter. The cases where it does matter—e.g., shell extensions such as TortoiseSVN—are available as 64-bit binaries.

I briefly flirted with using the 64-bit build of Python, but realized that I would have to recompile several eggs as 64-bit binaries. That was too painful and the 32-bit binary did everything I needed.

Building in Visual Studio 2005 is noticeably faster. I'm not sure how much of it was due to accumulated cruft after 18 months on Vista, but builds there were very slow.

The one exception was a major problem for the first week and a half. Whenever I ran our ASP.NET web application, it would go berserk, eat up all 4GB of my physical RAM, push the working set of IIS's w3wp.exe to 12GB, and max out one of my 4 cores! The only way to maintain any sanity was to run iisreset every 20 minutes to gently kill the process.

WinDbg and Process Explorer showed that the rogue thread was stuck in a loop in mscorjit!LifetimesListInteriorBlocksHelperIterative<GCInfoLiveRecordManipulator>. I passed a minidump on to my former colleagues in IIS, who sent it to the CLR team. They said:

The only thing I can tell is that it is Regex, and some regex expression compiled down to a method with 456KB of IL. That is huge, and yes 12GB of RAM consumed for something like that is expected.

With that clue, I was able to track down the problem, a particularly foul regex, built from a 10KB string, with 32 alternating expressions, each of which contains dozens of alternated subexpressions. The string is built from many smaller strings, so it's not obvious in the source just how ugly it is. I commented out the new Regex() and my problems went away.

Regardless of how ugly the regex is, this is a major regression in the CLR. This code has been working without blatant problems for two years on the 32-bit flavors of XP, Server 2003, Vista, and Server 2008. I've been meaning to try this code on 32-bit Windows 7, but have been too busy.

(The original, long-gone author was apparently aware that the regex is expensive to create because he runs a background thread to new the regex, which should have told him something. We'll fix the code that uses the regex to do something saner, soon.)

All that aside, I've been happy with the 64-bit version of Windows 7.