When Refactoring is YAGNI ...

I attended a Washington DC XP UG (WDCXP) meeting tonight for the first time.  It was a small turnout with Bryant Smith presenting on DIP (Dependency Inversion Principle) with a slant to persuading new developers to adopt agile techniques through design patterns.  The discussions were interesting and varied wildy across different aspects of design patterns and XP.

One question that has been bugging me with the project at one of my clients is the case for Refactoring.  As TDD developers, we all know that Refactoring is essential to atone for the sins of DoingTheSimplestThing.  But how do you reconcile the time spent refactoring with the features that the customer would have to give up for that time?  I am not talking about the little ExtractMethod or IntroduceExplainingVariable here - those are quick and easy ... I mean realizing a design pattern is now needed and implemented it.  The refactoring tools are very limited in the .NET world at the moment - meaning that this refactoring will have to be done by hand and then tested (with your existing suite of tests, of course!).  It is easier to just TODO the item and move on, especially when under a time crunch.  This accumulates a burden of TechnicalDebt as the project runs its course.  I wrote a little custom NAnt task a few months back to traverse our source code and put together a list of the all the TODO items - there are around 300!

Adam (a member of WDCXP) suggested two possible solutions:

1. Force refactorings to be done before checking the code in
2. Add needed refactorings when next a feature is added to that subsystem

Forcing the refactorings doesn't solve the lost time for other features problem.  I like the adding the refactorings as code is changed but I still feel a lot of those TODOs will be left undone.  Maybe that is OK?

Please note that this system and associated project is highly successful and is loved by developers and users alike (despite the many remaining TODOs).

How do you reconcile your time refactoring against lost value to the customer?

UPDATE:  More food for thought ... RefactoringTrumpsYagni ?