Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

If you've never used Microsoft Money, you can simply move on to other posts..
 
One of the things I've done lately is try to use Microsoft-Money to manage my business and home finances. It's quite a lovely program, really, and I wish it had better support for banks here in Israel. In fact, that's one of the biggest gripes about it here, and one of the main reasons Israelis don't use it. It's mainly meant to be used for the US/EU crowd, it seems. The program is able to download and update data directly from your local bank, but no Israeli Banks are available to be used with it.
We work with the "Poalim" bank here in Israel, the largest in the county if I'm not mistaken. The bank has a very good and secure website which allows practically running 90% of your tasks online,including reviewing latest transactions on your checking and credit card accounts.
I tried to find a way for Money to download data directly from my own bank but Nada. Money does have the ability to import "QIF" files - "Quicken Interchangeable Format" files, which contain the latest movements from your bank. IN fact, it seems that these are the kind of files that US and UK banks provide to MS Money for download and update.
If only my bank, Poalim, supported creating QIF files. It doesn't.
 
So, I set out to create my own QIF imported from my bank account. It's very much a hack, but it works. Here's how:
  1. I go to my bank's main page where I can see my transaction list (after login)
  2. I Copy the table of transactions: I select the table with the mouse and press CTRL-C
  3. I open my own little software hack windows form and paste the text into it.
  4. Using regular expressions, I parse the text and create a small dataset from it with a row for each transaction
  5. I then write the context of the dataset into a temp QIF file
  6. The program also copies the file's full path into the windows clipboard
  7. I open MS Money and click File-Import-Download statement
  8. I "paste" the file path to the new QIF file and press OK
  9. Money imports the data from my bank into its records.
That's it. The whole process takes less than 30 seconds to perform, and now I can actually manage my accounts without manually entering all that data which my bank already has in store for me.
A few more things about my little hack:
  • My bank has listings for credit card transactions and Checking account transactions. Both have different tables and fields. My app knows how to recognize "pasting" from both of these tables and performs different parsing based on the pasted text form. That way I can import details credit card transactions by copy pasting just the same.
  • Money does not have an extensibility mechanism, so I couldn't write this as a Money Addin. Nor does it have any public declared automation interfaces which I could have used. Shame!
Fixing the problem of the vanishing imported transactions
Here's one important bit that I found out the hard way and that other people may be encountering with money, regardless of my little hack. It seems that money sometimes has a problem importing QIF files - the transactions seem to be listed as unread in the status bar, but they are hidden in the register, never to be found again.
More descriptions of this problem here and here. they don't have the answer I needed.
The problem in my case seemed to be that I had double-quotes inside my QIF file as the payee name like so:
<---Example of part of QIF file--->
D02/04'2006
T100.00
PName of payee <--double quotes here screw things up in MS Money 2006
N000000100
^
<---End example----->
 
Hope this helps someone!
 
One last annoying thing about Money
Speaking of Money, can anyone explain to me why, when I create a budget in Money, I can't include any categories and accounts related to business, only those related to "private" accounts? It sees weird that this is not possible, and is a feature I could really use, especially as someone who is an independent consults - where my income is tightly related to our household expenses anyway.
Published Saturday, April 08, 2006 4:12 AM by RoyOsherove

Comments

Friday, April 07, 2006 10:15 PM by Alex

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

You can also use SWExplorerAutomation (SWEA)(http://alexfurman.net) to write the data extraction program. The SWEA TableDatExtractor returns data as DataTable (ADO.NET) - no pasting needed. SWEA supports very well Web sites with script and frames (Poalim Web site uses them a lot).
Friday, April 07, 2006 10:34 PM by Alex

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

I have just recorded the script for Hapoalim:

ExplorerManager explorerManager = new SWExplorerAutomation.Client.ExplorerManager();
explorerManager.Connect(-1);
explorerManager.LoadProject("D:\\alf\\JobSearch\\CJSearchService\\Hapoalim.htp");
explorerManager.Navigate("https://login.bankhapoalim.co.il/cgi-bin/poalwwwc?dt=924&nls=HE");
scene = explorerManager["Scene_0"];
scene.WaitForActive(30000);
((HtmlInputText)(scene["userid"])).Value = "XXXXX";
((HtmlInputText)(scene["id"])).Value = "XXXXX";
((HtmlInputText)(scene["password"])).Value = "XXXX";
((HtmlInputButton)(scene["inputSend"])).Click();
scene = explorerManager["Scene_1"];
scene.WaitForActive(30000);
System.Data.DataTable dataTable;
dataTable = ((HtmlContent)(scene["HtmlContent_0"])).TableDataExtractor.DataTable;
Saturday, April 08, 2006 6:55 AM by Roy Osherove

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

Alex: that's wonderful, and kinda ironic.
I actually started out by recording a SWExplorerAutomation script that does everything you mention, only I didn't know I could get a data table out of it, so I exported to Xml/Html and tried to parse that.
I finally decided it was easer to just copy paste, by your way seems waay more productive if I had to start from scratch again!
Saturday, April 08, 2006 9:10 AM by Alex

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

TableDataExtractor tranforms to DataTable any HTML repeating pattern. It uses XPath as "delimiters", so it couldn't be <TABLE> tag. The HtmlContent control should cover the "Table" area. The control has property TableDataExtractor which runs the extractor editor. The extractor editor allows visually define row and column (XPath) delimiters. Sometimes the delimiters should be edited manually to support empty cells or fix a table structure irregularities.
Saturday, April 08, 2006 9:21 AM by Alex

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

Just forgot to mention. I recommend to encrypt the password. SWEA supports a password encryption using DPAPI. In the script recorder select the password record and check [Encrypt] check box. The password value will be replaced by an encrypted value.
Sunday, April 23, 2006 10:24 AM by Amir

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

It's 'Transactions" not "Movements" :-)
Monday, April 24, 2006 8:23 AM by Alex Pinsker

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

Hi Roy,
May be there is a sense to make your app' public, so that pplz could extend it for other IL banks? I personally was volunteering for Leumi then…
Saturday, May 06, 2006 4:29 AM by mar

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

hi,

i'm new with money 2006.

can you supply a way to connect to leumi in israel?

thank you

mar
Tuesday, September 05, 2006 9:07 AM by Ben

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

How do u handle the Hebrew...Do u have a hebrew version of Money 2006?
Wednesday, September 06, 2006 4:00 AM by RoyOsherove

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

Ben: I actually reverse the charachters manually when parsing... (but not all of them, just the "special" ones).

Saturday, December 09, 2006 4:42 PM by Dov

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

Hi Roy,

I was looking for this kind of tool for a long time. where can I get your 'little software hack'?

P.S: I use MS Money 2004.

Thanks

Dov

Saturday, December 09, 2006 5:58 PM by RoyOsherove

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

Dov, I'm not making this available right now...

It's just too cluncky to be usable..

Roy

Sunday, December 10, 2006 2:46 AM by Dov

# re: Hacking Microsoft Money to import my own bank's data + Fix for hidden money transactions

Hi Roy,

I using MSMoney since 1997 but today it become very problematic to update my transactions. I just don't have time for it.

After seeing your blog I deiced to give it another try.

Is there other way that I can import transaction from bank hapoalim website?

Can you please give me the code an I will try to customize it for my need?

Thanks

Dov