Mike Diehl's WebLog

MS BI Conference: Monday Keynote

2008-10-07T01:31:00Z

Here are my notes on the Monday morning keynote:

About 3000 attendees at the conference, over 60 countries represented.
There is BI in Halo3: whenever you look at competitor stats or weapon effectiveness, this is implemented using BI tech

Madison - MS has acquired DATAllegro, a company that was accomplishing low TCO MPP (massively parallel processing) scale out of BI. Using standard enterprise servers, you can process queries on very large data warehouse databases very quickly. They demonstrated a hardware setup of a MPP cluster: one control node, 24 compute nodes, and at least as many storage nodes (ie. shared disks). They loaded 1 trillion (yes trillion) rows in the fact table, and a bunch of dimension tables, such that the data warehouse contained over 150 TeraBytes of data. Then they sliced the fact table up onto the 24 SQL instances on the compute nodes (each compute node then had 1/24 of the trillion rows) and replicated the dimension tables to all compute nodes. Using SQL 2008 (and its new star join optimization) they then issued a query on the fact table and the related dimension tables to the cluster, where the control node passed the query along to the compute nodes, they each processed it, and returned the results back to the client.

On one screen they had Reporting Services (the client app) and on another, a graphic display of the CPU and disk stats for the control node and all 24 compute nodes, each node having 8 CPUs. When the Report was being displayed, the query got processed, and you could see the CPU usage go up on many of the nodes, then disk usage on each of the nodes, then the activity would subside and the reporting view would then display the results. It was all done in under 10 seconds. It was truly impressive. Now, that was with essentially read only data, so you could probably "roll your own" MPP system, given the time and hardware. It's not a huge technical problem to scale out read-only data. If they could show the same demonstration except with a SSIS package *loading* a trillion rows into the cluster, that would have been astounding - it's a much different and more difficult problem. Still, I was impressed.

Gemini - this is "BI Self Service" - the first evidence of this is an Excel addin that the always-entertaining Donald Farmer demonstrated. He used the addin to connect to a data warehouse and in a spreadsheet showed 20 million rows. We didn't *see* all 20 million rows, but he did sort it in under a second, and then filtered it (to UK sales only, about 1.5 million rows) in under a second. That performance and capacity was on what he said was a <$1000 computer with 8 GB RAM, similar to what he purchased for home a few weeks ago.

Aside from the jaw-dropping performance, he used the addin to dynamically link the data from analysis services with another spreadsheet of user-supplied data (I think it was "industry standard salary" or something). The add-in was able to build a star-schema in the background automatically and then make it available in the views they wanted in Excel ( a graph or something? I can't remember). So it was showing the fact that sometimes the data warehouse doesn't have all the data needed for users to make decisions, so they got the data themselves, rather than wait for IT to get it in the DW. Ok, cool. So then he published that view into Sharepoint using Excel services, and the user-supplied data went along with it. So centrally publishing that view means it can be utilized by others in the enterprise, rather than sharing via email or a file share or something.

From the IT perspective, he showed a management view (dashboard) in SharePoint showing usage stats of "Sandboxes" (the thing they are currently calling these publications) and they could see how popular this particular sandbox was, and then take steps to formalize it into the enterprise. The tantalizing link on that web page was "Convert to Performance Point" - the idea was that you could take the sandbox view and convert it into a PPS web part. That looked cool too.

So Gemini looks very interesting.

Timeframes: the next major release of SQL will be 24-36 months from release of SQL 2008, but in the meantime, there are a number of releases coming: Madison and Gemini will be coming in the first half of 2010, and CTP's will be available sometime early next year. There are some incremental releases of Analysis Services, Integration Services and Reporting Services coming - the next gen of Reporting Services in particular will become available in a Feature Pack "real soon now".

MS BI Conference 2008 - First Impressions

2008-10-06T21:24:00Z

It has been over a year since I last blogged, but I want to restart with some posts about the BI Conference I am attending this week.

Chris and I flew to Vancouver yesterday and drove down to Seattle in a Camry Hybrid. Sitting in the lineup at the US border for an hour drained the batteries on the Camry so it had to restart the engine to recharge a couple of times, for about 10 minutes each time. Seemed odd to discharge so much battery just sitting in a lineup and moving 10 feet every five minutes. Anyway...the trip display shows that our fuel efficiency was under 8 liters/100km on the trip down. That also seems a little poor compared to my Golf TDI that gets 4.5 liters/100 km regularly.

We registered last night and wandered the Company Store for a bit - saw uber-geek stuff there and we thought of getting something for Cam, our uber-geek on the team at Imaginet. The conference package was predictable: a nice back-pack, a water bottle, a pen, a 2 GB USB stick, a SQL Server magazine, not as many sales brochures as last year, and a conference guidebook.

Last year's guide book was a small coil bound notebook with a section of blank pages at the end for taking notes. This year's edition has the same content - a description of all the sessions and keynotes and speakers, as well as sponsor ads, but it is missing the note-taking section. I really liked that section last year, so today I found myself scribbling notes on loose paper, and running out. I specifically left my (paper) notebook at home because I liked the smaller conference book instead, but now I am going back to the Company Store to buy a small notebook for the rest of the sessions.

The conference is trying to be more environmentally friendly - in the backpack was a water bottle and they encouraged you to refill that at the water stations rather than having bottled water. That's cool. For me, I would have preferred a coffee mug, since I had three cups of coffee over the day (in paper cups, and no plastic lids). In a strange twist, the breakfast and lunch dishes were on paper plates and not the real dishes like last year - one step forward, two steps back I guess. I can't figure it out.

In the main hall before the keynote address, there was an live band playing 80's hits. They were pretty good, but it seemed odd to have a bouncy energetic group on stage at 8:30 on a Monday morning, everyone was filing in and sitting down, morning coffee still just starting to kick in. The bass player was one or two steps beyond bouncy-happy. It reminded me of someone on a Japanese game show.

How to rename a Build Type in Team System (and a suggested naming convention)

2007-05-24T14:33:00Z

I suppose this might be in the manual, but...

If you want to rename a Build Type that you have created in a Team System Project, you need to open the Source Control Explorer window, dig down into the TeamBuildTypes folder under the project, and rename the folder that corresponds to the build type you want to change. After you check in that change, refresh the Team Builds folder in Team Explorer and you'll see your newly named Build Type.

Remember to change any scheduled tasks you may have created to run your builds automatically.

One more thing about naming Build Types - because we like to have an email sent out to the team members after a build, we have found that a naming convention for the build types helps make it easier to easily recognize and organize the build notifications. We use a standard that includes the environment, the Team Project name, and the sub-solution as the name of the build. So we have build names like

DEV Slam Customer Website - This builds the CustomerWebsite.Sln in the $\Slam\DEV branch.

QA Slam Customer Website - This builds the CustomerWebsite.Sln in the $\Slam\QA branch.

DEV Slam Monitor Service - This builds the MonitorService.Sln in the $\Slam\DEV branch.

QA Slam Monitor Service - This builds the MonitorService.Sln in the $\Slam\QA branch.

Having the project name in the build type helps because if you are a subscriber of lots of different builds for different projects, you cannot tell by looking at the email (other than this naming convention) which project the build is from.

MS BI Conference 14: Evening Reception

2007-05-11T04:26:00Z

The evening reception was at the Experience Music Project/SciFi Museum Hall of Fame. Gary and I walked through the SciFi Museum. It was really great, lots of memorabilia from all the TV series, movies, as well as books, comics, magazines, scripts, photos, videos. Really cool. The only underrepresentaed Sci Fi series was Dr. Who - I saw one thing from that, the "Fun Gun". No Daleks. Gary has read a lot of sci-fi I found out.

I thot it was going to be a banquet and awards ceremony - they had awards, but it was more like a standup reception. No tables except in a tent outside. It was pretty stuffy inside, so I hung out with Gary in the fresh air (well, he was smoking, and so was a lot of others around me, but for the most part it was fresh air).

It would have been a great night to have my wife along - she would have loved the sci-fi museum (and the Experience Music Project, very little of which I saw), and she would have been a classy-looking woman to have with me too. We'll come see this place another time.

Tomorrow morning keynote is Steve Ballmer. Somehow the chant "Business Intelligence, Business Intelligence, Business Intelligence" or "Information worker, information worker, information worker" doesn't really roll off the tongue. What will be his hook tomorrow?

MS BI Conference 13: BI Power Hour

2007-05-11T00:33:00Z

Apparently the Power Hour has been something that has been happening at TechEd in past years. I vaguely recall seeing something about it once.

This was a great session - two slides in total I think. All demos, and the demos were different - kinda crazy, but still educational. Lots of free stuff thrown into the crowd. I got something for my daughter.

1st demo - Magic 8-ball vs Data Mining Neural Net algorithm.

In an Integration package, the guy took a table of customer demographics, and ran it in parallel through two different algorithms to predict whether the customer was a homeowner or not. One algorithm was a DM NeuralNet algorithm, the second was a Script that launched the Magic 8-ball window. Looking at the results, the 8-ball didn't do too badly. The demo was interesting in that it showed you could solicit feedback from the user who was executing it (the 8-ball was in a Windows Form, created on the fly in the package).

2nd Demo - by Hitachi Consulting, he demoed an implementation of Analytics for mobile devices. The framework they built helped push out reports, alerts, forms, to a mobile device. They used MS Communication Server to send an SMS text message to the phone, and when the phone received the text message, it used web services to pull back the content (alert, report, form, etc). So he sent out a "Price Change" alert. An RMA authorization form. A Sales report. He said they also had a method to ping the phone and tell it to erase all its content, in case it got lost or stolen. Very cool.

3rd demo - the guy said he wanted to find the geekiest thing to do with Integration Services. He took two sets of a million random numbers between 0 and 1, and through selection of them and applying an algorithm, he basically calculated the value of PI. He didn't tell us what it was until at the end it became obvious. Terribly geeky.

4th demo - The guy had built a custom reporting services item, which took in a dataset (a summary of sales amounts for three sales reps in three categories), then was an interactive KPI mechanism for displaying the data. It presented the categories and reps in a 3x3 matrix, with a green and red button in the corner of each cell. If you decided the amount was good, you clicked in the red button and the cell got an X. If you thot it was a good amount, you clicked the green button, and the cell got a green O. (Get it? X's and O's in a 3x3 matrix?)

Last demo - Using Performance Point Server, they showed a web page with ten suitcases on it, and they invited someone to come up and play Deal or No Deal. She won $10 (in play money I think). Her highest offer was nearly $485,000.

Between each demo they threw out schwag, like t-shirts and hats and stuff.

We should definitely try this the next demo we do at Imaginet.

MS BI Conference 12: MOSS 2007

2007-05-11T00:21:00Z

A rolling stone grows no MOSS.

This guy was very MOSSy, because he didn't roll very much. I was nodding off in this session because the presenter wasn't very passionate, or funny, or showing me anything that hadn't already been shown in the keynotes, or at the MSDN tour in December.

(Microsoft Office SharePoint Server, btw).

Cool thing about SQL Server 2005 SP2 is that it adds much better integration of SS Reporting Services into MOSS, it contains the reports repository and the published reports become a document library, with much better web parts for integrating into Sharepoint.

I snoozed a little during this session. Good thing I did, because I was really glad I was awake for the next session.

MS BI Conference 11: chalk talk on MDX

2007-05-10T21:11:00Z

Two thumbs down.

Well, it probably was a great session, but I got there five minutes early, and already there was 30 people waiting outside the "room" it was in. So I went for lunch instead.

Chalk talks are the 2nd or 3rd class citizens in the sessions here at the conference, but they have the potential to be the most valuable. At least from my perspective. These sessions are real world, not lovey-dovey like the main sessions.

Microsoft, the chalk talks deserve a 1st-class upgrade. Please, I'm begging you.

MS BI conference 10: ProClarity

2007-05-10T21:03:00Z

A woman who was formerly from ProClarity, now a product manager in the Performance Point Server group, presented this session on ProClarity.

"Interface to insight" - answering the WHY? in BI.

Tools for decision makers to explore large amounts of data and get rapid insight.

Simple data navigation, powerful calculations, and advanced visualizations of data.

Reports tell you what happened. Dashboards tell you what is happening now, and ProClarity Analytics helps understand Why it is happening.

ProClarity Analytics Server (PAS) is an IIS App. There is also a SQL database of business metadata. The clients are thin-client web-based, thick client web-based (ActiveX control), and Windows-based thick client.

This product, to me, addresses a lot of the "last mile" gap between SQL Server Analysis Services cubes and stuff, and the user.

the KPI builder helps make calculated measures with no MDX at all, easily, and publish to the PAS.

Advanced visualizations: heat map, like spaceMonger, shows boxes stacked together, with the size indicating one measure (sales amount), and the colour indicating another KPI (profit margin, good/warning/bad).

Decomposition tree - view a measure, break it down by category, then by another dimension, and so on. Hard to describe, cool to see.

She was great - perfect balance of architecture slides to show how the thing fits together, with lots of demo time with the product itself.

MS BI Conference 9: Thursday keynote

2007-05-10T20:34:00Z

This morning's first keynote had content about Katmai, the next release of SQL Server. (there was other stuff before that, about the BI platform and pervasiveness and yadda yadda.)

SQL 2005 SP2 includes stuff for Excel for data mining. This tool takes an ordinary spreadsheet and applies a data mining algorithm to it, such as categorization. It submits the data to SSAS, builds a mining model, trains it with the data, and adds the results of the mining as a new column in the spreadsheet. All without the user needing to know anything about mining, other than what kind of scenario they want.

I remember seeing this in yesterday's keynote, where a tall, blonde, smart woman (she may have been from Canada, I saw her at the MS Canada thing at Fox Sports Bar last night) demoed a scenario. She took a table in Excel which was a list of prospects and their demographics. Someone who generated the list had started ranking the prospects, but we didn't know how he decided their ranking. he had maybe 10% of the rows ranked. She took those rows as an example, submitted it for data mining, then it determined the rest of the rankings for the other rows, based on the example rankings. Very cool.

This morning's demo showed another bunch of marketing prospects and their demographics. He submitted them for mining, asking it to categorize them into three groups. Group 1's demographics suggested that they were good prospects for SUV's (#children, age, income). Group 2's demos was more "poor student" starter vehicle types, and the third group was good for selling bicycles to.

The coolest thing I thot was that this used the data mining engine in SSAS without needing a cube or anything, it left the spreadsheet as a plain spreadsheet, and the user didn't need to understand all the data mining stuff to do it.

Katmai will be shipped in 2008. He didn't say WHEN.

Some new datatypes natively supported - filestream, spatial coordinates, new date/times. It will include an Entity Data platform in .NET managed types, and LINQ of course. It'll support the occassionally-connected database (like mobile databases) and handle the synch stuff better.

Microsoft bought OfficeWriter from Artisan, which is a set of tools that lets users author reports in Excel or Word, using the features of Word/Excel, and then publish the report to Reporting Services. Very nice. It was a "whyt didn't I think of that!" moment. Seemed simple enough to do.

The spatial datatypes will be supported in the query optimizer and the indexes, so you can do geographical queries very quickly.

Dr. Robert Kaplan, Harvard Business School, creator of the Balanced Score Card methodologies.

Balanced score cards help business determine/measure their performance on more than just financial metrics. It helps to measure the more intangible assets, like quality, customer relationships, employee skills.

There is a BSC for non-profit organizations too, which adds the mission perspective (how do we have an impact?)and support perspective (how do we attract resources and support for our mission?).

Most organizations do not know how to execute a strategy.

Principles:

1. Mobilize change through executive leadership.

2. Translate strategy to operational terms.

3. Align the organization to the strategy.

4. Motivate to make the strategy everyone's job.

5. Govern to make strategy a central process.

Mission - why we exist

Values - what is important to us

Vision - what we want to be

Strategy - our game plan.

Usually there is a gap from the Strategy to operations. We need to link the strategy to the operations (and it is a two-way street). the balanced scorecard and the strategy map are ways to bridge the gap.

Not all loyal customers are profitable. With time-driven, activity-based costing, you determine the actual cost of your customers. 20% of most-profitable customers generate 180% of the profit, and 20% least-profitable customers lost 80% of the profit.

Strategy map has perspectives: Financial, Customer, Process, Learning.

Motivate so strategy is everyone's job.

CEO walkthrough with strategy map - asks a random employee, what is this? (should identify the strategy map - if not, it indicates a problem with that employee's *supervisor*). Can you explain it to me? Sorry I interrupted your work. How does what you were just doing link to the strategy map?

Communicate the strategy seven times, in seven ways. Brand the strategy!

MS BI Conference 7: Wednesday evening

2007-05-10T06:26:00Z

The evening was the Partner Pavillion Expo reception. Open bar and light supper and you wander around the booths. Microsoft has areas where you can talk with the product managers. They have tables marked "Reporting and Analysis", "Integration and Data Warehousing", "Database engine", "ProClarity", "Performance Point Server", yadda yadda yadda. I wanted to talk to some of them from the Analysis Services team, to talk about the mutli-developer scenarios that I had been going through with my customer, and some of the problems I have had with team development in Analysis services. But I couldn't tell which guys were the SSAS ones, which MS people were just wandering around themselves, and I'm not great at starting up conversations with people I don't know anyway. And I wasnt' with anyone who would help bolster my courage. So I wandered around the tables, looking like I wanted to talk to someone if only they would come up to me and introduce themselves. It sounds stupid I know.

I decided to go back to the hotel at that point, and chat with my DW for a while, and my boss was asking me about the day too. MS Canada was hosting a party at the Fox Sports Bar a block away this evening, and Gary, the sales guy from Imaginet who is also here, said he was going to go, so I headed down there about 9.

I guess MS Canada just had an area of the bar, because there was still other "non-geek" types there - as evidenced by their lack of conference lanyards. It wasn't immediately obvious which area was the MS Canada reception, but there was a busy corner so I went over there, looking for Gary. It soon became apparent that Gary wasnt' there, and there was no one there I knew. Ramon, a manager at Imaginet, and former MS Canada guy, had sent me some contact info for the MS Canada BI tech lead, whose name I have since forgotten, so he was hoping I would connect with him and introduce myself. The bar was fairly loud, I didn't really know anyone, and pretty much everyone was already chatting amongst themselves, so I'm not one to stand at the edge of a group and horn in. Or go up to a complete stranger and introduce myself. If Ramon had been there, he would have known probably 75% of the people there, and he probably would have introduced me (actually, he would have "talked me up") to anyone that mattered. His reasons to go to a conference like this (and Gary's reasons too I bet) would be quite different than mine. He would have gone to network, to make contacts, to find business, and to come home with $150k worth of leads.

Me, I come to these things to soak up the knowledge. To find the best sessions and learn stuff I don't know. I don't really like the vendor booths because I don't much like the sales pitch. I do try to think about how I would apply the things I am learning within Imaginet, or with my customers and on future projects. But I don't really think "hey, if we took that idea to Customer X, we might get $50k of work out of it". I suppose I should. My value equation, I think, is using the knowledge I gain from events like this to do my work better, to recognize ways I can add value to customers or leads when they come to me with a problem. Joel, on the other hand, is much better at this than I am, he comes out of these thigns with ten new product ideas, and a strategy to talk with a dozen of our customers about what they are missing because they haven't done X or Y yet, and look at what you could do! ("and the villagers dance," as he would say).

Today I was wondering at a couple points about my last customer - I view them as at the "tactical" level of maturity, because they see the value of BI at a department level, at the IT level there is support for it, but it isn't particularly engrained at hte management level. I am left thinking "it's too bad they don't think at the broader scale about this." I imagine that Joel would be thinking "how can I talk to the right people there, so that I can convince them they need to think of this at a broader scale." There's a difference there. I'm not sure I'll ever get to where Joel is. I could be wrong, I suppose.

MS BI Conference 6: Chalk Talk on SQL Server Integration Services - moving from Dev to Test to Prod

2007-05-10T06:09:00Z

This was the best session of the day for me. The guy from SQM (Solid Quality Mentors) was Spanish with a thick accent, but I found him easy enough to understand. Again, the chalk talk venue was bad, couldn't hear or see, so I sat in the front row.

He talked about what he had learned and put into practice for SSIS packages.

At the Dev level, you treat packages like source code. They should be in source control. You edit them with BITS (Visual Studio). You test them with local or dummy databases.

At Test and Prod, you treat packages like executables, compiled code. You never edit them directly (with VS for exmaple).

He likes to store packages in the SQL Server store (essentially, in MSDB), when you deploy them from source control to Test or Prod. That way they get backed up with MSDB.

Packages should have "Package configuration" enabled, and you should use consistent naming conventions throughout - starting with the package name itself, which should include the Solution and Project name, so that when deployed, you can trace the package back to the source control system.

Usually the connection managers in packages should be consistently named - they usually represent logical names for data sources and destinations, which are configured at execution time with physical names (according to the environment).

He uses the SQL Server configuration option to store much of the package configuration in a table in the database - you can have a configuration database in each instance of SQL Server for Dev, Int, Test, Prod, and they each have their own values for configuration. So the Test Config database has config values that point logical sources and destinations to the Test source and Test destination, and so on.

Then, he adds a second configuration which is an XML configuration file, that configures the ConnectionManager for the SQL Configuration database. The XML file on the Test instance then points the package config to the Test Configuration database, and so on. You must have the xml configuration for the Config connection manager listed above the entry for the SQL Config configuration for this to work properly.

At the customer I have been working with, we used Xml configuration files and environment variables, and stored our packages in the file system instead of MSDB. We ran into a roadblock because they have both Test and Prod SQL instances on the same server, and SSIS does not have the concept of named instances. I was trying to use Xml config files like source controlled files, and that doesn't quite work. I like the ideas he came up with and I'll start implementing them when I return to that customer next week.

MS BI Conference 5: Practical Design Techniques for SQL Server Analysis Services

2007-05-10T05:50:00Z

This was a level 300 session.

It reminds me how much I don't know about SSAS. There's tons of stuff in the AdventureWorks analysis database that are good, meaty examples of design practices. Things like dealing with multiple currencies, charts of account, semi-additive measures, extrapolating measures at a smaller grain than the fact.

He showed dealing with multiple currencies - the measure in the fact table was always USD, but there was an original Currency ID as a dimension of the fact, so with an exhcange rates measure and a many to many relationship, you could have a conversion done using a measure expression. I've done this before, but he showed a few things about setting the IsAggregatable false, and the default member, and setting the Locale to show the proper currency symbol (which I'd tried but couldn't get to work right) that were useful.

Somehow he was debugging the calculated measures and expressions when he was using the cube browser! I've never thot of trying to set a break point on those expressions. Sheesh. It's like working int he dark for a long time, then someone comes in and turns onthe light you didn't know you had.

He showed extrapolation - he had a Sales Quota cube that was ont he Quarter grain of the time dimension - sales quota amounts were set quarterly. Then he used some expressions to extrapolate, in a weighted fashion, based on the previous year's sales, the monthly values. So if the quarterly sales quota was $10,000, and July had 40%, Aug had 35%, Sept 35%, the monthly quotas were $4000, $3500 and $3500.

Attribute relationships are extremely important in dimensional design. They can make or break your performance, and I need to understand them much much better.

He showed stuff around chart of account stuff, in a dimension, to make sure they aggregate properly - depending ont he account type. Revenue/Expense accounts are additive while some other types are subtractive or not additive at all. Again, I realized there's a lot I don't know here.

I wasn't lost in this session, but I was recognizing that I have encountered some of those issues, and in some ways managed to overcome them and sometimes I chose non-optimal ways I think. And there are fine distinctions that I don't understand, but I know make big differences.

I like getting humbled by sessions like this. There's always a bigger fish.

MS BI Conference 4: Best Practices for Migration to Microsoft BI

2007-05-10T05:24:00Z

This was a joint presentation by Ideaca (a Canadian consulting company) and General Mills Canada. I went for the CanCon I guess.

I didn't get much out of it. The presenters weren't very good, particularly the IT guy from General Mills - he got off topic several times, and used rather simplified analogies to describe the concepts of their project, which almost were condescending - you're at a BI conference with BI professionals, you should expect that we understand the concepts, so don't explain it like you're trying to convince the execs or the users at your company.

They spent too much time repeating what was on the slides.

Take-aways from this: they used some agile practices on this - particularly "show early, show often". And they had rigorous testing. And they used an Excel spreadsheet for their data dictionary, and had a macro that created a CREATE TABLE statmenet, and added a whack of extended properties for documentation sake in the database.

Types of tests they did:

1. automated data integrity routines (this would have been great to see some examples)

2. record counts (for sanity checks. again, would have been nice to see how they implemented this - did they keep a record of their results for tests over time?)

3. Scenario reconciliation - sounds like a complex unit test.

4. Report comparison - compare the new report to the original report.

5. Business hands on.

I asked him afterward if they did their tests in an automated way, he said they did, they had a buid process, and testing, and so on. THAT would have been great to see.

MS BI Conference 3: MDM Chalk Talk

2007-05-10T05:01:00Z

The chalk talks are held in the main exhibition hall, "rooms" with about 50 chairs in them, fabric drapes enclosing them, with a small sound system and a large monitor (small widescreen TV size) on a stand at the front.

They are supposed to be more interactive and more demo and whiteboard based than the main breakout sessions are, and that's promising.

Unfortunately, there's usually not enough seats, you can't hear the speaker for the noise from the chalk talks happening on the other side of the fabric, and you can't really see what is being demoed on the screen.

The first chalk talk I went to today was about Master Data Management. I hadn't heard of this term before, so I thot I'd better find out what it is. It turns out it is the same problem we've been dealing with for a long time now, having the same entities in different repositories and trying to have a consistent view of that entity. It's just that now there's a trendy label for it, MDM.

Roger Wolter introduced himself and said that the session would be a few slides and then mostly demos.

Master data management is about having a single view of the "master data" you have in your enterprise, things like Customers, Products, and so on. It usually also encompasses the hierarchy management of that too - all the different ways that the org slices/dices/rolls up/groups the master items.

Master data, if it wasn't already obvious to you, usually is the basis for your dimensions in your OLAP cubes, but maintaining it usually means an ETL process to build the master data in the first place (and dedupe and version and consolidate and make consistent) and then a process to continue to maintain a single view of the entities.

We did this with a customer when I first started with IMaginet, we made a process to keep their Active Directory consistent with their HR database, their accounting database, and some other sources of "people". I can see it being an issue at the furniture manufacturer that is our customer now too - maintaining their product list across two different systems with two different representations of it.

Master data must be surrounded by stewardship and governance. Stewardship means that the data has owners that are responsible for its quality. Governance means that there are rules and processes and quality and service expectiations to ensure the master data stays consistent. There is usually workflow processes around MDM, where the stewards of the data participate in maintaining the quality, particularly in the conflict resolution process.

MDM usually has four P's:

Politics

Policies

Processes

Products

What was most unfortunate about this session is that he ran out of time and he was still on the slides. Almost no demo at all. We spent all the time discussing the issues and what I was hoping to learn was some of the solutions to these things.

MS BI Conference 2 - Gartner session

2007-05-10T04:50:00Z

A guy from the Gartner group talked on "Building a Plan of Success and avoiding the Five Fatal Flaws of Business Intelligence".

His slides were amateurly formatted, and he had lots_of random_underscores on them_so I was left wondering_why? He also crammed too much stuff on his slides - not strictly text, but diagrams and clipart and labels and arrows.

He had a table that was promising, essentially it was a BI maturity model, the characteristics of each level of maturity, what are the likely ways to fail at each level. The first level was Opportunistic. The 2nd was "tactical" and the third level was "strategic". I'd put the furniture manufacturer I was working with on BI a few months ago at the tactical level.

His five fatal flaws were:

1. If you think there are only five flaws, you are destined to fail.

2. "If we build it, they will come." We like to anticipate the needs of users, and we often think that users don't understand what they do need, so we will solve their problems.

3. "Managers need to dance with their numbers." He often hears "we need a flexible spreadsheet interface that allows us to work with the latest numbers." His experience is that most managers don't like to make decisions anyway, if the numbers point them to a particular decision, they will dispute the numbers, rather than make a decision. Often, managers who make decisions can get fired.

4. "Our enterprise application vendor will deliver the best BI solution." He hears "we just spent $10M on an ERP, and now you are telling me we need a BI solution? A data warehouse is free, integration means its easy."

5. "Data Quality. What data quality problem?" Your boss doesn't like you if he makes you responsible for data quality. IT tries to solve this with a tool, but its more about governance and ownership. There should be a "Data Quality Firewall" with SLA"s and quality metrics.

6. "Darwin was wrong. Our BI apps don't need to evolve."

7. "We can just outsource the whole thing."

8. "Just give me a dashboard. How hard can it be?"