Manage Hierarchical data in SharePoint lists - Building a Project Team site template

No matter how hard you try, or time you spend... there is always something new to learn on SharePoint. Not just as Developer, Architect, or as IT Pro, but more so as User! I have learnt along that users can best be classified in slabs whereby some adopt a new product quickly, while others find it difficult to grasp new features no matter how usable... and most get to a point where they learn most (usable!) features and form their comfort zone around what they have learnt by cognitive and settle with it. They just won't move ahead of curve, and you can either blame users or the product... go figure!

I think SharePoint has many such boundaries, and I keep seeing them. I wonder, why didn't O'Reilly come up with a title like "SharePoint Hacks" as part of their popular series, despite the product being so full of them and also being undisputedly popular. Perhaps publishers and authors never settled on content and/or number of volumes needed for such title.

Opinions aside, I want to bring focus of this post to one of most common case of SharePoint usage... Project Management. While there are Fantastic 40 templates by MS that includes PM template, but have their story of issues. Users continue to struggle to best use built-in Lists for doing ad-hoc and light-weight project management. It is disappointing that such template was missed out of RTM, as templates in a generic product like SharePoint are a great means of delivering product-know and best practice guidance in shortest cycle. Apart from instant productivity gains.

While utilizing most out-of-the-box features to build such template, I'm going to focus on a specific tip of using SharePoint lists for managing Hierarchical data through maximized use of content-types. This will also bring in light usefulness of this not-so-well-discovered new feature of 2007, and how it can be applied to varied Master-Detail list data scenario's.

Problem Case

To summarize, I want to:

1. Build a simple PM site template for a Project team, only by utilizing standard features and no coding.
2. Instead of creating multiple lists for managing Requirements, Tasks, Risks, Defects, and Releases, I am going to maximize usage of multiple content-types in a list feature of SharePoint 2007. While doing, I'll share findings, hacks, and limitations as it exists.

Most list data doesn't live alone, and is related to another list that we commonly connect through Lookup columns. While Lookups are easy to use and does work for most scenarios, there are cases when there is need for managing Master-Detail without compromising context of Master while working on Detail data. SharePoint doesn't handle such cases well, or at all. Entering a Detail record and also to a specific Master, that resides in another list, through a lookup becomes a difficult usability issue. Particularly when you have large list of masters, SharePoint converts drop-down into a less usable one (when items>20). Problem heightens when Masters Title isn't unique; there is no way to correctly pick the record to ensure the Detail record is associated to correct Master. Take case in point, there is long list of Requirements and for each there are Work-Items, Defects, Releases, and Risks that you want to manage. How many lists will you switch, when all you want to do is narrow-down on a specific Requirement at a time.

Now there are some tricks that you can apply through SharePoint Designer to Master forms, to create child Detail records. This still requires context switching, and only works for forms. I am looking for something which gives better usability for Hierarchical data, without Relational quirks. SharePoint lists aren't treated well if assumed as mere Tables with relationships through Lookups alone; this is a slight paradigm shift for most users, used to considering lists as web extension of Excel files or Access tables!

Rules of the game!

In order to solve problem at hand and also to come up with something more useful than is functionally obvious to common users of SharePoint, I am going to restrict myself to following:

1. Do NOT write any code, or apply hacks that would otherwise be out of reach of normal SharePoint user.
2. Do NOT customize the site using SharePoint Designer. After all how many users have a licence purchased!
3. Use everything possible/offered on UI to deal with the requirements given above.

Project Team site Requirements

Keeping it simple, team wants to:

1. Manage Requirements given by Product owners.
2. Manage and track Work associated with each Requirement, for associated Work-Items, Defects, Releases and also any Risks.
3. Maximize use of out-of-the-box SharePoint (WSS) features to get most out of Project Team that's geographically distributed and heavily depends on SharePoint to collaborate and communicate. Use Wiki, Discussions, Calendar, Contacts, Libraries with Web-Parts and Views as a ready-to-use template.

Building a Project Team site template

I created following lists, ordered as in Navigation on site:

1. Project Requirements: Single list for Requirements, Tasks/Work, Defects, Release-schedules, and Risks.
2. Project Documents: All documentation pertaining to Project goes here.
3. Release Calendar: View all releases scheduled in a Calendar.
4. Project Releases: Version tracking Build Releases across their stage lifecycle.
5. Project Wiki: Wiki library for Project, so team can collaboratively author content such as technical design.
6. Team Work: Snapshot view of all the Work, team is working on.
7. Team Calendar: Event calendar for prime events on project, such as meetings, holidays, and vacations.
8. Team Contacts: Contact information for Project stakeholders and other contacts.
9. Team Discussions: Discussion list for team working on Project, with an aim to to get out of emails for many conversations.
10. Home: My Work status, Requirements in Progress, Announcements, Discussions, Project Team, Documents, Contacts and Team Calendar.

While 2-10 of above was relatively straight which most common users already understand, like using Views, Web-Part Views, Computed Columns etc. You can self-discover these in site template as it is attached to this post. I'm instead going to focus on Requirements list.

Hierarchical Data in Requirements list

SharePoint supports multiple content-types within same list. These content types can have varying columns/schema. Also folder content-type is available by default in each list. So in order to support Hierarchy of data types, we are going to leverage Folder content-type as container.

List Settings > Advanced Settings > Enable Content-Types Management

Go to Site Settings > Site Content-Types

Defined new content-types for Work (inheriting from Task), Defect, Release (inheriting from Work) and Requirement.

Define a new content-type, Requirement, inheriting from Folder so that Requirements can become container for other Work items. This will allow us avoid switching context across lists and various usability improvements. Wait for a moment...

Similarly, other content-types defined and they are then included into List Settings.

As a result of adding above content-types, new columns are added into list.  Notice union, and their source content-type.

I created Requirements list based on Tasks list template, in order to get feature of being able to send mails when item is Assigned To as user. After adding above content-types I removed Task content-type and also removed any left-over Column, such as Task Group.

Also I added some new Views as below. Notice the "Show-In" status and how 2 Views can be marked Default. Not that Views are default at different levels, ie Requirement and All. If you are browsing at Top root-level, for Requirement, Requirements view is going to be default. If you are browsing inside a specific Requirement, by default you'll see Work Progress.

To make it work, you have to configure "In Folders" section of Standard view template when creating or editing a view.

Requirements View (Top level)

Work Progress View (items under respective Requirements)

You can not only show items inside respective Folders, you can also have a Flattened view to show All items in all/any folder.

Team Work view (all Work items)

Release Calendar (all Release items in all folders, in a Calendar view)

Now going back to List, in default Requirements view, I have added on Requirement. By default, New button is given as Priority=1 from content-type management section in List settings... so that New button default to Requirement content-type in root level. At lower levels (views!) it will now show up as its been filtered out due to view settings and runner-up Work content-type will take over the New button behavior by default.

Click the requirement, and you will notice the View flip to Work Progress view since you are now one level down under Requirement content-type (a folder).

This is elegant as you now very well know which Requirement you are under and need to work for. All child Work, Defects etc are now scoped exclusively for that requirement only. Saving you ton of effort with clicks and flipping across lists and forms.

I never liked Gantt views in SharePoint 2007 as they are unusable when periods are long, do not allow grouping at anything other than week. Horizontal scrolling is never a favorable web experience! BUT, in this case I have created a view that Defaults to Gantt. First it gives a better sense of time, scoped at a requirement, and horizontal scrolling issue is minimized since a single Requirement is anywas not going to take longer than 3 months! (exceptions aside)

When one level down, you see following Views in menu. Notice, Requirements view is missing. This is because when you are level down, you can only switch to a View that is at same level or a flattened "All".

While when you are at Top-level, you see following. Notice that, flattened, All views - Release Calendar and Team Work are visible here as well, while lower-level view isn't. And Requirements is now showing as default, separated.

Limitations and Caveats

Some caveats to take note of:

1. Data-sheet view does not work when you are one-level down in a view. Though it shows up as an option in Action menu.
2. No way to hide content-types other than Requirement, when at top-level. Hiding from list settings, hides them from all levels!
3. You can hide specific content-types available under a specific requirement from New menu. Open context menu of a Requirement and select "Change New Button Order". There is no way to set this globally on a content-type basis, say Requirement in our case.

Conclusion

In my view, using multiple content-types in a list is a great way to manage data hierarchy, enhancing usability and portability of data. In order for you to jump-start and play with above example, I'm including Project Site Template (.stp) file, with and without sample content.

Feel free to feedback through comments and do share your own ideas, suggestions, tips, and hacks to make it Requirement list or site template any better. Thanks.

Hope you find it useful. Phew! :-)

-- Sharad

Download: Project Site v1.0 | Project Site v1.0 (with sample content)

Update (Aug 13th 2009): Since release, there seems to have been an issue and templates stopped working (throwing error while creating site). Reported by many readers, I finally got chance to check on a fresh machine with WSS/SP2 bits and indeed it failed. So I've recreated these manually (on a WSS SP2 installation), instead of trying to figure the underlying cause - in the interest of time! Links are updated, please download and let me know. Thanks for everyones feedback. Keep it coming... Sharad.