Eli Robillard's World of Blog.

From Wall Street to Dubai: How Infusion is evolving

2008-09-27T12:44:00Z

When the Wall Street herd began to stumble over their own their mortgage inventory, there were signs of rockier times ahead. Now that several giants lay collapsed, people like our CEO Greg Brill are writing about what it was like to watch the race to self-destruction unfold, and how they read the signs that clearly said, "now is a good time to evolve."

Read it here: From Wall Street to Dubai: Diary of a tech entrepreneur's adventures in the hot zone

A good adjunct to Greg's article is the lesson that that there were people who had the common sense to get out of the sub-prime mortgage business. A Bloomberg article (Toronto-Dominion Avoids Subprime as Banks' Costs Rise, May 2008) describes how Ed Clark, CEO of TD Bank sold off TD Waterhouse's exposure in 2005. "'I'm an old-school banker,' Clark told reporters last month in Calgary after the annual shareholder meeting. 'I don't think you should do something you don't understand, hoping there's somebody at the bottom of the organization who does. . . . The whole thing didn't make common sense to me.'"

So there's the lesson for the day, it's ageless, and it takes many forms. There are no dumb questions. When something doesn't make sense, resolve it. Traditions and so-called "best practices" come into being because they work for us, but they should never be frozen or used in place of thought and common sense. Each of us in responsible for our own actions and their effects. Traditions, including best-practices and business rules, must be continually questioned and allowed to evolve. A tradition without a means to evolve is merely a symbolic gesture that has lost its original purpose. Many would say, "assess the risk and resolve the question in accordance with that risk," but my mantra remains, "If it's worth doing, it's worth over-doing." Miyamoto Musashi wrote in 1643, "If you do not pursue a genuine path to its consummation, then a little bit of crookedness in the mind will later turn into a major warp. Reflect on this" (The Book of Five Rings, Thomas Cleary translation).

Infusion's path is great technology, not financial services, not the Microsoft stack. It's great to work in a company that evolves and asks the questions it takes to keep us on that path. What's your path?

TSPUG Fall Session Starts Tonight (2008-09-28)

2008-09-24T17:58:00Z

Members received an update last week, I just wanted to mention here that summer's over and meetings start back up tonight, Wednesday, September 28!

Topic: Jignesh Shaw, Applications Development Manager at Cyberplex Inc. will be doing a presentation on InfoPath 2007 forms development for Forms Server in MOSS 2007 - including tips, tricks and best practices for developing InfoPath forms. Examples will include building an InfoPath form for data collection via a public facing site where anonymous users can submit their data.

Date:                Wednesday, September 24, 2008
Address:          Nexient Learning
                        2 Bloor Street West
Time:               6:00pm

To RSVP, please email susie.ibbotson@cyberplex.com

Next month look for another great session from Bill Brockbank on STSDEV. But first, tonight!

SharePoint Security: Hard limits and recommended practices

2008-09-12T03:24:00Z

This summarizes the hard limits and recommended guidance for Groups, Access Control Lists (ACLs) and securable objects in SharePoint 2007.

Unique accounts or groups per SharePoint Group: ~2000. This is the identical with the guidance for any large SharePoint list as covered by the Working with large lists in Office SharePoint Server 2007 whitepaper. Lists can handle thousands of items, but performance of a Views (i.e. while viewing or updating membership) will degrade after 200 items and become unacceptable towards 2000 because the underlying SQL queries are not paged or filtered. Note that the next limitation (see Users per SharePoint ACL, below) will affect the ability to index any item (or list) you apply such a large ACL to. Resolution: Use a third-party tool to manage large SharePoint Groups or re-think your design to include AD groups. Mitigation: When possible, rather than assigning users membership in a SharePoint Group, assign users to an AD group and then assign the AD group membership to the SharePoint Group.

Users per SharePoint ACL: Query results must not exceed 64k, or ~1000 users per ACL. When exceeded, the "Parameter is incorrect" error is thrown causing crawling to fail on the item. This issue affects indexing, but does not otherwise affect SharePoint. The limit is noted by Joel Oleson in the comments of a "2003 to 2008 security changes" post and the Best Practices for SharePoint Search article on TechNet. The issue is not SharePoint-specific and will affect any content crawled with large ACLs including file system objects like Network Shares. KB article 885482 describes the cause as "The maximum buffer size of the InitializeAcl function is 64 KB. Therefore, the maximum size of an ACL in Windows, including the access control entries (ACEs) that are contained in the ACL, is 64 KB." Resolution: Either exclude the item from indexed content, or remove entries from the ACL. Mitigation: This is a rare issue normally avoided through sound design. However, since AD groups are not expanded when the ACL is read, assigning individuals to AD Groups rather than SharePoint Groups will mitigate the limit.

Active Directory groups per-user: 1024. Each membership increases a user's "SID count" and the limit on the token bag is 1024. When the limit is exceeded the error reads, "During a logon attempt, the user’s security context accumulated too many security IDs." The whitepaper Addressing Problems Due to Access Token Limitation discusses the issue in depth, and KB 906208 describes the error. Consider this limit if AD groups are to be used to manage SharePoint groups. Resolution: Remove the users from one or more AD groups to reduce the SID count. You can verify the issue with the Group Membership Evaluation feature of the Ntdsutil.exe tool. Mitigation: Manage users in SharePoint Groups rather than AD groups if growth would require any user to be a member of more than 1024 groups.

Recommended Guidance
The Plan for Software Boundaries whitepaper contains guidelines for "people objects" but its guidance for security principals is (politely) less-than-useful. For example, the guidance that "You can add millions of people to your Web site by using Microsoft Windows security groups to manage security instead of using individual users" says nothing about how many people can (or should) be added to a SharePoint Group and ignores the SID limits of AD groups. It would also lead you to believe that it's an acceptable option to use local Windows security groups to manage permissions rather than domain accounts. In a nutshell, that would be a bad idea if your farm will ever have more than one server.

There are many MSDN and TechNet articles with recommendations on what granularity of permissions is "right" but the one simply named Plan site security is the most detailed and provides the best example. To summarize and disambiguate the best practices and recommendations:

If many securable objects share the same ACL, then group them into a container with that ACL. This applies equally to list items within lists, or libraries within sites. If the permission change is a temporary state in a securable object's lifecycle (for example to assign reviewer permissions during a workflow) then reset the permissions to inherited when the temporary state is complete (e.g. at the end of the workflow). [Update 2008-09-12 TB] Avoid unique permissions on folders, as Flat views will not secure folder content as expected.
The more ACLs you create, the more ACLs you have to manage. I know, I know, it's self-evident, no? How I wish it were. A problem of some articles is that they can be interpreted to recommend moving the problem rather than solving it. For example: "Minimize the use of custom or fine-grained permissions. The more fine-grained permissions that are applied, the more difficult it is to track who has access to what." Well duh. The problem is that not applying permissions does not remove the business need to have permissions. If the issue is the tools and not the limits of the platform, the correct guidance should be "if management or reporting tools don't work the way you want. . . ." Better, it deserves to be a recommendation of its own.
If management or reporting tools don't work the way you want, then build or buy better management or reporting tools. DeliverPoint is one option. [I'll continue update this post as others are suggested.]
Understand the strengths and weaknesses of managing users in SharePoint Groups vs. Active Directory. A strength of SharePoint Groups is allowing the same people responsible for building teams in an enterprise to form their own groups. IT has been the bottleneck for too long. Advantages of SharePoint groups are decentralized maintenance, isolation by Site Collection, the ability to query it with the API without adding to DC load, and the ability to integrate it as necessary with external Authorization Providers (AD, FBA, etc.). This last point makes SharePoint groups preferable for extranets and mixed authentication zones, allowing internal users to authenticate through AD and external users through another AuthZ provider.

AD Domain Groups may be preferred for central management and a standardization of tools. Unfortunately these tools are not usually built for or available to users outside of IT and support services. AD groups are also not visible from within SharePoint without custom code or third-party components.

Unless your AD groups are subject to strict discipline so that "Help Desk" only contains people from Help Desk, and not people who for any reason one day needed access to the Help Desk's files (e.g. senior management or Gladys from accounting), it is best to start fresh with new AD groups for SharePoint. Since 95% of companies did not have that discipline when securing network shares, you should start fresh unless you can both prove that your AD groups are clean, and policies are in place so groups will not be corrupted when people start applying them to SharePoint.

[Update 2008-09-12: This entire section was rewritten based on TB's suggestions. The earlier version preferred ADGroups to SPGroups solely by virtue of central management.]
If you break inheritance on many lists for the same users, clean up duplicate Limited Access entries in the containing site. When you configure a list or list item to use unique permissions and add a user to it who is not listed as a member of the site, so as to not break navigation the user is automagically given Limited Access -- bare-minimum read access -- to the site that contains the list. All fine and dandy until it hapens again and a duplicate entry is created. Resolution: Remove the duplicate entries by hand or automated utility.
Do right for the Business. Sometimes to meet a business need you need to break from the recommended guidance. There is "recommended guidance," and then there are hard limits imposed by the platform that you should not break without raising flags and shooting off fireworks. The appropriate solution may be at odds with "recommended guidance" and that's okay. Your goal is to simulate the business, not to change the business. When conflict happens, help the business understand its choices. Make any conflict crystal clear to the people paying the bills and suggest alternatives. If the conflict is a hard platform limit with an eventual breaking point, provide all the options to mitigate the issue and make sure that the person with the power to make the decision understands the eventual consequence of the decision. You cannot stop someone above you in the food chain from blowing his or her own foot off. This is natural selection. You can only prove that you asked them not to, and escalated alerts to the issue as danger became imminent.

Thanks to Dennis Shtemberg from Infusion for getting the ball rolling with his research on uniquely-permissioned items per list, Todd Klindt for a great dialogue, corrections and help to tease apart the issues, Todd Bleeker for taking the time to review this post and make it better, and Joel Oleson and Keith Richie for their clarifying the ACL issue and their earlier work on the subject. Todd and Keith are co-authors of DeliverPoint.

Fixed or Unproved [Last Updated: 2008-09-26]

This section contians the guidance that either no longer applies, or never did.

~~Unique list-item permissions per list: 600 to 1000~~. Resolved by WSS and MOSS SP1. When you assign unique permissions to list items in a given list, when the critical point is exceeded the error is displayed: "Operation is not valid due to the current state of the object" and the following event is logged in Event Viewer: "Unknown SQL Exception 156 occured. Additional error information from SQL Server is included below. Incorrect syntax near the keyword 'SET'. Incorrect syntax near ')'. Incorrect syntax near the keyword 'with'. If this statement is a common table expression or an xmlnamespaces clause, the previous statement must be terminated with a semicolon." Resolution: When any list item with unique permissions is deleted, the list works again. When any list item with inherited permissions is deleted, the issue persists. Mitigation: If many items will share the same ACLs, then group them into a separate list or library and apply the ACL to the list or library. If the permission change is a temporary state in a list item's lifecycle (for example to assign reviewer permissions during a workflow) then reset the permissions to inherited when the temporary state is complete (e.g. at the end of the workflow).

~~Users listed in a Site Collection's User Info Gallery (aka SPWeb.SiteUsers): 1500 to 2000~~. No evidence can be found to support this guidance, and believe it to be a misinterpretation of the limit on ACL size. If you've been affected, please comment with further detail.

A user is added to this list (and assigned an ID unique to the Site Collection) when a user is a) explicitly assigned a permission level on a securable object, or b) when a user is named as a member of either a SharePoint group or an Active Directory (AD) Group and that group is assigned a permission level on a securable object and the user then contributes to the site. Note that prior to WSS 3.0, just visiting a site added the member to the list, and now the user needs to actually contribute. When a user is deleted from the list either through the API or the User Info Gallery, the entry is marked "Deleted" but not removed [Update 2008-09-12: Corrected by KR]. Resolution: If you pass the limit, you need to remove users from the list. The difficulty is figuring out which are no longer listed in Created By and Last Modified By fields. Displaying or editing a list item with a reference to a deleted entry results in an error. Mitigation: 1) Create Site Collections such that each will serve up to 1000 contributors. [Deleted: "Create AD groups for readers." Not necessary as readers aren't added to the list.] Monitor the size of the SiteUsers list, and consider routines to prune rows from it where contributors are no longer named on sites in the collection (e.g. in Created By, Modified By, or Assigned To columns). 2) Either assign AD groups, or a combination of AD groups and individual users (the exceptions who do not logically belong in the AD group) to SharePoint Groups unless this will lead to SID count issues (see Active Directory Groups per User, below).

Why care about Google Chrome?

2008-09-04T00:33:00Z

In the last 24 hours there's been a lot of conversation about Chrome. When Safari was released for Windows, why was so little written about Safari's SharePoint compatibility? I used Opera for years, but why never a post about Opera and SharePoint (summary: it stinks, even drop-down menus fail to render)? What's the big deal about Chrome? Web developers certainly don't need another browser to support, unless this is the one that finally gets it right, and the odds of that are way high against. So why did I bother?

Unlike earlier entries -- and Firefox is the only measure of success to compare anything against -- Chrome has a chance of grabbing enough market share to make a difference against MSIE. The first win with long-term implications is that Google did a great job of designing a browser core, and while the first cartoon was aimed at developers, you can bet that its next features and marketing will be aimed squarely at users. Chrome is the first contender since Netscape with even a snowball's chance in Furnace Creek of unseating MSIE. Even though it's hot and the snowball isn't like to make it, this is an event.

The release of Chrome is also an opportunity to point out what's wrong with the browser market. Browser choice (that is, for any browser that bothers to adhere to standards) should be as much a matter of style as Word vs. WordPerfect used to be or Zune vs. iPod vs. Sansa vs. Zen is today. Say it again, web developers don't need another browser to support, web designers should be writing to standards, not to brands. A brand can become a de facto standard, but that's still a sign of either an immature market or a space that no one cares enough to compete in. I'd like to think we've come further than this since 1993, and that a new browser release should have little more effect on web developers than a new MP3 player does for musicians. Why are so many of today's conversation about standards and compatibility? That's a problem.

I do expect that as soon as browscap.ini (or whatever the equivalent is today) is updated we’ll see better behavior out of Chrome against existing sites including SharePoint. My guess is that Chrome would render existing .js better, but it’s being served a safe fall-back version by sites that don't yet recognize it as a client. Opera provides a switch to identify itself as different browsers against any given site, and that was a great trick when Opera worked better against some versions of IE-targeted code than others. Full Silverlight support will be coming soon. I don't know that for certain, but ScottGu's entire team is obsessed about cross-platform and they consistently surprise the skeptics, so it would be more surprising if it doesn't come to pass.

As for SharePoint and standards, the unfortunate reality is that when your product is deployed at companies that can limit browser choice and have consultants who can bend your product to meet the low percentage of organizations with accessibility standards, you get to live in a bubble and standards aren't yet a priority. Keep pushing, maybe one day a release like this really won't matter.

Until then, what I wrote yesterday stands. This is a new product that needs to accelerate through a lifecycle that other browsers have lived for years. It isn't ready for prime-time today. And if you needed another reminder not to use beta products in production, Chrome even had its own Day Zero Security Flaw. Since malicious hackers tend to target the clients people use most, perhaps the clearest signal of Chrome's importance is that people are bothering to look. On to the next question: "how long before Chrome tells me that a security update is ready for download?"

[Updated 2008-08-04]

Chrome's EULA ~~will prevent it from being blessed as a corporate browser anytime soon~~ was fixed within a day so now you can own the content you write in Chrome, now there's a happy update, issue resolved.

Privacy concern - Chrome sends every URL you visit to toolbarqueries.google.com by default (you can watch it with Fiddler). You can turn it off through Options, Under the Hood, and uncheck "Help make Google Chrome better by automatically sending usage statistics and crash reports to Google." How does tracking my clicks improve Chrome? Good question.

Google Chrome works for SharePoint users, less so for administrators

2008-09-02T19:58:00Z

Today I downloaded and installed the just-released Google Chrome browser, ran it through some preliminary tests with SharePoint 2007 and so far, acceptable but missing a few key things. Chrome supports NTLM authentication, uploads (though not multiple uploads), renders all the usual menus correctly, and generally does a good job of rendering SharePoint pages. And it's screaming fast.

On the downside, when you click a file you're asked for a Save location rather than opening it with the associated application. So if you're in a Doc Lib and click a document, you're asked for a location to save it. If you open the ECB menu and click "Edit in Microsoft Word" you get the message that "'Edit Document' requires a Windows SharePoint Services-compatible application and Microsot Internet Explorer 6.0 or greater." And the back button sometimes asks you to reload / re-post, even if there wasn't a user-driven POST and you'd expect it to work, like like opening an image in a library and then hitting Alt-left. Maybe I'm just used to this behaviour in other browsers.

Administrators will especially want to hang on to MSIE or Firefox for a while. Web Parts don't drag and drop while a page is in Edit mode, and even the Minimize/Close/Delete/Modify This Web part menu oddly shows as a right-hand column rather than inline with each web part itself, perhaps this is default behaviour for unrecognized browsers. Because SharePoint's UI was designed to provide all it's functionality to unknown or unsupported browsers (e.g. Opera), you can still assemble and rearrange pages, but niceties like drag and drop don't work here yet.

So for WCM sites, Chrome will work fine. For Collaboration sites, hold off until Chrome supports opening files with their associated applications. For administration, you may want to hang onto MSIE or Firefox for a while.

And if only Chrome would render the rich text box controls used in my blogging engine, I could have used it to write this post. . .

My general (non-SharePoint reaction to Chrome -- It's fast and clean. I wouldn't be surprised if they heard from Hasbro about possible trademark infringement against Simon for that logo. There are a few odd things in like missing borders on text boxes. It supports NTLM, that's a plus. Silverlight 2 doesn't support it yet so no NBCOlympics.com video. YouTube is fine though, I suppose you'd expect them to get the most popular sites right.

It saves paswords but there doesn't seem to be a master key file that I have any control over (Firefox does), so no idea whether it's actually encrypting my secrets on disk.

Conclusion: not bad for an initial beta, but when you write anything from the ground up in a mature industry you can expect several releases to get the important parts right.

SharePoint WFE memory allocation on 32-bit Windows Server

2008-08-11T23:20:00Z

How much memory can my SharePoint web front-end (WFE) servers use? It's a common question, and this post is an attempt to answer it for common scenarios. Briefly: 4 GB is the maximum recommended for most scenarios, though you may be able to use more if you're hosting other applications on the same server, including other web applications.

On 32-bit Windows Server 2003 each application pool can only address 2 GB of RAM because that's the limit of user-mode address space in a 32-bit application. You can reduce this effectively to ~1.5 GB after the CLR and core Framework are loaded. Joel recommends configuring 800 Mb per worker process and he has a slew of other great tips for configuring your Application Pools. Note that the /3GB switch is not supported with SharePoint for allocating more memory to the IIS worker process since it leads to memory instability.

If hosting multiple web applications, you can allocate the web applications into multiple application pools, and make use of more than 4 GB RAM. Note that a maximum of 8 application pools is recommended. For example, it's preferable to run Central Administration on a separate server (usually a MOSS Application Server), but you could run it on a WFE in a different application pool and even with its own Application Pool Identity. The same goes for in-house web service applications, though again, to isolate IIS issues caused by SharePoint and your web services it's advisable to always host custom code away from your SharePoint farm. But if you have a good reason to run your code locally (i.e. you need SharePoint interaction and can't get the job done efficiently with SharePoint's web services), it's possible to create your own application pool and host your custom code there. If more than one Application Pool hosts a SharePoint Web Application (i.e a site with SharePoint content, not one of the administrative or WS interfaces) you'll just need to be sure that the Application Pool Identities remain in synch.

If you're only hosting one SharePoint Web Application, there is no benefit to adding memory beyond 4 GB. If that's your scenario it's probably better to move try to move services off that machine so it can at least take full advantage of what it has. Two candidates to move off the WFE (after the obvious APP / SSP services) are the Central Administration application and Query services.

Other ways to mitigate 32-bit memory issues:

Follow Joel's guidance on configuring Application Pool Settings particularly: recycle application pools daily at an off-peak time, adjust the ceiling to 800 MB per WP, and keep the number of worker processes at one per Application Pool.
In all custom code, follow Mike's guidance to dispose of all of the disposable objects that you create, and understand where you're creating them inadvertently. If you don't leak memory, you'll have more memory.
Avoid using standard views of large lists and lists with a lot of metadata. Think in terms of cells being returned, not just list size, and write custom queries or web parts where lists are unavoidably large. See the whitepaper on dealing with large lists for more information.
Use queries to avoid walking the object model wherever possible. Don't use the memory-hogging "foreach (SPListItem item in mySPList)" when you can use GetItems() or GetItemsInFolder(),
Web Gardens. Leave worker process at 1 per application pool, or according to TechNet, "Do not use Web gardens." But. If you see frequent out-of-memory issues (e.g. on a 4 GB WFE with thousands of users), they may be possible to resolve by increasing worker processes to 2 (aka create a Web Garden) and cap your memory ceiling to 600 to 800 MB (i.e. divide the available RAM) per process. This can mitigate OOM errors but two things to consider: 1) performance will degrade, and 2) you should also disable page output caching and the BLOB cache since only one process can get a lock to manage either of these at a time, so successful use of the cache now depends on which thread made the call. That said, if memory is a concern you should probably minimize these caches on collaborative sites anyway since they're intended for sites with relatively static content. Heck, this could be a point of its own.
Disable page output caching and BLOB caching on collaborative sites, these features are best-suited for relatively static WCM sites.
Migrate to 64-bit Windows Server

64-bit Windows Server doesn't have this 2 GB-per-process limitation (the user-mode address space goes up to 8 terabytes) so memory added beyond 4 GB will always help. Note that 64-bit doesn't remove the performance risks of large lists, which really stem from inefficient default query strategies and the resulting amount of information coming back from SQL Server. Guidance for configuring the Application Pools is similar; recycling daily is still important, but the ceiling can be much higher. Just beware that the higher you let it go, the inevitable and regular garbage collection will simply have that much more to do.

Further reference

Optimizing Office SharePoint Server for WAN environments
JoelO's Recommendations for SharePoint Application Pool Settings
JoelO's Why WSS 3.0 x64 and MOSS 2007 x64... What's the deal?

[Update: Thanks to Todd Klindt for the guidance against mixing web gardens and BLOB caching, and to Nadeem for stopping me from iterating over lists.]

SharePoint Trace Logs and the Unified Logging Service (ULS)

2008-08-01T01:10:00Z

This is to explain what the ULS logs are, why they exist, and how to read and write 'em. In a nutshell, the Unified Logging Service (ULS) writes WSS 3.0 and MOSS events to SharePoint’s Trace Logs, and these are stored in the file system in ...\12\LOGS. Collectively this location and its files are commonly referred to as the “ULS Logs” though MSDN calls them the Trace Logs.

According to the MSDN Trace Log docs:

<“Developers can program their applications to write custom operations messages to the trace log, the same log that Windows SharePoint Services uses to log developer-oriented information and events. Writing to the same trace log Windows SharePoint Services employs enables third-party developers to view their custom application traces in the larger context of Windows SharePoint Services operations, without having to correlate multiple trace logs. In addition, the trace log provides a central destination for information concerning development processes, such as debugging. This alleviates the need for developers to log their development information in other places, such as the Windows Event Log, which are more commonly used by system administrators.”

Configuring ULS Logs

Operations àDiagnostic Logging. The Diagnostic Logging options are described on TechNet, and in the excellent "Taming SharePoint Trace Logs" blog.

Recommended configuration:

Sign up for the Customer Experience Improvement Program: Disabled
Error Reports: Ignore errors and don’t collect information.
Event Throttling: Recommend no throttling in development environments, and throttling from Audit Failure and above to capture warnings and errors in Production environments.
Trace Log Path: Create a new logging location on a drive (i.e. a physical spindle) that does not contain your OS or data. It is a good practice to create such a location for all such logs so that if the drive fills up, it doesn't jeopardize the health of either the OS or your data. Yes, of course you can also use it for your IIS logs. Note that if you change this location, the location must exist for all servers in the farm.
Number of log files and minutes to use a log file: The default gets you two days, how far you go is up to you.

The administrator can select events to log either in the Diagnostic Logging page, or more selectively with the STSADM command line utility. Verbosity should be set according to the environment and intended audience. For example: more verbose in development where developers actively debug applications, and as-needed in production where a network operations team uses the logs to monitor application health. For further information see the Taming article (cited above) on the topic search the internet for “SharePoint verbosity diagnostic logging.”

Reading the ULS Logs

The raw ULS logs can be difficult to navigate, and SharePoint provides no built-in viewer. To fill the gap and ease development and administration, free third-party tools exist like SPTraceView and WSS/MOSS Log File Reader (which plugs into Central Administration). Some utilities including SPTraceView will aggregate log data from more than one server in the farm.

Writing ULS Logs

What to write? The ULS logs are appropriate for recording any application lifecycle and diagnostic events not recorded elsewhere, including messages during feature installation, removal, activation, deactivation, startup, shutdown, and exceptions. Message types are identified through the Trace Severity property: Unassigned, Critical Event, Warning Event, Information Event, Exception, Assert, Unexpected, Monitorable, High, Medium, and Verbose.

To write Trace Logs you need a Trace Provider. MSDN provides a sample TraceProvider here. And that one was either adapted or copied into the Community Kit for SharePoint available here. I'm providing both, even though they're (virtually?) identical is that I would hope the CKS code will evolve and I don't expect that of the MSDN content.

Assuming the CKS version works for you (it's buried in the CKS.EWE subtree), you could build a wrapper like this for exception logging:

using System;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics;
using CKS.Utilities.UlsLogs;
using System.Reflection;

namespace EliRobillard.SharePoint.Diagnostics
{
    class ULS
    {
        /// <summary>
        /// Log an error to the ULS log
        /// </summary>
        /// <param name="productName">The name of the application or project.</param>
        /// <param name="errorMessage">The text of the error message.</param>
        /// <param name="errorCategory">The category of the error, usually the feature or solution being implemented.</param>
        public static void LogError(string productName, string errorMessage, string errorCategory)
        {
            TraceProvider.WriteTrace(0, TraceProvider.TraceSeverity.CriticalEvent, Guid.NewGuid(), Assembly.GetExecutingAssembly().FullName, productName, errorCategory, errorMessage);
        }
    }
}

Or just as simple (and without the overhead of reflection), just add your reference and call WriteTrace() directly. Look for the LogMessage methods inside the CKS.EWE.WikiDiscussion.CoordinateActions receiver, there are good examples of descriptive logging and those methods could be used in your own MyName.SharePoint.Utilities.Diagnostics project.

International SharePoint Professionals Association: It's alive!

2008-07-18T18:05:00Z

On Wednesday Bob Fox announced the launch of the ISPA and its website at http://www.sharepointpros.org/. Bob's been putting this together for well over a year so first off, congratulations to Bob on launching and thank-you for all the hard work you do to help user groups world-wide.

What does the ISPA do? Initially its goal is to support existing SharePoint user groups and help people kick-start new ones. There's a small board and a number of evangelists world-wide. Reza and I are the first two "Canadian Evangelists" and we can't wait to have counterparts in the West and East. Canada only has one SharePoint User Group (Toronto) but there is demand from coast-to-coast. If you've been wishing there was somewhere you could go to learn the latest about SharePoint, maybe the ISPA can bring out the user group leader in you.

To learn more or to learn how to start a SharePoint user group in your area, you're welcome to contact me through this blog, or e-mail the ISPA at contactus@sharepointpros.org.

Readers Guide to the SharePoint Scalability White Paper

2008-06-18T15:50:00Z

A SharePoint Server 2007 Scalability and Performance whitepaper was recently released "to provide strategic information about designing a high-volume, high-availability enterprise solution that can easily grow." it was announced yesterday in the SharePoint Product Team blog.

There is plenty of good content here, lots of good ideas, and many attractive diagrams. As for the tests, these are idyllic goals to shoot for if you want great performance – minimize (or eliminate) inserts and deletes, keep fewer than 200 files per folder (if you do the math, they appear to cap it at 130), don’t use more than 5 WFEs, and spread your databases over many physical volumes.

Note that the testers assign 2 (or 3 in the case of H:) business divisions, each with its own content database, per 1 TB physical volume (Fig. 12), which is more SAN management than most shops are aware they should provide. This allows ~500 GB per division. It’s interesting that while the content databases stay under 200 GB for a combined total of <400GB, the used disk space averages 600.375 GB (Fig. 19).

To restate the point: separate physical disks remain the best path to efficient I/O, and sets of local disks kick the tar out of any SAN that doesn’t take the separation into account. Just because you move “the storage problem” to a network service doesn’t mean you should forget about intelligent design. The best practice to provide separate physical volumes for the OS, data, logs and temporary files remains.

On to the test methodology. Note that in these tests they loaded the data beforehand, and the “user load testing” consisted of modifying existing documents, not inserts or deletes. No files, sites or site collections were created or harmed in the course of this study. Search the document for “delete,” you won’t find it. Why not? Because for each content database, list items inserts are O(n), and site creation and and deletion are (politely) non-deterministic. It takes an incrementally longer period to insert an item as it did for previous items. When the icon is spinning during site creation, other requests may (or may not) be fulfilled until the operation is complete. Deleting a site may also have an effect on response time. Again, this performance factor affects requests being served from the same content database, requests served by other content dbs are not affected.

Conclusion

For constructing a document repository with relatively static content, or a Publishing Site for WCM, this is an excellent and thorough document. This whitepaper describes a "large-scale content storage scenario" rather than a "large scale collaboration scenario." That doesn't mean you cannot build a scalable SharePoint collaboration environment, but this whitepaper does not claim to describe it. Keep that in mind when you look at the performance graphs. As to the architecture itself and the guidance provided in constructing the test farms, this whitepaper is worth a look. Thank-you to the team who put it together, there's some good stuff here, but for the future I'd really like to see testing go beyond browsing, search, and file updates.

[Update 2009-09-05] Changed title from "Review of the SPSWP" to "Readers Guide to the SPSWP"

TSPUG June 25: Future-Ready SharePoint From Taxonomy to Deployment

2008-06-17T20:53:00Z

Toronto SharePoint User Group
2 Bloor West, Toronto

Wednesday, June 25
6:00pm Registration and Social
6:30pm Meeting
7:00pm Feature Presentation (description below)
8:30pm Closing

This is our last meeting of the 2007-2008 season. After June 25, TSPUG is on summer break until September 17.

Future-Ready SharePoint Architecture: From Taxonomy to Deployment
by Eli Robillard (Infusion Development), Ruven Gotz (Ideaca), and Craig Lussier (Torys LLP).

In last month's episode (by Mindsharp, with several courses in Toronto the week of July 14 to 18), we learned the benefits of designing an effective taxonomy. This month we'll continue with the how-to's of designing taxonomies, and then
turn our business requirements and corporate taxonomy into an effective SharePoint deployment. Discover the techniques, learn about great software to support the design process, and see a case study of a sucessful deployment.

To attend, please RSVP.

Thanks to our sponsors for another great year!
Nexient - Cyberplex - Microsoft

TSPUG Takes Requests: Comment on this post to tell me what you'd like to see!

Stay tuned for the announcement of the 2nd Annual SharePoint Camp, coming in Fall 2008!

WCAG and CLF 2.0 Compliance for SharePoint

2008-06-05T20:45:00Z

Here's a nugget. At the May Toronto SharePoint User Group meeting I met Mike Maadarani who got me excited about his company's CLF 2.0 SharePoint Toolkit. Orangutech is based in our nation's capital and they've been working with a government department and Microsoft to help SharePoint meet CLF 2.0 and WCAG requirements. From the documentation:

The CLF 2.0 Team Site Feature will allow departments and agencies to create SharePoint team web sites [and publishing sites] that conform to the new Common Look and Feel Standards 2.0 (CLF 2.0) that conforms to the World Wide Web (W3C) Web Content Accessibility Guidelines 1.0 Priority 1 and 2 checkpoints, which are collectively known as WCAG 1.0 AA.

The toolkit includes two solutions, one with Publishing Site features, the other with Team Site features. Each include both a WSP for an automatic install, and a set of files with instructions for manual deployment. The Publishing site features include the configuration of English and French variations. The actual elements provided include a List Rollup web part, compliant English and French master pages, three page layouts (1,2, and 3 column), several style sheets, and supporting image files and javascript, mainly for the compliant menus.

It looks like a good piece of work that should have wide appeal on its own or as a starting point for further customized solutions. I located a WCF CLF 2.0 slide deck on the SharePoint Community Portal but there is precious little documentation apart from the installation files and tutorial included within the download.

Congratulations to Orangutech on the toolkit's release, and thank-you from the community for sharing!

Update: I forgot to mention the Accessibility Toolkit for SharePoint published on Codeplex. Of course Mike is convinced that Orangutech's is better, I'll leave it to the Commenters to chime in.

The rule of good technology

2008-05-07T04:12:00Z

Technology provides value to business by simulating and automating business processes. The measure of business software, whether client or web-based, is its ability to transform Business State A into Goal State B with the least amount of human intervention. Done right, it's about process engineering, not decoration. Business fundamentals haven't changed in ten thousand years, but they're accelerating as we automate. This rule never breaks.

Maybe it's just me, but I can't remember an instance where sticking plastic hearts or stars on file folders helped me find "good content" faster. To each his own criteria, yours probably isn't mine (also a rule by the way). The little green "this answer solved the problem" on Microsoft newsgroups gets it right for me maybe 25% of the time, it's not bad. I don't doubt it solved the thread for whomever put it there, but three out of four times the answer I need is either on some other page or isn't yet documented.

Now if social computing finds me an expert two floors up and down the hall who can solve Business State Q faster than I could with a coffee room bulletin board or a week of "asking around," fantastic. Type-written index cards with subjects (keywords) to cross-reference libraries, that was useful metadata. Top ten lists a la "here are the ten best books on subject x," those are great.

Which brings me to Twitter, which is where this train of thought began. I'm thinking "Links to links? Now that's just silly." I'll click three levels deep if there are actual choices along the way. Start me down a click tunnel and I'll avoid your links even if you one day find beer on Mars.

Is Twitter the Blackberry of blogs? The Blackberry turned e-mail into instant e-mail. Twitter instantly broadcasts the state of your ego (or web browser) to the masses in <141 byte-sized pieces. Like any good tool, it's exactly as useful as the person wielding it. When it simulates what goes on in the real world and makes it better by reaching the goal faster, it's good technology. When it's used to say "here's where I was today, maybe you want to go there too" it's about as entertaining and as productive as TV.

Eleven years ago on my first web page I joked that the Web was the bastard child of television and CB radio. Funny how we keep coming back to that. All I can add is that: if now you can use the rule of good technology to slice through junk and figure out which tools will help reach goals faster, then maybe you can figure out how to automate it.

Toronto SharePoint User Group Tonight (and next month)

2008-04-23T15:13:00Z

Topic: SharePoint and Silverlight – Bringing the best of both worlds together!

In this session we'll explore different scenarios for integrating SharePoint 2007 and Silverlight. We’ll explore different ways on how you can leverage the SharePoint 2007 amazing framework to expose content and data in a much richer way.

Speaker:

Andy Nogueira (MCTS, MCPD, MCAD) has over 10 years of professional experience in the IT industry. He is the Team Lead for the Microsoft Practice Group at nonlinear creations Inc. (nonlinear.ca), a Microsoft Gold Partner. He is responsible for designing and implementing several successful technology solutions based on MOSS 2007 (Microsoft Office SharePoint Server) integrated with technologies such as Enterprise Search, Windows Workflow, K2, Windows Communication Foundation (WCF), and Silverlight. He is also a Microsoft Certified Technology Specialist (MCTS) in WSS 3.0 (Application Development) and MOSS 2007 (Configuration and Application Development).

Address: Nexient Learning
2 Bloor Street West, room 11 on the 8th floor

Time: 6:00pm

To RSVP, please email susie.ibbotson@cyberplex.com

Next Month: Mindsharp speaks on site planning and taxonomies!

What to know about SmartPart, and LoadControl()

2008-03-04T18:52:00Z

At Infusion ASP.NET developers regularly ask how to easily build a web part or how to host a user control (.ascx) in SharePoint. Someone invariably replies "SmartPart!" at which point my job is to make sure they understand what they're getting into.

SmartPart is a cleverly coded web part by fellow MOSS MVP Jan Tielens. Once installed, you can drag an instance of SmartPart into a SharePoint web part zone and configure the new web part to host an .ascx file stored in the file system. And like magic, you can host user controls in SharePoint. Recent versions (v1.3+) are AJAX-aware, support web part connections, and improved the deployment story, and the underlying code is really quite good.

But, there are drawbacks, some of which may be overcome, some of which are realities.

1. SmartPart examples set web.config to Full Trust. Not for the web part, but for the entire SharePoint application. You do not want to do this in Production, and happily this is avoidable. However, it does mean that you can't avoid the pain of creating a CAS policy (e.g. if your controls require access to the SharePoint API or unmanaged resources) by using the SmartPart.

2. SmartPart is an open source project, but the download contains only its classes so you can't compile it without some work of your own. If you work for clients who want to use SmartPart, I strongly recommend building your own SmartPart project as a starting point, and maintaining that code base as new features are added to the CodePlex project. While you won't "own" the code (copyright rules would say it's still Jan's and you should attribute it as such), you will now have full control and accountability for the code running on your servers, as it should be.

3. The SmartPart disclaimer defines exactly what to expect should things go wrong, or should you need to extend its capabilities: ". . . the Software comes 'as is', with no warranties. None whatsoever. This means no express, implied or statutory warranty, including without limitation, warranties of merchantability or fitness for a particular purpose or any warranty of title or non-infringement." Part of the license is to distribute this license whenever SmartPart is used.

To be fair, this disclaimer is similar to many software licenses, and all open source licenses. Linux is open source and offers no support. When things go wrong with Linux, Linus Torvalds will not show up at your door to hold your hand. To make Linux palatable for Production, Red Hat sells support. If you pay extra, they may even hold your hand.

SmartPart carries no support, and no commercial version exists that would provide support. But, if you follow the advice of the last point and create you own project so you have control of the code, you can support your own. This is what you want to avoid: there was a period of several months after GotDotNet was decommissioned and SmartPart did not yet exist on CodePlex. During this time it was impossible to download any "official" version of SmartPart, current or otherwise. had the source been released earlier (it wasn't) this wouldn't have inconvenienced anyone. From now on should the source become unavailable for any reason, you'll only have yourself to blame.

4. Building a custom web part that uses LoadControl() to host a user control is not difficult, and SmartPart's source code is a great way to learn how! The sample code below shows you how to load a user control and interact with its content from a web part, in this case to retrieve the content of a SharePoint list and implement paging.

Using the sample code

This is intended to roughly parallel the Smartpart - the user control will go into a usercontrols folder at the root of your application and the web part's assembly can be deployed to the GAC unless you'd like to write a CAS policy to deploy it to the bin folder.

Place the ascx in a \usercontrols folder into the Web Application's root (e.g. C:\Inetpub\wwwroot\wss\VirtualDirectories\80\usercontrols).

Create the web part, sign the project and add a line like this to the SafeControls section of web.config. Remember to replace the Assembly attribute with your own, best obtained by opening your assembly with Reflector:
<SafeControl Assembly="LoadControlWP, Version=1.0.0.0, Culture=neutral, PublicKeyToken=myToken" Namespace="LoadControlWP" TypeName="LoadControlWebPart" Safe="True" />

Then get your web part's assembly into the GAC and recycle the application pool, or build a CAS policy (if you control requires it) and place it in the web application's bin folder.

Sample code (LoadControlWP.cs):

using System;
using System.Collections.Generic;
using System.Text;

using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Data;
using System.Web.UI.WebControls.WebParts;
using System.Diagnostics;
using Microsoft.SharePoint;

namespace LoadControlWP
{
    public class LoadControlWebPart : System.Web.UI.WebControls.WebParts.WebPart
    {
        private UserControl usercontrol;
        private GridView gvDemo;
        private const string defaultlist = "";
        private string _listtolink = defaultlist;
        protected DataTable dtDemo = null;

        [Personalizable(), WebBrowsable(),
        WebDisplayName("List to display"),
        WebDescription("Name of the list in this site to display")]
        public string ListToLink
        {
            get { return _listtolink; }
            set { _listtolink = value; }
        }

        protected override void CreateChildControls()
        {
            try
            {
                base.CreateChildControls();
                this.Controls.Clear();
                this.GetData();
                usercontrol = (UserControl)Page.LoadControl(@"/usercontrols/wpgrid.ascx");

                gvDemo = (GridView)this.usercontrol.FindControl("gvDemo");
                gvDemo.AllowPaging = true;
                gvDemo.DataSource = dtDemo;
                gvDemo.PageSize = 3;
                gvDemo.PageIndexChanging += new GridViewPageEventHandler(gvDemo_PageIndexChanging);
                this.Controls.Add(usercontrol);

                gvDemo.DataBind();
            }
            catch (Exception ex)
            {
                EventLog.WriteEntry("WebParts", "UCWebPart" + ex.ToString());
            }
        }

        void gvDemo_PageIndexChanging(object sender, GridViewPageEventArgs e)
        {

            gvDemo.PageIndex = e.NewPageIndex;
            gvDemo.DataBind();
        }

        private void GetData()
        {
            try
            {
                if (ListToLink.Length > 0)
                {
                    dtDemo = new DataTable();
                    dtDemo.Columns.Add("Title", Type.GetType("System.String"));

                    SPWeb site = SPContext.Current.Web;
                    SPList list = site.Lists[_listtolink];

                    foreach (SPListItem item in list.Items)
                    {
                        DataRow newRow = dtDemo.NewRow();
                        newRow["Title"] = item["Title"];
                        dtDemo.Rows.Add(newRow);
                    }
                }
            }
            catch (Exception ex)
            {
                EventLog.WriteEntry("WebParts", "UCWebPart - Retrieving items from " + _listtolink + "-" + ex.ToString(), EventLogEntryType.Error);
            }
        }
    }
}

Sample code (wpgrid.ascx):

<%@ Control Language="C#" ClassName="WebUserControl" %>

<script runat="server">

</script>

<asp:GridView ID="gvDemo" runat="server" AutoGenerateColumns="False" CellPadding="4" ForeColor="#333333" GridLines="None">
    <Columns>
        <asp:BoundField DataField="Title" HeaderText="Title" />
    </Columns>
    <FooterStyle BackColor="#5D7B9D" Font-Bold="True" ForeColor="White" />
    <RowStyle BackColor="#F7F6F3" ForeColor="#333333" />
    <EditRowStyle BackColor="#999999" />
    <SelectedRowStyle BackColor="#E2DED6" Font-Bold="True" ForeColor="#333333" />
    <PagerStyle BackColor="#284775" ForeColor="White" HorizontalAlign="Center" />
    <HeaderStyle BackColor="#5D7B9D" Font-Bold="True" ForeColor="White" />
    <AlternatingRowStyle BackColor="White" ForeColor="#284775" />
</asp:GridView>

"How to build a SharePoint dev box" updated

2008-02-27T17:39:00Z

I've updated the article on how to build a development machine for SharePoint 2007.

The best tool I've discovered is the SharePoint Manager 2007 which takes over from the SharePoint Inspector as the object model browser of choice.

Notable tools not yet on the how-to page are the U2U CAML Creator tool and Ms. White's Event Receiver Manager.

Several of the recommended tools were moved into a new optional section, the most notable of which is the VSeWSS component which I really can't recommend. There are so many better project templates available on CodePlex that the sole bright spot of VSeWSS is the Solution Generator. In another tragic turn, VSeWSS was "updated" and baked into VS 2008 in a way that actually made a few templates worse. So while the workflow designer is better, it's even harder to deploy your custom workflow to another machine now.

As always, if you see ways to improve this how-to guide I'd love to hear from you!