Eli Robillard's World of Blog.

Apologies all, notification was to go out sooner but apparently we're having technical difficulties so please help spread the word:

The May meeting (tonight) is cancelled, leaving you free to enjoy the summery Toronto evening. Enjoy!

Next month on Wednesday, June 16, TSPUG is back with not one, but two, count'em two presentations: First up will be Matthew Pakula from AvePoint talking about their terrific tools for MOSS 2007 and SPS 2010. Then I will reveal the secret toolkit I use to help companies build out SharePoint delivery centres in a session preliminary titled "How We Build SharePoint at Infusion." Basically I'll distill the best practices and tools gathered from Microsoft, the history of project management, and years of implementation, development and maintenance projects. 

And yes, TSPUG.com is down. It too will return and we want to come back on SP2010, but being a non-profit org we're still working out how to cover the licensing so all is legit. In the meantime, tune into the Twitter feed at http://twitter.com/tspug for all the latest.

SharePoint 2007 was the best demo-ware ever. It’s like going to the pet store and seeing a great dog that does backflips all kinds of tricks – and it really is a smart dog and it does all those tricks – but when you get it home you realize that what you need is a dog that gets the paper. SharePoint 2007 can be trained, but is fundamentally a platform where Microsoft's priority was to get the infrastructure right – to make it trainable and extensible. Because it was great demo-ware it caught on like nothing ever before and became a billion dollar product. But adapting it to specific uses did take serious effort because the priority was the infrastructure rather than the tools to build solutions on that infrastructure. Thankfully Microsoft did spend time explaining how to train it (a lesson they learned after SPS 2003), but the effort was a significant part of delivery. 

When building SharePoint 2010, the infrastructure was there so Microsoft focused on making it easier for people to build solutions. You still need a vision of what the solution will be and a design process to work out the details, and you still want people who deeply understand the platform in order to develop effective solutions. But the development story is now much more about configuration and less to do with programming. In both cases, both configuration and programming are supported with much-improved user interfaces, especially for common scenarios.

A gut estimate of the difference in effort goes like this. Counting "lines of code" and "effort of configuration" solutions consisted of ~50% configuration and 50% code with SP&T 2007, and programming took up 80% of the time. With SharePoint 2010, the same solutions can be composed with 90% configuration and 10% code, and on average programming will account for about 20% of the build time.

To consider an actual project, we delivered a knowledge management solution using MOSS 2007 including customized collaborative sites and a document repository. The bulk of programming time was spent on: custom field types (some cascading) to pull lookup values from a web service, rewriting content type propagation (Soren's code and Gary's extension do not cover important cases), building custom web parts to aggregate information from disparate sources, building workflows to approve and provision new sites, writing event receivers to tag and route documents, and building a custom site directory to display the sites a user belongs to. Thinking through these components, easily 90% of it would now be covered by configuration, and a 2010 solution would take about 50% less effort to maintain and update post-launch.

It’s hard to say how much shorter delivery time will average. If lines of code accounted for 80% of time before and 20% now, and a “unit of configuration” takes half the effort of development, then all-code would be an index of 1 (= 100% * 1), 2007 presents a relative effort of .9 (= 20% * .5 + 80% * 1) and 2010 falls to .42 (= 80% x .5 * 20% * 1). That's about half the effort (or resources). The architecture and design cycle should change little, if at all. Prototypes are more likely to provide a starting point for development (e.g. Visio flowcharts and SharePoint Designer workflows can now be imported to Visual Studio for finishing). Branding will change little, though there will be less redundancy (e.g. no additional effort to rebrand application pages). All work still requires testing and a feedback loop with sponsors and users.

Remember that these are all from-the-gut, invented numbers and non-scientific equations, and what counts are our real experiences and projects just now underway. So what do you think - close or far off? Was your index lower for 2007? Do the SharePoint 2010 components better support your scenarios and if not, why?

This year we plan to add one room and expand the number of sessions slightly from about 20 to about 25. As in past years while everyone is welcome to submit more than one abstract for consideration, each speaker will be selected for a single session in order to involve more people and encourage first-time speakers. Also as in past years, we’re looking for more content for attendees representing IT Pro (server administration), Architects, and "The Business." You can learn more about our tracks and audiences in the Call for Speakers document.

If your company is interested in sponsorship opportunities during the TSP Camp, please contact me through this site and I will connect you with sponsorship chair Ruven Gotz. Registration will also begin this month, stay tuned for details.

We do not provide travel or accommodation, but do provide lunch, a speaker shirt, local fame for a full hour, and a lifetime of fond memories. Hope to see you there!

I was going to hold off on posting this until the public beta drops, but anyone preparing for the drop will want to get the right hardware, OS and optionally virtualization in place now. Then when the beta drops I'll write more about specific steps to get SQL, SharePoint and your development tools installed.

Hardware and Operating System

The "official" hardware or virtualization requirements for running either SharePoint Server 2010 or WSS 4.0  are: a 64-bit dual-core 3 GHz CPU, 4 GB of RAM, an 80 GB hard drive, and a DVD drive (presumably for installation). My first virtual environment ran fine with 2 GB RAM and a 20 GB (virtual) hard drive, though as always more is better. Remember that like any product, SharePoint and Visual Studio 2010 are not fully optimized during the betas, and performance will continue to improve through RTM.

You do need a 64-bit OS and for virtualized hosting therefore have the choice between Microsoft Hyper-V, VMWare Workstation, and Sun’s VirtualBox (and likely other third-party apps, though I know these work). Out of the game and being deprecated in deference to Hyper-V are Virtual PC and Virtual Server, neither of which support a 64-bit guest OS. There are a few options for operating system with Windows 2008 R2 being the clear favourite, though Windows 7 and Vista – for the first time non-server operating systems – are supported for WSS 4.0 and SharePoint Server development.

Minimum SQL flavours are SQL Server 2005 with SP2 or SQL Server 2008 Standard Edition, though standalone installations can run on SQL Server 2008 Express (included). Note that some features (like converting a Site Collection to read-only mode) require SQL Server 2008 Standard. While for performance reasons it is not generally recommended to run SQL Server (or the SharePoint Index role) virtualized, it's normal in a development environment.

Additional requirements included in (or downloaded by) the installer are: .NET 3.5 SP1, Silverlight 2, IDFX “Geneva”, Powershell 1.0, and IIS 7.0. Supported client browsers are IE 7, Firefox 3 and Safari 3.

Windows 2008 R2 with Hyper-V is a good choice for virtualization on desktop or server installs, but is not supported by Microsoft for use on laptops. Hyper-V was really never intended for use on laptops, and so conveniences like Sleep and Hibernation aren’t there. Other notable things not usable on Windows 2008 R2 are Bluetooth, Zune software and Call of Duty, and there are others. I really like Windows 7 but I don't want to do development on my daily-use OS, so third-party virtualization like VirtualBox or VMWare are the clear winners for me, both of which support a 64-bit client OS. That said, many of the SharePoint MVPs who presented at SharePoint Conference 2009 (#spc09) use a configuration that can dual-boot into either Windows Server for development and presentations, or into Windows 7 for personal use. The fact that the boot times are great for both minimizes their dependency on Sleep.

Average hard disk sizes are larger, so virtualization solutions that support differencing disks - Hyper-V and VMWare - have a clear advantage. A differencing disk lets you create a single base image (a virtual hard disk) that you can then branch with many additional drive images - for example one for every project you work on. So your base image with SharePoint Server, SQL Server, Visual Studio 2010, and Office might be 20 to 30 GB, but additional machines will only be as large as your project and content, perhaps 5 to 10 GB apiece.

And for your physical hard disk do yourself a favour and get an Intel SSD (solid-state) drive installed. The average sizes are smaller (~120 GB is large as of writing this), but performance is absolutely killer with reports of up to +40% in boot times and regular operations; these drives are the reason people don't mind using Hyper-V to boot into their dev images. It's also nice that my laptop doesn't sound like a small aircraft anymore. All concerns about virtualizing on SSD have been wiped away (unlike when they first appeared on the market), so if you can get one, do it. As always any reliable 7200 RPM drive will perform well, but SSD is definitely the new "best-of" around. For external storage, my "building a machine" post for 2007 still applies with 7200 RPM eSATA through an ExpressCard interface probably your best bang-for-the-buck.

Popular Hardware

This section describes a few known configurations used by early adopters, feel free to comment with your own!

Hewlett-Packard HP8530. Quad-core with up to 8GB RAM and 7200 RPM disk(s), running Windows Server 2008 R2 and Hyper-V. Recommend dedicating 2 cores and 6 Gb to the guest OS (either Windows Server 2008 R2 or Windows 7).

Dell M6400. Intel Core 2 Duo P8600 (2.40GHz, 3M L2 Cache, 1067MHZ) Dual Core, up to 16 GB RAM, 17" screen, SSD optional, ~8.5 lbs [Full specs].

IBM Lenovo T500 and W500. Intel Core 2 Duo T9600 / 2.8 GHz Dual Core, up to 8 GB RAM, 15.4" screen, SSD optional, ~6.2 lbs. These are nearly the same laptop, the W500 has a better screen and video sub-system (find a detailed comparison here). The differences weren't important to me so I'm using the T500 and it's been great. My current virtual machine is assigned a 20 GB hard drive, 4 GB RAM and one processor (I tried two but since I only have two it was unstable, at least with VirtualBox; VMWare generally does better sharing CPU cycles with the host).

Servers and Desktop Workstations. With relative costs, remoting into a dedicated machine or a virtual server hosting many guests is sounding better and better these days, especially for development and test farms. The principles are the same, the requirements are the same, but it no longer matters how or from where you connect, and hardware costs are significantly lower. For anyone who doesn't need to demo or present on a laptop, I'd seriously consider it.

Until the Public Beta

With any luck that should get you started, have fun!

Someone recently asked about test plans and how to test components during development so you can be comfortable they'll perform well when hosted on large farms. The short answer is that you want to create the best simulation you can, and that means creating a test farm as close to production as possible, and testing scenarios with patterns and data as close to production as possible. With mission-critical apps the test environment should be identical with production, but in most cases it won’t be. Recent versions of LoadRunner do well for building the tests, earlier versions have issues (e.g. with javascript and with scripting against dynamically named / generated file sets). Visual Studio 2010 contains load-testing tools that work great against SharePoint 2010, I'm really looking forward to testing these when beta 2 is released next month. The Developer Dashboard is another great tool for breaking down the load times of each component on your page, the performance of methods in your call stack, and the latency of calls to background services; this will be an indispensible tool for checking performance.

Another great but under-used way to test is the million monkey method - get as many people in a room hammering at the test farm as you can. The first client I know of who identified what turned out to be a SharePoint performance bottleneck (row-level locks escalating to table-level locks) tested this way with just 14 people. Happily this any many other database performance issues are resolved in SharePoint 2010, but this targeted brute-force testing laid these issues bare where they hadn't come up at all in months of automated testing at other companies.

As an example of why you want the test farm to be identical with production, there was a SharePoint app that a colleague built and tested at a large bank, and they had the benefit of an identical test farm. The app was stress tested over a full 24 hour period, and issues only showed up in the ninth or tenth hour. While that translated to the issue occurring only after weeks of actual production, it would have ground the system to a halt had they not caught it before release. When SharePoint becomes mission critical, building the right test farm is worth the investment.

But because duplicating production can cost a lot, few companies actually do it. When the test farm doesn’t match production you can only guarantee that tests will provide a baseline of performance to compare with other apps or versions run in the same test environment; it really doesn’t tell you much about how the app will behave in production or how many users it will handle. You can make estimates in the ballpark - within an order of magnitude - and extrapolate to a degree, but too many factors conspire against a reliable interpretation; there’s simply no such thing. Whether it’s the number of servers in different farm roles, the use of virtualized vs. real servers for testing, differences in SQL implementation (is it clustered or mirrored, how may spindles are available to split OS/logs/data, are other apps sharing the SAN, etc.) , the effects of load balancing schemes (and admins whose understanding stops at round-robin), the availability and latency of network services, and the actual behavior of users (it’s hard to guess what peaks will be until the app’s been in the wild for a while) all conspire against relying on extrapolated results. Experience helps makes better predictions, but it equally tells you that it’s hard to guess where an app will fail until it does. And then service packs and upgrades inevitably throw your baselines off unless you retest every application after every upgrade.

So unless you can stress test a farm identical with production, you just try to get in the ballpark and pick other practical goals – are web parts or application pages running logic that kill performance, how does the farm respond to a mix of operations, are web service calls stalling pages, where should you be caching parts or data, where could you benefit from AJAX, how does response time on the local subnet compare to requests from around the world, how are these affected by authentication and the location of the domain controllers, how does the respones time of your custom pages compare with out-of-box pages, do use-of-memory patterns show leaks, etc.

Once an app is in production for a while you can get a few numbers to inform the tests – how often do you see peak usage, how many concurrent users does that mean, and how long does it typically last. What’s the mix of reader / writer / administrative operations, what pages do users hit (e.g. list views vs. home pages vs. app pages), where are users calling from, and what’s the mix of file sizes and types being uploaded or read. All of these help you build tests to more accurately simulate production use.

So test early, test often, and gather as much information as you can in order for your tests to approximate "the truth." Where you can't get enough information, or the hardware does not match production, be realistic about what testing will prove. And even if you can't predict the circumstances where production will fail, you can still use load and stress tests to build a better product.

The default experience when you press F5 in Visual Studio 2010 is to Create, Build, Package and Deploy your solution, all at once, automagically, pretty cool. As long as you don't want to control that process. But wait, you can do that too. You can customize exactly what happens when you press F5 to meet your own needs. From copying things into specific locations, to retracting or installing solutions, to calling MS-Build to do something special, to resetting the Application Pool, to stopping and restarting services, to whatever your heart and project desires or requires. This is huge and goes far beyond pre and post-build actions we had before. You save your custom Deployment Configuration in the Project Properties panel in a new SharePoint tab.

So if you want to Quick Deploy a web part into the application \bin or GAC, or update your application page without having to recycle the App Pool, you can write a Deployment Configuration just for any of these scenarios. You can build Deployment Configurations to copy specific files for particular environments (DEV/ACC/PRD), do your packaging and FTP the files out for code review without adding a Deployment step. Truly cool. You get two for free - Default deployment (the aforementioned Create, Build, Package and Deploy), and No Activation which does pretty much what the name implies.

You can take this a step further by creating custom actions to reuse in any Deployment Configuration, perhaps to execute SharePoint commands. Since VS is 32-bit and SharePoint is 64-bit, for this you'll use a new thing called a SharePointCommand that marshals your code into a 64-bit thread and executes it on your behalf. It actually binds to a thread identical to the one that serves the URL that your project is affiliated with. There's some cool stuff going on here - you don't need to attach the debugger to the worker process anymore (w3wp.exe ), that wire-up happens for you. And the same goes for code you marshal into that context using a SharePointCommand - any VS extensions you care to write can execute code in the SharePoint context associated with your project. Ted Pattison's seasion also went deep on the next steps - packaging the new component (in this case a Deployment Configuration option) into a VSIX file for passing on to anyone else who wants to use your option in their IDE.

More great nuggets

• No more DDF files! The Package Builder takes care of all the gory details.
• Automatic generation of Features, Manifests, .webpart files and supporting files.
• While the Feature manifest generates all you need, you can provide your own nodes to merge into the generated manifest, or override the process completely (but like, why)

 All in all, there'ssome pretty awesome control to be had over the deployment process.

* Notes based on a presentation by Eric Shupps at SharePoint Conference 2009 (#SPC09)

Extensibility points

• Already had: Macros, add-ins and packages
• New extensions based on MEF
• VSIX model simplifies distribution and deployment

VSIX Package

• A zip package
• Contains am .XML manifest
• Install by double-clicking

Managed Extensibility Framework (MEF)

• Part of .NET 4.0
• An extensible app "imports" functionality
• An extensible componet "exports" its functionality
• An application catalog tracks instances of imported component
• An application is composed by dynamically loading components

SharePoint Tools export extensibility interfaces

• The interfaces: ISharePointProjectExtension, IProjectItemExtension, IDeploymentStep, IExplorerNodeTypeExtension (though these may change for beta 2 release)
• Your custom extensions export these interfaces, e.g. :

[Export(typeof(ISharePointProjectExtension))] 
internal class myClass : ISharePointProjectExtension {
     public void Initialize(...)
    
     /* add event handlers */
    
     void projectService_ProjectMenuItemsRequested(...) {
          /* Add your options to the project menu */
     }
}

• Requires applying MEF attributes to export each interface
• MEF attributes written to the catalog at installation

Calls to SharePoint

• The trick is that VS is 32-bit and SharePoint is 64-bit, so if you want VS to execute SharePoint operations, you need to use a special SharePointCommand type that will marshal the code for you for execution in a 64-bit thread. The Client OM would be a simpler option if you don't need to elevate permissions.

 Designers

• You can build WPF extensions to VS2010 and there will be a number of community projects that do exactly that, or provide templates for custom designers.

* Notes from a SharePoint Conference 2009 (#spc09) presentation by Ted Pattison and John Flanders.

Look out TSPUG, there's a new user group in town! Led by Ray Outair, all the pieces are finally in place and the first meeting is:

This Monday from 6 to 8:30 at Microsoft Canada's Mississauga office!

Presented by Rob Windsor (ObjectSharp)

[Register Now!]

Yes, MSPUG (and this is their temporary site) is getting the scoop on TSPUG and this will be Canada's first look at SharePoint 2010. The TSPUG meeting next Wednesday will also be SharePoint 2010 presentation delivered by Savash Alic of Microsoft Canada. However our registration is over capacity, full up, SOLD OUT, and so we're pleased to announce that we'll be streaming it live over LiveMeeting. Don't miss out, register for next Wednesday's TSPUG LiveMeeting feed today! So there you go, next week will be full of great SharePoint content. There's so much in SharePoint 2010 that I guarantee Rob and Savash won't be covering the same ground, so sign up, and see you all live or virtually next week. 

And a big bold thank-you to Ray, Wanda Yu and Damir Bersinic at Microsoft Canada, Bill Brockbank, and all the other volunteers and sponsors who made this new user group a reality. I know it's going to be a great one.