Soledad Pano's Blog

Media from Sharepoint to the Cloud

On my last post I talked about a media processing component developed for Sharepoint 2010. Besides the processing features (video encoding, thumbnail generation, validations, etc…) there was an asset storage manager that enabled us to store the files on a configurable place, inside or outside Sharepoint. I said we initially started with three storage flavors: a Sharepoint library, an FTP server or a shared folder/virtual directory.

My co-worker Dwight Goins has extended the options by adding a cloud based storage manager. In this case he uses the Amazon Simple Storage Service (Amazon S3), which resulted in a great performing storage solution. In his post, he digs deeper in the issues of storing BLOBs (Binary Large Objects) in Sharepoint and the solution that Sharepoint provides, SQL’s Remote BLOB Storage (RBS). Then he talks about our solution and the details of the AmazonS3 client. Check this out!

Media Processing Component for Sharepoint

I have been working recently on the task of building a media processing component for the Sharepoint project I am working on. The requirements for the component are more or less the following:

• The Site Content Creators must be able to upload media assets (images, audio and video) to a Sharepoint list, and these assets must accomplish certain validation rules.
• The final destination where the assets are saved after upload must be configurable and extensible. To begin with, we are supporting saving to a Sharepoint library, a network share or an FTP server.
• The videos must be encoded to MP4 format, and thumbnail and poster images must be generated. The encoding process must be run asynchronously and the user must be notified by email when it is finished.

Today I want to share the design of the component and the key pieces of code. I will focus on the video upload process which is the more complex one because of the encoding. Audio and image uploads are more straightforward.

The main parts that build the solution are:

1. Custom Upload Process: This is the front end of the solution. It consists of a custom list with a custom upload form. The list has the link to the media file and more metadata fields (title, author, date, keywords, etc). When you click on create a new item on the list the custom upload form is opened and you can browse for a file to upload. The form has the required validation logic and it serves to save the assets to the configured location, which can be a Sharepoint library or an external location, like File System or FTP server. When the upload finishes you are redirected to the list item edit form so you can enter the metadata. The experience is similar to uploading a file to a Sharepoint document library.
2. Media Processing Backend Process: This consists of a timer job that queries the Media Assets list for items to process. It encodes the videos, generates thumbnail and poster images and uploads everything to the final destination. Finally, it notifies the user of the result of the process by email. For the video encoding we used the Microsoft Expression Encoder SDK. As I will explain later, this SDK cannot be used inside a Sharepoint process, so it runs in a separated process that is invoked from the timer job.
3. Storage Manager: this is a flexible and extensible component that abstracts the logic of saving (and deleting) a file to the final location depending on the flavor chosen thru configuration (File System, Sharepoint library or FTP). This component is used both by the front end upload mechanism and the back end media processing job.

Here is a diagram of the overall design for the video processing:

Now I will explain in a little more detail each component:

1. Custom Upload Process

The Media Assets List

This is a Sharepoint list that stores the metadata of the media assets, but not the asset itself (the assets are stored in the definite storage, which can be a Sharepoint assets library, a network shared folder, or an FTP server). The list is based on three custom content types, WebVideo, WebAudio and WebImage, all three inheriting from a base MediaAsset content type. This content type has the required fields for saving the asset metadata. The more important ones for the processing being:

• Location: the URL of the asset in its definite location (in the example on the picture it is a sharepoint library called MediaAssetsLib).
• Temp Location: As videos needs asynchronous processing, they are saved in a temporary location on upload. It is the timer job that uploads them to the definite location after encoding. The temp location is a shared folder on the network.
• Processing Status: It is Success for assets successfully uploaded to the definite storage, Pending for assets waiting for encoding in the back end process and Error in case of encoding fail.

The list has an event receiver attached in order for deleting the assets from the final destination or temporary folder when the items are deleted from the list.

To achieve the storage flexibility, a custom upload form was developed and hooked to the MediaAssets list. When you click on the “Add new item” link of the picture above, the custom upload form is launched. The form is attached to the base content type definition in the Elements file as this:

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <!-- Parent ContentType: Item (0x01) -->
  <ContentType ID="0x01004e4f21afc14c487892253cb129dd5001"
               Name="MyMediaAsset" Group="MyContent Types"
               Description="My Media Asset" Inherits="TRUE"
               Version="0">
    <FieldRefs>
    …    </FieldRefs>
    <XmlDocuments>
      <XmlDocument NamespaceURI="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">
        <FormUrls xmlns="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms/url">
          <New>_layouts/MyMedia/Upload.aspx</New>
        </FormUrls>
      </XmlDocument>
    </XmlDocuments>
  </ContentType>
</Elements>

The Upload Form

The form was created as an Application Page (Upload.aspx) in the Layouts folder. It contains the browse control to upload the file.

A useful tip here is how to achieve the same look and feel as the Sharepoint OOB forms. The InputFormSeccion, InputFormControl and ButtonSection controls were used for that matter.

In order to use these controls, you need to register the namespace on the top of the page:

<%@ Register TagPrefix="wssuc" TagName="ButtonSection" Src="/_controltemplates/ButtonSection.ascx" %>
<%@ Register TagPrefix="wssuc" TagName="InputFormSection" Src="/_controltemplates/InputFormSection.ascx" %>
<%@ Register TagPrefix="wssuc" TagName="InputFormControl" Src="/_controltemplates/InputFormControl.ascx" %>

And then include them in the page like this:

<wssuc:InputFormSection ID="InputFormSection1" runat="server"
      Title="Upload Document" Description="Browse to the media asset you intend to upload." >
  <template_inputformcontrols>
    <wssuc:InputFormControl runat="server" LabelText="" >
      <Template_Control>                   
        <div class="ms-authoringcontrols">
          Name: <br />
          <input id="FileToUpload" class="ms-fileinput" size="35" type="file" runat="server">    
          <asp:RequiredFieldValidator id="RequiredFieldValidator" runat="server" ErrorMessage="You must specify a value for the required field." ControlToValidate="FileToUpload"></asp:RequiredFieldValidator>
          <br />
          <asp:RegularExpressionValidator id="FileExtensionValidator" runat="server" ErrorMessage="Invalid file name." ControlToValidate="FileToUpload"></asp:RegularExpressionValidator>
          …                                                                          
        </div>
      </Template_Control>
    </wssuc:InputFormControl>
  </template_inputformcontrols>
</wssuc:InputFormSection>

You can read more about how to use these controls here.

So, what happens in the code behind?

On the OK button submit handler, the file to upload is processed. The logic is different depending on the asset type. Images are copied to the final destination by using the Storage Manager. A thumbnail is also generated and uploaded for them. Duration is calculated for Audio and Video files (Microsoft.WindowsAPICodePack API is used for that). Audio files are also copied to the final destination. Videos instead are leaved in a temporary storage (a network shared folder), because they need to be processed later by the timer job.

In all three cases, a list item is created with the asset metadata, and inserted into the MediaAssets list. Then the user is redirected to the list item Edit form, so he can complete filling the rest of the metadata.

Since the upload process may take a long time, all this happens in the context of an SPLongOperation. This is a Sharepoint class provided to display the “Processing…” dialog with the rotating gear image in it.

So, here is part of the code:

protected void btnOk_Click(object sender, EventArgs e)
{
    if (FileToUpload.PostedFile == null || String.IsNullOrEmpty(FileToUpload.PostedFile.FileName))
        return;     //FileToUpload is the HtmlInputFile control
    var originFileInfo = new FileInfo(FileToUpload.PostedFile.FileName);
    SPWeb web = SPContext.Current.Web;
    try
    {
        //create a MediaAsset object to save all asset metadata
        MediaAsset asset = MediaAsset.FromFile(originFileInfo, web.Url, mediaConfig);
        //start long operation to show the user the "Processing..." message
        using (SPLongOperation longOperation = new SPLongOperation(this.Page))
        {
            longOperation.Begin();

            string newFileUniqueName = String.Concat(Guid.NewGuid().ToString(), originFileInfo.Extension);

            SPSecurity.RunWithElevatedPrivileges(delegate()
            {
                //save to file system. Need to elevate privileges for that
                var tempFileInfo = new FileInfo(Path.Combine(mediaConfig.TempLocationFolder, newFileUniqueName));
                FileToUpload.PostedFile.SaveAs(tempFileInfo.FullName);

                asset.TempLocation = tempFileInfo.FullName;
                asset.Duration = MediaLengthCalculator.GetMediaLength(tempFileInfo);
                ...
            });
            
            var list = web.Lists[mediaConfig.MediaAssetsListName];
            int id;
            string contentTypeId;
            //insert new item in the MediaAssets list
            mediaRepository.Insert(list, asset, out id, out contentTypeId);
            //build url of Edit Form to redirect 
            string url = String.Format("{0}?ID={1}&ContentTypeId={2}", list.DefaultEditFormUrl, id, contentTypeId);
            //long operation ends, redirecting to the Edit Form of the new list item
            longOperation.End(url);
        }
    }
    catch (ThreadAbortException) { /* Thrown when redirected */}
    catch (Exception ex)
    {
        logger.LogToOperations(ex, Categories.Media, EventSeverity.Error,
            "Error uploading file to MediaAssets list. FileName: '{0}'.", originFileInfo.Name);

        SPUtility.TransferToErrorPage(ex.Message);
    }
}

Ok, now let’s see how the asynchronous processing part works.

2. Media Processing Backend Process

The backend process consists of a Sharepoint Timer Job that orchestrates the video processing and a console application that performs the actual encoding and generate the images. The console application is invoked by the timer job.

Encoder Console Application

The tool chosen for encoding videos was Microsoft Expression Encoder 4. We used the Pro version (paid) which includes support for H.264 (this means can encode videos to mp4 format as required by our client).

The encoder comes with an SDK, so you can programmatically encode your videos. The thing is that this API depends upon .Net Framework 4.0, and it is also 32-bit only. This is incompatible with a Sharepoint process (either web or timer job), since Sharepoint relies upon .Net 3.5 and runs in 64 bits. Hence the need to build a separate process outside of the Sharepoint Timer Job. The console application was a simple solution, and it could be configured to target .Net Framework 4.0 and x86 Platform.

This application expects four input parameters: the path to the original video, the desired path for the thumbnail image to generate, the desired path of the poster image to generate and the desired path of the encoded video to generate.

The Encoder SDK provides full flexibility for setting the encoding parameters (like format, size, bitrate, etc). It also provides a set of presets, that let’s you implement the encoding very easily. For example, here is the code for encoding using the H264VimeoSD Preset:

public void EncodeVideo(FileInfo inputFile, FileInfo outputFile)
{
    Microsoft.Expression.Encoder.MediaItem mediaItem = new MediaItem(inputFile.FullName);
    int bitrate = GetBitrate(mediaItem);

    using (Microsoft.Expression.Encoder.Job job = new Job())
    {
        job.OutputDirectory = outputFile.Directory.FullName;
        job.CreateSubfolder = false;
        job.MediaItems.Add(mediaItem);
        //H264VimeoSD preset settings: Output Format: MP4. Container: MP4. Video Codec: H.264 - Main. 
        //Video size: 640, 480. Video Bitrate: 2500 Kbps. Video Encoding: CBR SinglePass. 
        //Audio Codec: AAC. Audio Channels: Stereo. Audio Bitrate: 128 Kbps. Audio Encoding: CBR Single Pass
        job.ApplyPreset(Microsoft.Expression.Encoder.Presets.H264VimeoSD);
        job.Encode();
    }
}

public void GenerateVideoImage(FileInfo mediaFile, string imageFilePath, int width, int height)
{
    var video = new MediaItem(mediaFile.FullName);
    using (var bitmap = video.MainMediaFile.GetThumbnail(
        new TimeSpan(0, 0, 5),
        new System.Drawing.Size(width, height)))
    {
        bitmap.Save(imageFilePath, ImageFormat.Jpeg);
    }
}

Media Processing Timer Job

Sharepoint supports asynchronous processing of data through Timer Jobs. These jobs run within the context of a windows service, and are easily managed and deployed using the same tools as any other Sharepoint solution.

As the requirement was to run the job only in one of the application servers, it inherits from SPServerJobDefinition. Here is the Timer Job code:

public class MediaProcessingTimerJob: SPServerJobDefinition
{
    privateLoggerlogger = newLogger();

    publicMediaProcessingTimerJob() : base()
    {
    }
       
    publicMediaProcessingTimerJob(stringname,SPServer server):base(name,server)
    {
        this.Title = "MediaProcessingTimerJob";
    }

    public override voidExecute(SPJobState jobState)
    {
        stringwebUrl = String.Empty;
        try
       {
            webUrl = this.Properties["webUrl"].ToString();
            var mediaProcessor = newMediaProcessor(webUrl, MediaConfig.FromConfigRepository(webUrl));
            mediaProcessor.ProcessMedia();
        }
        catch(Exceptionex)
        {
            logger.LogToOperations(ex,Categories.Media, EventSeverity.Error, "Error executing MediaProcessingTimerJob in web '{0}'", webUrl);
        }
    }
}

During the deployment process, the MediaProcessingTimerJob is installed on the required server. The URL of the website for processing the assets are passed thru the job properties.

Here is part of the code of the helper tool that installs the job and sets it to run every 15 minutes:

private static void CreateMediaJob(string webUrl,SPServer server)
{
    var job = new MediaProcessingTimerJob("my-job-media-processing", server);
            
    job.Properties.Add("webUrl", webUrl);
            
    var schedule = new SPMinuteSchedule();
    schedule.BeginSecond = 0;
    schedule.EndSecond = 59;
    schedule.Interval = 15;
    job.Schedule = schedule;
    job.Update();
}

The logic of the timer job resides in the MediaProcessor::ProcessMedia method. It essentially queries the Media Assets List for assets in the “Pending” status and for each of these items it invokes the Encoder process and then uploads the resulting mp4 video and the generated thumbnail and poster images to the final destination. Finally it notifies the user of the result by email.

This is the code that the job uses to call the console application. Since the job tells the console application the output parameters (the path to the images and encoded video files), it doesn’t need to read any output from the console application. It must only read the standard error in case the console application fails.

private void ExecuteMediaProcess(string inVideoPath,string outThumbnailPath, string outPosterPath,string outVideoPath)
{
    string args = String.Format("\"{0}\" \"{1}\" \"{2}\" \"{3}\"", inVideoPath, outThumbnailPath, outPosterPath, outVideoPath);

    ProcessStartInfo startInfo = new ProcessStartInfo(config.EncoderExePath);
    startInfo.Arguments = args;
    startInfo.CreateNoWindow = true;
    startInfo.UseShellExecute = false;
    startInfo.RedirectStandardError = true;
    startInfo.RedirectStandardOutput = true;

    Process process = new Process();
    process.StartInfo = startInfo;
    process.Start();
    string error = process.StandardError.ReadToEnd();
    process.WaitForExit(MaxWaitingProcessMillisecs);

    if (process.ExitCode != 0)
    {
        //the application failed, get error from message from standard error
        throw new MediaProcessingException(String.Format("Video encoder process returned with exit code '{0}'. Error was: '{1}'",
            process.ExitCode, error));
    }
}

3. Storage Manager

The Storage Manager is the piece of code used by both the upload media form and the backend process. It is just a file manager used for abstracting from the actual destination of the files, which is configurable. As I said, we started supporting saving assets to File System, to a Sharepoint library or to an FTP server, but this can be further extended to support other places, like some storage on the Cloud.

The need for a flexibility in the location to store the files may come from bandwidth or space limitations, for a need to share assets with other applications or a need to manage a centralized file store.

Anyway, the interface is very simple:

public interface IAssetStorageManager
{
    void Delete(string fileUrl);
    string Save(System.IO.FileInfo file);
    string Save(string fileName, System.IO.Stream fileStream);
}

There is a factory that will give you the particular storage manager framework depending on configuration (all configuration is saved in a Sharepoint list, and the Sharepoint Config Store is used for retrieving it). Here is part of the factory code:

public class AssetStorageFactory
{    …
    static public IAssetStorageManager GetStorageManager(string configCategory,string webUrl)
    {
        var configHelper = new ConfigHelper(webUrl);
        string storageMethod = configHelper.GetValue(configCategory, StorageMethodConfigKey);
        if ("SPLibrary".Equals(storageMethod, StringComparison.InvariantCultureIgnoreCase))
        {
            return new SPLibraryAssetStorageManager(webUrl, mediaLibraryName);
        }
        else if ("FileSystem".Equals(storageMethod, StringComparison.InvariantCultureIgnoreCase))
        {
            return new FileSystemAssetStorageManager(storageFolderPath,storageBaseAddress);
        }   
        else if ("FTP".Equals(storageMethod, StringComparison.InvariantCultureIgnoreCase))
        {
            return new FTPAssetStorageManager(ftpServerUrl,ftpServerPullAdress,ftpServerUsername,ftpServerPassword);
        }
        throw new ArgumentException(String.Format("Incorrect configuration Value '{0}' in ConfigStore for category '{1}' and key '{2}'. Supported options are: '{3}'",
            storageMethod, configCategory, StorageMethodConfigKey, "FileSystem|FTP|SPLibrary"));
    }       
}

The implementation for a particular flavor is simple. For example, this is how the FTPAssetStorageManager saves a file stream:

public string Save(string fileName, System.IO.Stream fileStream)
{
    string fileUrl = ftpServerUrl + fileName;
    FtpWebRequest request = (FtpWebRequest)WebRequest.Create(fileUrl);
    request.Method = WebRequestMethods.Ftp.UploadFile;
    request.Credentials = new NetworkCredential(username, password);
    using (Stream ftpStream = request.GetRequestStream())
    {
        FileUtils.CopyStream(fileStream, ftpStream);
    }
    return pullBaseAddress + fileName;
}

And this is how the storage manager is invoked from the Upload form or the Timer Job:

//save to final location
IAssetStorageManager storage = AssetStorageFactory.GetStorageManager("Media", web.Url);
asset.Location = storage.Save(newFileUniqueName, fileInputStream);

Conclusion

Having talked about the most important parts of the Media Processing Component for Sharepoint, I think I’m done here. The code is too much to show everything in a post, but I’ve chosen the most important parts. I might dig deeper in some other post. I will probably write about how to display videos in a web page, too.

The interesting part is that even if this whole component doesn’t apply to another project, maybe some of its pieces can be reused, like the video encoding thing, or the custom upload form and storage manager to save files outside Sharepoint. So I hope it results useful to someone else!

Sharepoint Upload File Error: ‘The remote server returned an error: (409) Conflict’

I’ve been using the following code to upload a file to a Sharepoint document library using the Client Object Model:

where

But I kept getting the 409 Conflict Error:

ExceptionType: 'WebException'  ExceptionMessage: 'The remote server returned an error: (409) Conflict.'   StackTrace: ' 
at System.Net.HttpWebRequest.GetResponse()     
at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute()     
at Microsoft.SharePoint.Client.File.SaveBinary(ClientContext context, String serverRelativeUrl, Stream stream, String etag, Boolean overwriteIfExists, SaveBinaryCheckMode checkMode)     
at Microsoft.SharePoint.Client.File.SaveBinaryDirect(ClientContext context, String serverRelativeUrl, Stream stream, Boolean overwriteIfExists)     

After struggling for a while, I figured out that the problem was the library name. It contained a dash on it, like “My-LibraryName”. When I renamed it without the dash it started working.

The error in the code is how I built the URL. I was using the library name to build the library URL, but it seems that Sharepoint doesn’t like some characters in the URL, so when you create a library called “My-LibraryName”, the URL is actually http://<server>/MyLibraryName, without the dash.

Sharepoint @ Buenos Aires Codecamp 2011

BA Codecamp 2011 is coming soon and Tellago is going to be part of it! More than 10 architects from our team will be presenting the most innovative technology sessions on next October 15. I’m very happy to be speaking at this event! Along with my colleague Mariano Escurra, we will be presenting Best Practices using Javascript and JQuery in Sharepoint 2010. Bellow is the abstract. Register!!

Mejores prácticas utilizando Javascript y JQuery en SharePoint 2010

El uso de Javascript es una parte importante en la construcción de cualquier sitio web interactivo, siendo JQuery una de las bibliotecas Javascript más utilizadas. Al crear este tipo de sitios sobre SharePoint 2010, los desarrolladores pueden combinar el uso esta biblioteca con el modelo de objetos cliente Javascript (ECMAScript) mejorando notablemente la experiencia del usuario final.
Durante esta charla abordaremos los aspectos más importantes del modelo de objetos cliente ECMAScript de SharePoint 2010 y su integración con JQuery siguiendo las mejores prácticas.

Fixing Sharepoint’s “Cannot connect to the configuration database” error

I have Sharepoint Server 2010 installed in my development environment. It happened a couple of times that the computer hangs and the Sharepoint Configuration dabatase remains in a corrupted state. When I try to access the Sharepoint local site from the browser it gives the following error:

So, how did I fix it?

I first used this addselftosqlsysadmin.cmd script to add my account to the SQLServer sysadmin group.

Then performed the following commands from SQL Server Management Studio:

select state_desc from master.sys.databases where name = 'SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45'

The result was: SUSPECT, confirming we had a problem.

Then run:

ALTER DATABASE [SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45]  SET EMERGENCY;

GO

ALTER DATABASE [SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45]  SET SINGLE_USER;

GO

DBCC CHECKDB ([SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45] , REPAIR_ALLOW_DATA_LOSS) WITH NO_INFOMSGS, ALL_ERRORMSGS;

GO

ALTER DATABASE [SharePoint_Config_5ffca14b-db6f-4b30-9dd8-8317ee0f4f45]  SET MULTI_USER;

GO

That’s it! I run the state_desc command again and this time it says: ONLINE

Solved!

Azure Bootcamp @MS Argentina

I participated yesterday in the Azure Bootcamp that took place at Microsoft Argentina. Along with Vanesa Guccione, my colleague at Lagash, we talked about advanced roles. The contents of the presentation included the new features of the platform, such as full IIS mode, startup tasks, remote desktop plugin and local storage. We also talked about input and internal endpoints and inter-role communication, showing some demo code as well. You can download the material from here.

How to find Visual Studio command bars

When developing a Visual Studio Addin it is a common task to add custom commands to existing menus and toolbars. It is a common problem too, not to find the proper command bar (we have to traverse all command bars to see its names, the names are not unique, etc). In these two posts: Using IVsProfferCommands to retrieve a Visual Studio CommandBar and  Using EnableVSIPLogging to identify menus and commands with VS 2005 + SP1 it is very well explained how to solve this problem.

The answer relies on the fact that every toolbar and menu is uniquely identified in Visual Studio by a GUID,Id pair. In order to see which is the GUID, Id pair for a given command, you must:

• Add or change the registry key HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\General\EnableVSIPLogging to 1.

• Click on the toolbar or menu you want identify while keeping CTRL+SHIFT pressed. This will show a dialog with the command bar properties. Take note of the Guid and the CmdID:

• From the Addin, use the following code to add your command in the desired command bar:

private void AddMyCustomCommand()

 

{

    …

 

 

    object[] contextGuids = new object[0];

 

    myCommand = ((Commands2)this._applicationObject.Commands).AddNamedCommand2(this._addInInstance, 

             partialCommandName,displayName, tooltip, true, null, ref contextGuids,

 

            (int)vsCommandStatus.vsCommandStatusSupported + (int)vsCommandStatus.vsCommandStatusEnabled,

 

            (int)vsCommandStyle.vsCommandStyleText,vsCommandControlType.vsCommandControlTypeButton);

 

    CommandBar ownerBar = FindCommandBar(new Guid("{9AEB9524-82C6-40B9-9285-8D85D3DBD4C4}"), 1280);

 

    myControl = workflowCommand.AddControl(ownerBar, 1) as CommandBarButton;   

}

 

private CommandBar FindCommandBar(Guid guidCmdGroup, uint menuID)

 

{

 

   // Retrieve IVsProfferComands via DTE's IOleServiceProvider interface

 

   IOleServiceProvider sp = (IOleServiceProvider)_applicationObject;

 

   Guid guidSvc = typeof(IVsProfferCommands).GUID;

 

   Object objService;

 

   sp.QueryService(ref guidSvc, ref guidSvc, out objService);

 

   IVsProfferCommands vsProfferCmds = (IVsProfferCommands)objService;

 

   return vsProfferCmds.FindCommandBar(IntPtr.Zero, ref guidCmdGroup, menuID) as CommandBar;

 

}

[ComImport,Guid("6D5140C1-7436-11CE-8034-00AA006009FA"),

 

InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)]

 

internal interface IOleServiceProvider

 

{

 

    [PreserveSig]

 

    int QueryService([In]ref Guid guidService, [In]ref Guid riid,

 

       [MarshalAs(UnmanagedType.Interface)] out System.Object obj);

 

}

CodeCamp Buenos Aires 2008

I attended yesterday to the Microsoft Code Camp event at Buenos Aires. It was fun!

Here's a picture of the Lagash stand:

UI Automation Article Published on Level Extreme

On this month's edition of the Level Extreme .Net Magazine, my article about Building an UI Automation Client Application was published. The article walks through building a client application for automating a win 32 target application (the Windows Address Book). The idea is to show through simple code snippets how to use the UI Automation API for manipulating a target application programmatically through its UI. It shows how to find the UI elements, and how to work with the UI Automation patterns and handle events. The sample client application code contains further examples and can be downloaded from the article's page. Hope you read it!

Read more on the magazine from Martin Salias.

CCF: Listening to External Application Events

It is often necessary to listen to external application UI events when automating applications in CCF. For example, we may need to save a value in the Context if the user presses a button, or do something when an alert dialog appears. There are many ways we can handle events. In CCF 2008, the Hosted Application Toolkit (HAT) provides an easy way to listen to them. If not using HAT we can hook to events using the UIAutomation or MSAA (Microsoft Active Accessibility) APIs within our own application adapters.

Events in HAT

When we configure a hosted application for using an Automation Adapter, this adapter listens to the application events. We can hook and unhook to these events using the CCF's WF activities RegisterActionForEvent (see figure bellow)  and UnregisterActionForEvent respectively.

It is the Data Driven Adapter (DDA) the one who throws the ControlChanged event (to which the activity subscribes). The arguments of the event consists of eventTypeName, controlName and controlValue.

For example, the WinDataDrivenAdapter that CCF provides supports the following event types:

1. SetControlValue: When the DDA set's a control's value.
2. ExecuteControlAction: When the DDA executes a control.
3. ContextChanged: When the context changes.
4. MenuShown: When a menu is shown
5. WindowShown: When a window is shown
6. WindowDisappeared: When a window is closed
7. LostFocus
8. SetFocus
9. CheckBoxSet
10. CheckBoxCleared
11. RadioButtonSet
12. RadioButtonCleared
13. ButtonPressed
14. ButtonReleased

There are some tricks about how to use this events, for example:

• 4-6: For menu and window events we must not register with the control's friendly name that we set in the databindings section of the AppInitString. We must specify the window or menu's caption instead. 
• 7-14: For these events the related control must be previously found on the UI. You can use the FindControl activity to ensure that the control is successfully found. When the DDA finds the control it registers it in a KnownControls collection. Only for those registered controls the events are thrown.

Custom DDAs or Legacy Adapters

If we need to handle events from a custom DDA or from a legacy adapter we can use win32 or accessibility APIs. If we use a custom DDA we can throw a ControlChanged event in order to use the WF/Automation activities mentioned before.

WinEvents

A low level approach is to use the SetWinEventHook and UnhookWinEvent functions from user32.dll. When we hook for events we specify the callback function that will handle the events. You can see the list of supported events and its constants in the MSAA SDK documentation on msdn.

In order to show some sample code, here are the user32 functions imports:

[Flags]

Here's how we can subscribe to win events from our custom adapter:

And finally how to react to events in the event handler method:

UIAutomation

Another option is to use the UIAutomation accessibility API, included on .Net framework 3.0 to hook to events from custom adapters. The Automation element provides the following static methods to hook to different event on different controls:

The UIAutomation API is much more friendly to use, but some times it takes too long for the event to reach the handler. You should be very precise when setting the element and scope to subscribe the event to. You can find more information about this API and sample code on my previous post about WPF Accessibility.