WPF Accessibility
Windows Presentation Foundation (WPF) provides a very interesting API for Accessibility called Microsoft UI Automation. It allows programmatic access to most user interface elements on the desktop, addressing the needs of assistive technology products and also for User Interface (UI) tests automation.
The framework provides solutions for both accessibility providers and clients, and it is conformed of four main components (see UI Automation Overview):
- The Provider API (UIAutomationProvider.dll and UIAutomationTypes.dll) defines a set of interfaces that are implemented by UI Automation providers, objects that provide information about UI elements and respond to programmatic input.
- The Client API (UIAutomationClient.dll and UIAutomationTypes.dll) is a set of types for managed code that enables UI Automation client applications to obtain information about the UI and to send input to controls.
- The UI Automation Core (UiAutomationCore.dll) has the underlying code that handles communication between providers and clients.
- The UIAutomationClientsideProviders.dll that has a set of UI Automation providers for standard legacy controls. (WPF controls have native support for UI Automation.) This support is automatically available to client applications.
We will typically use the Provider API to create support for our WPF custom controls, since they don't provide accessibility support by default. We will use the Client API for creating applications that need to communicate with UI elements and eventually automate the use of other program's UI.
You can find good documentation in the WPF Accessibility section on MSDN, and several code samples in the SDK Sample Applications.
Just to mention, Microsoft’s earlier solution for Accessibility was Active Accessibility. The main advantage of UI Automation over Active Accessibility is the technology in which it is based on (managed code vs COM) and, as a consequence, the programming languages that can be used. You can read more of the differences in UI Automation and Microsoft Active Accessibility.
A few words about the API's Object Model: Every piece of UI, such as a window, a button, etc, is represented by the AutomationElement class in the System.Windows.Automation namespace of the UIAutomationClient assembly. An AutomationElement corresponds to a piece of UI regardless of the underlying implementation (WPF or Win32). All automation elements are part of a tree, in which the root element is the Desktop. Through the AutomationElement.RootElement static property you can obtain a reference to the Desktop and find any child piece of UI from there to access.
AutomationElements expose Control Patterns that provide properties specific to their Control Types (window, button, checkbox, etc). Control Patterns also expose methods that enable clients to get further information about the element and to provide input. We are going to see how this works in the examples below.
A very useful tool for finding the element's properties in the tree is UISpy, which comes with the Windows SDK. It lets you see all the UI tree and each element's properties. In the Focus Tracking mode, you can select the real UI element in the Desktop and UISpy will automatically show its properties in the tree view.
Code sample
This is an example of a client application for automating a win32 application (in this case the Windows Address Book). For this example, we only need a Windows Console Application and references to the UIAutomationClient and UIAutomationTypes assemblies. From the code it can be seen how to match a control to a given pattern depending on what we want to do with the control.
The controls' names and/or ids where obtained using UISpy.
These are the UI artifacts that we want to automate:
And here is the code to:
- Get a reference to the Address Book application main window:
using System.Windows.Automation;
...
AutomationElement mainWindow = FindWindowByName(AutomationElement.RootElement, "Address Book - Main Identity");
...
private static AutomationElement FindWindowByName(AutomationElement rootElement, string name)
{
PropertyCondition nameCondition = new PropertyCondition(AutomationElement.NameProperty, name);
PropertyCondition typeCondition = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Window);
AndCondition andCondition = new AndCondition(nameCondition,typeCondition);
return rootElement.FindFirst(TreeScope.Element | TreeScope.Descendants, andCondition);
}
- Press the Find People button:
AutomationElement findPeopleButton = FindElementById(mainWindow, "Item 8084");
if (findPeopleButton != null)
{
InvokePattern invPattern = findPeopleButton.GetCurrentPattern(InvokePattern.Pattern) as InvokePattern;
if (invPattern != null)
{
invPattern.Invoke();
}
}
...
private static AutomationElement FindElementById(AutomationElement parentElement, string automationID)
{
PropertyCondition propertyCondition = new PropertyCondition( AutomationElement.AutomationIdProperty, automationID);
return parentElement.FindFirst(TreeScope.Element | TreeScope.Descendants, propertyCondition);
}
- Attach to the Find People dialog Closed event:
AutomationElement dialog = FindWindowByName(AutomationElement.RootElement, "Find People");
if (dialog != null)
{
Automation.AddAutomationEventHandler(WindowPattern.WindowClosedEvent, dialog, TreeScope.Element,
dialogEventHandler = new AutomationEventHandler(OnWindowClosedEvent));
}
...
private static void OnWindowClosedEvent(object src, AutomationEventArgs e)
{
Console.WriteLine("On window closed event");
}
- Attach to the Dialog's Close Button Clicked event:
AutomationElement closeButton = FindElementById(dialog, "68");
if (closeButton != null)
{
Automation.AddAutomationEventHandler(InvokePattern.InvokedEvent,closeButton, TreeScope.Element,
clickedEventHandler = new AutomationEventHandler(OnCloseButtonClickedEvent));
}
...
private static void OnCloseButtonClickedEvent(object src, AutomationEventArgs e)
{
Console.WriteLine("On button clicked event");
}
- Get and set the value of the Name text box of the Find People dialog:
AutomationElement text = FindElementById(mainWindow, "1152"); //Automation Id = 1152 of Name text box find with UISpy
if (text != null)
{
//get content
TextPattern txtPattern = text.GetCurrentPattern(TextPattern.Pattern) as TextPattern;
if (txtPattern != null)
{
string content = txtPattern.DocumentRange.GetText(-1);
Console.Out.WriteLine(content);
}
//write content
ValuePattern valuePattern = text.GetCurrentPattern(ValuePattern.Pattern) as ValuePattern;
valuePattern.SetValue("new value");
}
- Set a combo box value:
AutomationElement combo = FindElementById(dialog, "99");
if (combo != null)
{
AutomationElement item = FindListItemByName(combo,"Active Directory");
SelectionItemPattern itemPattern = item.GetCurrentPattern(SelectionItemPattern.Pattern) as SelectionItemPattern;
itemPattern.Select();
}
Well...that's it. I hope this helps for a start. Enjoy!
Sole