Solution to minimize the loading of too much data to a DataGrid– WCF RIA Services and Silverlight
Note: The WCF RIA Services examples in this post is based on the WCF RIA Services PDC Beta for VS 2008 and VS 2010 Beta 2 Preview.
I have notice that a lot of developers are passing a lot of data over the wire, like they have forgot or don’t see the network between the client and the server. In this post I’m going to show how some solutions can be used to minimize the loading of data. I’m not going to mention about using techniques like GZIP compression with IIS to reduce the size, instead how to use the DataPager and also do aync. calls to load data on demand etc.
The following example uses WCF RIA Services and the well known Northwind database. The example also uses the LinqToEntitesDomainService to just passing DAL types directly to the client over the network, but all the examples will work fine even with DTO/”Presentation Model”.
Note: By using DTO instead or create a Entity Data Model for presentation purpose only,you can reduce the number of values passed to the client and also focus on the data that should be displayed by the current View, so no extra data is passed. Passing too much data that shouldn’t be displayed on the view can affect performance badly if the app has many users.
Here is the code of the DomainService:
[EnableClientAccess()] public class CustomerDomainService : LinqToEntitiesDomainService<NORTHWNDEntities> { public IQueryable<Customer> GetCustomerSet() { return this.ObjectContext.CustomerSet; } }
It simply returns all the Customers located in the Northwind’s Customers table. The View has a DataGrid to display all the Customers and here is the XAML for the View:
<UserControl xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:controlsToolkit="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Toolkit" ...> <Grid x:Name="LayoutRoot"> <data:DataGrid x:Name="customersDataGrid"/> <controlsToolkit:BusyIndicator x:Name="busyIndicator"/> </Grid> </UserControl>
Note: It’s a good practice to add some kind of ProgressBar while loading data, the reason is because all communications to the Server services are made asynchronous and we can’t predict how long time it will take to load the data.
In the example above the Silverlight 3 November 09 Toolkit’s BusyIndicator is used as an activity indicator.
This example don’t use the WCF RIA Services DomainDataSource control, instead code-behind to load the Customers, the DomainDataSource will be mentioned later in this post.
Here is the code-behind of the View:
public partial class MainPage : UserControl { public MainPage() { InitializeComponent(); this.Loaded += new RoutedEventHandler(MainPage_Loaded); } void MainPage_Loaded(object sender, RoutedEventArgs e) { var customerDomainContext = new CustomerDomainContext(); busyIndicator.IsBusy = true; customerDomainContext.Load<Customer>( customerDomainContext.GetCustomerSetQuery(), lo => busyIndicator.IsBusy = false null); customersDataGrid.ItemsSource = customerDomainContext.Customers; } }
The code above will show the BusyIndicator before the Loading of Customers takes place, and will hide the indicator when the Customers is returned from the server by passing a callback to the Load method of the customerDomainContext. The customerDataGrid’s ItemsSource property is set to the customerDomainContext’s Customers property. A very simple example of binding a DataGrid with data, but there are several problems with the above code.
1) The DataGrid in Silverlight 3 can take some time to add all rows and update the UI. (As far as I know the DataGrid in Silverlight 3 will not asynchronous add the rows. In a future version of Silverlight the rows may be added asynchronous, haven’t yet confirmed if the DataGird in Silverlight 4 will do it).
2) There are a lot of data passed over the wire because all Customers and the properties of a Customer entity will be passed over the network from the server to the client. If this is a public application we can’t know what kind of bandwidth the different users have etc. Passing too much data can affect performance badly.
3) The users will probably never walk through all the Customers in the list, so what’s the reason to add them all to the DataGrid from start?
The following part of the post will take each three issues above and show different solutions to reduce the above problems.
How to solve the problem where the DataGrid will take some time to display all rows?
One simple way to make sure that not all rows will be added to the DataGrid is by using paging. With Silverlight 3 there is a DataPager control which can be used for paging. The following shows how a DataPager can be added to XAML and be used with a DataGrid:
<UserControl xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:controlsToolkit="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Toolkit" ...> <Grid x:Name="LayoutRoot"> <Grid.RowDefinitions> <RowDefinition/> <RowDefinition/> </Grid.RowDefinitions> <data:DataGrid x:Name="customersDataGrid"/> <data:DataPager Grid.Row="1" PageSize="20" Source="{Binding ItemsSource, ElementName=customersDataGrid}" IsTotalItemCountFixed="True" VerticalAlignment="Top"/> <controlsToolkit:BusyIndicator Grid.RowSpan="2" x:Name="busyIndicator"/> </Grid> </UserControl>
The DataPager will only work if its Source property is set to a PagedCollectionView, so the Customers added to the customersDataGrid ItemsSource property needs to be converted to a PagedCollectionView, this can be done by creating an instance of the PagedCollectionView and pass the Customers as an argument to the PageCollectionView’s constructor.
public partial class MainPage : UserControl { public MainPage() { InitializeComponent(); this.Loaded += new RoutedEventHandler(MainPage_Loaded); } void MainPage_Loaded(object sender, RoutedEventArgs e) { var customerDomainContext = new CusotmerDomainContext(); busyIndicator.IsBusy = true; customerDomainContext.Load<Customer>( customerDomainContext.GetCustomerSetQuery(), lo => busyIndicator.IsBusy = false, null); customersDataGrid.ItemsSource = new PagedCollectionView(customerDomainContext.Customers);
} }
The above code will only make sure the DataGrid will show data a lot faster by not adding all rows at once to the DataGrid, but it will still pass all Customers to the client. Some users may not want the paging support, instead they want to have all the Customers listed in the DataGrid. I try to make them understand what kind of problems that can lead to and try to suggest some other solutions, for example by adding a filter to make sure to only list about 500 items, and add a TextBlock to the View with a text like this: “Total number of Customers is 500 out of 25 000 ….”, then make sure they use some kind of filter to load some specific Customers, for example maybe filter on Country and City etc, everything to reduce the number of data passed from the server to the client. With WCF RIA Services there is a DomainDataSource Control which can be used with the DataPager and can asynchronous load the Customers the current page is showing. No code-behind is needed when the DomainDataSource is used. The following is an example where the DomainDataSource is used with a DataPager:
<UserControl xmlns:riaControls="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Ria" xmlns:riaData="clr-namespace:System.Windows.Data;assembly=System.Windows.Controls.Ria" xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:myData="clr-namespace:SilverlightApplication53.Web" ...> <Grid x:Name="LayoutRoot"> <Grid.RowDefinitions> <RowDefinition/> <RowDefinition/> </Grid.RowDefinitions> <data:DataGrid x:Name="customersDataGrid" ItemsSource="{Binding Data, ElementName=dds}"/> <data:DataPager Grid.Row="1" PageSize="20" Source="{Binding Data, ElementName=dds}" IsTotalItemCountFixed="True" VerticalAlignment="Top"/> <riaControls:DomainDataSource x:Name="dds" AutoLoad="True" PageSize="20" LoadSize="30" QueryName="GetCustomerSet"> <riaControls:DomainDataSource.DomainContext> <myData:DomainService1/> </riaControls:DomainDataSource.DomainContext> <riaControls:DomainDataSource.SortDescriptors> <riaData:SortDescriptor PropertyPath="CompanyName" /> </riaControls:DomainDataSource.SortDescriptors> </riaControls:DomainDataSource> </Grid> </UserControl>
By using the DomainDataSource LoadSize property, we can specify how many Customers that should be loaded at a time. The PageSize is used to specify the size of the Page, in this case the each page should display 20 Customers.
The solution by using the DomainDataSource control will solve the three problems mentioned earlier in this post. It will reduce the number of data passed to the client, it will also make sure the DataGrid will show the data much faster and if the users don’t need to see all Customers, they don’t need get all Customers loaded and added to the DataGrid. The only limitation is the use of a DataPager which some users may not want to use.
Another solution to minimize the data passed to the client can be to only get 20 Customers and add them to the DataGird, and when the user moves the DataGrid Scrollbar an asynchronous call is made to get the next 20 Customers and add them dynamically to the DataGrid. By doing so the DataPager isn’t needed and the DataGrid can show the Customers a lot faster, it will also reduce the number of Customers passed from the server to the client.
How to minimize the number of items passed over the wire?
The DomainContext’s Load method takes a Query as an argument, the Query will be passed to the DomainService and will be executed on the server-side. So by using this Query feature we can make sure to filter the data and only make sure the Query method will give us the items we are interested in. The following example will create a Query and pass it down to the server for execution, the query will make sure to Skip a number of Customers and only take 21 Customers:
EntityQuery<Customer> query = _customerDomainContext.GetCustomerSetQuery()
.OrderBy(customer => customer.CompanyName)
.Skip(0)
.Take(21);
_customerDomainContext.Load<Customer>(
query,
lo =>
{
busyIndicator.IsBusy = false;
},
null);
To reduce the number of items passed over the wire and skip the use of a DataPager we can create our own Custom DataGrid where we will only show the 21 first Customers and when the user moves the DataGrid’s Vertical Scroller we load the next 21 Customers asynchronous and add them to the DataGrid. The reason why we need to create our own custom DataGrid is because we will not have access to the Vertical Scroller from the DataGrid itself. The DataGrid will only add a Scroller if some items will not fit into the height of the DataGrid. If we only get 21 Customers and bind it to the DataGrid and they all will fit into the DataGrid no scroller will be visible, and the users will think they got all Customers. What we need to do is to add a Scroller as if all the Customers are already added to the DataGrid. To make sure to make such as Scroller we need to get the actual height of a DataGrid row and also get the total number of Customers the DataGrid should normally display, and take the height times the number of Customers to fake the Scroller. By using the WCF RIA Services service operation we can do a really fast call to get the total number of Customers, then we can create a Query to load a the 21 first Customers. You may wonder why 21 why not 20, the reason is that we will set the height of the DataGrid so only 20 items will be listed, but to get access to the Scroller and set a its range we need to make it appear. So a simple hack to get access to the Scroller is to just show 20 items but add 21 to the DataGrid. The following code is the new CustomerDomainService with a Service operation to get the total number of Customers:
[EnableClientAccess()] public class CusotmerDomainService : LinqToEntitiesDomainService<NORTHWNDEntities> { public IQueryable<Customer> GetCustomerSet() { return this.ObjectContext.CustomerSet; } [Invoke] public int NumberOfCustomers() { return this.ObjectContext.CustomerSet.Count(); } }
The following code is the code-behind of the View that will show the Customers:
public partial class MainPage : UserControl { CustomerDomainService _customerDomainContext = new CustomerDomainService(); int _currentPage = 0; const int NumberOfItemsToLoad = 20; public MainPage() { InitializeComponent(); this.Loaded += new RoutedEventHandler(MainPage_Loaded); } void MainPage_Loaded(object sender, RoutedEventArgs e) { customersDataGrid.ItemsSource = _customerDomainContext.Customers; LoadCustomers(); } private void LoadCustomers() { busyIndicator.IsBusy = true; EntityQuery<Customer> query = _customerDomainContext.GetCustomerSetQuery() .OrderBy(customer => customer.CompanyName) .Skip(_currentPage * NumberOfItemsToLoad) .Take(NumberOfItemsToLoad+1); _customerDomainContext.NumberOfCustomers( io => { customersDataGrid.TotalNumberOfRows = io.Value; _customerDomainContext.Load<Customer>( query, lo => { busyIndicator.IsBusy = false; }, null); }, null); } private void customersDataGrid_LoadNewItems(object sender, System.Windows.Controls.Primitives.ScrollEventArgs e) { _currentPage++; LoadCustomers(); } }
I will not focus so much on this code, I assume you can figure out what it does. The NumberOfCustomers Service operation will be executed before loading the Customers to make sure the custom DataGrid’s TotalNumberOfRows property is set before the Customers is added to the DataGird’s ItemsSource. You can also see in the code that the Skip and Take is used to create a Query, the _currentPage will be increased by 1 when the custom DataGrid will trigger its LoadNewItems event handler (The LoadNewItems event will be trigged when the Scroller of the DataGrid reach a specific value, in this case when 20 rows are displayed in the DataGrid and the next 20 must be loaded and added to the DataGrid). The LoadNewItems event handler will also make a call to the LoadCustomers method to Load the next 21 Customers. The following code is the XAML of the View where the custom DataGrid is used:
<UserControl xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:controlsToolkit="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Toolkit" xmlns:myData="clr-namespace:SilverlightApplicationSample" ...> <Grid x:Name="LayoutRoot"> <myData:AsyncDataGrid x:Name="customersDataGrid" Height="505" LoadNewItems="customersDataGrid_LoadNewItems" /> <controlsToolkit:BusyIndicator Grid.RowSpan="2" x:Name="busyIndicator"/> </Grid> </UserControl>
The following code is the custom DataGrid:
public class AsyncDataGrid : DataGrid { public event ScrollEventHandler LoadNewItems; private double _rowHeight = 0.0; private double _oldScrollValue = 0.0; private ScrollBar _verticalScrollBar = null; public AsyncDataGrid() { this.LoadingRow += AsyncDataGrid_LoadingRow; } public int TotalNumberOfRows { get; set; } private double GetNextLoadingPosition() { var numberOfItems = 0; foreach (var item in ItemsSource) numberOfItems++; return _rowHeight * numberOfItems; } public override void OnApplyTemplate() { base.OnApplyTemplate(); _verticalScrollBar = this.GetTemplateChild("VerticalScrollbar") as ScrollBar; if (_verticalScrollBar != null) _verticalScrollBar.Scroll +=scrollBar_Scroll; } private void AsyncDataGrid_LoadingRow(object sender, DataGridRowEventArgs e) { e.Row.SizeChanged += Row_SizeChanged; this.LoadingRow -= AsyncDataGrid_LoadingRow; } private void Row_SizeChanged(object sender, SizeChangedEventArgs e) { _rowHeight = e.NewSize.Height; ((DataGridRow)sender).SizeChanged -= Row_SizeChanged; if (_verticalScrollBar != null) { _verticalScrollBar.Maximum = TotalNumberOfRows * _rowHeight; _verticalScrollBar.UpdateLayout(); } }
private void scrollBar_Scroll(object sender, ScrollEventArgs e) { if (LoadNewItems != null && e.NewValue > _oldScrollValue) { _verticalScrollBar.Maximum = TotalNumberOfRows * _rowHeight; if (e.NewValue >= GetNextLoadingPosition()) { _oldScrollValue = e.NewValue; LoadNewItems(this, e); } } } }
By override the OnApplyTemplate method, we can get access to the DataGrid’s Default ControlTemplate’s VerticalScrollbar and hook up to the Scroller’s Scoll event. By using the LoadingRow event and then the SizeChanged on a DataGridRow we can get the height of the DataGridRow (I couldn’t figure out a simple way of doing it, because the LoadingRow event can’t be used to get the ActaulHeight of the row loaded, it’s to early and there is no Rows property to get the UIElement of the DataGrid’s Row, the only way to get a DataGridRow is by using the LoadingRow event). Within the SizeChanged event handler of the DataGridRow, the Maximum property of the Scroller is set to the TotalNumberOfRows times the height of the row, this will make sure the Scroller has a range that will make sure it looks like several of rows are added but they aren’t. Withing the Scoller’s Scroll event handler (scrollBar_Scroll) the LoadNewItems event will be trigged only if the Scroller reach a position when new Customers should be loaded.
Note: I haven’t take care of the key board events, if a user uses the keyboard to walk through all the Customers added and reach the 21 Customer, no Customers will be loaded, but it’s easy to add. I left it out in this example because the hard part is to figure out how to handle the Scroller.
This solution will solve the three problems mentioned earlier in this post. It will reduce the number of data passed to the client, it will also make sure the DataGrid will show the data much faster and if the users don’t need to see all Customers, they don’t need get all Customers loaded and added to the DataGrid.
Summary
This post covered some solutions to minimize the data loaded form the server to the client, and also speed up the DataGrid. This was done by using the DataPager control, or the DomainDataSoure or to load Customers while the users moves a DataGrid’s Scrollbar.
If you want to know when I post a new blog post or just follow my me, you can follow me on twitter: http://www.twitter.com/fredrikn