Solution to minimize the loading of too much data to a DataGrid– WCF RIA Services and Silverlight
Note: The WCF RIA Services examples in this post is based on the WCF RIA Services PDC Beta for VS 2008 and VS 2010 Beta 2 Preview.
I have notice that a lot of developers are passing a lot of data over the wire, like they have forgot or don’t see the network between the client and the server. In this post I’m going to show how some solutions can be used to minimize the loading of data. I’m not going to mention about using techniques like GZIP compression with IIS to reduce the size, instead how to use the DataPager and also do aync. calls to load data on demand etc.
The following example uses WCF RIA Services and the well known Northwind database. The example also uses the LinqToEntitesDomainService to just passing DAL types directly to the client over the network, but all the examples will work fine even with DTO/”Presentation Model”.
Note: By using DTO instead or create a Entity Data Model for presentation purpose only,you can reduce the number of values passed to the client and also focus on the data that should be displayed by the current View, so no extra data is passed. Passing too much data that shouldn’t be displayed on the view can affect performance badly if the app has many users.
Here is the code of the DomainService:
[EnableClientAccess()] public class CustomerDomainService : LinqToEntitiesDomainService<NORTHWNDEntities> { public IQueryable<Customer> GetCustomerSet() { return this.ObjectContext.CustomerSet; } }
It simply returns all the Customers located in the
Northwind’s Customers table. The View has a DataGrid to
display all the Customers and here is the XAML for the View:
<UserControl xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:controlsToolkit="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Toolkit" ...> <Grid x:Name="LayoutRoot"> <data:DataGrid x:Name="customersDataGrid"/> <controlsToolkit:BusyIndicator x:Name="busyIndicator"/> </Grid> </UserControl>
Note: It’s a good practice to add some kind of
ProgressBar while loading data, the reason is because
all communications to the Server services are made
asynchronous and we can’t predict how long time it will
take to load the data.
In the example above the Silverlight 3 November 09 Toolkit’s BusyIndicator is used as an activity indicator.
This example don’t use the WCF RIA Services DomainDataSource
control, instead code-behind to load the Customers, the
DomainDataSource will be mentioned later in this post.
Here is the code-behind of the View:
public partial class MainPage : UserControl { public MainPage() { InitializeComponent(); this.Loaded += new RoutedEventHandler(MainPage_Loaded); } void MainPage_Loaded(object sender, RoutedEventArgs e) { var customerDomainContext = new CustomerDomainContext(); busyIndicator.IsBusy = true; customerDomainContext.Load<Customer>( customerDomainContext.GetCustomerSetQuery(), lo => busyIndicator.IsBusy = false null); customersDataGrid.ItemsSource = customerDomainContext.Customers; } }
The code above will show the BusyIndicator before the Loading of Customers takes place, and will hide the indicator when the Customers is returned from the server by passing a callback to the Load method of the customerDomainContext. The customerDataGrid’s ItemsSource property is set to the customerDomainContext’s Customers property. A very simple example of binding a DataGrid with data, but there are several problems with the above code.
1) The DataGrid in Silverlight 3 can take some time to add all rows and update the UI. (As far as I know the DataGrid in Silverlight 3 will not asynchronous add the rows. In a future version of Silverlight the rows may be added asynchronous, haven’t yet confirmed if the DataGird in Silverlight 4 will do it).
2) There are a lot of data passed over the wire because all Customers and the properties of a Customer entity will be passed over the network from the server to the client. If this is a public application we can’t know what kind of bandwidth the different users have etc. Passing too much data can affect performance badly.
3) The users will probably never walk through all the Customers in the list, so what’s the reason to add them all to the DataGrid from start?
The following part of the post will take each three issues
above and show different solutions to reduce the above
problems.
How to solve the problem where the DataGrid will take some time to display all rows?
One simple way to make sure that not all rows will be added
to the DataGrid is by using paging. With Silverlight 3 there
is a
DataPager
control which can be used for paging. The following shows
how a DataPager can be added to XAML and be used with a
DataGrid:
<UserControl xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:controlsToolkit="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Toolkit" ...> <Grid x:Name="LayoutRoot"> <Grid.RowDefinitions> <RowDefinition/> <RowDefinition/> </Grid.RowDefinitions> <data:DataGrid x:Name="customersDataGrid"/> <data:DataPager Grid.Row="1" PageSize="20" Source="{Binding ItemsSource, ElementName=customersDataGrid}" IsTotalItemCountFixed="True" VerticalAlignment="Top"/> <controlsToolkit:BusyIndicator Grid.RowSpan="2" x:Name="busyIndicator"/> </Grid> </UserControl>
The DataPager will only work if its Source property is
set to a
PagedCollectionView, so the Customers added to the customersDataGrid
ItemsSource property needs to be converted to a
PagedCollectionView, this can be done by creating an
instance of the PagedCollectionView and pass the Customers
as an argument to the PageCollectionView’s constructor.
public partial class MainPage : UserControl { public MainPage() { InitializeComponent(); this.Loaded += new RoutedEventHandler(MainPage_Loaded); } void MainPage_Loaded(object sender, RoutedEventArgs e) { var customerDomainContext = new CusotmerDomainContext(); busyIndicator.IsBusy = true; customerDomainContext.Load<Customer>( customerDomainContext.GetCustomerSetQuery(), lo => busyIndicator.IsBusy = false, null); customersDataGrid.ItemsSource = new PagedCollectionView(customerDomainContext.Customers);
} }
The above code will only make sure the DataGrid will show data a lot faster by not adding all rows at once to the DataGrid, but it will still pass all Customers to the client. Some users may not want the paging support, instead they want to have all the Customers listed in the DataGrid. I try to make them understand what kind of problems that can lead to and try to suggest some other solutions, for example by adding a filter to make sure to only list about 500 items, and add a TextBlock to the View with a text like this: “Total number of Customers is 500 out of 25 000 ….”, then make sure they use some kind of filter to load some specific Customers, for example maybe filter on Country and City etc, everything to reduce the number of data passed from the server to the client. With WCF RIA Services there is a DomainDataSource Control which can be used with the DataPager and can asynchronous load the Customers the current page is showing. No code-behind is needed when the DomainDataSource is used. The following is an example where the DomainDataSource is used with a DataPager:
<UserControl xmlns:riaControls="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Ria" xmlns:riaData="clr-namespace:System.Windows.Data;assembly=System.Windows.Controls.Ria" xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:myData="clr-namespace:SilverlightApplication53.Web" ...> <Grid x:Name="LayoutRoot"> <Grid.RowDefinitions> <RowDefinition/> <RowDefinition/> </Grid.RowDefinitions> <data:DataGrid x:Name="customersDataGrid" ItemsSource="{Binding Data, ElementName=dds}"/> <data:DataPager Grid.Row="1" PageSize="20" Source="{Binding Data, ElementName=dds}" IsTotalItemCountFixed="True" VerticalAlignment="Top"/> <riaControls:DomainDataSource x:Name="dds" AutoLoad="True" PageSize="20" LoadSize="30" QueryName="GetCustomerSet"> <riaControls:DomainDataSource.DomainContext> <myData:DomainService1/> </riaControls:DomainDataSource.DomainContext> <riaControls:DomainDataSource.SortDescriptors> <riaData:SortDescriptor PropertyPath="CompanyName" /> </riaControls:DomainDataSource.SortDescriptors> </riaControls:DomainDataSource> </Grid> </UserControl>
By using the DomainDataSource LoadSize property, we can specify how many Customers that should be loaded at a time. The PageSize is used to specify the size of the Page, in this case the each page should display 20 Customers.
The solution by using the DomainDataSource control will solve the three problems mentioned earlier in this post. It will reduce the number of data passed to the client, it will also make sure the DataGrid will show the data much faster and if the users don’t need to see all Customers, they don’t need get all Customers loaded and added to the DataGrid. The only limitation is the use of a DataPager which some users may not want to use.
Another solution to minimize the data passed to the client
can be to only get 20 Customers and add them to the
DataGird, and when the user moves the DataGrid Scrollbar an
asynchronous call is made to get the next 20 Customers and
add them dynamically to the DataGrid. By doing so the
DataPager isn’t needed and the DataGrid can show the
Customers a lot faster, it will also reduce the number of
Customers passed from the server to the client.
How to minimize the number of items passed over the wire?
The DomainContext’s Load method takes a Query as an
argument, the Query will be passed to the DomainService and
will be executed on the server-side. So by using this Query
feature we can make sure to filter the data and only make
sure the Query method will give us the items we are
interested in. The following example will create a Query and
pass it down to the server for execution, the query will
make sure to Skip a number of Customers and only take 21
Customers:
EntityQuery<Customer> query = _customerDomainContext.GetCustomerSetQuery()
.OrderBy(customer => customer.CompanyName)
.Skip(0)
.Take(21);
_customerDomainContext.Load<Customer>(
query,
lo =>
{
busyIndicator.IsBusy = false;
},
null);
To reduce the number of items passed over the wire and
skip the use of a DataPager we can create our own Custom
DataGrid where we will only show the 21 first Customers and
when the user moves the DataGrid’s Vertical Scroller we
load the next 21 Customers asynchronous and add them to the
DataGrid. The reason why we need to create our own custom
DataGrid is because we will not have access to the Vertical
Scroller from the DataGrid itself. The DataGrid will only
add a Scroller if some items will not fit into the height of
the DataGrid. If we only get 21 Customers and bind it to the
DataGrid and they all will fit into the DataGrid no scroller
will be visible, and the users will think they got all
Customers. What we need to do is to add a Scroller as if all
the Customers are already added to the DataGrid. To make
sure to make such as Scroller we need to get the actual
height of a DataGrid row and also get the total number of
Customers the DataGrid should normally display, and take the
height times the number of Customers to fake the Scroller.
By using the WCF RIA Services service operation we can do a
really fast call to get the total number of Customers, then
we can create a Query to load a the 21 first Customers. You
may wonder why 21 why not 20, the reason is that we will set
the height of the DataGrid so only 20 items will be listed,
but to get access to the Scroller and set a its range we
need to make it appear. So a simple hack to get access to
the Scroller is to just show 20 items but add 21 to the
DataGrid. The following code is the new
CustomerDomainService with a Service operation to get the
total number of Customers:
[EnableClientAccess()] public class CusotmerDomainService : LinqToEntitiesDomainService<NORTHWNDEntities> { public IQueryable<Customer> GetCustomerSet() { return this.ObjectContext.CustomerSet; } [Invoke] public int NumberOfCustomers() { return this.ObjectContext.CustomerSet.Count(); } }
The following code is the code-behind of the View that will
show the Customers:
public partial class MainPage : UserControl { CustomerDomainService _customerDomainContext = new CustomerDomainService(); int _currentPage = 0; const int NumberOfItemsToLoad = 20; public MainPage() { InitializeComponent(); this.Loaded += new RoutedEventHandler(MainPage_Loaded); } void MainPage_Loaded(object sender, RoutedEventArgs e) { customersDataGrid.ItemsSource = _customerDomainContext.Customers; LoadCustomers(); } private void LoadCustomers() { busyIndicator.IsBusy = true; EntityQuery<Customer> query = _customerDomainContext.GetCustomerSetQuery() .OrderBy(customer => customer.CompanyName) .Skip(_currentPage * NumberOfItemsToLoad) .Take(NumberOfItemsToLoad+1); _customerDomainContext.NumberOfCustomers( io => { customersDataGrid.TotalNumberOfRows = io.Value; _customerDomainContext.Load<Customer>( query, lo => { busyIndicator.IsBusy = false; }, null); }, null); } private void customersDataGrid_LoadNewItems(object sender, System.Windows.Controls.Primitives.ScrollEventArgs e) { _currentPage++; LoadCustomers(); } }
I will not focus so much on this code, I assume you
can figure out what it does. The NumberOfCustomers Service
operation will be executed before loading the Customers to
make sure the custom DataGrid’s TotalNumberOfRows property
is set before the Customers is added to the DataGird’s
ItemsSource. You can also see in the code that the Skip and
Take is used to create a Query, the _currentPage will be
increased by 1 when the custom DataGrid will trigger its
LoadNewItems event handler (The LoadNewItems event will be
trigged when the Scroller of the DataGrid reach a specific
value, in this case when 20 rows are displayed in the
DataGrid and the next 20 must be loaded and added to the
DataGrid). The LoadNewItems event handler will also make a
call to the LoadCustomers method to Load the next 21
Customers. The following code is the XAML of the View where
the custom DataGrid is used:
<UserControl xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data" xmlns:controlsToolkit="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Toolkit" xmlns:myData="clr-namespace:SilverlightApplicationSample" ...> <Grid x:Name="LayoutRoot"> <myData:AsyncDataGrid x:Name="customersDataGrid" Height="505" LoadNewItems="customersDataGrid_LoadNewItems" /> <controlsToolkit:BusyIndicator Grid.RowSpan="2" x:Name="busyIndicator"/> </Grid> </UserControl>
The following code is the custom DataGrid:
public class AsyncDataGrid : DataGrid { public event ScrollEventHandler LoadNewItems; private double _rowHeight = 0.0; private double _oldScrollValue = 0.0; private ScrollBar _verticalScrollBar = null; public AsyncDataGrid() { this.LoadingRow += AsyncDataGrid_LoadingRow; } public int TotalNumberOfRows { get; set; } private double GetNextLoadingPosition() { var numberOfItems = 0; foreach (var item in ItemsSource) numberOfItems++; return _rowHeight * numberOfItems; } public override void OnApplyTemplate() { base.OnApplyTemplate(); _verticalScrollBar = this.GetTemplateChild("VerticalScrollbar") as ScrollBar; if (_verticalScrollBar != null) _verticalScrollBar.Scroll +=scrollBar_Scroll; } private void AsyncDataGrid_LoadingRow(object sender, DataGridRowEventArgs e) { e.Row.SizeChanged += Row_SizeChanged; this.LoadingRow -= AsyncDataGrid_LoadingRow; } private void Row_SizeChanged(object sender, SizeChangedEventArgs e) { _rowHeight = e.NewSize.Height; ((DataGridRow)sender).SizeChanged -= Row_SizeChanged; if (_verticalScrollBar != null) { _verticalScrollBar.Maximum = TotalNumberOfRows * _rowHeight; _verticalScrollBar.UpdateLayout(); } }
private void scrollBar_Scroll(object sender, ScrollEventArgs e) { if (LoadNewItems != null && e.NewValue > _oldScrollValue) { _verticalScrollBar.Maximum = TotalNumberOfRows * _rowHeight; if (e.NewValue >= GetNextLoadingPosition()) { _oldScrollValue = e.NewValue; LoadNewItems(this, e); } } } }
By override the OnApplyTemplate method, we can get access to the DataGrid’s Default ControlTemplate’s VerticalScrollbar and hook up to the Scroller’s Scoll event. By using the LoadingRow event and then the SizeChanged on a DataGridRow we can get the height of the DataGridRow (I couldn’t figure out a simple way of doing it, because the LoadingRow event can’t be used to get the ActaulHeight of the row loaded, it’s to early and there is no Rows property to get the UIElement of the DataGrid’s Row, the only way to get a DataGridRow is by using the LoadingRow event). Within the SizeChanged event handler of the DataGridRow, the Maximum property of the Scroller is set to the TotalNumberOfRows times the height of the row, this will make sure the Scroller has a range that will make sure it looks like several of rows are added but they aren’t. Withing the Scoller’s Scroll event handler (scrollBar_Scroll) the LoadNewItems event will be trigged only if the Scroller reach a position when new Customers should be loaded.
Note: I haven’t take care of the key board events, if a user uses the keyboard to walk through all the Customers added and reach the 21 Customer, no Customers will be loaded, but it’s easy to add. I left it out in this example because the hard part is to figure out how to handle the Scroller.
This solution will solve the three problems mentioned
earlier in this post. It will reduce the number of data
passed to the client, it will also make sure the DataGrid
will show the data much faster and if the users don’t need
to see all Customers, they don’t need get all Customers
loaded and added to the DataGrid.
Summary
This post covered some solutions to minimize the data loaded form the server to the client, and also speed up the DataGrid. This was done by using the DataPager control, or the DomainDataSoure or to load Customers while the users moves a DataGrid’s Scrollbar.
If you want to know when I post a new blog post or just follow my me, you can follow me on twitter: http://www.twitter.com/fredrikn