Manish Dalal's blog

Exploring .net!

Stealth Paging : DataGrid

It is quite common to see paging of data in today’s web application. It allows developer to show only a fixed set of rows at a time and pulling next set of rows on demand, improving overall user experience.

In a desktop application, user normally does not have to deal with explicit paging. User experience is that of a continuous data stream, user just scrolls to see additional data.

It is certainly possible to replicate explicit paging functionality in Silverlight. However Silverlight is supposed to be a RIA platform, combining best features of web and desktop. So let see how we can combine these two concepts, efficient chunked data retrieval of the web, with continuous scroll experience of the desktop. Enter Stealth Paging.

Stealth Paging

Stealth paging refers to the concept of retrieving data in background just before it is needed. You start by loading DataGrid with only one or two pages worth of data, just enough to cover the screen and some more. Now when user starts scrolling, and nears the end of available rows to show, we fetch additional data and add it to the data collection. User continues scrolling, without ever realizing that data was fetched in the background.

We will start with a simple cumulative paging and then enhance it to implement the stealth mode.

Simple Cumulative Paging

In traditional web paging scenario, user is only shown one page of data at a time. Each new page replaces previous page, forcing user to page back to see the previous data. Instead of replacing one page of data with next page of data, we will just add to the existing data. This way user has access to all the current and previous data, and only needs to page to get next set of data.

Create a new Silverlight application project. Also create the corresponding web application to host and test the Silverlight application

Step 1 – PeopleService

We will use WCF service to provide data to client. First add reference to System.Runtime.Serialization. Next create a data contract, as a Person class that will be sent back to the client as shown:

using System.Runtime.Serialization;

namespace Silverlight.Web {
    [DataContract]
    public class Person {

        private string _firstName;
        [DataMember]
        public string FirstName {
            get { return _firstName; }
            set { _firstName = value; }
        }

        private string _lastName;
        [DataMember]
        public string LastName {
            get { return _lastName; }
            set { _lastName = value; }
        }

        private int _age;
        [DataMember]
        public int Age {
            get { return _age; }
            set { _age = value; }
        }

        private string _city;
        [DataMember]
        public string City {
            get { return _city; }
            set { _city = value; }
        }
    }
}

Note that Person class is marked with DataContract attribute and individual fields with DataMemeber attribute. Next add a simple service to provide data for the client. Add Silverlight enabled WCF service and call it PeopleSevice. Add method GetData to the code behind PeopleService.svc.cs as shown.

[OperationContract]
public List<Person> GetData(int startRow, int endRow) {
    List<Person> personList = new List<Person>();
    for (int i = startRow; i < endRow; i++) {
        personList.Add(new Person() {
            FirstName = string.Format("First Name {0}", i),
            LastName = string.Format("Last Name {0}", i),
            Age = i,
            City = string.Format("City {0}", i)
        });
    }
    //
    return personList;
}

Here we are just creating test data on demand. In an actual production application, you will be calling database and paging, perhaps using Linq Skip and Take functionality or Top and RowNumber features in SQL.

Step 2 - Client

Now add the service reference to the Silverlight client and call is PeopleServiceReference. (use Discover button to see PeopleService). Next add System.Windows.Control.Data reference to the Silverlight project.

Add following xaml to page.xaml.

<UserControl x:Class="Silverlight.Page"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
    xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data"
    >
    <Grid x:Name="LayoutRoot" Background="White">
        <Grid.RowDefinitions>
            <RowDefinition Height="*"/>
            <RowDefinition Height="Auto"/>
        </Grid.RowDefinitions>
        <data:DataGrid x:Name="peopleDataGrid" AutoGenerateColumns="True" Grid.Row="0" Margin="10"/>
        <Button x:Name="nextPageButton" Content="Next Page" Grid.Row="1" HorizontalAlignment="Right" Margin="2,2,20,2"/>
    </Grid>
</UserControl>

Here we are adding DataGrid and also button to fetch the next set of data. Note that unlike traditional web application, we do not have previous button as we are accumulating all the data.

Add following code to page.xaml.cs code behind

 
public partial class Page : UserControl {
    
    int _startRow;
    int _pageSize = 20;
    ObservableCollection<Person> _people;

    public Page() {
        InitializeComponent();
        this.Loaded += new RoutedEventHandler(Page_Loaded);
        this.nextPageButton.Click += new RoutedEventHandler(nextPageButton_Click);
    }

    void Page_Loaded(object sender, RoutedEventArgs e) {
        _startRow = 0;
        _people = new ObservableCollection<Person>();
        peopleDataGrid.ItemsSource = _people;
        nextPageButton.IsEnabled = false;
        GetData(_startRow, _pageSize);
    }

    void nextPageButton_Click(object sender, RoutedEventArgs e) {
        GetData(_startRow, _startRow + _pageSize);
    }
    
    private void GetData(int startRow, int endRow) {
        PeopleServiceClient proxy = new PeopleServiceClient();
        proxy.GetDataCompleted += new EventHandler<GetDataCompletedEventArgs>(proxy_GetDataCompleted);
        nextPageButton.IsEnabled = false;
        this.Cursor = Cursors.Wait;
        proxy.GetDataAsync(startRow, endRow);
    }

    void proxy_GetDataCompleted(object sender, GetDataCompletedEventArgs e) {
        this.Cursor = Cursors.Arrow;
        if (null != e.Error) {
            MessageBox.Show(e.Error.Message, "Error Getting Data", MessageBoxButton.OK);
            return;
        }
        _startRow += _pageSize;
        foreach (Person person in e.Result) {
            _people.Add(person);
        }
        nextPageButton.IsEnabled = true;
    }
}

Here we are loading 20 rows at startup. When user clicks on next page button, we call backend service and get additional 20 rows of data.

image

F5 and test the application.

image

Stealth Paging

Now let make the paging automatic. Instead of asking user to click on the next button to fetch next set of data, we will get next set of data automatically when required, providing perception of one continuous scrollable page, similar to what user is already accustomed in a desktop application.

Comment out nextPageButton in xaml and code behind. Add an event handler for DataGrid.LoadingRow event. Also create class level variable _loading to track when data is being loaded.

 
public Page() {
    InitializeComponent();
    this.Loaded += new RoutedEventHandler(Page_Loaded);
    //this.nextPageButton.Click += new RoutedEventHandler(nextPageButton_Click);
    this.peopleDataGrid.LoadingRow += new EventHandler<DataGridRowEventArgs>(peopleDataGrid_LoadingRow);
}

void peopleDataGrid_LoadingRow(object sender, DataGridRowEventArgs e) {
    if (_loading || _people.Count < _pageSize) {
        return;
    }
    if (_people.Count - 5 < e.Row.GetIndex()) {
        GetData(_startRow, _startRow + _pageSize);
    }
}

What enables stealth paging is the detection of need for more data. It would be great if there was some way to let DataGrid know of total number of rows and event that will trigger when DataGrid needs next set of data, but current implementation does not have that feature. But we can engineer our own functionality using LoadingRow event.

image

LoadingRow event is fired just before the DataGrid is about to display the row. This event enable us to be stealthy by providing us with the crucial row number (using e.Row.GetIndex()). We are constantly looking for the row being displayed. When the last from the 5th row is about to be display, we know that we are near the end (only 4 rows are left to be shown) and make call to backend service to get additional data. Newly fetched data is added to the existing data collection, forcing DataGrid to add new rows and in turn increasing the scrollable area. As user continues scrolling down and every time we are near the end of available rows to show, we fetch additional data.

If you run the application, and start scrolling down, you will see that size of scroll thumb reduces progressively as we add more data.

image image

One caveat is a little jerkiness in the scrollbar position and obvious change to height of scroll thumb. Other caveat is that we have removed random access to data that traditional explicit paging provides.

Hopefully DataGrid will evolve to provide features to enable stealth paging in future, but till then you can use LoadingRow technique to provide user with a stealth paging experience.

Source Code: StealthPaging.zip

Technorati Tags:

Comments

Dave said:

So as I'm thinking the ultimate is as I scroll down, you throw the records away the records that  are no longer in view?  Otherwise memory would be an issue - right?

# October 9, 2008 3:43 PM

manish.dalal said:

Dave,

You are correct in assuming that memory would be an issue as we do not throw away records. If the data set that you are displaying is very large, it is best to stick with traditional explicit paging. (though you can use the same technique also when user scroll up to fetch previous page)

Currently I am using this technique with a relatively large data set (35+ columns) and with 300+ rows and at least in IE7 and FireFox3 it works without issues. (avg. memory footprint: 87 mb)

# October 9, 2008 4:18 PM

unruledboy said:

@Bart Czernicki

could you write an article showing how your grid performs?

# October 12, 2008 11:49 PM

geordie jenner said:

this appears to have to create a new query for each server hit.  is there a way to have a recordset on the server-side, and with a cursor, FETCH records as needed.  that would eliminate new queries

# March 9, 2009 9:28 PM

manish.dalal said:

geordie jenner,

Reason for paging is to reduce initial / incremental load time and hence new query is desired. This prevents getting lots of data from server when user only needs couple of pages! You can certainly cache data on demand on server if it is relatively static.

# March 10, 2009 10:46 AM

dsoltesz said:

Very cool post, thanks

# March 13, 2009 5:53 PM

sat said:

Nice example. Do you think it will be better to use RIA so we get all the data and do the size manipulation in the domain object.

Thanks

# November 19, 2009 3:01 PM

Sabarinathan Arthanari said:

Thanks. Good idea!

# January 18, 2010 3:13 AM