.NETTER Characters...

Software Designers often have to face a common decision factor, that whether they need to design automated test (unit test, integration test etc) infrastructure for data access layer code, specially when the data access layer codes are written using code generator tool. Basically the straight forward answer as ‘Yes’ or ‘No’ in this regard depends of several situations/ factors such the size and budget of the project etc. Here are my 5 top reasons to write automated test (unit test, integration test etc) for generated data access layer code.

1. When the code generation template/logic itself contains bug

While using the code generators, it’s possible that the underlying code/logic of the code generators may contain bug! Having the automated test for the generated data access layer code greatly helps of indentify the ‘generated’ bugs!

2. When the code is not re-generated after to change of underlying database object

Well I swear my generated code engine is perfect (i.e. no bug on my generator engine/template logic)! Cool. I have generated the code perfectly. However I have changed the underlying database, but really forgot to re-generate the code using the code generator! Well that could be possible and can be detected if we have corresponding automated tests!

3. When the re-generated code is not perfect for the new changes of underlying database object

Well I swear my code has been re-generated after I have changed by database. Cool. But the initial version of generated code may work perfectly with the initial version of database objects. However, new bugs can be found/introduced in the generated code for the new changes of underlying database objects, which results updating the code generation logic. So, still you need automated tests for your data access layer code!

4. When custom code is added in the generated code

Sometimes custom code needs to be placed on the generated code. In those cases automated tests really helps a lot of indentify the bugs possibly placed in custom code in data access layer and/or in database stored procedure, function etc.

5. To check the perfect integration with database objects

Even the codes are generated/written for data access layer perfectly, but it’s still possible of failing the code running properly while the data access layer code and database objects are integrated. For instance using wrong connection string that points to wrong version of database etc. Having a well designed automated testing infrastructure really helps us in this regard!

Microsoft Visual Studio Team System/Test Edition provides an excellent tool to perform web site load testing. Using this load testing tool, you can monitor and measure the site performance along with system status with respect to a given load/stress.

Fortunately VSTS provides a support for wide range of performance counters, from web page request per second to condition of physical disk, memories. Unfortunately, they are too huge that, initially testers/designers get overwhelmed with all of those, to find out a clear idea about the performance of the site they built.

The number of counter parameters to be considered by the load tester/designers is greatly varies based on the type and size of the web application to be tested. Here is my favorite top 10 performance counters that I use on my each load tests, regardless of project size. These counters are based two primary categories: Web Site end and Hardware end.

Web Site Related Performance Counters

Web site related performance counters are the counters that provide valuable information about the health of web site that is under test. These parameters are categorized as Requests, Pages, Tests and Errors.

1. Request - Avg Req/Sec

Desired value range: High

This is the average number of requests per second, which includes failed and passed requests, but not cached requests, because they are not issued on web server. Please note that, all http requests, such as image, java-script, aspx, html files generates separate/individual/single request .

2. Request -    Avg Req Passed/Sec

Desired value range: High

While “Request - Avg Req/Sec” provides an average with respect to all passed and failed request, “Request -    Avg Req Passed/Sec” provided the average of passed requests. This info also helps to determine the average number of failed requests/sec.

3. Page -        Avg Page Time (Sec)

Desired value range: Low

While a single request refers to request to a single http elements (such as css, java-script files, images, aspx, html etc), a page is the container of all of the corresponding requests generated when a web page is requested (for instance via the browser address bar). “Page -        Avg Page Time (Sec)” counter refers to the average of total time taken to load a page with all of its http elements.

4. Test -        Total Test

Desired value range: High

For instance, we have created a web test, that contains two web pages, pushing on a button on the first page will re-direct the user to the second page, although there will be multiple entries will be involved for Requests and Pages counters, but the whole process will be considered as a single Test.

This counter considers the total number of tests (which includes passed and failed tests) during the test period.

5. Scenario -        User Load

Desired value range: High

This counter considers the maximum user load that has been provided during the test run. Please note that, for Step Load pattern, where more user volume is added on step by step basis, the maximum user load will be counted through this counter parameter.

6. Errors -    Errors/Sec

Desired value range: Low

Includes average number of errors occurred per second, which includes all types of errors.

Hardware Related Performance Counters

7. Processor -    % Processor Time

Desired value range: Low

This is the number of processor time being utilized in percentage.

8. Memory -    Available MBytes

Desired value range: High

This the amount of Memory available in Mega byte.

9. Physical Disk -        Current Disk Queue Length

Desired value range: Low

It shows how many read or write requests are waiting to execute to the disk. For a single disk, it should idle at 2-3 or lower.

10. Network Interface -    Output Queue Length

Desired value range: Low

This is the number of packets in queue waiting to be sent. A bottleneck needs to be resolved if there is a sustained average of more than two packets in a queue.

Download the SQL Script which selects all of the parameters as mentioned above with respect to the latest load test, from here:

Microsoft Visual Studio Team System/Test Edition provides an excellent tool to perform web site load testing. Using this load testing tool, you can monitor and measure the site performance along with system status with respect to a given load/stress.

Fortunately VSTS provides a support for wide range of performance counters, from web page request per second to condition of physical disk, memories. Unfortunately, they are too huge that, initially testers/designers get overwhelmed with all of those, to find out a clear idea about the performance of the site they built. Today we’ll be discussing about three counter sets that are closely related to each other, but provides a meaningful information about the health of the target web site to be load tested.

Definitions

Requests: Requests are the smallest load testing parameter with respect to a web page.

It contains details for individual requests issued during a load test. This includes all HTTP requests, and dependent requests such as images, css and java-script files.

Meaning, each refereeing contents in a web page, such as images, css files, java-script files will generate separate request along with the request for the web page which includes the actual textual content (text, tags etc). It will include the same number of additional entries for a postback. 

Pages: Right after “Requests” we can consider “Pages” as the next level of load testing counter set, which is defined in MSDN as “Displays a list of pages accessed during a load test run. Some data in this table is available only after a load test has completed.”

For instance, we have a web page, which has a button, clicking on which shows a message on that page, which generate two entries for Page counter.

One important note is, any redirection to a separate page will not  be counted with additional value, however the corresponding Requests for the redirected page will be counted. 

Tests: Contains the details for individual tests run during a load test.

For instance, we have created a web test, that contains two web pages, pushing on a button on the first page will re-direct the user to the second page, although there will be multiple entries will be involved for Requests and Pages counters, but the whole process will be considered as a single Test.

Case Study

To have better idea, lets consider 4 different cases to see the corresponding entries with respect to the counters we are discussing.

Text only:

We have a web page, that contains only textual content, no external content such as image, css, java-script file referred there. However it may contain html controls, but no postback (button click etc) will be considered here.

Counter Status: number of request = number of pages = number of tests

For a single hit by a single user:

- 1 entry in “request” counter will be added
- 1 entry in “page” counter will be added
- 1 entry in “test” counter will be added

Text and Image:

We have a web page, which contains an image, besides the textual contents.

Counter Status: number of pages = number of test

For a single hit by a single user:

- 2 entries in “request” counter will be added
- 1 entry in “page” counter will be added
- 1 entry in “test” counter will be added

Text with Postback:

We have a web page, which has a button control only, besides the textual contents. Clicking on the button will how a message in the page “Hello World” with a postback.

Counter Status: number of pages = 2 * number of test

For a single hit by a single user:

- 2 entries in “request” counter will be added
- 2 entries in “page” counter will be added
- 1 entry in “test” counter will be added

Text and Image with Postback:

We have a web page, which has a button and a image, besides the textual contents. Clicking on the button will how a message in the page “Hello World” with a postback.

Counter Status: number of pages = 2 * number of test

For a single hit by a single user:

- 4 entries in “request” counter will be added
- 2 entries in “page” counter will be added
- 1 entry in “test” counter will be added

From Text only page to another text only page using postback:

We have  a page, which has a button, besides the textual contents. Clicking on the button will to a another “Text only” page.

Counter Status: "number of failed request = number of failed test" (i.e. if one of the request in a test fails, the entire test would be considered as failed test)

For a single hit by a single user:

- 3 entries in “request” counter will be added
- 2 entries in “page” counter will be added
- 1 entry in “test” counter will be added

Microsoft Visual Studio Team System 2008 provides an excellent data synchronization tool, synchronize data and schema between two database. It saves lots of developer time to sync database objects. Among two possible synchronization techniques, VSTS 2008 uses the unidirectional synchronization technique. In this consideration, as safety measurement will be helpful for developers before start synchronization.

The security measurement is considered whether it would create any loss of data or not. Here few useful cases with respect to database table schema synchronization has been discussed one by one.

• New Table in Source Database: Safe. The new table will be added to the destination database.
• Remove Table in Source Database: Safe. The removed tabled in source database will also be removed from destination database.
• Modify Table in Source Database, Add New Field: Safe. However the new field needs to have the ‘allow null’ property as true.
• Modify Table in Source Database, Remove Old Field: Safe. The old column as well as the corresponding data in the destination database will be removed. However, as the old column has been removed, so the removal of corresponding data is expected.
• Modify Table in Source Database, Modify Old Field: Not safe. Data Loose in VSTS, as it does alter table add + drop field. To retain existing data due to sync process, external script would be useful.  Below is some sample sql statement in this regard:

-- sql script to change data type of a table field, without data loss

alter table Contact

      alter column Comment nvarchar(10) null

-- sql script to rename a table field 'Address' of a table 'Contact', to 'FullAddress' -- without data loss

EXEC sp_rename 'Contact.Address', 'FullAddress', 'COLUMN';

Data or content synchronization is one of the classic problems in software world. It becomes very trivial point while working on software production, where production data and schema needs to be synchronized with live data and schema. Having some basic conceptual complexity, developers often get afraid to use any automated tool considering the risk factors to lose of data or content. Mostly, in these cases a manual process has to involve making sure a safe content synchronization. However as human is also error prone, there still exists risk factor to lose of content, but also includes a huge human time and effort on it. Having a clear specific idea on content synchronization will greatly help to reduce such overheads. Although, the synchronization concept exists in disk space, network, database etc sectors, today we’ll basically focus on database synchronization concept, which will also help to understand synchronization concept from a generic point of view.

What is synchronization?

So, what is synchronization? This is a process that ensures the same content among two participating entities, having different set of content possibly.

For instance, a database table, named Employee, which has two instance on two different databases, and exact same schema definition, after a synchronization process, both table will contact the identical number of data rows and column values.

In a synchronization process, there involves two participants, generally, termed as source and destination, where the content will be placed from source entity to destination entity.

Based on requirement and characteristics of data, the synchronization process can be categorized in two ways

1. Unidirectional synchronization: replacing destination entity with the source entity
   In a unidirectional synchronization, all of the contents from source entity will be placed to destination entity, which also implies, any content in the destination entity, that doesn’t exist in source entity, will be deleted.
2. Bidirectional synchronization: merging data from both participating entities

Before understanding both synchronization processes clearly, let’s consider three sample states of data entities:

a) Initial state: where both source and destination entity contains exact same number of records and column values.

b) Data change state: the state where data get changed in both source and destination entity.

c) The synchronized state: where data has been synchronized among source and destination state.

In a unidirectional synchronization, all of the contents from source entity will be placed to destination entity, which also implies, any content in the destination entity, that doesn’t exist in source entity, will be deleted.

There is a high degree of data lost risk factors in unidirectional synchronization, as all of the data contents will be deleted in the destination entity, which don’t exist in source entity. In the above sample, #2 and #5 row item has been deleted due to data synchronization process. So, database administrators need to be cautious to confirm that if this data lose is expected.   
 
In bidirectional synchronization, all of the rows and column values in source and destination entity will be merged data from both participating entities.

Thus, on bidirectional synchronization, no data will be deleted neither in source or destination entity during the synchronization process. However the only data lose risk factors in bidirectional synchronization can be considered when same data row (identified by primary key) that has been modified in source entity, get replaced in the destination entity.
In the above sample, #1 row has been updated in destination entity, from ‘Ashraf’ to ‘Ashraful’. So, database administrators need to be cautious to confirm that if this data replacement is expected.