We are passing some fairly large datsets around using remoting with IIS as the host. We have seen a lot of performance issues with doing this. My research has revealed that a DataSet hard-codes the serialization as a diffgram regardless of the characteristics of the underlying stream. I have heard that MS recognizes the design issue with this and is working on doing something.
In the meantime, what are other folks doing to reduce the size of large datasets? Any kind of compression that works?
I stumbled on
this today. Rather dated, but none the less worth looking over.
In follow-up, I think it is important to remember this is a public forum and all of us on here are intelligent, well-spoken folks who wish to share their experiences, else we would not be blogging. I do not see where any good can come from a blog that suggests one is a lesser because of a language choice, or for that matter, for anything we do in our development lives.
Share your excitement and experiences. Comments like "but they don't even care, or at least don't make an effort", "Maybe they're just used to the magic hand-waving of Visual Basic", " I certainly wouldn't be heartbroken if many in the latter crowd stopped programming" are negative, irrogant, and uncalled for. The point was well taken, you must know and understand OOP to big a great .net developer. You must also know how to turn your computer on, but I am not going to comment on those who do not know how to. If it is in their heart to be a great programmer, than they will, in their own regard.
I have always looked at it like this, there will always be someone better, smarter, wealthier, faster, better looking (well, maybe not :-) ), etc than me. This just makes me work harder. Competition is good, it makes us grow. Lets use it in a positive tone.
I was saddened to hear the news
I grew up with Mr. Rogers and Captain Kangaroo! They will be missed and I am sorry today's kids do not have shows like this to watch.
Why do I continue to read blogs from C, C++ and C# snobs that have to degrade and knock down VB developers? http://dotnetweblogs.com/AKrowczyk/posts/post.aspx?ID=3126
Why do these discussions ALWAYS start from a C, C++ and C# developer's mouth? Do they not have anything better to think about other than how inferiror VB and VB.NET developers are? Why not write some good code with all that spare thinking time! God this angers me.
I feel lucky and fortunate enough to have an incredible job that allows me to do what I love, develop applications. I have no time and no concern with anything other than that. I chose VB.NET as this is what my 1st three clients wanted. In fact, all of my clients are using VB.NET. Are these systems inferior to an application written in C#? Doubtful, and, they do not care. They need business problems solved, not a language war. Get over it an move on C# snobs!
I drive a car to the office. It gets me from point A to point B. Your car is no better than mine in that respect. If I chose to take a taxi, I still get to work. If I chose to bike it, I still get to work. heck, if I chose to walk, I still get to work. I still get to do what I truly love, code and develop.
I banged my head on this for 2 hours yesterday. I am passing an array of Structures over http to a remoted object using the Binary Formatter. Well, seems there is a
bug with doing this.
I have been struggling with passing SqlParameters to remoted data access object. Why? MarshalByRefObject and server callbacks...what a mess. So, I finally decided to build a value type that mirrors a SqlParameter and pass this to the remoted object. The problem was, how do I get Output params back to the client? With a SqlParameter type it is easy, since the AppDomain that creates the SqlParameter passes the SqlParameter as a ref type.
The solution? Well, in my case I have a local object whose sole purpose in life is to determine if there is a LAN or WAN connection available. If LAN, the data access code runnings locally. If WAN, we need to connect via HTTP to a remoted object on IIS.
The client exe builds the following structure:
<Serializable()> _
Public Class DataPortalGeneral
<Serializable()> _
Public Enum Direction
Input
InputOutput
Output
ReturnValue
End Enum
<Serializable()> _
Structure ParmData
Dim ParmName As String
Dim ParmValue As String
Dim ParmType As SqlDbType
Dim ParmSize As Integer
Dim ParmDirection As String
Sub New(ByVal Name As String, _
ByVal Value As String, _
ByVal Type As SqlDbType, _
ByVal Size As Integer, _
ByVal Direction As Direction)
Me.ParmName = Name
Me.ParmValue = Value
Me.ParmType = Type
Me.ParmSize = Size
Me.ParmDirection = Direction
End Sub
End Structure
End Class
Like this:
Dim ParmGroup(3) As ParmData
ParmGroup(0) = New ParmData("@ProductID", "1", SqlDbType.Int, 4, Direction.Input)
ParmGroup(1) = New ParmData("@ProductName", "", SqlDbType.VarChar, 40, Direction.Output)
ParmGroup(2) = New ParmData("@UnitPrice", "", SqlDbType.Money, 8, Direction.Output)
ParmGroup(3) = New ParmData("@QtyPerUnit", "", SqlDbType.VarChar, 20, Direction.Output)
The client exe passes this structure to a local client object. This local client object check for LAN or WAN, then passes this struct to the remoted server object (if WAN). The remoted server object uses this struct to build a SqlParameter array like this:
Private Shared Function BuildSqlParamArray(ByVal ParmData() As ParmData) As SqlParameter()
Dim arParms() As SqlParameter = New SqlParameter(ParmData.Length - 1) {}
Dim parm As ParmData
Dim i As Integer = 0
Dim myParam As SqlParameter
For Each parm In ParmData
myParam = New SqlParameter(parm.ParmName, parm.ParmType, parm.ParmSize)
Select Case parm.ParmDirection
Case Direction.Input
myParam.Direction = ParameterDirection.Input
Case Direction.InputOutput
myParam.Direction = ParameterDirection.InputOutput
Case Direction.Output
myParam.Direction = ParameterDirection.Output
Case Direction.ReturnValue
myParam.Direction = ParameterDirection.ReturnValue
End Select
myParam.Value = parm.ParmValue
arParms(i) = myParam
i += 1
Next
Return arParms
End Function
Now, I have a sqlParameter array I can pass to my generic Data Access Class.
The Generic Data Access component can build a Command object and add the SqlParameters and Execute.
The next issue is how to get the Output params stored in the SqlParameter back to the client after the Command Execute. Well, back in the calling remoted server object we still have our Reference Type SqlParameter array. So, I now take the SqlParameter array and build a ParmData structure array like this:
Private Shared Function BuildParmData(ByVal sqlParameter() As SqlParameter) As ParmData()
Dim ParmsDataArray(sqlParameter.Length - 1) As ParmData
Dim parm As ParmData
Dim i As Integer = 0
Dim myParam As sqlParameter
For Each myParam In sqlParameter
parm = New ParmData(myParam.ParameterName, myParam.Value, myParam.DbType, myParam.Size, myParam.Direction)
ParmsDataArray(i) = parm
i += 1
Next
Return ParmsDataArray
End Function
The remoted server object can now pass a Value Type, ParmData, back to the calling client. In the client, instead of doing something like this:
Textbox1.Text = mySqlParms(1).Value.Tostring
I do this:
Textbox1.Text = ParmGroup(1).ParmValue.ToString
So far, so good. I would love for someone to break this or offer critiques as we are still in testing.