I had hosted a webchat on serialization in .Net today, will be posting the link to the transcript as soon as it is up.
Bascially the areas I discussed were, Runtime serialization and Xml Serialization. Versioning issues, common problems with serialization and what's new in .Net 2.0
Here are the notes I used for the webchat, pretty unorganized. Hope to put all this into an article soon.
Serialization in .Net
What is serialization?
Serialization is the process of converting an object or a graph of objects into a linear sequence of bytes for either storage or transmission to another location. Deserialization is the process of taking in stored information and recreating objects from it.
What can it be used for?
Preserve the state of your application to persistent storage.
Distributed communication (Remoting and web services use serialization)
Deep copies of objects
Storing config information
Site settings using XmlSerializer
http://codebetter.com/blogs/david.hayden/archive/2005/03/01/56226.aspx
XmlSerializerSectionHandler
http://pluralsight.com/wiki/default.aspx/Craig/XmlSerializerSectionHandler.html
In .Net the support for serialization is in two namespaces
System.Runtime.Serialization – The classes in this namespace facilitate “runtime serialization”. The goal here is to preserve as much type information as possible and then recreate the object with the type information on Deserialization. Requires that the objects have the Serializable attribute. And the Serialization Formatter permission is required which is only granted to local apps by default.
System.Xml.Serialization – The classes in this namespace facilitate XML serialization. The goal here is to provide a high performance mechanism to convert objects into XML. Only public fields and properties of the object are preserved. Used if communication between heterogeneous systems is required. Requires no special security permissions. Has a lot of restrictions as to what can be serialized and what cannot.
In runtime serialization
Two aspects of serialization
What needs to be serialized
How is the serialized data formatted
In .net we can do serialization using the Serializable attribute or by implementing ISerializable interface.
In case we choose Serializable attribute the formatters use reflection to get the data that needs to be serialized. Binary and SoapFormatters will serialize every field in the object including private and protected ones. It does not serialize fields marked with the NonSerialized attribute.
If the object implements ISerializable it gets complete control over the serialization and deserialization process.
ISerializable has only one method GetObjectData(), for Deserialization a special ctor is required. Which should be made protected and private if the class is sealed.
BinaryFormatter
Writes the data to be serialized in binary format. Most efficient formatter but not useful for open systems.
SoapFormatter
Writes the data to be serialized in the SOAP 1.1 format. Less efficient but a must if interoperability is a requirement. Used by remoting.
StreamingContext
An enumeration that can be used to decide what to serialize based on the context in which serialization is taking place. For example if the context is cross process then system wide handles like files etc. can be saved as is. The context can be specified in the constructor of the formatter. Contains information about the destination of the serialized message. It is passed to objects that handle their own serialization through ISerializable.
Versioning
SerializationBinder
For Assembly and Type resolution. Allows you to handle things like change in class name, change in namespace etc.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemruntimeserializationserializationbinderclasstopic.asp
SerializationSurrogate
Allows another class to handle the serialization of an object. Allows you to handle the case where there is an object that does not support serialization and cannot be inherited or wrapped. The problem here is that the surrogate receives a uninitialized object i.e. no ctors have run. Also can only access public fields and properties, has to resort to reflection to access private, protected members.
An implementation of serialization surrogate that uses reflection
http://weblogs.asp.net/rosherove/archive/2005/03/03/384267.aspx
Implementation of surrogates for binary serialization of datasets
http://msdn.microsoft.com/msdnmag/issues/04/10/CuttingEdge/
http://www.topxml.com/xmlserializer/surrogateselectors.asp
IDeserializationCallback
Allows objects to do post serialization initialization.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemruntimeserializationideserializationcallbackclasstopic.asp
XmlSerializer
Allows objects to be stored in XML format. Webservices internally use XmlSerializer to convert objects into SOAP format. Focuses on mapping .Net classes to arbitrary XML formats or formats defined via a schema. Optimized for performance. Takes the type to serialize in the ctor and uses the type information to give high performance while serializing and deserializing. Works only with public field and properties(provided they have both get and set properties), this is to facilitate working in restricted conditions like the intranet and internet.
Restrictions on types that can be serialized using XmlSerializer
http://www.topxml.com/xmlserializer/serializable_classes.asp
Controlling Xml serialization via attributes
http://www.topxml.com/xmlserializer/serialization_attributes.asp
Attrbutes that control Xml Serialization
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/html/cpconattributesthatcontrolserialization.asp
Controlling Xml serialization using attributes
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpguide/html/cpconcontrollingserializationbyxmlserializerwithattributes.asp
When type is passed as ctor, it creates a type mapping table which it uses for serialization.
XmlIncludeAttribute is applied on the base class to specify all derived types. Or we can use XmlElementAttribute on a field to specify the types of objects that we expect by setting the DataType property.
For Arrays we can either use the XmlElement attribute in which case the root node for arrays is not used. Or we can use XmlArray and XmlArrayItem to control how array and array elements are serialized.
Any class that implements IEnumerable or ICollection can be serialized and serialization can be controlled using XmlArray and XmlArrayItem attributes. When XmlSerializer detects a collection it does not serialize the public properties of the class.
Xsd.exe can be used to generate .Net classes from an xsd schema
Advanced Xml serialization
This can be used if we cannot attach attributes to the class, or it is not known beforehand.
Global types – can be specified in the ctor by passing an array of additional types.
XmlAttributeOverrides – again is specified via the ctor. XmlAttributeOverrides has two overloads of Add to apply attributes to type or to an element within a type. XmlAttributes class needs to be passed to Add() this contains strongly typed properties for the various attributes that can be applied.
Choice elements
We can use an enumeration with XmlChoiceIdentifier to handle these situations.
Namespaces
XmlSerializerNameSpaces class, and XmlNamespaceDecalaration or can be passed in the ctor of the XmlSerializer.
XmlSerializerNameSpaces used to map prefixes to namespaces.
Soap encoding using XmlSerializer
Using SoapReflectionImporter to generate TypeMapping which is passed to ctor of XmlSerialzier
Can be customized using SoapAttributeOverrides which takes a SoapAttributes corresponding to a type or to a member in the type.
Handling versioning problems with XmlSerializer, provides events when it encounters unknown element, unknown node etc. Alternatively you can apply the XmlAnyElement or XmlAnyAttribute attribute to an array of type XmlElement and that will contain all unresolved elements and attributes.
Troubleshooting common problems with XmlSerialzier
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnxmlnet/html/trblshtxsd.asp
*XmlPreSerializer, XmlReflectionImporter, SoapReflectionImporter
Serialization and remoting
Remoting uses serialization to transport marshal-by-value objects.
Serialization and datasets
DataSets convert to XML by implementing the IXmlSerializable interface
Common problems in serialization
Delegates and serialization, solved by using the field:NonSerialized option. Versioning. Objects that cannot be serialized via XmlSerializer. Permission problems.
What’s new in serialization in .Net 2.0
http://msdn.microsoft.com/msdnmag/issues/04/10/AdvancedSerialization/default.aspx
OptionalFieldAttribute
OnDeserializing/OnDeserialized OnSerializing/OnSerialized ,
these events are only defined for BinaryFormatter. No new stuff has been added for the SoapFormatter.
XmlFormatter class is now supporting all these Deserialization events. It also supports two modes SharedContract and SharedType.
More info at
http://weblogs.asp.net/cschittko/archive/2003/10/29/34414.aspx
http://www.douglasp.com/CommentView.aspx?guid=bff35b26-4739-4971-9a05-507e5aaddd7a
SoapFormatter has been deprecated in .Net 2.0 no new features have been added. XmlFormatter is the successor to that
IXmlSerializable is now documented and can be used in applications to get control over xml serialization.
GetSchema(), ReadXml() and WriteXml() are the methods. XmlSchemaProvider attribute
Sample code illustrating use of IXmlSerializable
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnvs05/html/wsnetfx2.asp
How does that address some of the problems present in the current implementation?
OptionalField allow us to specify a field as optional and a version in which it was added. Thus when a new version is deserialized the formatter does not thrown an exception if the type does not contain that field. The serialization events give more control.
Example of OptionalField
http://fredrik.nsquared2.com/viewpost.aspx?PostID=174
Links:
How to serialize a hashtable
http://codebetter.com/blogs/geoff.appleby/archive/2004/11/13/32005.aspx
Handling non serializable events in .Net 2.0 using VB.Net
http://www.lhotka.net/WeBlog/PermaLink.aspx?guid=8f8ae33f-f3b3-4864-a146-4f852d78783e
And here are the code snippets I used during the chat
/* Creating deep copies using serialization
*
[Serializable]
public class Customer
{
public string Name;
public Address Address = new Address();
public Customer(string name)
{
this.Name = name;
}
public Customer DeepCopy()
{
MemoryStream stream = new MemoryStream();
BinaryFormatter formatter = new BinaryFormatter();
formatter.Serialize(stream, this);
stream.Seek(0, SeekOrigin.Begin);
Customer clone = (Customer)formatter.Deserialize(stream);
return clone;
}
public Customer ShallowCopy()
{
return (Customer)this.MemberwiseClone();
}
}
[Serializable]
public class Address
{
public string Street;
public string Pin;
public string City;
}
[STAThread]
static void Main(string[] args)
{
Customer obj1 = new Customer("Henry");
obj1.Address.City = "Bangalore";
obj1.Address.Street = "Banergatta";
obj1.Address.Pin = "560076";
Customer obj2 = (Customer)obj1.ShallowCopy();
Customer obj3 = (Customer)obj1.DeepCopy();
Debug.Assert(Object.ReferenceEquals(obj1.Name, obj2.Name));
Debug.Assert(Object.ReferenceEquals(obj1.Address, obj2.Address));
Debug.Assert(! Object.ReferenceEquals(obj1.Name, obj3.Name));
Debug.Assert(! Object.ReferenceEquals(obj1.Address, obj3.Address));
System.Console.ReadLine();
}
*/
/* Storing user settings
*
public class UserSettings
{
private Point _mainFormPosition = Point.Empty;
public Point MainFormPosition
{
get{ return _mainFormPosition; }
set{ _mainFormPosition = value; }
}
private Size _mainFormSize = Size.Empty;
public Size MainFormSize
{
get{ return _mainFormSize; }
set{ _mainFormSize = value; }
}
private float _conversionFactor = float.MinValue;
public float ConversionFactor
{
get{ return _conversionFactor; }
set{ _conversionFactor = value; }
}
}
[STAThread]
static void Main(string[] args)
{
UserSettings settings = new UserSettings();
settings.ConversionFactor = 10;
settings.MainFormPosition = new Point(100,100);
settings.MainFormSize = new Size(640, 480);
XmlSerializer serializer = new XmlSerializer(typeof(UserSettings));
XmlTextWriter writer = new XmlTextWriter("settings.xml", System.Text.Encoding.UTF8);
writer.Formatting = Formatting.Indented;
serializer.Serialize(writer, settings);
writer.Close();
System.Console.ReadLine();
}
/* Output
<?xml version="1.0" encoding="utf-8"?>
<UserSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<MainFormPosition>
<X>100</X>
<Y>100</Y>
</MainFormPosition>
<MainFormSize>
<Width>640</Width>
<Height>480</Height>
</MainFormSize>
<ConversionFactor>10</ConversionFactor>
</UserSettings>
*/
/* Simple runtime serialization with SoapFormatter
[Serializable]
public class Customer
{
public string Name;
public ArrayList Orders = new ArrayList();
[NonSerialized]
public string DataThatShouldNotBeSerialized;
public Customer(string name)
{
this.Name = name;
}
}
[Serializable]
public class Order
{
public string Product;
public int Quantity;
public Customer Customer;
public Order(string product, int quantity, Customer customer)
{
this.Product = product;
this.Quantity = quantity;
this.Customer = customer;
}
}
Customer customer = new Customer("James");
customer.DataThatShouldNotBeSerialized = "Random data";
Order order1 = new Order("Motherboard", 2, customer);
Order order2 = new Order("CPU", 2, customer);
customer.Orders.Add(order1);
customer.Orders.Add(order2);
SoapFormatter formatter = new SoapFormatter();
FileStream stream = new FileStream("customer.xml", FileMode.Create);
formatter.Serialize(stream, customer);
stream.Close();
*/
/* Custom runtime serialization with SoapFormatter
[Serializable]
public class Customer : ISerializable
{
public string Name;
public ArrayList Orders = new ArrayList();
public string DataThatShouldNotBeSerialized;
public Customer(string name)
{
this.Name = name;
}
#region ISerializable Members
public void GetObjectData(SerializationInfo info, StreamingContext context)
{
info.AddValue("Name", Name);
info.AddValue("Orders", Orders);
}
private Customer(SerializationInfo info, StreamingContext context)
{
Name = info.GetString("Name");
Orders = (ArrayList)info.GetValue("Orders", typeof(ArrayList));
}
#endregion
}
[Serializable]
public class Order
{
public string Product;
public int Quantity;
public Customer Customer;
public Order(string product, int quantity, Customer customer)
{
this.Product = product;
this.Quantity = quantity;
this.Customer = customer;
}
}
Customer customer = new Customer("James");
customer.DataThatShouldNotBeSerialized = "Random data";
Order order1 = new Order("Motherboard", 2, customer);
Order order2 = new Order("CPU", 2, customer);
customer.Orders.Add(order1);
customer.Orders.Add(order2);
SoapFormatter formatter = new SoapFormatter();
FileStream stream = new FileStream("customer.xml", FileMode.Create);
formatter.Serialize(stream, customer);
stream.Close();
*/
/* New serialization events in .Net 2.0
[Serializable]
public class Employee
{
private string _name;
public string Name
{
get{ return _name; }
set{ _name = value; }
}
public Employee(string name)
{
_name = name;
}
[OnSerializing]
private void OnSerializing(StreamingContext context)
{
Console.WriteLine("Serializing");
}
[OnSerialized]
private void OnSerialized(StreamingContext context)
{
Console.WriteLine("Serialized");
}
[OnDeserializing]
private void OnDeserializing(StreamingContext context)
{
Console.WriteLine("DeSerializing");
}
[OnDeserialized]
private void OnDeserialized(StreamingContext context)
{
Console.WriteLine("DeSerialized");
}
}
Employee emp = new Employee("Mark");
BinaryFormatter formatter = new BinaryFormatter();
FileStream stream = new FileStream("employee.dat", FileMode.Create);
formatter.Serialize(stream, emp);
stream.Seek(0, SeekOrigin.Begin);
emp = (Employee)formatter.Deserialize(stream);
stream.Close();
/*
This will output
Serializing
Serialized
Deserializing
Deserialized
*/
/* Simple Xml serialization
public class Employee
{
public int ID;
public string Name;
public string Department;
}
Employee emp = new Employee();
emp.ID = 1001;
emp.Name = "Mark";
emp.Department = "Accounts";
XmlSerializer serializer = new XmlSerializer(typeof(Employee));
XmlTextWriter writer = new XmlTextWriter("employee.xml", System.Text.Encoding.UTF8);
writer.Formatting = Formatting.Indented;
serializer.Serialize(writer, emp);
writer.Close();
Output is
<?xml version="1.0" encoding="utf-8"?>
<Employee xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ID>1001</ID>
<Name>Mark</Name>
<Department>Accounts</Department>
</Employee>
[XmlRoot("employee")]
public class Employee
{
[XmlAttribute("id")]
public int ID;
[XmlElement("name")]
public string Name;
[XmlElement("department")]
public string Department;
}
<?xml version="1.0" encoding="utf-8"?>
<employee xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="1001">
<name>Mark</name>
<department>Accounts</department>
</employee>
*/