Fabrice's weblog

Testing object databases for .NET: Eloquera, STSdb, Siaqodb, Ninja Database Lite

Given all the activity around databases lately with the NOSQL movement, I decided to give a try to some of the new contenders.

Lately, I've added some object database management systems to SharpToolbox. You can find them all in the DBMS category.
I downloaded some of them to see what they look like and determine whether they are stable enough to start developments with them or not.

Here are the databases I tested: Eloquera, STSdb, Siaqodb, Ninja Database Lite. Most of them support LINQ, which is great.

Eloquera Database

I tested versions 2.8 and 3.0 Beta.
It seems that development is active, with new features and bug fixes being released often.
I received replies quickly to my questions in the forum.
Documentation exists but is somewhat light. XML documentation for IntelliSense would have been helpful.

Here are some of Eloquera Database's features:

• Storing any .NET object without implementing any interface and without inheriting from a specially designed class
• Queries can be expressed with LINQ or a query language based on SQL
• Indexes
• In-memory database support
• Supports arrays in queries
• Regular expressions support in queries
• Bulk inserts and updates
• Generic object support
• Inheritance support in queries
• Client/server or embedded
• Unique identifier for any object within a database
• Native GUID identifiers support

Eloquera can be used in client/server mode or embedded. When run in c/s, Eloquera runs as a Windows service. When you want to embed Eloquera in your application, you'll just add references to the Eloquera DLLs and use the Desktop mode. There's no real difference between the two modes from your program's perspective.

My small testing consisted of two classes:

public class Company
{
  public string Name { get; set; }
}
 
public class Job
{
  public Company Company { get; set; }
  public String Title { get; set; }
}

Here is how I used them to persist and save objects:

const string connectionString = "server=localhost;password=pwd;options=none;";
const string databaseName = "Base1";
 
using (var db = new DB(connectionString))
{
  db.DeleteDatabase(databaseName, true);
  db.CreateDatabase(databaseName);
  db.OpenDatabase(databaseName);
  db.Store(new Job {
    Company = new Company { Name = "Company 1" },
    Title = "Design Engineer"
  });
  db.Store(new Job {
    Company = new Company { Name = "Company 2" },
    Title = "Project Manager"
  });
}

using (var db = new DB(connectionString))
{
  db.OpenDatabase(databaseName);
  var query =
    from Company company in db
    where company.Name.Contains("1")
    join Job job in db on company equals job.Company into jobs
    select new { Company = company, Jobs = jobs }; 

  Console.WriteLine("Jobs by company");
  ObjectDumper.Write(query, 2);
}

Here is a sample SQL-like query:

db.ExecuteQuery("SELECT Company WHERE Name LIKE '%1%'")

The API is simple and it was not difficult to get started.

STSdb

I tested version 3.5.6.
I'm not able to tell whether development is active. I haven't seen any release notes or roadmap.
There isn't much activity in the forum. Though, I received a reply to a message I posted there in about 24 hours.
Documentation consists of a partial API reference. XML documentation is available but isn't complete either.

Here are some of STSdb's features:

• 100% managed
• Supports different storage modes (on-disk, in-memory and combined)
• Compression of columns of data
• ACID transactions
• LINQ support
  

STSdb is an embedded database. When you download it, what you get is a 181 KB DLL that you'll reference in your project.

I was not able to use the same classes because instances are not shared between Job and Company classes. Here are the classes I used this time:

public class Company
{
  public uint ID { get; set; }
  public string Name { get; set; }
}
 
public class Job
{
  public uint CompanyID { get; set; }
  public String Title { get; set; }
}

You can notice that there is no direct reference between the two classes. Object will then be joined by ID only.

Here is my test code:

const String filename = "Test.stsdb";
 
using (var repository = StorageEngine.FromFile(filename))
{
  var jobsTable = repository.Scheme.CreateOrOpenXTable<ulong, Job>(
    new Locator("Jobs"));
  var companiesTable = repository.Scheme.CreateOrOpenXTable<ulong, Company>(
    new Locator("Company"));

  companiesTable[0] = new Company { ID = 1, Name = "Company 1" };
  companiesTable[1] = new Company { ID = 2, Name = "Company 2" };
  jobsTable[0] = new Job { CompanyID = 1, Title = "Design Engineer" };
  jobsTable[1] = new Job { CompanyID = 2, Title = "Project Manager" };

  repository.Scheme.Commit();
  jobsTable.Commit();
  companiesTable.Commit();

  // ...

  jobsTable = repository.Scheme.OpenXTable<ulong, Job>(
    new Locator("Jobs"));
  companiesTable = repository.Scheme.OpenXTable<ulong, Company>(
    new Locator("Company"));
  var query =
    from companyEntry in companiesTable
    let company = companyEntry.Record
    where company.Name.Contains("1")
    join jobEntry in jobsTable on company.ID equals jobEntry.Record.CompanyID
    into jobs
    select new { Company = company, Jobs = jobs.Select(job => job.Record) };
 
  ObjectDumper.Write(query, 2);
}

As you can see, STSdb introduces notions such as XTable and Locator, and it requires you to work explicitly with indexes to add items to the database.
This may make it powerful or fast, but it makes coding less intuitive than with Eloquera.

The fact that I had to remove the reference from Job to Company requires dealing with joins each time, which is something I'd like to avoid. I'm not in a relational world, but objects should be connected together in my book.

Siaqodb

I tested version 3.5.6.
Development and forum seems to be active.
Documentation consists of the API reference, a tutorial and sample projects. XML documentation is missing, which is a mistake because it seems that it exists as the API reference shows.

Here are some of Siaqodb's features:

• Objects are stored as they are without any conversion/mapping
• LINQ is the only query engine that can be used to retrieve objects or members of objects from database
• Full POCO support
• Automatic object schema refactoring
• Import/export from/to XML
• Thread safe
• Any object has unique OID (object identifier) for its Type

Siaqodb is an embedded database for .NET, Mono and Silverlight.
The download installs a 135 KB DLL for .NET, as well as DLLs for Silverlight and Windows Phone 7. It also installs a tool for running queries against Siaqodb databases. The UI is similar to the one of LINQPad.

Again, I had to create new classes because only primitive types are supported:

public class Company
{
  public int OID { get; set; }
 
  public uint ID { get; set; }
  public string Name { get; set; }
}
 
public class Job
{
  public int OID { get; set; }
 
  public uint CompanyID { get; set; }
  public String Title { get; set; }
}
Here is my sample test code:
var dbPath = Path.Combine(
  Path.GetDirectoryName(Environment.GetCommandLineArgs().First()),
  "Data");
 
if (Directory.Exists(dbPath))
  Directory.Delete(dbPath, true);
Directory.CreateDirectory(dbPath);
var db = new Sqo.Siaqodb(dbPath);
 
db.StoreObject(new Company { ID = 1, Name = "Company 1" });
db.StoreObject(new Company { ID = 2, Name = "Company 2" });

db.StoreObject(new Job { CompanyID = 1, Title = "Job 1" });
db.StoreObject(new Job { CompanyID = 2, Title = "Job 2" });

// ...

db = new Sqo.Siaqodb(dbPath);

var query =
  from Company company in db
  where company.Name.Contains("1")
  join Job job in db on company.ID equals job.CompanyID into jobs
  select new {Company = company, Jobs = jobs };
 
var companyJobs = query.ToArray();
 
Console.WriteLine("Direct from DB:");
ObjectDumper.Write(companyJobs, 1);
 
Console.WriteLine();
 
var realCompanies = companyJobs.Select(item =>
  {
    var company = new RealCompany { Name = item.Company.Name };
    company.Jobs =
      item.Jobs
        .Select(job => new RealJob { Company = company, Title = job.Title })
        .ToList();
    return company;
  });
 
Console.WriteLine("After conversion:");
ObjectDumper.Write(realCompanies, 1);
 
db.Close();

As you can see, given that there can be no automatic reference between objects, I use some conversion code to convert what comes out of the database to another model.
Here are the target classes:

public class RealCompany
{
  public IList<RealJob> Jobs { get; set; }
  public string Name { get; set; }
}
 
public class RealJob
{
  public RealCompany Company { get; set; }
  public String Title { get; set; }
} 

Ninja Database Lite

I tested version 1.0.
As it's a brand new product, it's not possible yet to tell whether development and forum will be active.
There documentation is light, with reference help, a small tutorial, and sample projects. XML documentation is provided.

Here are some of Ninja Database Lite's features:

• Compact binary serialization
• Compression
• Encryption
• Automatic type registration
• Supports Windows Phone 7, Silverlight, .NET 3.5 and .NET 4

You can find more details on Ninja Database Lite's SharpToolbox page.

Ninja is an embedded database for .NET, Silverlight and WP7.
The download installs a 58 KB DLL for .NET 4, as well as DLLs for the other platforms.

I had to use IDs again, because although complete object graphs are persisted, the instances don't seem to be shared between the Job and Company classes.

Here is the code:

var db = new NinjaDb { DatabaseName = "Base 1" };
 
var companies = new List<Company> {
  new Company {ID = 1, Name = "Company 1"},
  new Company {ID = 2, Name = "Company 2"}
};
db.Save(companies);
 
var jobs = new List<Job> {
  new Job { CompanyID = 1, Title = "Job 1" },
  new Job { CompanyID = 2, Title = "Job 2" }
};
db.Save(jobs);
 
// ...
 
companies = db.Load<List<Company>>();
jobs = db.Load<List<Job>>();
 
var query =
  from company in companies
  where company.Name.Contains("1")
  join job in jobs on company.ID equals job.CompanyID into companyJobs
  select new { Company = company, Jobs = companyJobs };
 
ObjectDumper.Write(query, 2);

Conclusion

That's about all I had time to do with these databases for now.
It was enough to get a feel for their usability and stability. It's not enough though to tell whether they are efficient and stable.

For the moment, I consider that they are not mature enough, but they're worth keeping under my radar. I may use them in the future.

PS: I know other object databases exist for .NET (db4o, Versant Object Database, Objectivity, Objectivity/DB, Perst), but I don't like their pricing and/or find them too heavy for the job.

Sesame: Microsoft Dallas support

With the release of Microsoft Codename "Dallas" CTP3 and the features announced in the previous post, it became easier to add support for Dallas to Sesame Data Browser.
The new version of Sesame published today allows you to browse Dallas datasets, as demonstrated below.

Creating a connection to a Dallas dataset in Sesame is easy. You just need a dataset URL and an account key.

The Dallas portal

In order to get an account key and URLs, you need to visit https://www.sqlazureservices.com and sign in with a Windows Live ID. You'll then be able to subscribe to datasets, such as AP Online (Associated Press) or business information from InfoGroup:

Note: Only CTP3 services are supported in Sesame.

Once you have a subscription, you can visit its preview page. This is where you'll find the URL you'll use in Sesame.
Here is for example the page for AP Online:

The URL for AP Online is highlighted in blue on the above picture.

Your account key is available on the "Account keys" page:

Dallas in Sesame

Now that you have an account key and a URL, you can create an OData connection in Sesame:

Here is the result for AP Online if you click on GetBreakingNewsCategories:

Copy a category ID and click on GetBreakingNewsContentByCategory.
Paste the category ID in categoryId and type 5 for count:

After clicking on Open, you'll get data in a grid, as usual:

You can also locate the news items on a map:

Known bugs

• A bug in Dallas CTP3 prevents the "Load more" button to work correctly in all cases.
• Dallas CTP3 does not support all filter operations. Namely, the EndsWith, StartsWith, Contains, DoesNotContain and IsContainedIn don't work with Dallas datasets.

Please give this new version a try. As always, your feedback and suggestions are welcome!

Sesame: Spatial OData on Maps, Service Operations, HTTP Basic Authentication

Sesame Data Browser has just been updated to offer the following features for OData feeds:

• Maps
• Improved Service operations (FunctionImport) support
• HTTP Basic Authentication support
• Microsoft Dallas support
  

Maps

Sesame now automatically displays items on a map if spatial information is available in data.

This works when latitude and longitude pairs are provided.
Here is for example a map of drinking fountains in Vancouver:

This comes from DrinkingFountains in http://vancouverdataservice.cloudapp.net/v1/vancouver, which provides latitude/longitude for each fountain.

Here is another example, without latitude/longitude this time:

This is a map of the customers from the Northwind database, which are located based on their country, postal code, city, and street address.

Service operations (FunctionImport)

Support for service operations (aka FunctionImports) has been improved. Until now, only functions without parameters were supported.

It's now possible to use service operations that take input parameters. Let's take as an example the GetProductsByRating function from http://services.odata.org/OData/OData.svc.

This function is attached to Products, as you can see below:

A "rating" parameter is expected in order to open the function:

After clicking Open, you'll get data as usual:

HTTP Basic Authentication

New authentication options have been added: HTTP Basic and Dallas (more on the latter in the next post).

HTTP Basic authentication just requires a user name and a password, and is simple to implement.

Microsoft Dallas support

Please give Sesame Data Browser a try. As always, your feedback and suggestions are welcome!