Tales from the Evil Empire

Bertrand Le Roy's blog

Dates and JSON

JSON is a great data format and it's taken the Internet by storm for a number of good reasons. But because of a strange oversight in the EcmaScript specs, there is no standard way of describing dates in JSON. There's been a lot of discussion on this topic and it still remains a problem today.

In the Microsoft Ajax Library, we've tried a number of different approaches before we converged to the solution we're using today.

The first thing we tried was to inject Date constructors in the JSON string. This is a (very) bad idea for a number of reasons. First, it simply does not conform to the JSON specs. Second, any JSON parser that validates its input before parsing it will cough on such a thing. Finally, it establishes a precedent: why would it be allowed for dates and not for arbitrary types? This would just defeat the purpose of JSON.

The second approach, which is what most people settled on, is to adopt some string form of date and just interpret any string that conforms to this format as a date. We've been using "@1163531522089@" where the number is the number of milliseconds since January 1st 1970 UTC for a while (not super-readable) and some people are just using the ISO 8601 format. These are both almost acceptable compromises (if everyone agrees on them) but have a fundamental flaw which is that there can be false positives: what if you want to serialize the "1997-07-16T19:20:30.45+01:00" string, as a real string, not as a date?

To be perfectly honest, JSON Schema does solve the problem by making it possible to "subtype" a string as a date literal, but this is still work in progress and it will take time before any significant adoption is reached.

Our current approach is using a small loophole in the JSON specs. In a JSON string literal, you may (or may not) escape some characters. Among those characters, weirdly enough, there is the slash character ('/'). This is weird because there actually is no reason that I can think of why you'd want to do that. We've used it to our benefit to disambiguate a string from a date literal.

The new format is "\/Date(1198908717056)\/" where the number is again the number of milliseconds since January 1st 1970 UTC. I would gladly agree that this is still not super readable, which could be solved by using ISO 8601 instead.

The point is that this disambiguates a date literal from a string that looks like the same date literal, while remaining pure JSON that will be parsed by any standard JSON parser. Of course, a parser that doesn't know about this convention will just see a string, but parsers that do will be able to parse those as dates without a risk for false positives (except if the originating serializer escaped slashes, but I don't know of one that does).

Here's how you'd encode a string that contains the same literal I described above: "/Date(1198908717056)/".

Notice how a simple eval-based parser will just return the exact same string in both cases, but if you run a simple search for "\\/Date\((\d+)\)\\/" and replace with "new Date($1)" before the eval (but after validation), you'll get the dates right in the final object graph. The string I described above will just go through that filter undetected and will remain a string.

We're pretty much satisfied with this solution to the date problem, but of course for the moment very few serializers and parsers support that convention. It would be great if this could become the consensus across the industry.

I'd love to read any comments you may have on that subject.

Comments

Josh Stodola said:

>> It would be great if this could become the consensus across the industry. <<

I have been saying this about numerous Javascript inconsistensies for over 7 years. In that time span, I have yet to see a single consensus. The language is extremely desperate for standardization. It's about time somebody in authority puts their foot down! It seems that without a leader, it's going no where (and fast).

# January 18, 2008 4:20 PM

Bertrand Le Roy said:

Well, Josh, this is a little different in the sense that JSON (not directly talking about JavaScript date literals here) is in the hands of a few individuals, not a standards comittee so it may actually be achievable.

# January 18, 2008 4:24 PM

Joe Chung said:

One reason to escape slashes in JavaScript is that some browsers don't handle HTML close tags, e.g., </p>, well in JavaScript so you would use <\/p> instead.

# January 18, 2008 4:40 PM

Alex Osipov said:

For me, what gets confusing is DateTime serialization in regards to time zones.  .NET in general is not consistent when dealing with DateTime and time zones, so would love to see a more standard approach not just for Atlas but .NET in general.  An attribute allowing me to use local time on the client would be very helpful at times.  

# January 18, 2008 4:48 PM

Bertrand Le Roy said:

Thanks joe, that explains it.

Alex, do you have a specific example of such an inconsistency?

# January 18, 2008 4:50 PM

Kris Zyp said:

Bertrand, clever trick, and good job making it go with eval. However, I would still be hestitant to move away from the date serialization used by Crockford's library and ES4. I think your suggestion of defining a property as a date in a schema(JSON Schema or otherwise) outside of the actual data in order to explicitly declare dates, is a better technique, because the standard serialization/parsing libaries can still be used.

# January 18, 2008 5:30 PM

Bertrand Le Roy said:

I'll try to lure Doug to this post and get his thoughts. Good thing is that on the parser side, we can support several approaches. Again, Doug's convention will have false positives so most parsers will probably not do the special casing by default.

# January 18, 2008 5:36 PM

Nikhil Kothari said:

Actually, I should have updated my main post (a year ago I guess) but as discussed in the comments, there is a way to use the @ syntax without false positives.

Your post called my attention to it, and I updated it now. :-)

# January 18, 2008 10:54 PM

Rick Strahl said:

The question is who do we have to bribe to make *some* sort of standard for date literals :-}. This is a pretty big stumbling block in universal JSON usage, although the solution you came up with MS AJAX with the \/Date()\/ seems like a good way to go - it  works and is relatively easy to implement. I posted some additional thoughts and some code to that effect here a while back when we discussed this last month:

www.west-wind.com/.../214731.aspx

# January 19, 2008 8:01 AM

Michael Yeaney said:

I've worked around these problems in the past by using JSON to *only* transport formatted data - in other words, no reliance on the client browser native types at all for numbers and dates.  While it sounds a little odd, most of the enterprise products i've worked on supported the notion of a configurable timezone and/or culture, which makes the browsers cultural settings worthless.  So for dates and numbers, we do *all* formatting on the server (after setting the current thread culture code and format options as per the app config), and just ship the "ready for display" data to the browser.

IMO, until we can *force* the browser to use different time zone and number format options, it's pointless to do manipulation on the browser (and a *lot* more code to support - we tried it) - the .NET framework has most of these features built in (except for explicit DST support), so why not leverage them?

Any thoughts???

# January 19, 2008 12:31 PM

Eilon said:

Some languages out there already have Date literals - which ones are they? Why don't more languages have Date literals?

And for that matter, what are some other primitive types that lack literal support in languages?

# January 21, 2008 1:41 PM

rascunho » Blog Archive » links for 2008-01-21 said:

Pingback from  rascunho  &raquo; Blog Archive   &raquo; links for 2008-01-21

# January 21, 2008 3:24 PM

Wöchentliche Rundablage: .NET 3.5, WPF, LINQ, Tests, System.AddIn, SubSonic, Sandcastle | Code-Inside Blog said:

Pingback from  W&ouml;chentliche Rundablage: .NET 3.5, WPF, LINQ, Tests, System.AddIn, SubSonic, Sandcastle | Code-Inside Blog

# January 21, 2008 3:48 PM

Sweet said:

You guys are killin me. Why not put a few lines a code in here. Whatever you do make sure nobody can find this without spending an hour on google. Nice job

# February 12, 2008 5:38 PM

James Newton-King said:

I am really happy with how Json.NET 2.0 and LINQ to JSON is turning out. This release offers big improvements

# March 2, 2008 2:20 AM

Denny.NET said:

Passing a JSON object to a WCF service with jQuery

# March 3, 2008 8:55 PM

mike said:

Suggestion for your blog: each link for JSON is the same, I thought each link was to a relative article/source, so perhaps only link the first instance?

# April 22, 2008 4:59 PM

NZEYIMANA Emery Fabrice » Invalid JSON from NET - DateTime said:

Pingback from  NZEYIMANA Emery Fabrice &raquo; Invalid JSON from NET - DateTime

# May 14, 2008 2:21 PM
Leave a Comment

(required) 

(required) 

(optional)

(required)