Your JSON Is Lying to You
36 points by gkoos
36 points by gkoos
What behaviors are strictly related to json and not javascript ? Yes, json originated from javascript-land, but a lot of those problems look like they originates from the use of javascript, and not json per-se.
Agreed, the title is a bit misleading unless JavaScript is implied. Almost every issue in that post is due to JavaScript's JSON.stringify() spec, not the JSON format itself.
e.g. serializing NaN as null is purely a JavaScript spec decision, they could just as well have raised a TypeError (like it does for BigInt).
For example:
BigInt can hold integers beyond the safe Number range, but JSON has no corresponding value type.
Is not true, the original JSON spec has no specified limitation for the number type, and I'm round-tripping bigints through JSON just fine in Ruby.
Later on RFC7159 clarified that number range is a implementation limitation, and just somewhat note that many implementation are limited to the precision and range of a double, but that's about it.
e.g. serializing NaN as null is purely a JavaScript spec decision, they could just as well have raised a TypeError (like it does for BigInt).
This is a nitpick, but probably something like RangeError, not TypeError. The type of NaN is number, which is a valid type for serialization, but the value itself is out of range since the JSON spec doesn't support NaN, -Infinity, or Infinity.
I came here to say that. JSON does have things that aren't well-defined (numbers are famously a footgun), but everything in this article screams "JS is lossy when (de)serializing JSON".
It's also quite outdated these days. The reviver now gets context.source so you can do stuff like this for instance:
const data = JSON.parse(
'{"id": 9007199254740993}',
(key, value, context) => {
if (typeof value === "number" && /^-?\d+$/.test(context?.source)) {
const exact = BigInt(context.source);
if (
exact > BigInt(Number.MAX_SAFE_INTEGER) ||
exact < BigInt(Number.MIN_SAFE_INTEGER)
) {
return exact;
}
}
return value;
}
);
data.id then becomes 9007199254740993n.
The "date becomes a string" is Javascript, I think - I can happily serialise a time.Time to JSON and back to a time.Time object in Go.
Go refuses to serialise NaN to JSON (fair) but if you really need that, you can fudge around it with custom marshalling - https://go.dev/play/p/87vFzk5fj7s
The mistake is treating a deliberately tiny interchange format as a lossless snapshot of JavaScript state. A JavaScript object and its JSON representation should be treated as different data models connected by an explicit conversion.
I think this is the most important takeaway here.
Many developers use JSON to move state over expecting a verbatim copy, but unfortunately there's a myriad of footguns scattered all over the place, and it works fine until it doesn't.
This is why https://maml.dev exists)
... It's different, but I wouldn't call it better.
It refuses to encode the number or NaN, but it converts the undefined to null, and the date just gets converted to an empty object.
Neither behaviour is great, but failing to handle something as basic as a Date object does not give me much confidence in yet another standard.
Basically the only real problem here that isn't just JavaScripts JSON parser behaving in a surprising way is that JSON can't represent all float values. NaN is a real float value, but serializing/deserializing it is nothing short of a headache. (Date and time objects I would regard as complicated enough to just make them explicit ISO format serialization if you need timezone data or to use Unix timestamps, everything to do with time is complicated.)
An extension to JSON is unfortunately impossible/a nightmare to deploy, but actual support for NaN would be very helpful. As-is, you're stuck in the nightmare of "manually swap all your NaNs with null before serializing" or "extend the parser to special case NaNs to be nulls", which is both repeated code, can slow down the (de)serializer and 99/100 instances when using JSON (as a message format), you really don't care about the distinction between NaN and null anyways. It's a case of "the serializer complains about this for the extremely remote situation where it's an issue".
One of the niceties of orjson (in python) is that it does that for you (and also handles the NaN/NaT values of numpy in the same way with an option flag, because numpy operations can just return those values when mercury is in retrograde - in other words, almost completely randomly and without logic).