JSON5E - JSON5 for Humans
9 points by borisk
9 points by borisk
I'm the opposite, I feel like JSON5 is doing too much. To me, the only crucial things which are missing from JSON are:
JSON5 fixes all that, but also adds single quoted strings, hexadecimal numbers, NaN, Infinity, multi-line strings and more, which is too much for my taste.
But despite this, I still think JSON5 is the best general-purpose config/data exchange format.
I personally prefer jsonc for the fact that it doesn’t do anything extra, and only addresses the first two. quoted identifiers has never bothered me, but the other two are very legitimate quality of life improvements.
I agree. I also have one major gripe with JSON5: it relies on Unicode character classes, requiring that parsers include a whole Unicode database for full support. That's so unnecessary. Nobody would have minded if only ASCII characters could be in unquoted keys. In my own JSON5 library (https://github.com/mortie/json5cpp), I opted to accept all non-ASCII characters as long as they aren't in the ranges associated with whitespace, just so that I didn't have to bring in a database.
The Unicode character classes reliance also adds an implicit versioning system: what is and isn't a valid JSON5 document depends on what version of Unicode the parser uses.
But other than that, and the awkward surrogate pair \u escapes it inherits from JSON, JSON5 is a pretty good format. I would have made different choices but ultimately think it's better to standardize on JSON5 than have too many different almost-compatible config file formats.
Single-quoted strings are convenient in string literals in C-family languages. I’ve written a lot of JSON-related tests in and JSON5 made it easier.
I’ve also worked with JSON config files that can include (JS) code in strings, and they’re quite awkward if they have to be on a single line.
But I agree the number stuff is superfluous.
But I agree the number stuff is superfluous.
They are superfluous until you really, really want to write hex numbers. Like in a file describing hardware such as here: https://github.com/CHERIoT-Platform/cheriot-rtos/tree/main/sdk/boards
Tailscale use "HuJSON, also known as Human JSON" (https://github.com/tailscale/hujson) as a configuration format that does two out those three.
The github.com/tailscale/hujson package implements the JWCC extension of standard JSON.
The JWCC format permits two things over standard JSON:
- C-style line comments and block comments intermixed with whitespace,
- allows trailing commas after the last member/element in an object/array.
All JSON is valid JWCC.
Personally, I think that unquoted keys aren't worth it, because it limits what future changes you can make (while maintaining compatibility).
Allowing unquoted keys has the side effect of squatting on every possible keyword you might want to ever use later. Want to add non-string dictionary keys, like CBOR? Ok. Want to add undefined (distinguishable from null), like CBOR? Ok. Want to do both, like CBOR? Oops, an unquoted undefined key already means a string key.
Personally, I think that unquoted keys aren't worth it, because it limits what future changes you can make (while maintaining compatibility).
JavaScript solves this problem by wrapping computed (non-literal) keys in [ ].
HuJSON sounds confusingly similar to hjson, tho hjson is much more like json5e but ten years older.
HuJSON sounds confusingly similar to hjson, tho hjson is much more like json5e but ten years older.
FWIW, HuJSON's first commit is from 2019.
And also, personally, I prefer the name JWCC over HuJSON. Both names mean essentially the same format, but I think that JWCC is a clearer name than something "for Humans".
If I tell you that JWCC is literally an acronym for JSON With Commas and Comments, you immediately know what it is (if you know what JSON is).
JavaScript solves this problem by wrapping computed (non-literal) keys in [ ].
IIUC, I think computed keys is a different thing. You're saying that JS lets you do this:
let dynamicKey = undefined;
let myObject = {
[dynamicKey]: 12345
};
Which is fine, shrug, but it's not the same as "extend JSON syntax (while keeping out computation as a thing) with unquoted keys", which would let you say this (that I think is a bad idea):
{ undefined: 12345 }
Want to add non-string dictionary keys, like CBOR?
I thought of another example: want to allow numbers as keys (e.g. sparse vectors)? Oops, 42 is already a valid key, but it means 42-the-string, not 42-the-number.
So in other words - simplified YAML
The foo: 123 syntax does resemble YAML, but it's missing what is to me the quintessential YAML feature: whitespace-delimited nested structures. Being able to specify nested lists and maps without adding extra brackets (which, if I understand correctly, this JSON5E proposal would still require).
I'd really like to see a competitor to YAML that does that part well, without all the craziness.
Any reason strictyaml isn't a good fit?
Well that's a specific parser for a specific language, rather than a defined syntax. NestedText is similar but is a defined syntax.
Yeah, I was going to say the same thing.
Being a library does mean StrictYAML can make schema validation a mandatory part of parsing (if I understand it correctly). Whereas YAML syntax's parsing rules are sometimes surprising (1.2.3 is a string but 1.2 is a float), StrictYAML can change parsing rules based on the schema (weight: 1.2.3 is always an error and version: 1.2 is always a string).
I think my Platonic ideal of "YAML done right" would still keep the schema-less nature. Maybe true/false/null have a syntax like #t/#true, and strings starting with a digit or a # have to be quoted? Disclaimer: I've only thought about this for like 60 seconds, and no matter what you do, I think you have to make something more awkward.
Have you already looked at NestedText?
I have not; thanks for mentioning it!
My first impression is that making everything a string is too extreme of a solution. But it also feels like one of those ideas that will eventually make me think, "On second thought, that's actually really elegant." I'll have to ponder it some more!
I don’t get all this JSON-next things. Json is simple enough and mature. The only lacking thing is comments (which can be adressed with „_” field).
Sorry but a format without comments and without trailing commas just isn't suitable for configuration files. JSON is a data interchange format and nothing more.
JSON is pretty painful to write by hand and I don't think it's an outright bad idea to try and see if it can be extended to make it more palatable for humans. There are drawbacks of that, sure (compatibility is a major one) but the relative success of JSON5 suggests on balance it's a fruitful direction to explore.
JSON is pretty painful to write by hand
Is it? More specifically, what makes you say it is painful to write by hand?
I'm not them but without any question the "keys must be quoted" is a concession to a computer and not one for legibility; that's why both jq and JS itself don't require them:
node <<'JS'
console.log('ahh, nice', {
metadata: {
name: "muppet",
namespace: "default",
},
replicas: 1,
})
JS
jq -n '{similarly: {nice: true}}'
and only if one is in a brace-matching editor is one able to answer "how many } are required to make this snippet legal?"
{
apiVersion: "v1",
kind: "Deployment",
metadata: {
name: "muppet",
namespace: "default",
},
spec: {
template: {
containers: [
{
name: "alpha",
resources: {
requests: {
memory: "8Gi"
to say nothing of "how many closing braces does this require? - ha, trick question!"
apiVersion: v1
kind: Deployment
metadata:
name: muppet
namespace: default
spec:
template:
containers:
- name: alpha
resources:
requests:
memory: 8Gi
and, since I deviated from this thread's topic by introducing yaml into the discussion, the other pet-peeve is that one has not experienced pain unless trying to use a multi-line snippet in json
Let's consider a random example ->
[
{
"name": "Set up keychain",
"if": "${{ matrix.os.base == 'mac' && needs.setup.outputs.has_secrets == 'true' }}",
"run": "KEYCHAIN_PASSWORD=$(openssl rand -hex 32)\necho \"::add-mask::$KEYCHAIN_PASSWORD\"\nsecurity create-keychain -p \"$KEYCHAIN_PASSWORD\" build.keychain\nsecurity default-keychain -s build.keychain\nsecurity unlock-keychain -p \"$KEYCHAIN_PASSWORD\" build.keychain\nsecurity set-keychain-settings -lut 1200 build.keychain\n\nsecurity import \"$HOME/certificates/devid-app-cert.p12\" -k build.keychain -P \"\" \\\n -T /usr/bin/codesign -T /usr/bin/security -T /usr/bin/productbuild\n\nsecurity set-key-partition-list -S apple-tool:,apple:,codesign: -s -k \"$KEYCHAIN_PASSWORD\" build.keychain\n"
}
]
obvious, amirite?!
obvious, amirite?!
Well it wasn't to me, otherwise I wouldn't have asked :) It might be a blind spot as I have been working with json for well over a decade, but to me aren't that big of a deal.
I'll concede that quoted keys and no trailing commas are a bit of a annoyance. But brackets I personally never found annoying and actually helpful. Purely indentation based annotations always mess me up personally.
The top level starts looking like TOML. Maybe use TOML then? TOML values can be JSON-like objects.
Indeed, especially with the multiline inline tables of toml 1.1 even for people who don't like standard tables (and arrays thereof) they can just ignore that and use only inlines below the toplevel.
IMO TOML kinda killed its momentum by waiting for so many years to standardize 1.1. It has ossified now. There were a few years where it was the "up and coming new format" which still could've changed and parsers and Q&A site answers would've evolved with it, but they decided to do nothing during those years. 1.1 is too late.
And honestly, multi line inline tables should have been there since 1.0.
Feel free to critique.
Some further extensions I am considering:
Multi-line/raw values to allow JSON5E to serve as a container for other formats (pasting a Bash script is the canonical example).
Unquoted "name" values. It would be nice to be able to write:
connection-proxy: proxy.example.com
Instead of:
connection-proxy: "proxy.example.com"
You’re re-inventing yaml. Consider reading their spec :-)
The huge problem with YAML is really just the lack of braces and brackets around objects and arrays, and all the insane stuff in the spec like being able to run arbitrary constructors in the programming language that's parsing it. YAML without those things and with mandatory brackets and braces wouldn't be such a bad idea.
Now, unquoted name values is arguably also one of the bad ideas in YAML, giving rise to the norway problem.
and all the insane stuff in the spec like being able to run arbitrary constructors in the programming language that's parsing it.
That's more of an issue of how the parser is implemented.
YAML without those things and with mandatory brackets and braces wouldn't be such a bad idea
The new(?) kyaml encoding may interest you, although as very best I can tell it is more of a convention than a technical change: https://kubernetes.io/docs/reference/encodings/kyaml/
The example shows it with leading whitespace but the original design did not, so I don't know if that's a documentation bug or they backed down from that choice (and I don't have an example output in front of me right now)
Hey that looks pretty nice! Someone should make a proper spec for that, and maybe it could start to become an option in yaml libraries.
Feel free to critique.
The name is a lie. json5 extended the original JSON with javascript features (some added since json had been standardised), this is literally not JSON, as in it is not a JavaScript Object Notation for any version of javascript which has ever existed.
literally not JSON
Tangentially related fun fact: JSON itself technically was not JSON. But they later made JSON JSON by changing JS to match.
Eh I don't really agree with this criticism. It's not "JavaScript's Object Notation with some extensions", it's "JSON5 with some extensions". JSON5 itself isn't "version 5 of JavaScript's Object Notation", it's JSON with some extensions (and, yes, happens to be a subset of ECMAScript).
It's not "JavaScript's Object Notation with some extensions", it's "JSON5 with some extensions"
JSON5 is a JavaScript Object Notation.
and, yes, happens to be a subset of ECMAScript
That's not happenstance, that's why it's still a JSON, it's still a javascript object notation, which TFA's is not.
JSON5 itself isn't "version 5 of JavaScript's Object Notation"
The 5 is not a version number, it's a direct reference to ES5, because as the JSON5 spec explains
The JSON5 Data Interchange Format (JSON5) is a superset of JSON that aims to alleviate some of the limitations of JSON by expanding its syntax to include some productions from ECMAScript 5.1.
Most notably... trailing commas were specified in ES5.
JSON5 is a JavaScript Object Notation.
No, JSON5 is JSON with some extensions. JSON is JavaScript Object Notation.
No
Yes.
JSON5 is JSON with some extensions
Which were intentionally selected from JavaScript, making it still a JavaScript Object Notation. Unlike TFA which is very much not that.
JSON is JavaScript Object Notation.
You missed a critical word.