TOML and YAML both have the problem that if you receive an incomplete document, there's a decent chance you can't tell. JSON doesn't have that because of the closing curly.
this post was submitted on 01 Feb 2024
41 points (86.0% liked)
Programming
17313 readers
205 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities [email protected]
founded 1 year ago
MODERATORS
That's not a problem of a format and should be handled by transport or storage.
It very much is an aspect of the format. You may deem it unimportant, but it's a feature that is missing from toml and yaml.
It's not a responsibly of the format, so, at most, it's a mere side effect. In any practical process which could result with truncated data, even if it handles data with such property, it should be wrapped in a container which includes length. At the very least, it would allow to trace the source of truncation, e.g. was the data originally truncated, or was it truncated in the process, and change the format without shooting in oneselves foot. And the generating side should always provide success flag which should be properly handled, since it may produce syntactically correct, but semantically invalid data. Such as checking exit code of process which generates json/yaml in question
What about processes that terminate before writing the whole thing? You can't protect against everything. Blame other processes all you want but the language spec allows for confusion.
You just check the exit code, no? The other process may fail while generating syntactically correct data too, regardless of format.
Doesn't YAML have a (seldom used) feature of a start and end of document marker? The "YAML frontmatter" that a few markdown documents have, uses this.
Good point, I'd been interested in using toml
Every time I have reached for TOML I have ended up using JSON. The first reason is that Python standard library can read but not write TOML, which is generally useless for me. The second reason is TOML does not add any benefit over JSON. It’s not that much easier to read and IMO JSON is easier to write by hand because the syntax rules are completely obvious.
TOML is mainly for humans to write, certainly not a good choice if you're programmatically writing files - comments and formatting would be lost.
It all depends on the library you use. Rust has you covered with toml_edit. It is what is used for all the cargo commands editing the Cargo.toml file.
Agreed. Except that it’s not easier to write imo
Where do you put your comments in JSON files?
That doesn't really work when you need two comments at the same level, since they'd both have the same key
write json with comments. Use a yaml parser.
If you're reaching for yaml, why not use toml?
Every time i try to use toml, i end up going back to json
because of the cut and paste problem. It works in json.
Cut and paste problem?
cut out a random piece of your document. is it a partial or a complete document?
paste it somewhere else in the document. you have to fix the indentation because if not then the document won't work or mean something completely different
you have to fix the indentation because if not then the document won’t work or mean something completely different
Whitespace has no meaning in json. You can indent however you want, or not at all.
I'm assuming you're running into issues because you're writing json in a yaml file which does care about indentation, and you're only writing json in yaml to get access to comments.
In which case it circles back around to: why not use toml? Whitespace formatting doesn't corrupt the file, and it has built in comments.
i do use json instead of yaml precisely for the reasons you mentioned. That was my original point in the first place that json does not have these problems. something must have been lost in transmission
It still works since multiple identical keys are still valid json. Although that in itself isn't fantastic imo.
For settings files I always have an example file with sensible values filled in and along with descriptive keys that serves as reasonable documentation. If something is truly unknowable, I’ve probably done something wrong.
How would you mark a flag in your json settings file as deprecated?
In my opinion, the settings file isn’t where this information should be presented. I would put these notes in the release log and readme and example settings file. I have also written this information to logging during startup so a user knows what to do, or I write a migration that does the change automatically if that’s possible.
This is only my opinion and you can use the comment method described like “//“: “Deprecated” if desired.
The very first moment that I had to use JSON as a configuration format, and I was desperate to find a way to make a long string into a JSON field. JSON is great for many things, but it's not good at all for a configuration format where you need users to make it pretty, and need features like comments or multi-line strings (because you don't want to fix a merge conflict in a 400 character-wide line).
I really don't understand why people still insist on prohibiting trailing commas anywhere. The syntax is interesting but it looks like defining an array of objects would be needlessly difficult. I think the double square bracket syntax is far too easy to confuse.
but... trailing commas are ok in TOML
- edit 1: I now see the caveat of an inline table - though trailing commas are not that useful for an inline list of values anyways
- edit 2: they're changing this for TOML 1.1: https://github.com/toml-lang/toml/issues/516 .
The double square bracket is for an array of tables. A regular array looks "normal": https://toml.io/en/v1.0.0#array
Just gonna throw out HJSON as another alternative: https://hjson.github.io/
I thinks a great idea but I have never seen it used in the wild, unforunately.
I like the syntax so much, but I'm so missing variables like the ones in ConfigParser's .ini format, I wish there was a good format where they're actually standard
I’ll never understand why we don’t just use s-expressions.
Why would one pick toml over yaml?
Xkcd 927?
Also yaml is ugly as hell and I'm okay avoiding it.
What?
It's simple and readable. You literally put somebody that has never coded in their life, show them the YAML file and they will probably get it. Worked both with my boss and my girlfriend.
In Toml there are too many ways to do the same thing, which I don't like. Also unless you know it deeply, you have no idea how the underlying data structure is going to look.
In Toml there are too many ways to do the same thing, which I don’t like
ha
Wow. I've never used yaml or even looked at it but damn that is horrid. Why do people even use this? JSON and XML are so better.
I say this with all due respect, but XML can gargle nuts.
Because no one ever uses those. Literally > and | are the only ones I’ve ever seen in over a decade and you will never need to worry about the differences between the two.
XML as a configuration language is terrible. Yaml gets the point across in an easily readable way, which is exactly the point. Same for JSON except JSON you can’t even use comments (you need json5 or one of the numerous other alternatives to get those).
It's really unfortunate the devops world chose such a hot mess of a format. Extending JSON with comments would be a dumb choice and still do a better job for most config files.
Yaml is already pretty popular, so I don't think 927 applies here. It's actually more common in newer projects than toml.
Which begs the question, should I go with the flow or is there good reason to go with toml?
Perhaps they're Norwegian.
Significant whitespace is the devil's play thing.