-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recommend using RFC 3339 formats for dates in strings #4
Comments
I like this idea. However, instead of ISO 8601, I would say that dates should be converted to UTC and represented as strings in Internet Date/Time Format as defined by RFC 3339. ISO 8601 is not a free standard, and it supports a very broad range of date and time formats. RFC 3339 defines an ISO 8601 profile that provides a high level of interoperability. |
I added the following section to a new v1.1.0 branch. Dates and TimesIt is recommended that JSON5 generators convert date and time values to Coordinated Universal Time (UTC) and represent them as strings in internet format as defined by RFC 3339 in the interest of interoperability. Example 1 (Informative) {
billenium: '2001-09-09T01:46:40Z',
} Alternatively, JSON5 generators may represent dates and times as Unix time, which is the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970, minus the number of leap seconds that have taken place since then. Example 2 (Informative) {
json5Birthday: 1338150759534,
} If a JSON5 parser expects a value to be a date or time, it may attempt to convert the value from one of the earlier defined formats to a date or time. |
Most JSON encoders/stringify respect objects with a At this point, the best bet would be to support automatic hydration of full ISO strings that match a I would propose that if new |
@tracker1 I'm hesitant to have parsers automatically convert date-like strings to dates. Strings would no longer be just strings, but they'd also be maybe-dates. And if the platform doesn't have support for fractions of seconds, then the string The nature of strings is that they can be very precise if need be. For example, the number Developers are welcome to write parsers that convert strings to dates, and in fact I've tried the idea myself, but I don't think it should be the standard. |
I'm fine with not doing anything with it... just didn't want to add something to the spec regarding dates, as date encoding already has a defacto implementation. |
If you have a defacto implementation, other people are just going to implement it wrong. If you have a standard that people can test against, they know how to implement it right. This is what standards are for. |
The only safe solution, imo, is to extend the spec to have some form of tagged literals, as was proposed on the original discussion: json5/json5#3 (comment). Besides, this solution would allow supporting any custom types of literals. Probably, the best bet would be to use the same syntax as the new ECMAScript tagged template literals: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals. Note that supported tags and their evaluation functions would always need to be registered with the parser, thus allowing full control of supported types of literals and simultaneously ensuring the safety of the solution. |
My main problems with dates are inside API calls: I was thinking about allowing something like this in JSON: {
firstName: 'John',
lastName: 'Smith',
birthday: new Date('1986-08-04'),
createdAt: new Date('2019-07-24T13:37:34.752Z'),
} |
If the spec is extended, I'd prefer bogdan's solution... though, would probably be somewhat better to remain compatible as a string.
An encoded data record should start with:
If the string starts with In this case In this way, we could use existing escape characters for encoded data, and if there's a supported decoder, it can be added and handle any encoded complex type, while still complying with original JSON. For more complex data types, there's also...
https://gist.github.com/tracker1/84fd0c1a31b3d17b176495f37b3431ba |
I don't think it's a good idea to start treating strings as maybe-dates, especially when it breaks backward compatibility. On another note, I think putting non-printable characters in strings would make JSON5 less human readable than JSON. |
In my opinion, platforms and languages have too many differences when it comes to date/time storage, time-zone conversions, parsing, formatting, precision, etc. - and also, these are often subject to change on various systems as they evolve over time, adding/removing time-zone, changing precision, etc. I think that specifying anything about dates is probably unnecessary, as most systems already specify how they want dates/timestamps represented - and usually with good reason, as it has to match their back-end database or other systems, so it's most likely safer to stay away from this topic entirely. |
I disagree strongly with @mindplay-dk . Anyone who is very particular about how their dates are interpretered will always have the option of encoding them as strings or objects. But 99.99% of the use cases are already handled in rfc3339. People have already spent a lot of time and effort figuring out wire syntaxes for these things and we depend on those wire syntaxes literally every time we use the Web. These are solved problems. With respect to syntax: I think that a string prefix is the most backwards-compatible and yet recognizable thing. Something like: "jsondate:2009-01-02" People could turn on date handling in their parsers if they want them auto-parsed or turn it off if they prefer to leave them as strings. |
If parsing strings as maybe-dates is a parser option, then I don't think it belongs in the syntax. |
Honestly, I don't see the problem in a generic solution, such as: {
"customerId": "abcd",
"orderDate": {"_": "date", "value": "2019-11-25T16:27"}
} var parseClassMap = {
"date": function parseISODate(spec) {
return new Date(Date.parse(spec.value));
}
};
var data = JSON.parse(jsonString, parseClassMap); And something inverse for |
@jordanbtucker : Okay then, let's drop the idea of having it be a parser option. We can define a unambiguous and backwards-compatible syntax and just move forward. I could live with either the string-embedded solution or dcleao's solution. The string-embedded version is a lot less "syntax" but dcleao's is more generic for future extensions. |
@prescod Please see my comments at #4 (comment) Once you start treating strings as maybe-dates, it is no longer deterministic and it breaks backward compatibility. |
Hey Jordan: I did read your comment and I thought that my prefix addressed it. The chances that there exists a string prefixed with "jsondate:" in the wild, which is NOT a JSON date seems minuscule to me. The chances of your program being broken by a cosmic ray are much higher. And we could lower it even further by calling the prefix "json5date". Google can find only two bits of code that use that string of characters in any context. And they still wouldn't match because the case isn't the same. So strings aren't maybe-dates. They are dates if and only if they start with the prefix. Otherwise they are just strings. But if you want to avoid even that minuscule level of risk, then I would endorse @dcleao's syntax and would volunteer to create a pull request if that's what you suggest. |
@prescod What if you actually want a string that starts with "jsondate:" instead of a date? |
Read through this whole thread again, here are some thoughts. Ideally, I would like to have support for a date/time literal - I just don't see how we can remain compatible with JSON and ES5 if we do so. Following a simple recommendation (as OP proposed) would be easy and natural in JS though. Dates do convert to RFC 3339 format by default: new Date().toJSON() // "2019-11-28T17:22:21.490Z" Likewise, the new Date("2019-11-28T17:22:21.490Z").toJSON() // "2019-11-28T17:22:21.490Z" These formats are also natively supported by JSON schema, where they are also encoded as strings. So on the JS side (and other languages) arguably there is already a popular standard - even if this doesn't automatically give you native So that would be my suggestion. If we have to do anything for dates/times, it doesn't need to be more than a recommendation, possibly even mentioning JSON schema just to be helpful and to show that this is something we did consider and discuss? I think I'm with OP on this one. |
@jordanbtucker : Thanks for your question. It's a good one. Escaping is going to get ugly. The problem with:
Is that JSON parsers don't expect anything. JSON parsing is usually schema-less (one of the big benefits of JSON!). If I were a JSON5 parser author, I wouldn't know what to do with that line of the specification. What do you think about explicit type-tagging as proposed above:
Yes, it might "open the can of worms" of every other type like binary or regular expressions. But on the other hand, that's a decision you can make right now as a spec author. You can either decide to "let 100 flowers bloom" in which case JSON5 will be extensible, or you can decide to say that the key "_" (or some other clever identifier) is reserved for this use only, unless another use becomes urgent in the future. It is quite possible that some years in the future, having such an extension mechanism pre-reserved could allow some important differentiator between JSON and JSON5. But that's all a digression. My main point is that DATE is one of the basic types in every programming language, database and API and it remains a pretty big gap that JSON5 does not have an unambiguous way of handling it. Dragging in JSON Schema or something similarly complex to solve it seems like overkill. |
The term "parser" is intentionally left generic. It means anything that consumes JSON5 text. It does not necessarily mean a JSON5 parsing library like https://github.com/json5/json5. That being said, if a JSON5 parser library uses some form of schema, then it would be able to expect a date. That is the intention of that clause. JSON5 is in wide use, so making any breaking changes requires extra care, as it has the potential to disrupt all existing implementations. It also has the effect of causing fragmentation into versions. If first-class date handling is included in the JSON5 syntax, then we must define a new version of JSON5, and parsers and generators must know which version they are handling. Adding dates to the syntax, something that almost no other programming language or data interchange format has done, adds more complexity than it's worth. Dates are handled just fine by strings and schema. |
@prescod This was already commented on - same problem here, it's a breaking change: what if you actually want to encode that value? The essence of this argument is you can't change the meaning (semantics) of anything that could already exist in JSON data structures - doing so would be a breaking change.
I understand the desire for a date type (and, ideally, I'd love to have that) but any date encoding or convention that reuses existing JSON types is probably more or less out of the question, since it will inevitably change the meaning of an existing valid JSON expression. So I'm by no means disagreeing with what you said - there's definitely a gap here, and I would love it if we could think of a solution. I will mention I-JSON here - if the chosen approach was merely a recommendation, we could point to this. It deals not only with date/time (in the manner we've been discussing) but also with permitted number ranges, recommendations regarding binary data payloads, and a few other details. Again, those are only recommendations, so here's one idea for encoding dates and possibly other types of values in the future - template literal string syntax: {
"name": "Rasmus",
"birthday": $date`1975-07-07`
} This is JavaScript compatible in terms of syntax - you just need to have a template function
Of course, this is ES6 and not ES5 syntax, so this would somewhat change the mission statement of json5, though I don't imagine the world would care much if you can't copy/paste such an expression into an old version of IE. Also, the name would be a bit of a problem - we can't simply change it to json6, since there is already a json6 spec in the wild. (and arguably, this feature might be a better proposal to that standard - though, personally, I feel like that spec has already added far too many features and options; I would strongly prefer json5 with this addition over json6 with it's much higher complexity.) If you like the idea, I'd suggest supporting both Thoughts? |
Instead of |
@mindplay-dk, @prescod, @tracker1 the proposed, generic inline type syntax is not a breaking change, imo, in the sense that taking advantage of it would require the explicit specification of a "type parser map" to the In what concerns the mentioned "can of worms" that this would open, I disagree with this view. One way to see the worms is to map them to types. Users choose the worms that are allowed. Each allowed worm is added value to the user. That said, the "only" possible worm that I see with this approach is the management of type names. Standard type names should/could be reserved to match all of the existing JSON types and some future ones. The For completeness and regularity, a generic form could be defined for each standard type which would be semantically equivalent to its literal JSON value. Of course, the use of literals would always be preferred. As an example, in case I wasn't clear: {
"name": "Duarte"
} would be semantically equivalent to: {
"name": {"_": "string", "value": "Duarte"}
} With this generic mechanism in place, JSON5 could be easily extended to support other useful types that are slowly being introduced in JavaScript. With this generic mechanism in place, JSON5 could be easily used to serialize and de-serialize objects of any custom classes — without a supporting schema — armed only with the inline type annotations. Example with custom and standard types: {
"_": "customer",
"name": "Luciano Franzi",
"birthDate": {"_": "date", "value": "1976-06-26"},
"address": {
"_": "address",
"street": "St. John",
"door": 50,
"country": "UK"
}
} I've seen this pattern in use, throughout the years, in enterprise software. |
I'm closing this issue as it breaks the unofficially official "no new data types" rule. |
How many years would it take humanity to fix dates in json? |
This kind of reminds me of the Time Zones video by Tom Scott. |
That's a pity. |
@dcleao There is already a standard mechanism for defining data types. It's called JSON Schema. |
Reopening and renaming since the original issue proposed adding a recommendation for using an interoperable formatting for dates in strings. |
Okay, so returning to the original discussion, RFC 3339 is definitely the right choice in terms of interoperability - it's supported everywhere, sanctioned by the W3C, and so forth. The only thing I'm wondering is whether this covers real-world use-cases well enough for things like scheduling, planning, calendars, etc.? Given that it does not support time-zones:
DST is horribly complex, so it's hard to disagree with the conclusion here. But it seems like JSON5 has to prioritize interoperability and simplicity against the ability to represent time-zones, a commonly-used piece of data in many applications. Not sure, but maybe it's okay to forego support for time-zones as part of a recommendation? I mean, it is only a recommendation, so you are of course allowed to represent timestamps however you choose. But it does perhaps beg the question of whether this recommendation is going to be truly helpful? Given that, for a lot of well-known use-cases, the recommendation simply doesn't work. APIs already generally document the format they use for timestamps, and would need to do so either way, given that this is only a recommendation. Also, RFC 3339 timestamps are already a general W3C recommendation for anything web related, which implicitly includes JSON - it isn't limited to any particular protocol, encoding or format. So, is a recommendation going to have real value, or might it be redundant and distracting? |
@mindplay-dk Good points.
IMO, if you need real time zone information (not just the offset), just include it in its own property along with the date.
I think an important thing to remember is that JSON5 is JSON for humans, which means human readable dates are important. Many applications that communicate via JSON use Unix timestamps, and while that's great for machines, it's practically useless to humans. So the recommendation would help facilitate human readable and writable dates. |
Good answer, so let's put that in the spec? e.g. "date/time values should be represented as strings in RFC 3339 format", and "timezones may be included in a separate property from the date".
Good point, let's remove that whole section from your proposed text? I'm also not sure you should say anything about JSON 5 parsers or generators - keep it simple, stay on point: date/time values in RFC 3339 format is recommended in JSON5 files, done. We're already "above and beyond", since this is really not about the JSON5 format per se, but just a general recommendation on how to use it in practice, so I'd keep this as short and clear as possible. |
@mindplay-dk Good points. Maybe the JSON5 spec should have an "Authoring JSON5 Documents" section since JSON5 is meant for human-machine data transfer. |
I can see discussion is completed, but wanted to suggest an approach. I opened #34 asking support for the new ES Template’s strings. The new format add support for tagged strings. Assuming that issue 34 is added, than it can be extended to support the tagged string, e.g. date Parser will have the decision on how to represent tagged strings in their api. While certain languages have standard date objects, other do not have one accepted format. |
If we were going to add backtick-quoted strings, adding tagged strings would seem more reasonable. One small problem is then we're not strictly JSON5 anymore, but JSON6, which already exists. I'm not fond of JSON6, which adds far too many opinionated features and complexity - whereas I really appreciate the focus on simplicity with the JSON5 proposal, which doesn't go overboard on anything. (And JSON6 doesn't actually specify anything about dates in tagged strings, so...) 🤔 New standards for something like JSON don't come twice in a life-time, so I think it's worth entertaining the idea for a bit longer? If JSON5 were to someday proliferate and succeed JSON, it would definitely suck to go another 20 years without support for something as universal as dates. I would definitely still support the idea of just recommending a date format - but at the same time, it's impossible not to recognize the fact that it doesn't really solve the problem: a parser can't parse a JSON file and know whether a given string is a date or a string. It could guess, but at the risk of making a wrong guess, crashing some code that expects a string. Let's say we were to add support for tagged strings in general. How would we go about dealing with dates specifically? As I suggested earlier, I would propose reserving a const $date = value => new Date(value);
console.log({
"name": "Rasmus",
"birthday": $date`1975-07-07T23:20:50.52Z`
}); If this feature gets generalized to allow other function-names, anything without a In terms of integration with the JavaScript environment, const data = JSON.parse('{"birthday": mydate`1975-07-07T23:20:50.52Z`}', (key, value, reviver) =>
// key is "birthday"
// value is "1975-07-07T23:20:50.52Z"
// reviver is "mydate"
); Similarly, const json = JSON.stringify({ date: new Date() }, (key, value) => {
if (value instanceof Date) {
return { value, reviver: "mydate" }; // generates a tagged string
}
}); For the reserved const data = JSON.parse('{"birthday": $date`1975-07-07T23:20:50.52Z`}', (key, value, reviver) =>
// key is "birthday"
// value is a `Date` instance
// reviver is "$date"
);
const json = JSON.stringify({ date: new Date() }, (key, value) => {
if (value instanceof Date) {
return value; // generates a tagged string with the reserved $date name
}
}); Interestingly, the change to console.log(JSON.stringify({ date: new Date() })); // => '{"date":"2021-10-10T11:18:25.693Z"}' So on that side, it looks like a very natural fit. As for automatically reviving So even if we were to add support for tagged strings, reserving the name That's if we care about backwards compatibility with existing APIs at all, of course - an alternative would be to just add new Just spitballing here - any thoughts? |
Thanks for feedback. I was not aware that JSON6 exists. I think it’s worth posting the $date`` as a proposal there. I would suggest that it make sense to codify recommendation to use the $date, $time, and $datetime - non JS languages may have different types here. From a practical point if view, if json5 parser will adopt the tagged proposal, I think it should translate it to an object { $tag: “$date”, raw: “whatever” } with an option to try to apply the conversion at the parser level (create the date object). Different application may take different approach toward error handling, etc. This grammar can later be extended to include the ability fir parsed section, etc. Hope it make sense. I looked the json6 at the link. Can you highlight the complexities you refer to ? |
@yairlenga There's also a pet project of mine called JSONext. It's similar to JSON6 in that it brings ES5 and ES6 features to JSON, but it's not intended to be tied to one version of ES. It can evolve over time. @mindplay-dk Although I like your ideas about tagged templates for JSON-like formats that would allow for it (not JSON5), for the sake of keeping this thread on topic, I'll ask that you file an issue in a different repo if you want to continue discussion. |
Can we agree on adding a recommendation then? I would suggest: And perhaps, to be helpful, something along the lines of:
Let's get it done and let's close this discussion? No point in beating a dead horse. 😄 |
Please do not have the parser turn quoted text into anything other than a string! The application will convert the string to a date when it stores the data into it's own data structures and it has to do the conversion. It will also know it is (or is not) a date, there is absolutely no need to indicate the type in the string. |
@spitzak Thanks for your suggestions. I agree that converting date-like strings to dates would be a regression for JSON5. Keep in mind that this repo is for discussing the JSON5 specification, which deals the structure and semantics of the format, not the API. If we were to add verbiage around this topic, it would be to recommend that generators store dates as strings in RFC 3339 format to increase interoperability. Based on your comments, I think it might be wise to recommend that parsers do not automatically convert RFC 3339 strings into dates without providing some data contract or opt-in/opt-out functionality. |
Pertaining to the proposal, I would like to ask for more comments on time duration. Could that be specified in an interoperable way? |
@bogdan All of the classes in Temporal already have interoperable representations as strings, and those strings are returned by both |
The beauty of JSON is that it has remained intact after JSON 1.0. These enhancements with Dates and templates etc look cool and definitely help developers write better formatted files, but they also break a strong foundation of what JSON5 is about. I think enhancements should be closed, and the spec left as it is. That will make sure adoption for JSON5 will remain well around the tech industry. Follow the footsteps of JSON I say |
All you need is "It is recommended (but not required) that dates and times be represented as strings formatted per ISO 8601".
Then anyone who needs to build a library to handle dates and times can look up the standard and say "Oh, that's reasonable, I'll do that".
This is a known and persistent problem with existing solutions, and not using an existing solution seems unnecessarily obtuse.
The text was updated successfully, but these errors were encountered: