Hacker News

points by modulovalue 3 days ago | 0 comments

I'm working on better SIMD support in Dart and one of the motivations for that work is specializing parser specifications based on schema specifications of that language.

Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.

That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!

I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.

I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.

[–] lightandlight 3 days ago

This approach is called "staged parsing":

https://dl.acm.org/doi/pdf/10.1145/2714064.2660241

https://dl.acm.org/doi/pdf/10.1145/3314221.3314625

[–] weitendorf 3 days ago

I was working on metaparsing earlier this year (specifically over grpc because of its better support for types and better serialization format).

I think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.

A grpc-like protocol with more prescriptive type registries/discovery (over the network) and better interop with http (like grpc-gateway) would be the best approach IMO. Definitely not JSON schema

[–] tcfhgj 3 days ago

> think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.

I have been using serde without issues, and never resorted to `serde::Value`[0], the untyped option, so far.

I have yet to experience anything working against me.

[0]: https://docs.rs/serde_json/latest/serde_json/enum.Value.html

[–] thayne 3 days ago

> I think most people who use JSON don't want strong typing or to use schemas.

Not necessarily. It's biggest advantage is probably that it has very good interoperability. Basically every language has a readily accessible library for json, and even if you don't, it isn't that hard to write a parser and/or generator for it. And being human readable isn't something to sneeze at either.

[–] ahtihn 3 days ago

> I think most people who use JSON don't want strong typing or to use schemas.

We use JSON because it's standard, human-readable, mostly self-described and works natively with Javascript.

You can read JSON object without having to know the schema even if one exists.

Protobuf is unusablr garbage without a lot of supporting tooling.

[–] tclancy 3 days ago

I think it may be fairer to say, “most people pick up JSON because of the flexibility.” Those who continue to use it for anything where there’s a consumer of your API probably do come to want types. In my day job I am starting to enforce end-to-end typing as best I can (given it’s Python and JSON) because our payloads are partially content from agents and I want to be able to flag up weird shit if we can. Certainly not the only gate that we have for looking for prompt injection and not even a great one, but it provides some piece of mind.

[–] gwbas1c 3 days ago

> I think most people who use JSON don't want strong typing or to use schemas.

Pretty much anywhere I've come across JSON (and xml before that,) everyone expected strong typing. The fact that JSON (and xml) can represent different data than they anticipated "did not compute" in their heads.

In many cases, they were treating JSON (and xml) as dumb serializers and not even paying attention to the output and corner cases.

[–] dankobgd 3 days ago

I definitely want jsonschema, it's a standard and i can use it for many tools. openapi spec is also a standard and built on top of jsonschema. I can use jsonschema to generate models, types, servers, client sdks, automate forms etc. I think it's better than grpc for some of these things, plus protobufs and grpc have bunch of tooling that i can pretend to understand but in depth i don't really.