SICK: efficient deduplicating indexed binary JSON-like data structure

By Pavel Shirshov

Elevator Pitch

Fix application performance by switching to indexed JSON superset

Description

JSON is ubiquitous but might be a suboptimal solution for many problems. It has 2nd type grammar (in Chomsky terms) and can’t have an efficient partial and streaming parsers. Most times it’s being used with object mappers, which require further reduce efficiency. JSON-bound code might be hard to scale when datasets grow. The talk covers SICK, which is an efficient indexed binary superset of JSON. It allows just-in-time access, natively supports deduplication, doesn’t require parsing, and makes object mappers optional. SICK has O(1) amortized access complexity and is good for streaming and incremental streaming.