Data engineering and analytics, Slides of Computer science

Download Data engineering and analytics and more Slides Computer science in PDF only on Docsity!

TOON

TOONTOON

Brij Kishore Pandey @brijpandeyji

COMPREHENSIVE GUIDE

Token-Oriented

Object Notation

VS

JSON

JSONJSON

This document provides a comprehensive comparison

between JSON (JavaScript Object Notation) and TOON

(Token-Oriented Object Notation). As LLMs evolve into

agentic systems, the need for token-efficient, model-

friendly data formats increases. TOON is designed

specifically to address the limitations of JSON when

used inside LLM prompts, agent memory, RAG

workflows, and multi-agent orchestration.

Introduction

TOON (Token-Oriented Object Notation) is a new serialization format designed specifically for Large Language Models. It reduces token consumption by minimizing syntax noise and removing repetitive keys. It maintains full compatibility with JSON and can be converted back and forth losslessly. TOON excels in linear, uniform, tabular, or semi-structured data used in prompts. What is TOON?

Why TOON Was Created Reduce token cost for LLM inputs Improve LLM parsing accuracy by removing syntax clutter Optimize agent looping, tool output, and memory blocks Maintain JSON equivalence without complexity Provide a human- and machine-readable structure

Side-by-Side Comparison (Basic Object) JSON JSONJSON TOON TOONTOON

Nested Structure Comparison JSON JSONJSON TOON TOONTOON

Mixed Arrays JSON JSONJSON TOON TOONTOON

Complex Table (Telemetry, Logs, Events) JSON JSONJSON TOON TOONTOON

RAG Retrieval Example JSON JSONJSON TOON TOONTOON

Limitations of TOON

Performs best with

uniform data

Not intended as a

storage format

For deeply nested

objects, token savings

decrease

No native schema

language (JSON

Schema still used)

When To Use JSON vs TOON When To Use When To Use JSON JSONJSON TOON TOONTOON API communication Passing structured data to LLMs Tool I/O, agent memory, RAG chunks Logs, telemetry, catalogs, events Reducing token cost is a priority Complex nested structures Validation, schemas, storage

Conclusion JSON remains the universal standard for structured data, but it is not optimized for LLM interaction. TOON provides a token-efficient, LLM-friendly alternative that preserves structure while dramatically