JSON Formatting at Scale: Handling 100MB+ Files Without Crashing Your Browser

Last month I needed to inspect an API response dump from a data migration. The file was 140 MB of minified JSON — one long line, no whitespace, completely unreadable in any text editor. I opened it in three different online formatters. Two crashed immediately. The third loaded for about 90 seconds and then froze the tab.

Eventually I used jq. It formatted the file in under two seconds and I was looking at the structure I needed. I've been reaching for command-line tools for large JSON ever since.

Here's what's actually happening when those browser tools fail — and how to handle it properly.

---

Why browser tools fail on large files

The problem isn't the tool's quality — it's the architecture.

When a browser-based JSON formatter processes your file, here's what it does:

1. Load the raw text into a JavaScript string (~100 MB stays ~100 MB here)
2. Parse using JSON.parse() — this converts the string into a JavaScript object tree. A 100 MB JSON file can expand to 300–500 MB in memory at this step, because JavaScript objects carry significant overhead per key/value pair compared to raw text.
3. Stringify the object back to a formatted string with JSON.stringify(null, 2) — another copy in memory
4. Render the formatted text as DOM nodes with syntax highlighting — another multiplication of memory

By step 4, a 100 MB file may have consumed 600–800 MB of browser tab memory. Chrome's per-tab V8 heap cap is typically 1.5–2 GB, so files in the 50–150 MB range are in the danger zone. Add a MacBook with other tabs open, and the tab either crashes or the browser kills it.

The fundamental issue: this approach requires the entire file to be in memory simultaneously in multiple representations. There's no streaming — the browser waits for the full parse before it can do anything.

For files below about 10 MB, this is fine. For anything larger, you need a different approach.

---

jq: the right tool for the job

jq{:target="_blank"} is a command-line JSON processor. It's fast, streams data where possible, and handles arbitrarily large files for most operations.

Installation

# macOS
brew install jq

# Ubuntu/Debian
sudo apt-get install jq

# Windows (winget)
winget install jqlang.jq

Pretty-print a large file

jq . large-file.json

That's it. The . is jq's identity filter — output what came in, formatted. On a 140 MB minified JSON file, this runs in 1–3 seconds and outputs properly indented JSON to stdout.

To write the output to a new file:

jq . large-file.json > large-file-formatted.json

Inspect the structure first

Before formatting a 500 MB file, it helps to know what's in it. To see just the top-level keys:

jq 'keys' large-file.json

To see the first element of an array (if the root is an array):

jq '.[0]' large-file.json

To count items in an array:

jq 'length' large-file.json

Extract a specific field from every object

If your JSON is an array of objects and you only care about one field:

jq '.[].email' large-file.json

This is dramatically more efficient than loading the whole thing in a browser — jq can extract a field from a 1 GB file using a fraction of the RAM.

Minify (the reverse problem)

If you need to compress formatted JSON back to a single line for transmission:

jq -c . formatted-file.json

The -c flag means compact output — no whitespace.

---

Handling NDJSON (newline-delimited JSON)

A lot of large JSON datasets aren't a single giant object or array. They're NDJSON: one JSON object per line. Log files, database exports, and streaming API outputs commonly use this format.

NDJSON looks like:

{"id":1,"name":"Alice","email":"alice@example.com"}
{"id":2,"name":"Bob","email":"bob@example.com"}
{"id":3,"name":"Carol","email":"carol@example.com"}

Standard JSON.parse() and most online formatters will reject this as invalid JSON. jq handles it natively with the --slurp-raw or --raw-input flags, but the cleaner approach is:

# Process each line as a separate JSON object
jq -R 'fromjson?' large-file.ndjson

# Pretty-print all objects
jq -R 'fromjson? | .' large-file.ndjson

# Extract a field from every line
jq -Rc 'fromjson? | .email' large-file.ndjson

The ? suppresses errors on lines that aren't valid JSON — useful if the file has a header row or occasional blank lines.

---

Node.js: streaming for programmatic use

If you're processing large JSON files in a Node.js application (not just inspecting from the command line), loading the whole file with JSON.parse(fs.readFileSync(...)) will eventually cause memory problems at scale.

For NDJSON, streaming line-by-line is straightforward:

const fs = require('fs')
const readline = require('readline')

const rl = readline.createInterface({
  input: fs.createReadStream('large-file.ndjson'),
  crlfDelay: Infinity,
})

rl.on('line', (line) => {
  if (!line.trim()) return
  const obj = JSON.parse(line)
  // process each object here
  console.log(obj.email)
})

rl.on('close', () => {
  console.log('Done')
})

This reads the file one line at a time, keeping memory usage flat regardless of file size.

For standard JSON arrays too large to fit in memory, the stream-json npm package provides a streaming parser:

const { parser } = require('stream-json')
const { streamArray } = require('stream-json/streamers/StreamArray')
const fs = require('fs')

fs.createReadStream('large-array.json')
  .pipe(parser())
  .pipe(streamArray())
  .on('data', ({ key, value }) => {
    // value is one element from the top-level array
    console.log(value.id)
  })

This approach handles arrays that are gigabytes in size with constant memory usage.

---

Python: ijson for large structured files

Python's built-in json.load() has the same problem as JSON.parse() — it loads the whole file into memory. For large files, ijson provides a streaming parser:

import ijson

with open('large-file.json', 'rb') as f:
    # iterate over top-level array items
    for item in ijson.items(f, 'item'):
        print(item['email'])

The 'item' prefix tells ijson to yield each element of the top-level array one at a time. Memory usage stays constant. Install with pip install ijson.

For a quick pretty-print from the command line, Python's built-in module is faster to type than installing jq:

python3 -m json.tool large-file.json

This validates and formats the file. It's slower than jq and still loads the full file into memory, but it's available wherever Python is and requires no installation.

---

Choosing the right tool

File size	Situation	Tool
Under 10 MB	Quick inspection	Online formatter
10–100 MB	Inspect, format, extract fields	jq
100 MB+	Format or extract	jq (it handles this fine)
Any size NDJSON	Process line-by-line	jq -R 'fromjson?'
Programmatic, Node.js	Stream large arrays	stream-json
Programmatic, Python	Stream large arrays	ijson
Quick format, no install	Small–medium	python3 -m json.tool

For comparison of JSON with other config formats and when each is appropriate, JSON vs YAML vs TOML: which config format should you use? covers the trade-offs.

---

A word on validation

Formatting and validation are different operations. A formatter makes JSON readable. A validator confirms the JSON is syntactically correct.

jq validates as a side effect — if your JSON is malformed, jq will report the parse error and exit. This is often the most useful first step with a large file you can't inspect directly:

jq empty large-file.json && echo "Valid JSON" || echo "Invalid JSON"

jq empty parses the file and produces no output — it just succeeds or fails. Useful for CI pipelines or pre-processing checks.

---

---

FAQ

Why do browser JSON formatters crash on large files?

They load the entire file into memory, parse it into a JavaScript object tree, then re-serialise it as formatted HTML. A 100 MB JSON file expands to 300–500 MB in memory during parsing due to JavaScript object overhead, and syntax-highlighted DOM rendering adds more. Chrome's per-tab memory limits put files over 50 MB in the crash zone. It's a fundamental architectural constraint of in-browser processing — not a bug.

What is jq and how do I install it?

jq is a command-line JSON processor — lightweight, fast, and handles files of any size. Install on macOS with brew install jq, on Ubuntu with sudo apt-get install jq, on Windows with winget install jqlang.jq. The basic command is jq . file.json to pretty-print.

Can jq handle files over 1 GB?

For most operations (pretty-printing, field extraction, filtering), yes — jq processes files incrementally and uses far less memory than browser tools. Some transformations that require sorting the full dataset do load everything into memory, but basic formatting and extraction work well into the gigabytes.

What is NDJSON and how is it different from regular JSON?

NDJSON (Newline-Delimited JSON, also called JSON Lines) has one complete JSON object per line rather than one large root object or array. It's common in log files, database exports, and streaming APIs. Standard JSON parsers reject it; use jq -R 'fromjson?' to process it line-by-line.

Is there a size limit on MyEasyTools JSON formatter?

The free tier handles files up to 10 MB reliably — above that, browser memory constraints cause slow processing or crashes on older devices. This is an honest architectural limit, not a policy one. For files over 10 MB, the command-line tools in this post (jq, Python, Node.js) are faster and more reliable.