From d7fc281a0e678bf92f0e82f4ab1b8edfd7cb1675 Mon Sep 17 00:00:00 2001 From: Jordan Pittman Date: Thu, 26 Mar 2026 18:42:03 -0400 Subject: [PATCH] Use `json` preprocessor for new-line delimited JSON files (#19862) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary This specializes the `.jsonl` and `.ndjson` file extensions so they're preprocessed like JSON instead of by the standard scanner. This prevents them from creating thousands of sub machines and reduces scanning time (see #17125 where this was done for `.json` files). It seems reasonable to handle new-line delimited JSON files as well otherwise scanning these files can take quite a long time. It's quite unlikely that these will contain classes so, alternatively, these *could* go in the binary extensions list so they get ignored entirely. ## Test plan I ran manual tests inside the `oxide` crate against some large-ish JSONL files (5MB–15MB). These changes bring down scanning time from 2s–3s on my M3 Max (via `cargo test --release …`) to less than 20ms. I also ran tests through a full CLI build pipeline on a low-spec linux box. This change brought scanning time down from ~90s to ~300ms for a single ~15MB file. --- CHANGELOG.md | 1 + crates/oxide/src/scanner/mod.rs | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 89c462fc1..cdebce6d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,6 +32,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Canonicalization: migrate `end-full` → `inset-e-full`, `end-auto` → `inset-e-auto`, `end-px` → `inset-e-px`, and `end-` → `inset-e-` as well as negative versions ([#19849](https://github.com/tailwindlabs/tailwindcss/pull/19849)) - Canonicalization: move the `-` sign inside the arbitrary value `-left-[9rem]` → `left-[-9rem]` ([#19858](https://github.com/tailwindlabs/tailwindcss/pull/19858)) - Canonicalization: move the `-` sign outside the arbitrary value `ml-[calc(-1*var(--width))]` → `-ml-(--width)` ([#19858](https://github.com/tailwindlabs/tailwindcss/pull/19858)) +- Improve performance when scanning JSONL / NDJSON files ([#19862](https://github.com/tailwindlabs/tailwindcss/pull/19862)) ## [4.2.2] - 2026-03-18 diff --git a/crates/oxide/src/scanner/mod.rs b/crates/oxide/src/scanner/mod.rs index 6d6cabc2a..30724f162 100644 --- a/crates/oxide/src/scanner/mod.rs +++ b/crates/oxide/src/scanner/mod.rs @@ -474,7 +474,7 @@ pub fn pre_process_input(content: Vec, extension: &str) -> Vec { "heex" | "eex" | "ex" | "exs" => Elixir.process(&content), "cshtml" | "razor" => Razor.process(&content), "haml" => Haml.process(&content), - "json" => Json.process(&content), + "json" | "jsonl" | "ndjson" => Json.process(&content), "md" | "mdx" => Markdown.process(&content), "pug" => Pug.process(&content), "rb" | "erb" => Ruby.process(&content),