tailwindcss/packages/@tailwindcss-node/src/urls.ts

208 lines
5.9 KiB
TypeScript
Raw Normal View History

// Inlined version of code from Vite <https://github.com/vitejs/vite>
// Copyright (c) 2019-present, VoidZero Inc. and Vite contributors
// Released under the MIT License.
//
// Minor modifications have been made to work with the Tailwind CSS codebase
import * as path from 'node:path'
Generalize the `walk` implementation (#19126) This PR generalizes the `walk` implementations we have. What's important here is that we currently have multiple `walk` implementations, one for the AST, one for the `SelectorParser`, one for the `ValueParser`. Sometimes, we also need to go up the tree in a depth-first manner. For that, we have `walkDepth` implementations. The funny thing is, all these implementations are very very similar, even the kinds of trees are very similar. They are just objects with `nodes: []` as children. So this PR introduces a generic `walk` function that can work on all of these trees. There are also some situations where you need to go down and back up the tree. For this reason, we added an `enter` and `exit` phase: ```ts walk(ast, { enter(node, ctx) {}, exit(node, ctx) {}, }) ``` This means that you don't need to `walk(ast)` and later `walkDepth(ast)` in case you wanted to do something _after_ visiting all nodes. The API of these walk functions also slightly changed to fix some problems we've had before. One is the `replaceWith` function. You could technically call it multiple times, but that doesn't make sense so instead you always have to return an explicit `WalkAction`. The possibilities are: ```ts // The ones we already had WalkAction.Continue // Continue walking as normal, the default behavior WalkAction.Skip // Skip walking the `nodes` of the current node WalkAction.Stop // Stop the entire walk // The new ones WalkAction.Replace(newNode) // Replace the current node, and continue walking the new node(s) WalkAction.ReplaceSkip(newNode) // Replace the current node, but don't walk the new node(s) WalkAction.ReplaceStop(newNode) // Replace the current node, but stop the entire walk ``` To make sure that we can walk in both directions, and to make sure we have proper control over when to walk which nodes, the `walk` function is implemented in an iterative manner using a stack instead of recursion. This also means that a `WalkAction.Stop` or `WalkAction.ReplaceStop` will immediately stop the walk, without unwinding the entire call stack. Some notes: - The CSS AST does have `context` nodes, for this we can build up the context lazily when we need it. I added a `cssContext(ctx)` that gives you an enhanced context including the `context` object that you can read information from. - The second argument of the `walk` function can still be a normal function, which is equivalent to `{ enter: fn }`. Let's also take a look at some numbers. With this new implementation, each `walk` is roughly ~1.3-1.5x faster than before. If you look at the memory usage (especially in Bun) we go from `~2.2GB` peak memory usage, to `~300mb` peak memory usage. Some benchmarks on small and big trees (M1 Max): <img width="2062" height="1438" alt="image" src="https://github.com/user-attachments/assets/5ec8c22a-9de8-4e08-869a-18c0d30eb7e8" /> <img width="2062" height="1246" alt="image" src="https://github.com/user-attachments/assets/e89d4b8e-29ca-4aee-8fd2-b7c043d3bbf4" /> We also ran some benchmarks on @thecrypticace's M3 Max: <img width="1598" height="1452" alt="image" src="https://github.com/user-attachments/assets/3b06b6fe-2497-4f24-a428-1a0e2af3896a" /> In node the memory difference isn't that big, but the performance itself is still better: <img width="2034" height="1586" alt="image" src="https://github.com/user-attachments/assets/ef28ae14-b53e-4912-9621-531f3b02898f" /> In summary: 1. Single `walk` implementation for multiple use cases 2. Support for `enter` and `exit` phases 3. New `WalkAction` possibilities for better control 4. Overall better performance 5. ... and lower memory usage ## Test plan 1. All tests still pass (but had to adjust some of the APIs if `walk` was used inside tests). 2. Added new tests for the `walk` implementation 3. Ran local benchmarks to verify the performance improvements
2025-10-15 21:28:12 +02:00
import { toCss } from '../../tailwindcss/src/ast'
import { parse } from '../../tailwindcss/src/css-parser'
Generalize the `walk` implementation (#19126) This PR generalizes the `walk` implementations we have. What's important here is that we currently have multiple `walk` implementations, one for the AST, one for the `SelectorParser`, one for the `ValueParser`. Sometimes, we also need to go up the tree in a depth-first manner. For that, we have `walkDepth` implementations. The funny thing is, all these implementations are very very similar, even the kinds of trees are very similar. They are just objects with `nodes: []` as children. So this PR introduces a generic `walk` function that can work on all of these trees. There are also some situations where you need to go down and back up the tree. For this reason, we added an `enter` and `exit` phase: ```ts walk(ast, { enter(node, ctx) {}, exit(node, ctx) {}, }) ``` This means that you don't need to `walk(ast)` and later `walkDepth(ast)` in case you wanted to do something _after_ visiting all nodes. The API of these walk functions also slightly changed to fix some problems we've had before. One is the `replaceWith` function. You could technically call it multiple times, but that doesn't make sense so instead you always have to return an explicit `WalkAction`. The possibilities are: ```ts // The ones we already had WalkAction.Continue // Continue walking as normal, the default behavior WalkAction.Skip // Skip walking the `nodes` of the current node WalkAction.Stop // Stop the entire walk // The new ones WalkAction.Replace(newNode) // Replace the current node, and continue walking the new node(s) WalkAction.ReplaceSkip(newNode) // Replace the current node, but don't walk the new node(s) WalkAction.ReplaceStop(newNode) // Replace the current node, but stop the entire walk ``` To make sure that we can walk in both directions, and to make sure we have proper control over when to walk which nodes, the `walk` function is implemented in an iterative manner using a stack instead of recursion. This also means that a `WalkAction.Stop` or `WalkAction.ReplaceStop` will immediately stop the walk, without unwinding the entire call stack. Some notes: - The CSS AST does have `context` nodes, for this we can build up the context lazily when we need it. I added a `cssContext(ctx)` that gives you an enhanced context including the `context` object that you can read information from. - The second argument of the `walk` function can still be a normal function, which is equivalent to `{ enter: fn }`. Let's also take a look at some numbers. With this new implementation, each `walk` is roughly ~1.3-1.5x faster than before. If you look at the memory usage (especially in Bun) we go from `~2.2GB` peak memory usage, to `~300mb` peak memory usage. Some benchmarks on small and big trees (M1 Max): <img width="2062" height="1438" alt="image" src="https://github.com/user-attachments/assets/5ec8c22a-9de8-4e08-869a-18c0d30eb7e8" /> <img width="2062" height="1246" alt="image" src="https://github.com/user-attachments/assets/e89d4b8e-29ca-4aee-8fd2-b7c043d3bbf4" /> We also ran some benchmarks on @thecrypticace's M3 Max: <img width="1598" height="1452" alt="image" src="https://github.com/user-attachments/assets/3b06b6fe-2497-4f24-a428-1a0e2af3896a" /> In node the memory difference isn't that big, but the performance itself is still better: <img width="2034" height="1586" alt="image" src="https://github.com/user-attachments/assets/ef28ae14-b53e-4912-9621-531f3b02898f" /> In summary: 1. Single `walk` implementation for multiple use cases 2. Support for `enter` and `exit` phases 3. New `WalkAction` possibilities for better control 4. Overall better performance 5. ... and lower memory usage ## Test plan 1. All tests still pass (but had to adjust some of the APIs if `walk` was used inside tests). 2. Added new tests for the `walk` implementation 3. Ran local benchmarks to verify the performance improvements
2025-10-15 21:28:12 +02:00
import { walk } from '../../tailwindcss/src/walk'
import { normalizePath } from './normalize-path'
const cssUrlRE =
/(?<!@import\s+)(?<=^|[^\w\-\u0080-\uffff])url\((\s*('[^']+'|"[^"]+")\s*|[^'")]+)\)/
const cssImageSetRE = /(?<=image-set\()((?:[\w-]{1,256}\([^)]*\)|[^)])*)(?=\))/
const cssNotProcessedRE = /(?:gradient|element|cross-fade|image)\(/
const dataUrlRE = /^\s*data:/i
const externalRE = /^([a-z]+:)?\/\//
const functionCallRE = /^[A-Z_][.\w-]*\(/i
const imageCandidateRE =
/(?:^|\s)(?<url>[\w-]+\([^)]*\)|"[^"]*"|'[^']*'|[^,]\S*[^,])\s*(?:\s(?<descriptor>\w[^,]+))?(?:,|$)/g
const nonEscapedDoubleQuoteRE = /(?<!\\)"/g
const escapedSpaceCharactersRE = /(?: |\\t|\\n|\\f|\\r)+/g
const isDataUrl = (url: string): boolean => dataUrlRE.test(url)
const isExternalUrl = (url: string): boolean => externalRE.test(url)
type CssUrlReplacer = (url: string, importer?: string) => string | Promise<string>
interface ImageCandidate {
url: string
descriptor: string
}
export async function rewriteUrls({
css,
base,
root,
}: {
css: string
base: string
root: string
}) {
if (!css.includes('url(') && !css.includes('image-set(')) {
return css
}
let ast = parse(css)
let promises: Promise<void>[] = []
function replacerForDeclaration(url: string) {
if (url[0] === '/') return url
let absoluteUrl = path.posix.join(normalizePath(base), url)
let relativeUrl = path.posix.relative(normalizePath(root), absoluteUrl)
// If the path points to a file in the same directory, `path.relative` will
// remove the leading `./` and we need to add it back in order to still
// consider the path relative
if (!relativeUrl.startsWith('.')) {
relativeUrl = './' + relativeUrl
}
return relativeUrl
}
walk(ast, (node) => {
if (node.kind !== 'declaration') return
if (!node.value) return
let isCssUrl = cssUrlRE.test(node.value)
let isCssImageSet = cssImageSetRE.test(node.value)
if (isCssUrl || isCssImageSet) {
let rewriterToUse = isCssImageSet ? rewriteCssImageSet : rewriteCssUrls
promises.push(
rewriterToUse(node.value, replacerForDeclaration).then((url) => {
node.value = url
}),
)
}
})
if (promises.length) {
await Promise.all(promises)
}
return toCss(ast)
}
function rewriteCssUrls(css: string, replacer: CssUrlReplacer): Promise<string> {
return asyncReplace(css, cssUrlRE, async (match) => {
const [matched, rawUrl] = match
return await doUrlReplace(rawUrl.trim(), matched, replacer)
})
}
async function rewriteCssImageSet(css: string, replacer: CssUrlReplacer): Promise<string> {
return await asyncReplace(css, cssImageSetRE, async (match) => {
const [, rawUrl] = match
const url = await processSrcSet(rawUrl, async ({ url }) => {
// the url maybe url(...)
if (cssUrlRE.test(url)) {
return await rewriteCssUrls(url, replacer)
}
if (!cssNotProcessedRE.test(url)) {
return await doUrlReplace(url, url, replacer)
}
return url
})
return url
})
}
async function doUrlReplace(
rawUrl: string,
matched: string,
replacer: CssUrlReplacer,
funcName: string = 'url',
) {
let wrap = ''
const first = rawUrl[0]
if (first === `"` || first === `'`) {
wrap = first
rawUrl = rawUrl.slice(1, -1)
}
if (skipUrlReplacer(rawUrl)) {
return matched
}
let newUrl = await replacer(rawUrl)
// The new url might need wrapping even if the original did not have it, e.g. if a space was added during replacement
if (wrap === '' && newUrl !== encodeURI(newUrl)) {
wrap = '"'
}
// If wrapping in single quotes and newUrl also contains single quotes, switch to double quotes.
// Give preference to double quotes since SVG inlining converts double quotes to single quotes.
if (wrap === "'" && newUrl.includes("'")) {
wrap = '"'
}
// Escape double quotes if they exist (they also tend to be rarer than single quotes)
if (wrap === '"' && newUrl.includes('"')) {
newUrl = newUrl.replace(nonEscapedDoubleQuoteRE, '\\"')
}
return `${funcName}(${wrap}${newUrl}${wrap})`
}
Vite: Don't rebase urls that appear to be aliases (#16078) Closes #16039 This PR changes our URL rebasing logic used with Vite so that it does not rebase URLs that look like common alias paths (e.g. urls starting in `~`, `@` or `#`, etc.). Unfortunately this is only an approximation and you can configure an alias for a path that starts with a regular alphabetical character (e.g. `foo` => `./my/foo`) so this isn't a perfect fix, however in practice most aliases will be prefixed with a symbol to make it clear that it's an alias anyways. One alternative we have considered is to only rebase URLs that we know are relative (so they need to start with a `.`). This, however, will break common CSS use cases where urls are loaded like this: ```css background: image-set( url('image1.jpg') 1x, url('image2.jpg') 2x ); ``` So making this change felt like we only trade one GitHub issue for another one. In a more ideal scenario we try to resolve the URL with the Vite resolver (we have to run the resolver and can't rely on the `resolve` setting alone due to packages like [`vite-tsconfig-paths`](https://www.npmjs.com/package/vite-tsconfig-paths)), however even then we can have relative paths being resolvable to different files based on wether they were rebased or not (e.g. when an image with the same filename exists in two different paths). So ultimately we settled on extending the already existing blocklist (which we have taken from the Vite implementation) for now. ## Test plan - Added unit test and it was tested with the Vite playground. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2025-01-31 15:23:32 +01:00
function skipUrlReplacer(rawUrl: string, aliases?: string[]) {
return (
Vite: Don't rebase urls that appear to be aliases (#16078) Closes #16039 This PR changes our URL rebasing logic used with Vite so that it does not rebase URLs that look like common alias paths (e.g. urls starting in `~`, `@` or `#`, etc.). Unfortunately this is only an approximation and you can configure an alias for a path that starts with a regular alphabetical character (e.g. `foo` => `./my/foo`) so this isn't a perfect fix, however in practice most aliases will be prefixed with a symbol to make it clear that it's an alias anyways. One alternative we have considered is to only rebase URLs that we know are relative (so they need to start with a `.`). This, however, will break common CSS use cases where urls are loaded like this: ```css background: image-set( url('image1.jpg') 1x, url('image2.jpg') 2x ); ``` So making this change felt like we only trade one GitHub issue for another one. In a more ideal scenario we try to resolve the URL with the Vite resolver (we have to run the resolver and can't rely on the `resolve` setting alone due to packages like [`vite-tsconfig-paths`](https://www.npmjs.com/package/vite-tsconfig-paths)), however even then we can have relative paths being resolvable to different files based on wether they were rebased or not (e.g. when an image with the same filename exists in two different paths). So ultimately we settled on extending the already existing blocklist (which we have taken from the Vite implementation) for now. ## Test plan - Added unit test and it was tested with the Vite playground. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2025-01-31 15:23:32 +01:00
isExternalUrl(rawUrl) ||
isDataUrl(rawUrl) ||
!rawUrl[0].match(/[\.a-zA-Z0-9_]/) ||
functionCallRE.test(rawUrl)
)
}
function processSrcSet(
srcs: string,
replacer: (arg: ImageCandidate) => Promise<string>,
): Promise<string> {
return Promise.all(
parseSrcset(srcs).map(async ({ url, descriptor }) => ({
url: await replacer({ url, descriptor }),
descriptor,
})),
).then(joinSrcset)
}
function parseSrcset(string: string): ImageCandidate[] {
const matches = string
.trim()
.replace(escapedSpaceCharactersRE, ' ')
.replace(/\r?\n/, '')
.replace(/,\s+/, ', ')
.replaceAll(/\s+/g, ' ')
.matchAll(imageCandidateRE)
return Array.from(matches, ({ groups }) => ({
url: groups?.url?.trim() ?? '',
descriptor: groups?.descriptor?.trim() ?? '',
})).filter(({ url }) => !!url)
}
function joinSrcset(ret: ImageCandidate[]) {
return ret.map(({ url, descriptor }) => url + (descriptor ? ` ${descriptor}` : '')).join(', ')
}
async function asyncReplace(
input: string,
re: RegExp,
replacer: (match: RegExpExecArray) => string | Promise<string>,
): Promise<string> {
let match: RegExpExecArray | null
let remaining = input
let rewritten = ''
while ((match = re.exec(remaining))) {
rewritten += remaining.slice(0, match.index)
rewritten += await replacer(match)
remaining = remaining.slice(match.index + match[0].length)
}
rewritten += remaining
return rewritten
}