tailwindcss/integrations/utils.ts

872 lines
28 KiB
TypeScript
Raw Normal View History

Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
import dedent from 'dedent'
import fastGlob from 'fast-glob'
import { exec, execFile, spawn, type ChildProcess } from 'node:child_process'
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
import fs from 'node:fs/promises'
import { createServer } from 'node:net'
import { platform, tmpdir } from 'node:os'
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
import path from 'node:path'
import { promisify, stripVTControlCharacters } from 'node:util'
Add support for source maps (#17775) Closes #13694 Closes #13591 # Source Maps Support for Tailwind CSS This PR adds support for source maps to Tailwind CSS v4 allowing us to track where styles come from whether that be user CSS, imported stylesheets, or generated utilities. This will improve debuggability in browser dev tools and gives us a good foundation for producing better error messages. I'll go over the details on how end users can enable source maps, any limitations in our implementation, changes to the internal `compile(…)` API, and some details and reasoning around the implementation we chose. ## Usage ### CLI Source maps can be enabled in the CLI by using the command line argument `--map` which will generate an inline source map comment at the bottom of your CSS. A separate file may be generated by passing a file name to `--map`: ```bash # Generates an inline source map npx tailwindcss -i input.css -o output.css --map # Generates a separate source map file npx tailwindcss -i input.css -o output.css --map output.css.map ``` ### PostCSS Source maps are supported when using Tailwind as a PostCSS plugin *in development mode only*. They may or may not be enabled by default depending on your build tool. If they are not you may be able to configure them within your PostCSS config: ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "@tailwindcss/postcss": {}, }, } } ``` ### Vite Source maps are supported when using the Tailwind CSS Vite plugin in *development mode only* by enabling the `css.devSourcemap` setting: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { devSourcemap: true, }, }) ``` Now when a CSS file is requested by the browser it'll have an inline source map comment that the browser can use. ## Limitations - Production build source maps are currently disabled due to a bug in Lightning CSS. See https://github.com/parcel-bundler/lightningcss/pull/971 for more details. - In Vite, minified CSS build source maps are not supported at all. See https://github.com/vitejs/vite/issues/2830 for more details. - In PostCSS, minified CSS source maps are not supported. This is due to the complexity required around re-associating every AST node with a location in the generated, optimized CSS. This complexity would also have a non-trivial performance impact. ## Testing Here's how to test the source map functionality in different environments: ### Testing the CLI 1. Setup typical project that the CLI can use and with sources to scan. ```css @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 2. Build with source maps: ```bash bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map ``` 3. Open Chrome DevTools, inspect an element with utility classes, and you should see rules pointing to `input.css` or `node_modules/tailwindcss/index.css` ### Testing with Vite Testing in Vite will require building and installing necessary files under `dist/*.tgz`. 1. Create a Vite project and enable source maps in `vite.config.js`: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { // This line is required for them to work devSourcemap: true, }, }) ``` 2. Add a component that uses Tailwind classes and custom CSS: ```jsx // ./src/app.jsx export default function App() { return ( <div className="bg-blue-500 my-custom-class"> Hello World </div> ) } ``` ```css /* ./src/styles.css */ @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 3. Run `npm run dev`, open DevTools, and inspect elements to verify source mapping works for both utility classes and custom CSS. ### Testing with PostCSS CLI 1. Create a test file and update your PostCSS config: ```css /* input.css */ @import "tailwindcss"; @layer components { .card { @apply p-6 rounded-lg shadow-lg; } } ``` ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {} } } } ``` 2. Run PostCSS through Bun: ```bash bunx --bun postcss ./src/index.css -o out.css ``` 3. Inspect the output CSS - it should include an inline source map comment at the bottom. ### Testing with PostCSS + Next.js Testing in Next.js will require building and installing necessary files under `dist/*.tgz`. However, I've not been able to get CSS source maps to work in Next.js without this hack: ```js const nextConfig: NextConfig = { // next.js overwrites config.devtool so we prevent it from doing so // please don't actually do this… webpack: (config) => Object.defineProperty(config, "devtool", { get: () => "inline-source-map", set: () => {}, }), }; ``` This is definitely not supported and also doesn't work with turbopack. This can be used to test them temporarily but I suspect that they just don't work there. ### Manual source map analysis You can analyze source maps using Evan Wallace's [Source Map Visualization](https://evanw.github.io/source-map-visualization/) tool which will help to verify the accuracy and quality of source maps. This is what I used extensively while developing this implementation. It'll help verify that custom, user CSS maps back to itself in the input, that generated utilities all map back to `@tailwind utilities;`, that source locations from imported files are also handled correctly, etc… It also highlights the ranges of stuff so it's easy to see if there are off-by-one errors. It's easiest to use inline source maps with this tool because you can take the CSS file and drop it on the page and it'll analyze it while showing the file content. If you're using Vite you'll want to access the CSS file with `?direct` at the end so you don't get a JS module back. ## Implementation The source map implementation follows the ECMA-426 specification and includes several key components to aid in that goal: ### Source Location Tracking Each emittable AST node in the compilation pipeline tracks two types of source locations: - `src`: Original source location - [source file, start offset, end offset] - `dst`: Generated source location - [output file, start offset, end offset] This dual tracking allows us to maintain mappings between the original source and generated output for things like user CSS, generated utilities, uses of `@apply`, and tracking theme variables. It is important to note that source locations for nodes _never overlap_ within a file which helps simplify source map generation. As such each type of node tracks a specific piece of itself rather than its entire "block": | Node | What a `SourceLocation` represents | | ----------- | ---------------------------------------------------------------- | | Style Rule | The selector | | At Rule | Rule name and params, includes the `@` | | Declaration | Property name and value, excludes the semicolon | | Comment | The entire comment, includes the start `/*` and end `*/` markers | ### Windows line endings when parsing CSS Because our AST tracks nodes through offsets we must ensure that any mutations to the file do *not* change the lenth of the string. We were previously replacing `\r\n` with `\n` (see [filter code points](https://drafts.csswg.org/css-syntax/#css-filter-code-points) from the spec) — which changes the length of the string and all offsets may end up incorrect. The CSS parser was updated to handle the CRLF token directly by skipping over the `\r` and letting remaining code handle `\n` as it did previously. Some additional tweaks were required when "peeking" the input but those changes were fairly small. ### Tracking of imports Source maps need paths to the actual imported stylesheets but the resolve step for stylesheets happens inside the call to `loadStylesheet` which make the file path unavailable to us. Because of this the `loadStylesheet` API was augmented such that it has to return a `path` property that we can then use to identify imported sources. I've also made the same change to the `loadModule` API for consistency but nothing currently uses this property. The `path` property likely makes `base` redundant but elminating that (if we even want to) is a future task. ### Optimizing the AST Our optimization pass may intoduce some nodes, for example, fallbacks we create for `@property`. These nodes are linked back to `@tailwind utilities` as ultimately that is what is responsible for creating them. ### Line Offset Tables A key component to our source map generation is the line offset table, which was inspired by some ESBuild internals. It stores a sorted list of offsets for the start of each line allowing us to translate offsets to line/column `Position`s in `O(log N)` time and from `Position`s to offsets in `O(1)` time. Creation of the table takes `O(N)` time. This means that we can store code point offsets for source locations and not have to worry about computing or tracking line/column numbers during parsing and serialization. Only when a source map is generated do these offsets need to be computed. This ensures the performance penalty when not using source maps is minimal. ### Source Map Generation The source map returned by `buildSourceMap()` is designed to follow the [ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not completely finalized we consider the result of `buildSourceMap()` to be internal API that may change as the spec chamges. The produces source map is a "decoded" map such that all sources and mappings are in an object graph. A library like `source-map-js` must be used to convert this to an encoded source map of the right version where mappings are encoded with base 64 VLQs. Any specific integration (Vite, PostCSS, etc…) can then use `toSourceMap()` from `@tailwindcss/node` to convert from the internal source map to an spec-compliant encoded source map that can be understood by other tools. ### Handling minification in Lightning Since we use Lightning CSS for optimization, and it takes in an input map, we generate an encoded source map that we then pass to lightning. The output source map *from lighting itself* is then passed back in during the second optimization pass. The final map is then passed from lightning to the CLI (but not Vite or PostCSS — see the limitations section for details). In some cases we have to "fix up" the output CSS. When this happens we use `magic-string` to do the replacement in a way that is trackable and `@amppproject/remapping` to map that change back onto the original source map. Once the need for these fix ups disappear these dependencies can go away. Notes: - The accuracy of source maps run though lightning is reduced as it only tracks on a per-rule level. This is sufficient enough for browser dev tools so should be fine. - Source maps during optimization do not function properly at this time because of a bug in Lightning CSS regarding license comments. Once this bug is fixed they will start working as expected. ### How source locations flow through the system 1. During initial CSS parsing, source locations are preserved. 2. During parsing these source locations are also mapped to the destinations which supports an optimization for when no utilities are generated. 3. Throughout the compilation process, transformations maintain source location data 4. Generated utilities are explicitly pointed to `@tailwind utilities` unless generated by `@apply`. 5. When optimization is enabled, source maps are remapped through lightningcss 6. Final source maps are written in the requested format (inline or separate file)
2025-05-08 16:29:49 -04:00
import { RawSourceMap, SourceMapConsumer } from 'source-map-js'
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
import { test as defaultTest, type ExpectStatic } from 'vitest'
import * as Yaml from 'yaml'
Add support for source maps (#17775) Closes #13694 Closes #13591 # Source Maps Support for Tailwind CSS This PR adds support for source maps to Tailwind CSS v4 allowing us to track where styles come from whether that be user CSS, imported stylesheets, or generated utilities. This will improve debuggability in browser dev tools and gives us a good foundation for producing better error messages. I'll go over the details on how end users can enable source maps, any limitations in our implementation, changes to the internal `compile(…)` API, and some details and reasoning around the implementation we chose. ## Usage ### CLI Source maps can be enabled in the CLI by using the command line argument `--map` which will generate an inline source map comment at the bottom of your CSS. A separate file may be generated by passing a file name to `--map`: ```bash # Generates an inline source map npx tailwindcss -i input.css -o output.css --map # Generates a separate source map file npx tailwindcss -i input.css -o output.css --map output.css.map ``` ### PostCSS Source maps are supported when using Tailwind as a PostCSS plugin *in development mode only*. They may or may not be enabled by default depending on your build tool. If they are not you may be able to configure them within your PostCSS config: ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "@tailwindcss/postcss": {}, }, } } ``` ### Vite Source maps are supported when using the Tailwind CSS Vite plugin in *development mode only* by enabling the `css.devSourcemap` setting: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { devSourcemap: true, }, }) ``` Now when a CSS file is requested by the browser it'll have an inline source map comment that the browser can use. ## Limitations - Production build source maps are currently disabled due to a bug in Lightning CSS. See https://github.com/parcel-bundler/lightningcss/pull/971 for more details. - In Vite, minified CSS build source maps are not supported at all. See https://github.com/vitejs/vite/issues/2830 for more details. - In PostCSS, minified CSS source maps are not supported. This is due to the complexity required around re-associating every AST node with a location in the generated, optimized CSS. This complexity would also have a non-trivial performance impact. ## Testing Here's how to test the source map functionality in different environments: ### Testing the CLI 1. Setup typical project that the CLI can use and with sources to scan. ```css @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 2. Build with source maps: ```bash bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map ``` 3. Open Chrome DevTools, inspect an element with utility classes, and you should see rules pointing to `input.css` or `node_modules/tailwindcss/index.css` ### Testing with Vite Testing in Vite will require building and installing necessary files under `dist/*.tgz`. 1. Create a Vite project and enable source maps in `vite.config.js`: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { // This line is required for them to work devSourcemap: true, }, }) ``` 2. Add a component that uses Tailwind classes and custom CSS: ```jsx // ./src/app.jsx export default function App() { return ( <div className="bg-blue-500 my-custom-class"> Hello World </div> ) } ``` ```css /* ./src/styles.css */ @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 3. Run `npm run dev`, open DevTools, and inspect elements to verify source mapping works for both utility classes and custom CSS. ### Testing with PostCSS CLI 1. Create a test file and update your PostCSS config: ```css /* input.css */ @import "tailwindcss"; @layer components { .card { @apply p-6 rounded-lg shadow-lg; } } ``` ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {} } } } ``` 2. Run PostCSS through Bun: ```bash bunx --bun postcss ./src/index.css -o out.css ``` 3. Inspect the output CSS - it should include an inline source map comment at the bottom. ### Testing with PostCSS + Next.js Testing in Next.js will require building and installing necessary files under `dist/*.tgz`. However, I've not been able to get CSS source maps to work in Next.js without this hack: ```js const nextConfig: NextConfig = { // next.js overwrites config.devtool so we prevent it from doing so // please don't actually do this… webpack: (config) => Object.defineProperty(config, "devtool", { get: () => "inline-source-map", set: () => {}, }), }; ``` This is definitely not supported and also doesn't work with turbopack. This can be used to test them temporarily but I suspect that they just don't work there. ### Manual source map analysis You can analyze source maps using Evan Wallace's [Source Map Visualization](https://evanw.github.io/source-map-visualization/) tool which will help to verify the accuracy and quality of source maps. This is what I used extensively while developing this implementation. It'll help verify that custom, user CSS maps back to itself in the input, that generated utilities all map back to `@tailwind utilities;`, that source locations from imported files are also handled correctly, etc… It also highlights the ranges of stuff so it's easy to see if there are off-by-one errors. It's easiest to use inline source maps with this tool because you can take the CSS file and drop it on the page and it'll analyze it while showing the file content. If you're using Vite you'll want to access the CSS file with `?direct` at the end so you don't get a JS module back. ## Implementation The source map implementation follows the ECMA-426 specification and includes several key components to aid in that goal: ### Source Location Tracking Each emittable AST node in the compilation pipeline tracks two types of source locations: - `src`: Original source location - [source file, start offset, end offset] - `dst`: Generated source location - [output file, start offset, end offset] This dual tracking allows us to maintain mappings between the original source and generated output for things like user CSS, generated utilities, uses of `@apply`, and tracking theme variables. It is important to note that source locations for nodes _never overlap_ within a file which helps simplify source map generation. As such each type of node tracks a specific piece of itself rather than its entire "block": | Node | What a `SourceLocation` represents | | ----------- | ---------------------------------------------------------------- | | Style Rule | The selector | | At Rule | Rule name and params, includes the `@` | | Declaration | Property name and value, excludes the semicolon | | Comment | The entire comment, includes the start `/*` and end `*/` markers | ### Windows line endings when parsing CSS Because our AST tracks nodes through offsets we must ensure that any mutations to the file do *not* change the lenth of the string. We were previously replacing `\r\n` with `\n` (see [filter code points](https://drafts.csswg.org/css-syntax/#css-filter-code-points) from the spec) — which changes the length of the string and all offsets may end up incorrect. The CSS parser was updated to handle the CRLF token directly by skipping over the `\r` and letting remaining code handle `\n` as it did previously. Some additional tweaks were required when "peeking" the input but those changes were fairly small. ### Tracking of imports Source maps need paths to the actual imported stylesheets but the resolve step for stylesheets happens inside the call to `loadStylesheet` which make the file path unavailable to us. Because of this the `loadStylesheet` API was augmented such that it has to return a `path` property that we can then use to identify imported sources. I've also made the same change to the `loadModule` API for consistency but nothing currently uses this property. The `path` property likely makes `base` redundant but elminating that (if we even want to) is a future task. ### Optimizing the AST Our optimization pass may intoduce some nodes, for example, fallbacks we create for `@property`. These nodes are linked back to `@tailwind utilities` as ultimately that is what is responsible for creating them. ### Line Offset Tables A key component to our source map generation is the line offset table, which was inspired by some ESBuild internals. It stores a sorted list of offsets for the start of each line allowing us to translate offsets to line/column `Position`s in `O(log N)` time and from `Position`s to offsets in `O(1)` time. Creation of the table takes `O(N)` time. This means that we can store code point offsets for source locations and not have to worry about computing or tracking line/column numbers during parsing and serialization. Only when a source map is generated do these offsets need to be computed. This ensures the performance penalty when not using source maps is minimal. ### Source Map Generation The source map returned by `buildSourceMap()` is designed to follow the [ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not completely finalized we consider the result of `buildSourceMap()` to be internal API that may change as the spec chamges. The produces source map is a "decoded" map such that all sources and mappings are in an object graph. A library like `source-map-js` must be used to convert this to an encoded source map of the right version where mappings are encoded with base 64 VLQs. Any specific integration (Vite, PostCSS, etc…) can then use `toSourceMap()` from `@tailwindcss/node` to convert from the internal source map to an spec-compliant encoded source map that can be understood by other tools. ### Handling minification in Lightning Since we use Lightning CSS for optimization, and it takes in an input map, we generate an encoded source map that we then pass to lightning. The output source map *from lighting itself* is then passed back in during the second optimization pass. The final map is then passed from lightning to the CLI (but not Vite or PostCSS — see the limitations section for details). In some cases we have to "fix up" the output CSS. When this happens we use `magic-string` to do the replacement in a way that is trackable and `@amppproject/remapping` to map that change back onto the original source map. Once the need for these fix ups disappear these dependencies can go away. Notes: - The accuracy of source maps run though lightning is reduced as it only tracks on a per-rule level. This is sufficient enough for browser dev tools so should be fine. - Source maps during optimization do not function properly at this time because of a bug in Lightning CSS regarding license comments. Once this bug is fixed they will start working as expected. ### How source locations flow through the system 1. During initial CSS parsing, source locations are preserved. 2. During parsing these source locations are also mapped to the destinations which supports an optimization for when no utilities are generated. 3. Throughout the compilation process, transformations maintain source location data 4. Generated utilities are explicitly pointed to `@tailwind utilities` unless generated by `@apply`. 5. When optimization is enabled, source maps are remapped through lightningcss 6. Final source maps are written in the requested format (inline or separate file)
2025-05-08 16:29:49 -04:00
import { createLineTable } from '../packages/tailwindcss/src/source-maps/line-table'
import { escape } from '../packages/tailwindcss/src/utils/escape'
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
const REPO_ROOT = path.join(__dirname, '..')
const ROOT_PNPM_WORKSPACE = Yaml.parse(
await fs.readFile(path.join(REPO_ROOT, 'pnpm-workspace.yaml'), 'utf8'),
)
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
const PUBLIC_PACKAGES = (await fs.readdir(path.join(REPO_ROOT, 'dist'))).map((name) =>
name.replace('tailwindcss-', '@tailwindcss/').replace('.tgz', ''),
)
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
interface SpawnedProcess {
Make upgrade tooling more stable (#19846) This PR is an attempt to make the upgrade tooling more stable. ### TL;DR 1. When migrating from Tailwind CSS v3 → Tailwind CSS v4, only migrate files listed in the `config.content` instead of relying on v4's auto content detection feature 2. Skip writing files that have not been changed 3. Write changed files in a safe way: first write to a temporary file, then rename the file atomically 4. Never migrate files that are git ignored, even if they are listed in the `config.content` file 5. Always ignore `.env` and `.env.*` files when scanning for files. Most people will have this in their `.gitignore` file, but if not, then this is a fallback mechanism. --- Looking at the #18972 issue, it looks like some people are running into weird situations where some of the contents is just gone. I have never been able to reproduce this on my own devices and in my own projects unfortunately. But there is definitely _something_ happening that's not right that people are running into. Therefore, this PR is an attempt to fix what I think _might_ be wrong, but I'm not 100% sure if these fixes are enough, or if something else is still happening here. This builds on top of the #19779 PR which has some small fixes, but is incomplete to make this work. ### What's happening Looking at some of the comments, it looks like a few things are happening such as: 1. The upgrade tool is emptying out my files — it looks like these are only happening if you ctrl+c while the process is taking a while. It could be that a lot of files are being checked and therefore the tooling looks like its stuck. 6. The upgrade tool is looking at files it shouldn't look at — in Tailwind CSS v4 we have this concept of the auto-content detection. This means that we will look at any plain text file that is not git ignored. ### Fixes #### Emptying out files The files being emptied looks like it's because how `fs.writeFile` behaves by default. It opens the file handle with the `w` flag, which will first truncate the file before writing the new contents. This is not a single atomic operation, so a killed process in the middle will cause invalid state. When we migrate your template files, everything is happening in promises to migrate things at the same time. When a lot of files are being scanned, truncating might have happened already before we write the new content. Since we migrate a bunch of files in parallel, a ctrl+c could cause data loss in multiple files. To mitigate this, I switched to an alternative way of writing files. 1. First, we do some quick checks where if the contents didn't change we just bail out immediately. Files that don't include Tailwind CSS classes won't change, and therefore we don't need to override these files with the same contents. 2. When the migrated contents is empty, we bail out as well. I'm 100% sure that this is not the spot where the "emptying out" happens, I still believe it happens in the `writeFile` itself, but added it just in case. 7. Next, I introduced a safe write, where we first write to a temporary file in the same folder. We could write it to `/tmp`, but then we can't guarantee that we are on the same file system. If we ctrl+c at this stage, then the worst case scenario is that you have additional temporary files in your project, but your original files are still there. Once that file was written, we will use the atomic `fs.rename`. This should be atomic as long as we are on the same file system, so either the rename didn't happen yet, or it completed. I added an integration test for this, but I had to change the `writeFile` implementation slightly. In the test, we will truncate the file first, after that we will write the new contents. This is so that we have enough time to kill the current process and allows us to verify that we didn't clear out the file. Again, this is a hacky way of testing this, just because I can't reproduce this issue myself, let alone reproduce it reliable in a CI environment. Note: we are also using `realpath` to make sure that we are updating the real file. Otherwise, if we were dealing with a symlinked file, we would override the symlink with a "hard" copy instead. #### Touching files that should not be touched During the migration, we rely on the Tailwind CSS v4 auto detection logic which means that it will scan any plain text file that is not git ignored. Therefore changes to php files could happen because in theory they could contain Tailwind CSS classes. To solve this, when migrating from Tailwind CSS v3 to Tailwind CSS v4, we will _only_ take the sources into account that were listed in the `config.content` array. Since this was a requirement in Tailwind CSS v3, it should be safe to rely on this array. Additionally, this will make sure that we are dealing with way fewer files to migrate as well. On top of that, files that match the patterns in the content array that are git ignored will also be skipped. This is to prevent that we mutate files in `node_modules` for example. In one of the comments I read that `.env` files were emptied out. In most cases people will have these files gitignored but I explicitly added `.env` and `.env.*` as files to never ever touch by default when scanning. Last but not least, this also updates the output a little bit of the upgrade tool in case we skip content files (because of git ignore) and if we changed a file. <img width="1122" height="1376" alt="image" src="https://github.com/user-attachments/assets/318fdbbf-e319-4c7e-9648-ee9283842624" /> - "Git ignored folder, skipping: `./node_modules`": this is because the content array looks like this while the `node_modules` are being ignored: <img width="1090" height="398" alt="image" src="https://github.com/user-attachments/assets/7d694720-5671-47ec-bb2e-f24c5f2c4248" /> - "Migrated `./resources/views/vendor/filament-panels/components/logo.blade.php`": this is because **I** made a change to showcase this feature. Fixes: #18972 Closes: #19779 ### Test plan 1. Existing tests still pass 1. Added a dedicated integration test to ensure that we only take `config.content` into account when migrating from Tailwind CSS v3 to Tailwind CSS v4 projects. 1. Added a dedicated integration test to make sure that files listed in `config.content` that are also git ignored, will still be skipped. 1. Added a dedicated integration test to ensure that when `writeFile` is cancelled mid-write that our old files are still present. 1. Added a dedicated integration test to ensure that we ignore `.env` and `.env.*` files even if you didn't git ignore them. [ci-all] To verify on Windows --------- Co-authored-by: Sami <sychocouldy@gmail.com>
2026-03-24 17:24:12 +01:00
dispose: () => Promise<void>
flush: () => void
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
onStdout: (predicate: (message: string) => boolean) => Promise<void>
onStderr: (predicate: (message: string) => boolean) => Promise<void>
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
interface ChildProcessOptions {
cwd?: string
env?: Record<string, string>
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
}
Throw a useful error when `tailwindcss` is used as a PostCSS plugin (#14378) While upgrading a project to Tailwind CSS v4, I forgot to remove the `tailwindcss` import from the PostCSS config. As a result of this, I was greeted with the following message: ``` node:internal/process/promises:289 triggerUncaughtException(err, true /* fromPromise */); ^ [Failed to load PostCSS config: Failed to load PostCSS config (searchPath: /Users/philipp/dev/project): [TypeError] Invalid PostCSS Plugin found at: plugins[0] (@/Users/philipp/dev/project/postcss.config.js) TypeError: Invalid PostCSS Plugin found at: plugins[0] ``` I don't think this was particularly helpful, so I’m proposing we add a default function export to the `tailwindcss` package so when it's used inside PostCSS, we can control the error message. So I changed it to something along these lines: ``` It looks like you're trying to use the \`tailwindcss\` package as a PostCSS plugin. This is no longer possible since Tailwind CSS v4. If you want to continue to use Tailwind CSS with PostCSS, please install \`@tailwindcss/postcss\` and change your PostCSS config file. at w (/Users/philipp/dev/project/node_modules/tailwindcss/node_modules/tailwindcss/dist/lib.js:1:21233) at Object.<anonymous> (/Users/philipp/dev/project/node_modules/tailwindcss/postcss.config.cjs:3:13) at Module._compile (node:internal/modules/cjs/loader:1358:14) at Module._extensions..js (node:internal/modules/cjs/loader:1416:10) at Module.load (node:internal/modules/cjs/loader:1208:32) at Module._load (node:internal/modules/cjs/loader:1024:12) at cjsLoader (node:internal/modules/esm/translators:348:17) at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:297:7) at ModuleJob.run (node:internal/modules/esm/module_job:222:25) at async ModuleLoader.import (node:internal/modules/esm/loader:316:24) ``` This is also a good place to link to the migration guides once we have them 🙂 --------- Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-09-11 17:02:24 +02:00
interface ExecOptions {
ignoreStdErr?: boolean
stdin?: string
Throw a useful error when `tailwindcss` is used as a PostCSS plugin (#14378) While upgrading a project to Tailwind CSS v4, I forgot to remove the `tailwindcss` import from the PostCSS config. As a result of this, I was greeted with the following message: ``` node:internal/process/promises:289 triggerUncaughtException(err, true /* fromPromise */); ^ [Failed to load PostCSS config: Failed to load PostCSS config (searchPath: /Users/philipp/dev/project): [TypeError] Invalid PostCSS Plugin found at: plugins[0] (@/Users/philipp/dev/project/postcss.config.js) TypeError: Invalid PostCSS Plugin found at: plugins[0] ``` I don't think this was particularly helpful, so I’m proposing we add a default function export to the `tailwindcss` package so when it's used inside PostCSS, we can control the error message. So I changed it to something along these lines: ``` It looks like you're trying to use the \`tailwindcss\` package as a PostCSS plugin. This is no longer possible since Tailwind CSS v4. If you want to continue to use Tailwind CSS with PostCSS, please install \`@tailwindcss/postcss\` and change your PostCSS config file. at w (/Users/philipp/dev/project/node_modules/tailwindcss/node_modules/tailwindcss/dist/lib.js:1:21233) at Object.<anonymous> (/Users/philipp/dev/project/node_modules/tailwindcss/postcss.config.cjs:3:13) at Module._compile (node:internal/modules/cjs/loader:1358:14) at Module._extensions..js (node:internal/modules/cjs/loader:1416:10) at Module.load (node:internal/modules/cjs/loader:1208:32) at Module._load (node:internal/modules/cjs/loader:1024:12) at cjsLoader (node:internal/modules/esm/translators:348:17) at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:297:7) at ModuleJob.run (node:internal/modules/esm/module_job:222:25) at async ModuleLoader.import (node:internal/modules/esm/loader:316:24) ``` This is also a good place to link to the migration guides once we have them 🙂 --------- Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-09-11 17:02:24 +02:00
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
interface TestConfig {
fs: {
[filePath: string]: string | Uint8Array
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
timeout?: number
installDependencies?: boolean
retry?: number
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
interface TestContext {
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
root: string
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
expect: ExpectStatic
Throw a useful error when `tailwindcss` is used as a PostCSS plugin (#14378) While upgrading a project to Tailwind CSS v4, I forgot to remove the `tailwindcss` import from the PostCSS config. As a result of this, I was greeted with the following message: ``` node:internal/process/promises:289 triggerUncaughtException(err, true /* fromPromise */); ^ [Failed to load PostCSS config: Failed to load PostCSS config (searchPath: /Users/philipp/dev/project): [TypeError] Invalid PostCSS Plugin found at: plugins[0] (@/Users/philipp/dev/project/postcss.config.js) TypeError: Invalid PostCSS Plugin found at: plugins[0] ``` I don't think this was particularly helpful, so I’m proposing we add a default function export to the `tailwindcss` package so when it's used inside PostCSS, we can control the error message. So I changed it to something along these lines: ``` It looks like you're trying to use the \`tailwindcss\` package as a PostCSS plugin. This is no longer possible since Tailwind CSS v4. If you want to continue to use Tailwind CSS with PostCSS, please install \`@tailwindcss/postcss\` and change your PostCSS config file. at w (/Users/philipp/dev/project/node_modules/tailwindcss/node_modules/tailwindcss/dist/lib.js:1:21233) at Object.<anonymous> (/Users/philipp/dev/project/node_modules/tailwindcss/postcss.config.cjs:3:13) at Module._compile (node:internal/modules/cjs/loader:1358:14) at Module._extensions..js (node:internal/modules/cjs/loader:1416:10) at Module.load (node:internal/modules/cjs/loader:1208:32) at Module._load (node:internal/modules/cjs/loader:1024:12) at cjsLoader (node:internal/modules/esm/translators:348:17) at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:297:7) at ModuleJob.run (node:internal/modules/esm/module_job:222:25) at async ModuleLoader.import (node:internal/modules/esm/loader:316:24) ``` This is also a good place to link to the migration guides once we have them 🙂 --------- Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-09-11 17:02:24 +02:00
exec(command: string, options?: ChildProcessOptions, execOptions?: ExecOptions): Promise<string>
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
spawn(command: string, options?: ChildProcessOptions): Promise<SpawnedProcess>
Add support for source maps (#17775) Closes #13694 Closes #13591 # Source Maps Support for Tailwind CSS This PR adds support for source maps to Tailwind CSS v4 allowing us to track where styles come from whether that be user CSS, imported stylesheets, or generated utilities. This will improve debuggability in browser dev tools and gives us a good foundation for producing better error messages. I'll go over the details on how end users can enable source maps, any limitations in our implementation, changes to the internal `compile(…)` API, and some details and reasoning around the implementation we chose. ## Usage ### CLI Source maps can be enabled in the CLI by using the command line argument `--map` which will generate an inline source map comment at the bottom of your CSS. A separate file may be generated by passing a file name to `--map`: ```bash # Generates an inline source map npx tailwindcss -i input.css -o output.css --map # Generates a separate source map file npx tailwindcss -i input.css -o output.css --map output.css.map ``` ### PostCSS Source maps are supported when using Tailwind as a PostCSS plugin *in development mode only*. They may or may not be enabled by default depending on your build tool. If they are not you may be able to configure them within your PostCSS config: ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "@tailwindcss/postcss": {}, }, } } ``` ### Vite Source maps are supported when using the Tailwind CSS Vite plugin in *development mode only* by enabling the `css.devSourcemap` setting: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { devSourcemap: true, }, }) ``` Now when a CSS file is requested by the browser it'll have an inline source map comment that the browser can use. ## Limitations - Production build source maps are currently disabled due to a bug in Lightning CSS. See https://github.com/parcel-bundler/lightningcss/pull/971 for more details. - In Vite, minified CSS build source maps are not supported at all. See https://github.com/vitejs/vite/issues/2830 for more details. - In PostCSS, minified CSS source maps are not supported. This is due to the complexity required around re-associating every AST node with a location in the generated, optimized CSS. This complexity would also have a non-trivial performance impact. ## Testing Here's how to test the source map functionality in different environments: ### Testing the CLI 1. Setup typical project that the CLI can use and with sources to scan. ```css @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 2. Build with source maps: ```bash bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map ``` 3. Open Chrome DevTools, inspect an element with utility classes, and you should see rules pointing to `input.css` or `node_modules/tailwindcss/index.css` ### Testing with Vite Testing in Vite will require building and installing necessary files under `dist/*.tgz`. 1. Create a Vite project and enable source maps in `vite.config.js`: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { // This line is required for them to work devSourcemap: true, }, }) ``` 2. Add a component that uses Tailwind classes and custom CSS: ```jsx // ./src/app.jsx export default function App() { return ( <div className="bg-blue-500 my-custom-class"> Hello World </div> ) } ``` ```css /* ./src/styles.css */ @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 3. Run `npm run dev`, open DevTools, and inspect elements to verify source mapping works for both utility classes and custom CSS. ### Testing with PostCSS CLI 1. Create a test file and update your PostCSS config: ```css /* input.css */ @import "tailwindcss"; @layer components { .card { @apply p-6 rounded-lg shadow-lg; } } ``` ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {} } } } ``` 2. Run PostCSS through Bun: ```bash bunx --bun postcss ./src/index.css -o out.css ``` 3. Inspect the output CSS - it should include an inline source map comment at the bottom. ### Testing with PostCSS + Next.js Testing in Next.js will require building and installing necessary files under `dist/*.tgz`. However, I've not been able to get CSS source maps to work in Next.js without this hack: ```js const nextConfig: NextConfig = { // next.js overwrites config.devtool so we prevent it from doing so // please don't actually do this… webpack: (config) => Object.defineProperty(config, "devtool", { get: () => "inline-source-map", set: () => {}, }), }; ``` This is definitely not supported and also doesn't work with turbopack. This can be used to test them temporarily but I suspect that they just don't work there. ### Manual source map analysis You can analyze source maps using Evan Wallace's [Source Map Visualization](https://evanw.github.io/source-map-visualization/) tool which will help to verify the accuracy and quality of source maps. This is what I used extensively while developing this implementation. It'll help verify that custom, user CSS maps back to itself in the input, that generated utilities all map back to `@tailwind utilities;`, that source locations from imported files are also handled correctly, etc… It also highlights the ranges of stuff so it's easy to see if there are off-by-one errors. It's easiest to use inline source maps with this tool because you can take the CSS file and drop it on the page and it'll analyze it while showing the file content. If you're using Vite you'll want to access the CSS file with `?direct` at the end so you don't get a JS module back. ## Implementation The source map implementation follows the ECMA-426 specification and includes several key components to aid in that goal: ### Source Location Tracking Each emittable AST node in the compilation pipeline tracks two types of source locations: - `src`: Original source location - [source file, start offset, end offset] - `dst`: Generated source location - [output file, start offset, end offset] This dual tracking allows us to maintain mappings between the original source and generated output for things like user CSS, generated utilities, uses of `@apply`, and tracking theme variables. It is important to note that source locations for nodes _never overlap_ within a file which helps simplify source map generation. As such each type of node tracks a specific piece of itself rather than its entire "block": | Node | What a `SourceLocation` represents | | ----------- | ---------------------------------------------------------------- | | Style Rule | The selector | | At Rule | Rule name and params, includes the `@` | | Declaration | Property name and value, excludes the semicolon | | Comment | The entire comment, includes the start `/*` and end `*/` markers | ### Windows line endings when parsing CSS Because our AST tracks nodes through offsets we must ensure that any mutations to the file do *not* change the lenth of the string. We were previously replacing `\r\n` with `\n` (see [filter code points](https://drafts.csswg.org/css-syntax/#css-filter-code-points) from the spec) — which changes the length of the string and all offsets may end up incorrect. The CSS parser was updated to handle the CRLF token directly by skipping over the `\r` and letting remaining code handle `\n` as it did previously. Some additional tweaks were required when "peeking" the input but those changes were fairly small. ### Tracking of imports Source maps need paths to the actual imported stylesheets but the resolve step for stylesheets happens inside the call to `loadStylesheet` which make the file path unavailable to us. Because of this the `loadStylesheet` API was augmented such that it has to return a `path` property that we can then use to identify imported sources. I've also made the same change to the `loadModule` API for consistency but nothing currently uses this property. The `path` property likely makes `base` redundant but elminating that (if we even want to) is a future task. ### Optimizing the AST Our optimization pass may intoduce some nodes, for example, fallbacks we create for `@property`. These nodes are linked back to `@tailwind utilities` as ultimately that is what is responsible for creating them. ### Line Offset Tables A key component to our source map generation is the line offset table, which was inspired by some ESBuild internals. It stores a sorted list of offsets for the start of each line allowing us to translate offsets to line/column `Position`s in `O(log N)` time and from `Position`s to offsets in `O(1)` time. Creation of the table takes `O(N)` time. This means that we can store code point offsets for source locations and not have to worry about computing or tracking line/column numbers during parsing and serialization. Only when a source map is generated do these offsets need to be computed. This ensures the performance penalty when not using source maps is minimal. ### Source Map Generation The source map returned by `buildSourceMap()` is designed to follow the [ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not completely finalized we consider the result of `buildSourceMap()` to be internal API that may change as the spec chamges. The produces source map is a "decoded" map such that all sources and mappings are in an object graph. A library like `source-map-js` must be used to convert this to an encoded source map of the right version where mappings are encoded with base 64 VLQs. Any specific integration (Vite, PostCSS, etc…) can then use `toSourceMap()` from `@tailwindcss/node` to convert from the internal source map to an spec-compliant encoded source map that can be understood by other tools. ### Handling minification in Lightning Since we use Lightning CSS for optimization, and it takes in an input map, we generate an encoded source map that we then pass to lightning. The output source map *from lighting itself* is then passed back in during the second optimization pass. The final map is then passed from lightning to the CLI (but not Vite or PostCSS — see the limitations section for details). In some cases we have to "fix up" the output CSS. When this happens we use `magic-string` to do the replacement in a way that is trackable and `@amppproject/remapping` to map that change back onto the original source map. Once the need for these fix ups disappear these dependencies can go away. Notes: - The accuracy of source maps run though lightning is reduced as it only tracks on a per-rule level. This is sufficient enough for browser dev tools so should be fine. - Source maps during optimization do not function properly at this time because of a bug in Lightning CSS regarding license comments. Once this bug is fixed they will start working as expected. ### How source locations flow through the system 1. During initial CSS parsing, source locations are preserved. 2. During parsing these source locations are also mapped to the destinations which supports an optimization for when no utilities are generated. 3. Throughout the compilation process, transformations maintain source location data 4. Generated utilities are explicitly pointed to `@tailwind utilities` unless generated by `@apply`. 5. When optimization is enabled, source maps are remapped through lightningcss 6. Final source maps are written in the requested format (inline or separate file)
2025-05-08 16:29:49 -04:00
parseSourceMap(opts: string | SourceMapOptions): SourceMap
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
fs: {
write(filePath: string, content: string, encoding?: BufferEncoding): Promise<void>
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
symlink(dst: string, src: string): Promise<void>
create(filePaths: string[]): Promise<void>
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
read(filePath: string): Promise<string>
Ensure `@tailwindcss/cli` recovers from a deleted transitive dependency (#20137) This PR fixes an issue where the `@tailwindcss/cli` can get into a non-recoverable state when any of the transitive dependencies break. Tailwind CSS has 2 kinds of dependencies: 1. All your templates 2. All dependencies that contribute to your configuration such as the `input.css`, any plugins, any `tailwind.config.js` files and so on. When a template changes, we just have to scan for new Tailwind CSS classes and emit a new CSS file. But when the `input.css` file, or any of its dependencies changes, then we want to perform a full rebuild. The idea is that your `@theme` might have changed, or new plugins have been added, or old plugins have been removed. If you have an `input.css` file: ```css @import "tailwindcss"; @config "./tailwind.config.js"; ``` That relies on a custom config: `tailwind.config.js`: ```js const theme = require('./my-custom-theme.js'); module.exports = { theme } ``` If that file relies on yet another file: `./my-custom-theme.js`, then changes there should also trigger a full rebuild. Since we're dealing with JavaScript here, we want to clear the require cache and rebuild the dependency tree such that another change to any of these files triggers a full fresh build. However, if any of those (transitive) dependencies are deleted, then we will end up in an invalid state. Creating a new compiler will result in a build error. The compiler won't be able to figure out the entire dependency tree, and we're stuck. Once the user fixes the potentially missing dependency, the watchers will not be watching any of those files because we created a fresh compiler. With this PR, we fix that by keeping track of old paths and using those while we are still in an invalid state. The moment everything is fixed, a fresh dependency tree is created and everything starts working again without you having to restart the `@tailwindcss/cli` command. Fixes: #20113 Closes: #20114 Closes: #20133 ## Test plan - Added an integration test that removes the transitive dependency. Re-adding that file later will recover the CLI state.
2026-06-01 19:15:40 +02:00
delete(filePath: string): Promise<void>
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
glob(pattern: string): Promise<[string, string][]>
dumpFiles(pattern: string): Promise<string>
expectFileToContain(
filePath: string,
Vite: Don't track candidate changes for Svelte `<style>` tags (#14981) Closes #14965 This PR changes the way we register Tailwind CSS as a Svelte preprocessor when using the Vite plugin. The idea is to reduce the bookkeeping for interacting with CSS inside `<style>` tags so that we have a more consistent behavior and make sure the Svelte-specific post-processing (e.g. local class mangling) works as expected. Prior to this change, we were running Tailwind CSS as a Svelte preprocessor and then we would transform the file again when necessary inside the Vite `transform` hook. This is necessary to have the right list of candidates when we build the final CSS, but it did cause some situation to not apply the Svelte post-processors anymore. The repro for this seemed to indicate a timing specific issue and I did notice that specifically the code where we invalidate modules in Vite would cause unexpected processing orders. We do, however, not officially support rendering utilities (`@tailwind utilities;`) inside `<style>` tag. This is because the `<style>` block is scoped by default and emitting utilities will always include utilities for all classes in your whole project. For this case, we highly recommend creating as separate `.css` file and importing it explicitly. With this limitation in place, the additional bookkeeping where we need to invalidate modules because the candidate list has changed is no longer necessary and removing it allows us to reduce the complexity of the Svelte integration. ## Test Plan https://github.com/user-attachments/assets/32c8e91f-ab21-48c6-aeaf-2582273b9bac Not seen in the test plan above I also tested the `pnpm build --watch` step of the Vite project. This does require the `pnpm preview` server to restart but the build artifact are updated as expected.
2024-11-13 16:42:49 +01:00
contents: string | RegExp | (string | RegExp)[],
): Promise<void>
Fix `@source` with folders that are ignored (#20214) This PR fixes an issue where a `@source` that's pointing to a folder that is git ignored, is also ignored by the `@source` even if it's explicitly added. Internally, we convert `@source` directives from `PublicSourceEntry`s to `SourceEntry`s where we have dedicated enum branches for `Auto`, `Pattern`, `Ignored` and `External`. The `Auto` one accepts a `base` path, and will be used for auto content detection. However, these paths will make use of all the default auto content detection rules, which includes git ignore rules. We also have `External` where we link to something that's "external" to the current repo. We can probably improve this name, but it's external in the sense that it won't show up on GitHub for example, aka ignored. We have some content dirs that we ignore by default, such as the `node_modules` folder. When you do use `@source` with `node_modules` in the path, then we will mark it as an `external` resource which does not look at the `gitignore` related rules and allowing it to be included this way. The idea with this is that, even though the folder is ignored by default, you can still include files from the folder by explicitly using the `@source` directive. The issue as seen in #19844 is using `vendor/` instead of `node_modules/` which is _not_ ignored by default. While we can add `vendor/` to this same ignored dirs list, it will result in a breaking change because this folder is often used by the Laravel community to store some resources in. This PR fixes this problem by not only looking at the content dirs we ignore by default, but also looking at the actual git ignore state of this folder. If it turns out that this is ignored, then we promote the `Auto` source to an `External` source. Fixes: #19844 Closes: #20057 ## Test plan 1. Added integration tests for this situation 2. Ran the fix on the reproduction from #19844. If we run the CLI with the `DEBUG=*` environment variable, the log file produces these results: ```diff diff --git a/./tailwindcss-29207.log b/./tailwindcss-30381.log index bc3017c..921af0e 100644 --- a/./tailwindcss-29207.log +++ b/./tailwindcss-30381.log @@ -6,8 +6,9 @@ INFO tailwindcss_oxide::scanner: Source: PublicSourceEntry { base: "/Users/robin INFO tailwindcss_oxide::scanner: Optimized sources: INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.phtml" } INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.xml" } -INFO tailwindcss_oxide::scanner: Source: Auto { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" } +INFO tailwindcss_oxide::scanner: Source: External { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" } INFO tailwindcss_oxide::scanner: Source: Ignored { base: "/Users/robin/.fnm/node-versions/v26.1.0/installation/bin", pattern: "/node" } INFO discover_sources: tailwindcss_oxide::scanner: enter -INFO discover_sources: tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml" +INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml" +INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme/module/templates/component.phtml" INFO discover_sources: tailwindcss_oxide::scanner: exit ``` We're checking some `.gitignore` related files, so let's check on each OS [ci-all]
2026-06-10 18:44:14 +02:00
expectFileNotToContain(filePath: string, contents: string[]): Promise<void>
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
}
type TestCallback = (context: TestContext) => Promise<void> | void
interface TestFlags {
only?: boolean
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
skip?: boolean
debug?: boolean
concurrent?: boolean
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
type SpawnActor = { predicate: (message: string) => boolean; resolve: () => void }
export const IS_WINDOWS = platform() === 'win32'
const execFileAsync = promisify(execFile)
const TEST_TIMEOUT = IS_WINDOWS ? 120000 : 60000
const ASSERTION_TIMEOUT = IS_WINDOWS ? 10000 : 5000
// On Windows CI, tmpdir returns a path containing a weird RUNNER~1 folder that
// apparently causes the vite builds to not work.
const TMP_ROOT =
process.env.CI && IS_WINDOWS ? path.dirname(process.env.GITHUB_WORKSPACE!) : tmpdir()
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
export function test(
name: string,
config: TestConfig,
testCallback: TestCallback,
{ only = false, skip = false, debug = false, concurrent = false }: TestFlags = {},
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
) {
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
return defaultTest(
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
name,
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
{
timeout: config.timeout ?? TEST_TIMEOUT,
retry: config.retry ?? (process.env.CI ? 2 : 0),
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
only: only || (!process.env.CI && debug),
skip,
concurrent,
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
},
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
async (options) => {
let rootDir = debug ? path.join(REPO_ROOT, '.debug') : TMP_ROOT
await fs.mkdir(rootDir, { recursive: true })
let root = await fs.mkdtemp(path.join(rootDir, 'tailwind-integrations'))
if (debug) {
console.log('Running test in debug mode. File system will be written to:')
console.log(root)
console.log()
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
let context = {
root,
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
expect: options.expect,
Add support for source maps (#17775) Closes #13694 Closes #13591 # Source Maps Support for Tailwind CSS This PR adds support for source maps to Tailwind CSS v4 allowing us to track where styles come from whether that be user CSS, imported stylesheets, or generated utilities. This will improve debuggability in browser dev tools and gives us a good foundation for producing better error messages. I'll go over the details on how end users can enable source maps, any limitations in our implementation, changes to the internal `compile(…)` API, and some details and reasoning around the implementation we chose. ## Usage ### CLI Source maps can be enabled in the CLI by using the command line argument `--map` which will generate an inline source map comment at the bottom of your CSS. A separate file may be generated by passing a file name to `--map`: ```bash # Generates an inline source map npx tailwindcss -i input.css -o output.css --map # Generates a separate source map file npx tailwindcss -i input.css -o output.css --map output.css.map ``` ### PostCSS Source maps are supported when using Tailwind as a PostCSS plugin *in development mode only*. They may or may not be enabled by default depending on your build tool. If they are not you may be able to configure them within your PostCSS config: ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "@tailwindcss/postcss": {}, }, } } ``` ### Vite Source maps are supported when using the Tailwind CSS Vite plugin in *development mode only* by enabling the `css.devSourcemap` setting: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { devSourcemap: true, }, }) ``` Now when a CSS file is requested by the browser it'll have an inline source map comment that the browser can use. ## Limitations - Production build source maps are currently disabled due to a bug in Lightning CSS. See https://github.com/parcel-bundler/lightningcss/pull/971 for more details. - In Vite, minified CSS build source maps are not supported at all. See https://github.com/vitejs/vite/issues/2830 for more details. - In PostCSS, minified CSS source maps are not supported. This is due to the complexity required around re-associating every AST node with a location in the generated, optimized CSS. This complexity would also have a non-trivial performance impact. ## Testing Here's how to test the source map functionality in different environments: ### Testing the CLI 1. Setup typical project that the CLI can use and with sources to scan. ```css @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 2. Build with source maps: ```bash bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map ``` 3. Open Chrome DevTools, inspect an element with utility classes, and you should see rules pointing to `input.css` or `node_modules/tailwindcss/index.css` ### Testing with Vite Testing in Vite will require building and installing necessary files under `dist/*.tgz`. 1. Create a Vite project and enable source maps in `vite.config.js`: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { // This line is required for them to work devSourcemap: true, }, }) ``` 2. Add a component that uses Tailwind classes and custom CSS: ```jsx // ./src/app.jsx export default function App() { return ( <div className="bg-blue-500 my-custom-class"> Hello World </div> ) } ``` ```css /* ./src/styles.css */ @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 3. Run `npm run dev`, open DevTools, and inspect elements to verify source mapping works for both utility classes and custom CSS. ### Testing with PostCSS CLI 1. Create a test file and update your PostCSS config: ```css /* input.css */ @import "tailwindcss"; @layer components { .card { @apply p-6 rounded-lg shadow-lg; } } ``` ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {} } } } ``` 2. Run PostCSS through Bun: ```bash bunx --bun postcss ./src/index.css -o out.css ``` 3. Inspect the output CSS - it should include an inline source map comment at the bottom. ### Testing with PostCSS + Next.js Testing in Next.js will require building and installing necessary files under `dist/*.tgz`. However, I've not been able to get CSS source maps to work in Next.js without this hack: ```js const nextConfig: NextConfig = { // next.js overwrites config.devtool so we prevent it from doing so // please don't actually do this… webpack: (config) => Object.defineProperty(config, "devtool", { get: () => "inline-source-map", set: () => {}, }), }; ``` This is definitely not supported and also doesn't work with turbopack. This can be used to test them temporarily but I suspect that they just don't work there. ### Manual source map analysis You can analyze source maps using Evan Wallace's [Source Map Visualization](https://evanw.github.io/source-map-visualization/) tool which will help to verify the accuracy and quality of source maps. This is what I used extensively while developing this implementation. It'll help verify that custom, user CSS maps back to itself in the input, that generated utilities all map back to `@tailwind utilities;`, that source locations from imported files are also handled correctly, etc… It also highlights the ranges of stuff so it's easy to see if there are off-by-one errors. It's easiest to use inline source maps with this tool because you can take the CSS file and drop it on the page and it'll analyze it while showing the file content. If you're using Vite you'll want to access the CSS file with `?direct` at the end so you don't get a JS module back. ## Implementation The source map implementation follows the ECMA-426 specification and includes several key components to aid in that goal: ### Source Location Tracking Each emittable AST node in the compilation pipeline tracks two types of source locations: - `src`: Original source location - [source file, start offset, end offset] - `dst`: Generated source location - [output file, start offset, end offset] This dual tracking allows us to maintain mappings between the original source and generated output for things like user CSS, generated utilities, uses of `@apply`, and tracking theme variables. It is important to note that source locations for nodes _never overlap_ within a file which helps simplify source map generation. As such each type of node tracks a specific piece of itself rather than its entire "block": | Node | What a `SourceLocation` represents | | ----------- | ---------------------------------------------------------------- | | Style Rule | The selector | | At Rule | Rule name and params, includes the `@` | | Declaration | Property name and value, excludes the semicolon | | Comment | The entire comment, includes the start `/*` and end `*/` markers | ### Windows line endings when parsing CSS Because our AST tracks nodes through offsets we must ensure that any mutations to the file do *not* change the lenth of the string. We were previously replacing `\r\n` with `\n` (see [filter code points](https://drafts.csswg.org/css-syntax/#css-filter-code-points) from the spec) — which changes the length of the string and all offsets may end up incorrect. The CSS parser was updated to handle the CRLF token directly by skipping over the `\r` and letting remaining code handle `\n` as it did previously. Some additional tweaks were required when "peeking" the input but those changes were fairly small. ### Tracking of imports Source maps need paths to the actual imported stylesheets but the resolve step for stylesheets happens inside the call to `loadStylesheet` which make the file path unavailable to us. Because of this the `loadStylesheet` API was augmented such that it has to return a `path` property that we can then use to identify imported sources. I've also made the same change to the `loadModule` API for consistency but nothing currently uses this property. The `path` property likely makes `base` redundant but elminating that (if we even want to) is a future task. ### Optimizing the AST Our optimization pass may intoduce some nodes, for example, fallbacks we create for `@property`. These nodes are linked back to `@tailwind utilities` as ultimately that is what is responsible for creating them. ### Line Offset Tables A key component to our source map generation is the line offset table, which was inspired by some ESBuild internals. It stores a sorted list of offsets for the start of each line allowing us to translate offsets to line/column `Position`s in `O(log N)` time and from `Position`s to offsets in `O(1)` time. Creation of the table takes `O(N)` time. This means that we can store code point offsets for source locations and not have to worry about computing or tracking line/column numbers during parsing and serialization. Only when a source map is generated do these offsets need to be computed. This ensures the performance penalty when not using source maps is minimal. ### Source Map Generation The source map returned by `buildSourceMap()` is designed to follow the [ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not completely finalized we consider the result of `buildSourceMap()` to be internal API that may change as the spec chamges. The produces source map is a "decoded" map such that all sources and mappings are in an object graph. A library like `source-map-js` must be used to convert this to an encoded source map of the right version where mappings are encoded with base 64 VLQs. Any specific integration (Vite, PostCSS, etc…) can then use `toSourceMap()` from `@tailwindcss/node` to convert from the internal source map to an spec-compliant encoded source map that can be understood by other tools. ### Handling minification in Lightning Since we use Lightning CSS for optimization, and it takes in an input map, we generate an encoded source map that we then pass to lightning. The output source map *from lighting itself* is then passed back in during the second optimization pass. The final map is then passed from lightning to the CLI (but not Vite or PostCSS — see the limitations section for details). In some cases we have to "fix up" the output CSS. When this happens we use `magic-string` to do the replacement in a way that is trackable and `@amppproject/remapping` to map that change back onto the original source map. Once the need for these fix ups disappear these dependencies can go away. Notes: - The accuracy of source maps run though lightning is reduced as it only tracks on a per-rule level. This is sufficient enough for browser dev tools so should be fine. - Source maps during optimization do not function properly at this time because of a bug in Lightning CSS regarding license comments. Once this bug is fixed they will start working as expected. ### How source locations flow through the system 1. During initial CSS parsing, source locations are preserved. 2. During parsing these source locations are also mapped to the destinations which supports an optimization for when no utilities are generated. 3. Throughout the compilation process, transformations maintain source location data 4. Generated utilities are explicitly pointed to `@tailwind utilities` unless generated by `@apply`. 5. When optimization is enabled, source maps are remapped through lightningcss 6. Final source maps are written in the requested format (inline or separate file)
2025-05-08 16:29:49 -04:00
parseSourceMap,
Throw a useful error when `tailwindcss` is used as a PostCSS plugin (#14378) While upgrading a project to Tailwind CSS v4, I forgot to remove the `tailwindcss` import from the PostCSS config. As a result of this, I was greeted with the following message: ``` node:internal/process/promises:289 triggerUncaughtException(err, true /* fromPromise */); ^ [Failed to load PostCSS config: Failed to load PostCSS config (searchPath: /Users/philipp/dev/project): [TypeError] Invalid PostCSS Plugin found at: plugins[0] (@/Users/philipp/dev/project/postcss.config.js) TypeError: Invalid PostCSS Plugin found at: plugins[0] ``` I don't think this was particularly helpful, so I’m proposing we add a default function export to the `tailwindcss` package so when it's used inside PostCSS, we can control the error message. So I changed it to something along these lines: ``` It looks like you're trying to use the \`tailwindcss\` package as a PostCSS plugin. This is no longer possible since Tailwind CSS v4. If you want to continue to use Tailwind CSS with PostCSS, please install \`@tailwindcss/postcss\` and change your PostCSS config file. at w (/Users/philipp/dev/project/node_modules/tailwindcss/node_modules/tailwindcss/dist/lib.js:1:21233) at Object.<anonymous> (/Users/philipp/dev/project/node_modules/tailwindcss/postcss.config.cjs:3:13) at Module._compile (node:internal/modules/cjs/loader:1358:14) at Module._extensions..js (node:internal/modules/cjs/loader:1416:10) at Module.load (node:internal/modules/cjs/loader:1208:32) at Module._load (node:internal/modules/cjs/loader:1024:12) at cjsLoader (node:internal/modules/esm/translators:348:17) at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:297:7) at ModuleJob.run (node:internal/modules/esm/module_job:222:25) at async ModuleLoader.import (node:internal/modules/esm/loader:316:24) ``` This is also a good place to link to the migration guides once we have them 🙂 --------- Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-09-11 17:02:24 +02:00
async exec(
command: string,
childProcessOptions: ChildProcessOptions = {},
execOptions: ExecOptions = {},
) {
let cwd = childProcessOptions.cwd ?? root
let originalCommand = command
// Avoid `pnpm exec upgrade` on Windows because the upgrader runs
// `pnpm add`, which rewrites Windows `.cmd` shims while the current
// shim is still executing.
//
// Pretty sure this is a pnpm v11 bug on Windows.
if (IS_WINDOWS && command.includes('pnpm exec upgrade')) {
for (let base = cwd; ; base = path.dirname(base)) {
let upgradeBin = path.join(
base,
'node_modules',
'@tailwindcss',
'upgrade',
'dist',
'index.mjs',
)
try {
await fs.access(upgradeBin)
command = command.replace('pnpm exec upgrade', `node "${upgradeBin}"`)
break
} catch (error: any) {
if (error?.code !== 'ENOENT') throw error
}
let parent = path.dirname(base)
if (parent === base) {
throw new Error(`Unable to resolve @tailwindcss/upgrade from ${cwd}`)
}
}
}
if (debug && cwd !== root) {
let relative = path.relative(root, cwd)
if (relative[0] !== '.') relative = `./${relative}`
console.log(`> cd ${relative}`)
}
if (debug) console.log(`> ${command}`)
Rework Vite plugin to support lightningcss pre processor and fast rebuilds (#14269) Fixes #14205 Fixes #14106 This PR reworks the Vite extension in order to supprt `lightningcss` as the pre-processor, enable faster rebuilds, and adds support for `vite build --watch` mode. To make this change possible, we've done two major changes to the extension that have caused the other changes. ## 1. Tailwind CSS is a preprocessor We now run all of our modifications in `enforce: 'pre'`. This means that Tailwind CSS now gets the untransformed CSS files rather than the one already going through postcss or lightningcss. We do this because Tailwind CSS _is_ a preprocessor at the same level as those tools and we do sometimes use the language in ways that [creates problems when it's the input for other bundlers](https://github.com/tailwindlabs/tailwindcss/pull/14269). The correct solution here is to make Tailwind not depend on any other transformations. The main reason we were not using the `enforce: 'pre'` phase in Vite before was becuase we relied on the `@import` flattening of postcss so we now have to do this manually. `@import` flattening is now a concern that every Tailwind V4 client has to deal with so this might actually be something we want to inline into tailwindcss in the future. ## 2. A Vite config can have multiple Tailwind roots This is something that we have not made very explicit in the previous iteration of the Vite plugin but we have to support multiple Tailwind roots in a single Vite workspace. A Tailwind root is a CSS file that is used to configure Tailwind. Technically, any CSS file can be the input for `tailwindcss` but you have to add certain rules (e.g. `@import "tailwindcss";`) to make the compiler do something. A workspace can have multiple of these rules (e.g. by having different Tailwind configures for different sub-pages). With the addition of [support for `@source` rules](https://github.com/tailwindlabs/tailwindcss/pull/14078) and [JS config files](https://github.com/tailwindlabs/tailwindcss/pull/14239), Tailwind roots become more complex and can have a custom list of _dependencies_ (that is other JavaScript modules the compiler includes as part of these new rules). In order to _only rebuild the roots we need to_, we have to make this separation very clear. --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-09-04 10:09:24 +02:00
return new Promise((resolve, reject) => {
let child = exec(
Rework Vite plugin to support lightningcss pre processor and fast rebuilds (#14269) Fixes #14205 Fixes #14106 This PR reworks the Vite extension in order to supprt `lightningcss` as the pre-processor, enable faster rebuilds, and adds support for `vite build --watch` mode. To make this change possible, we've done two major changes to the extension that have caused the other changes. ## 1. Tailwind CSS is a preprocessor We now run all of our modifications in `enforce: 'pre'`. This means that Tailwind CSS now gets the untransformed CSS files rather than the one already going through postcss or lightningcss. We do this because Tailwind CSS _is_ a preprocessor at the same level as those tools and we do sometimes use the language in ways that [creates problems when it's the input for other bundlers](https://github.com/tailwindlabs/tailwindcss/pull/14269). The correct solution here is to make Tailwind not depend on any other transformations. The main reason we were not using the `enforce: 'pre'` phase in Vite before was becuase we relied on the `@import` flattening of postcss so we now have to do this manually. `@import` flattening is now a concern that every Tailwind V4 client has to deal with so this might actually be something we want to inline into tailwindcss in the future. ## 2. A Vite config can have multiple Tailwind roots This is something that we have not made very explicit in the previous iteration of the Vite plugin but we have to support multiple Tailwind roots in a single Vite workspace. A Tailwind root is a CSS file that is used to configure Tailwind. Technically, any CSS file can be the input for `tailwindcss` but you have to add certain rules (e.g. `@import "tailwindcss";`) to make the compiler do something. A workspace can have multiple of these rules (e.g. by having different Tailwind configures for different sub-pages). With the addition of [support for `@source` rules](https://github.com/tailwindlabs/tailwindcss/pull/14078) and [JS config files](https://github.com/tailwindlabs/tailwindcss/pull/14239), Tailwind roots become more complex and can have a custom list of _dependencies_ (that is other JavaScript modules the compiler includes as part of these new rules). In order to _only rebuild the roots we need to_, we have to make this separation very clear. --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-09-04 10:09:24 +02:00
command,
{
cwd,
...childProcessOptions,
Download platform specific package if `optionalDependencies` are skipped (#17929) Closes #15806 This PR adds a new `postinstall` script to `@tailwindcss/oxide` that will attempt to download the platform-specific optional dependency to avoid issues when the package manager does not do that automatically (see #15806). The implementation for this is fairly simple: The npm package is downloaded from the official npm servers and extracted into the `node_modules` directory of the `@tailwidncss/oxide` installation. ## Test plan Since we still lack a solid repro of #15806, the way I tested this change was to manually remove all automatically-installed optional dependencies and then running the postinstall script manually. The script then downloads the right version package which makes the import to `@tailwidncss/oxide` work. An appropriate integration test was added too. I furthermore also validated that: - This works across CI platforms [ci-all] - The postinstall script bails out when running `pnpm install` in the dev setup. This is necessary since doing the initial install won't have any binary dependencies yet so it would download invalid versions from npm (as the version numbers locally refer to the last released version). We can safely bail out here though since this was never an issue with local development. - The postinstall script does not do anything when the `@tailwindcss/oxide` library is added _unless_ the issue is detected. [ci-all] --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2025-05-09 14:55:02 +02:00
env: {
...process.env,
...childProcessOptions.env,
},
Rework Vite plugin to support lightningcss pre processor and fast rebuilds (#14269) Fixes #14205 Fixes #14106 This PR reworks the Vite extension in order to supprt `lightningcss` as the pre-processor, enable faster rebuilds, and adds support for `vite build --watch` mode. To make this change possible, we've done two major changes to the extension that have caused the other changes. ## 1. Tailwind CSS is a preprocessor We now run all of our modifications in `enforce: 'pre'`. This means that Tailwind CSS now gets the untransformed CSS files rather than the one already going through postcss or lightningcss. We do this because Tailwind CSS _is_ a preprocessor at the same level as those tools and we do sometimes use the language in ways that [creates problems when it's the input for other bundlers](https://github.com/tailwindlabs/tailwindcss/pull/14269). The correct solution here is to make Tailwind not depend on any other transformations. The main reason we were not using the `enforce: 'pre'` phase in Vite before was becuase we relied on the `@import` flattening of postcss so we now have to do this manually. `@import` flattening is now a concern that every Tailwind V4 client has to deal with so this might actually be something we want to inline into tailwindcss in the future. ## 2. A Vite config can have multiple Tailwind roots This is something that we have not made very explicit in the previous iteration of the Vite plugin but we have to support multiple Tailwind roots in a single Vite workspace. A Tailwind root is a CSS file that is used to configure Tailwind. Technically, any CSS file can be the input for `tailwindcss` but you have to add certain rules (e.g. `@import "tailwindcss";`) to make the compiler do something. A workspace can have multiple of these rules (e.g. by having different Tailwind configures for different sub-pages). With the addition of [support for `@source` rules](https://github.com/tailwindlabs/tailwindcss/pull/14078) and [JS config files](https://github.com/tailwindlabs/tailwindcss/pull/14239), Tailwind roots become more complex and can have a custom list of _dependencies_ (that is other JavaScript modules the compiler includes as part of these new rules). In order to _only rebuild the roots we need to_, we have to make this separation very clear. --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-09-04 10:09:24 +02:00
},
(error, stdout, stderr) => {
if (error) {
if (command !== originalCommand) {
error.message = error.message.replace(command, originalCommand)
let mappedError = error as any
mappedError.cmd = originalCommand
}
Throw a useful error when `tailwindcss` is used as a PostCSS plugin (#14378) While upgrading a project to Tailwind CSS v4, I forgot to remove the `tailwindcss` import from the PostCSS config. As a result of this, I was greeted with the following message: ``` node:internal/process/promises:289 triggerUncaughtException(err, true /* fromPromise */); ^ [Failed to load PostCSS config: Failed to load PostCSS config (searchPath: /Users/philipp/dev/project): [TypeError] Invalid PostCSS Plugin found at: plugins[0] (@/Users/philipp/dev/project/postcss.config.js) TypeError: Invalid PostCSS Plugin found at: plugins[0] ``` I don't think this was particularly helpful, so I’m proposing we add a default function export to the `tailwindcss` package so when it's used inside PostCSS, we can control the error message. So I changed it to something along these lines: ``` It looks like you're trying to use the \`tailwindcss\` package as a PostCSS plugin. This is no longer possible since Tailwind CSS v4. If you want to continue to use Tailwind CSS with PostCSS, please install \`@tailwindcss/postcss\` and change your PostCSS config file. at w (/Users/philipp/dev/project/node_modules/tailwindcss/node_modules/tailwindcss/dist/lib.js:1:21233) at Object.<anonymous> (/Users/philipp/dev/project/node_modules/tailwindcss/postcss.config.cjs:3:13) at Module._compile (node:internal/modules/cjs/loader:1358:14) at Module._extensions..js (node:internal/modules/cjs/loader:1416:10) at Module.load (node:internal/modules/cjs/loader:1208:32) at Module._load (node:internal/modules/cjs/loader:1024:12) at cjsLoader (node:internal/modules/esm/translators:348:17) at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:297:7) at ModuleJob.run (node:internal/modules/esm/module_job:222:25) at async ModuleLoader.import (node:internal/modules/esm/loader:316:24) ``` This is also a good place to link to the migration guides once we have them 🙂 --------- Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-09-11 17:02:24 +02:00
if (execOptions.ignoreStdErr !== true) console.error(stderr)
Auto source detection improvements (#14820) This PR introduces a new `source(…)` argument and improves on the existing `@source`. The goal of this PR is to make the automatic source detection configurable, let's dig in. By default, we will perform automatic source detection starting at the current working directory. Auto source detection will find plain text files (no binaries, images, ...) and will ignore git-ignored files. If you want to start from a different directory, you can use the new `source(…)` next to the `@import "tailwindcss/utilities" layer(utilities) source(…)`. E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss/utilities' layer(utilities) source('../../'); ``` Most people won't split their source files, and will just use the simple `@import "tailwindcss";`, because of this reason, you can use `source(…)` on the import as well: E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss' source('../../'); ``` Sometimes, you want to rely on auto source detection, but also want to look in another directory for source files. In this case, yuo can use the `@source` directive: ```css /* ./src/index.css */ @import 'tailwindcss'; /* Look for `blade.php` files in `../resources/views` */ @source '../resources/views/**/*.blade.php'; ``` However, you don't need to specify the extension, instead you can just point the directory and all the same automatic source detection rules will apply. ```css /* ./src/index.css */ @import 'tailwindcss'; @source '../resources/views'; ``` If, for whatever reason, you want to disable the default source detection feature entirely, and only want to rely on very specific glob patterns you define, then you can disable it via `source(none)`. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Only look at .blade.php files, nothing else */ @source "../resources/views/**/*.blade.php"; ``` Note: even with `source(none)`, if your `@source` points to a directory, then auto source detection will still be performed in that directory. If you don't want that, then you can simply add explicit files in the globs as seen in the previous example. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Run auto source detection in `../resources/views` */ @source "../resources/views"; ``` --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
if (only || debug) {
console.error(stdout)
}
Rework Vite plugin to support lightningcss pre processor and fast rebuilds (#14269) Fixes #14205 Fixes #14106 This PR reworks the Vite extension in order to supprt `lightningcss` as the pre-processor, enable faster rebuilds, and adds support for `vite build --watch` mode. To make this change possible, we've done two major changes to the extension that have caused the other changes. ## 1. Tailwind CSS is a preprocessor We now run all of our modifications in `enforce: 'pre'`. This means that Tailwind CSS now gets the untransformed CSS files rather than the one already going through postcss or lightningcss. We do this because Tailwind CSS _is_ a preprocessor at the same level as those tools and we do sometimes use the language in ways that [creates problems when it's the input for other bundlers](https://github.com/tailwindlabs/tailwindcss/pull/14269). The correct solution here is to make Tailwind not depend on any other transformations. The main reason we were not using the `enforce: 'pre'` phase in Vite before was becuase we relied on the `@import` flattening of postcss so we now have to do this manually. `@import` flattening is now a concern that every Tailwind V4 client has to deal with so this might actually be something we want to inline into tailwindcss in the future. ## 2. A Vite config can have multiple Tailwind roots This is something that we have not made very explicit in the previous iteration of the Vite plugin but we have to support multiple Tailwind roots in a single Vite workspace. A Tailwind root is a CSS file that is used to configure Tailwind. Technically, any CSS file can be the input for `tailwindcss` but you have to add certain rules (e.g. `@import "tailwindcss";`) to make the compiler do something. A workspace can have multiple of these rules (e.g. by having different Tailwind configures for different sub-pages). With the addition of [support for `@source` rules](https://github.com/tailwindlabs/tailwindcss/pull/14078) and [JS config files](https://github.com/tailwindlabs/tailwindcss/pull/14239), Tailwind roots become more complex and can have a custom list of _dependencies_ (that is other JavaScript modules the compiler includes as part of these new rules). In order to _only rebuild the roots we need to_, we have to make this separation very clear. --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-09-04 10:09:24 +02:00
reject(error)
} else {
Auto source detection improvements (#14820) This PR introduces a new `source(…)` argument and improves on the existing `@source`. The goal of this PR is to make the automatic source detection configurable, let's dig in. By default, we will perform automatic source detection starting at the current working directory. Auto source detection will find plain text files (no binaries, images, ...) and will ignore git-ignored files. If you want to start from a different directory, you can use the new `source(…)` next to the `@import "tailwindcss/utilities" layer(utilities) source(…)`. E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss/utilities' layer(utilities) source('../../'); ``` Most people won't split their source files, and will just use the simple `@import "tailwindcss";`, because of this reason, you can use `source(…)` on the import as well: E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss' source('../../'); ``` Sometimes, you want to rely on auto source detection, but also want to look in another directory for source files. In this case, yuo can use the `@source` directive: ```css /* ./src/index.css */ @import 'tailwindcss'; /* Look for `blade.php` files in `../resources/views` */ @source '../resources/views/**/*.blade.php'; ``` However, you don't need to specify the extension, instead you can just point the directory and all the same automatic source detection rules will apply. ```css /* ./src/index.css */ @import 'tailwindcss'; @source '../resources/views'; ``` If, for whatever reason, you want to disable the default source detection feature entirely, and only want to rely on very specific glob patterns you define, then you can disable it via `source(none)`. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Only look at .blade.php files, nothing else */ @source "../resources/views/**/*.blade.php"; ``` Note: even with `source(none)`, if your `@source` points to a directory, then auto source detection will still be performed in that directory. If you don't want that, then you can simply add explicit files in the globs as seen in the previous example. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Run auto source detection in `../resources/views` */ @source "../resources/views"; ``` --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
if (only || debug) {
console.log(stdout.toString() + '\n\n' + stderr.toString())
}
resolve(stdout.toString() + '\n\n' + stderr.toString())
Rework Vite plugin to support lightningcss pre processor and fast rebuilds (#14269) Fixes #14205 Fixes #14106 This PR reworks the Vite extension in order to supprt `lightningcss` as the pre-processor, enable faster rebuilds, and adds support for `vite build --watch` mode. To make this change possible, we've done two major changes to the extension that have caused the other changes. ## 1. Tailwind CSS is a preprocessor We now run all of our modifications in `enforce: 'pre'`. This means that Tailwind CSS now gets the untransformed CSS files rather than the one already going through postcss or lightningcss. We do this because Tailwind CSS _is_ a preprocessor at the same level as those tools and we do sometimes use the language in ways that [creates problems when it's the input for other bundlers](https://github.com/tailwindlabs/tailwindcss/pull/14269). The correct solution here is to make Tailwind not depend on any other transformations. The main reason we were not using the `enforce: 'pre'` phase in Vite before was becuase we relied on the `@import` flattening of postcss so we now have to do this manually. `@import` flattening is now a concern that every Tailwind V4 client has to deal with so this might actually be something we want to inline into tailwindcss in the future. ## 2. A Vite config can have multiple Tailwind roots This is something that we have not made very explicit in the previous iteration of the Vite plugin but we have to support multiple Tailwind roots in a single Vite workspace. A Tailwind root is a CSS file that is used to configure Tailwind. Technically, any CSS file can be the input for `tailwindcss` but you have to add certain rules (e.g. `@import "tailwindcss";`) to make the compiler do something. A workspace can have multiple of these rules (e.g. by having different Tailwind configures for different sub-pages). With the addition of [support for `@source` rules](https://github.com/tailwindlabs/tailwindcss/pull/14078) and [JS config files](https://github.com/tailwindlabs/tailwindcss/pull/14239), Tailwind roots become more complex and can have a custom list of _dependencies_ (that is other JavaScript modules the compiler includes as part of these new rules). In order to _only rebuild the roots we need to_, we have to make this separation very clear. --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-09-04 10:09:24 +02:00
}
},
)
if (execOptions.stdin) {
child.stdin?.write(execOptions.stdin)
child.stdin?.end()
}
Rework Vite plugin to support lightningcss pre processor and fast rebuilds (#14269) Fixes #14205 Fixes #14106 This PR reworks the Vite extension in order to supprt `lightningcss` as the pre-processor, enable faster rebuilds, and adds support for `vite build --watch` mode. To make this change possible, we've done two major changes to the extension that have caused the other changes. ## 1. Tailwind CSS is a preprocessor We now run all of our modifications in `enforce: 'pre'`. This means that Tailwind CSS now gets the untransformed CSS files rather than the one already going through postcss or lightningcss. We do this because Tailwind CSS _is_ a preprocessor at the same level as those tools and we do sometimes use the language in ways that [creates problems when it's the input for other bundlers](https://github.com/tailwindlabs/tailwindcss/pull/14269). The correct solution here is to make Tailwind not depend on any other transformations. The main reason we were not using the `enforce: 'pre'` phase in Vite before was becuase we relied on the `@import` flattening of postcss so we now have to do this manually. `@import` flattening is now a concern that every Tailwind V4 client has to deal with so this might actually be something we want to inline into tailwindcss in the future. ## 2. A Vite config can have multiple Tailwind roots This is something that we have not made very explicit in the previous iteration of the Vite plugin but we have to support multiple Tailwind roots in a single Vite workspace. A Tailwind root is a CSS file that is used to configure Tailwind. Technically, any CSS file can be the input for `tailwindcss` but you have to add certain rules (e.g. `@import "tailwindcss";`) to make the compiler do something. A workspace can have multiple of these rules (e.g. by having different Tailwind configures for different sub-pages). With the addition of [support for `@source` rules](https://github.com/tailwindlabs/tailwindcss/pull/14078) and [JS config files](https://github.com/tailwindlabs/tailwindcss/pull/14239), Tailwind roots become more complex and can have a custom list of _dependencies_ (that is other JavaScript modules the compiler includes as part of these new rules). In order to _only rebuild the roots we need to_, we have to make this separation very clear. --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-09-04 10:09:24 +02:00
})
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
},
async spawn(command: string, childProcessOptions: ChildProcessOptions = {}) {
let resolveDisposal: (() => void) | undefined
let rejectDisposal: ((error: Error) => void) | undefined
let disposePromise = new Promise<void>((resolve, reject) => {
resolveDisposal = resolve
rejectDisposal = reject
})
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
let cwd = childProcessOptions.cwd ?? root
if (debug && cwd !== root) {
let relative = path.relative(root, cwd)
if (relative[0] !== '.') relative = `./${relative}`
console.log(`> cd ${relative}`)
}
if (debug) console.log(`>& ${command}`)
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
let child = spawn(command, {
cwd,
detached: !IS_WINDOWS,
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
shell: true,
...childProcessOptions,
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
env: {
...process.env,
...childProcessOptions.env,
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
},
})
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
let disposed = false
async function dispose() {
if (disposed) return disposePromise
disposed = true
await killProcessTree(child)
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
let timer = setTimeout(() => {
forceKillProcessTree(child)
rejectDisposal?.(new Error(`spawned process (${command}) did not exit in time`))
}, ASSERTION_TIMEOUT)
disposePromise.then(
() => clearTimeout(timer),
() => clearTimeout(timer),
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
)
return disposePromise
}
disposables.push(dispose)
Make upgrade tooling more stable (#19846) This PR is an attempt to make the upgrade tooling more stable. ### TL;DR 1. When migrating from Tailwind CSS v3 → Tailwind CSS v4, only migrate files listed in the `config.content` instead of relying on v4's auto content detection feature 2. Skip writing files that have not been changed 3. Write changed files in a safe way: first write to a temporary file, then rename the file atomically 4. Never migrate files that are git ignored, even if they are listed in the `config.content` file 5. Always ignore `.env` and `.env.*` files when scanning for files. Most people will have this in their `.gitignore` file, but if not, then this is a fallback mechanism. --- Looking at the #18972 issue, it looks like some people are running into weird situations where some of the contents is just gone. I have never been able to reproduce this on my own devices and in my own projects unfortunately. But there is definitely _something_ happening that's not right that people are running into. Therefore, this PR is an attempt to fix what I think _might_ be wrong, but I'm not 100% sure if these fixes are enough, or if something else is still happening here. This builds on top of the #19779 PR which has some small fixes, but is incomplete to make this work. ### What's happening Looking at some of the comments, it looks like a few things are happening such as: 1. The upgrade tool is emptying out my files — it looks like these are only happening if you ctrl+c while the process is taking a while. It could be that a lot of files are being checked and therefore the tooling looks like its stuck. 6. The upgrade tool is looking at files it shouldn't look at — in Tailwind CSS v4 we have this concept of the auto-content detection. This means that we will look at any plain text file that is not git ignored. ### Fixes #### Emptying out files The files being emptied looks like it's because how `fs.writeFile` behaves by default. It opens the file handle with the `w` flag, which will first truncate the file before writing the new contents. This is not a single atomic operation, so a killed process in the middle will cause invalid state. When we migrate your template files, everything is happening in promises to migrate things at the same time. When a lot of files are being scanned, truncating might have happened already before we write the new content. Since we migrate a bunch of files in parallel, a ctrl+c could cause data loss in multiple files. To mitigate this, I switched to an alternative way of writing files. 1. First, we do some quick checks where if the contents didn't change we just bail out immediately. Files that don't include Tailwind CSS classes won't change, and therefore we don't need to override these files with the same contents. 2. When the migrated contents is empty, we bail out as well. I'm 100% sure that this is not the spot where the "emptying out" happens, I still believe it happens in the `writeFile` itself, but added it just in case. 7. Next, I introduced a safe write, where we first write to a temporary file in the same folder. We could write it to `/tmp`, but then we can't guarantee that we are on the same file system. If we ctrl+c at this stage, then the worst case scenario is that you have additional temporary files in your project, but your original files are still there. Once that file was written, we will use the atomic `fs.rename`. This should be atomic as long as we are on the same file system, so either the rename didn't happen yet, or it completed. I added an integration test for this, but I had to change the `writeFile` implementation slightly. In the test, we will truncate the file first, after that we will write the new contents. This is so that we have enough time to kill the current process and allows us to verify that we didn't clear out the file. Again, this is a hacky way of testing this, just because I can't reproduce this issue myself, let alone reproduce it reliable in a CI environment. Note: we are also using `realpath` to make sure that we are updating the real file. Otherwise, if we were dealing with a symlinked file, we would override the symlink with a "hard" copy instead. #### Touching files that should not be touched During the migration, we rely on the Tailwind CSS v4 auto detection logic which means that it will scan any plain text file that is not git ignored. Therefore changes to php files could happen because in theory they could contain Tailwind CSS classes. To solve this, when migrating from Tailwind CSS v3 to Tailwind CSS v4, we will _only_ take the sources into account that were listed in the `config.content` array. Since this was a requirement in Tailwind CSS v3, it should be safe to rely on this array. Additionally, this will make sure that we are dealing with way fewer files to migrate as well. On top of that, files that match the patterns in the content array that are git ignored will also be skipped. This is to prevent that we mutate files in `node_modules` for example. In one of the comments I read that `.env` files were emptied out. In most cases people will have these files gitignored but I explicitly added `.env` and `.env.*` as files to never ever touch by default when scanning. Last but not least, this also updates the output a little bit of the upgrade tool in case we skip content files (because of git ignore) and if we changed a file. <img width="1122" height="1376" alt="image" src="https://github.com/user-attachments/assets/318fdbbf-e319-4c7e-9648-ee9283842624" /> - "Git ignored folder, skipping: `./node_modules`": this is because the content array looks like this while the `node_modules` are being ignored: <img width="1090" height="398" alt="image" src="https://github.com/user-attachments/assets/7d694720-5671-47ec-bb2e-f24c5f2c4248" /> - "Migrated `./resources/views/vendor/filament-panels/components/logo.blade.php`": this is because **I** made a change to showcase this feature. Fixes: #18972 Closes: #19779 ### Test plan 1. Existing tests still pass 1. Added a dedicated integration test to ensure that we only take `config.content` into account when migrating from Tailwind CSS v3 to Tailwind CSS v4 projects. 1. Added a dedicated integration test to make sure that files listed in `config.content` that are also git ignored, will still be skipped. 1. Added a dedicated integration test to ensure that when `writeFile` is cancelled mid-write that our old files are still present. 1. Added a dedicated integration test to ensure that we ignore `.env` and `.env.*` files even if you didn't git ignore them. [ci-all] To verify on Windows --------- Co-authored-by: Sami <sychocouldy@gmail.com>
2026-03-24 17:24:12 +01:00
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
function onExit() {
resolveDisposal?.()
}
let stdoutMessages: string[] = []
let stderrMessages: string[] = []
let stdoutActors: SpawnActor[] = []
let stderrActors: SpawnActor[] = []
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
function notifyNext(actors: SpawnActor[], messages: string[]) {
if (actors.length <= 0) return
let [next] = actors
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
for (let [idx, message] of messages.entries()) {
if (next.predicate(message)) {
messages.splice(0, idx + 1)
let actorIdx = actors.indexOf(next)
actors.splice(actorIdx, 1)
next.resolve()
break
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
let combined: ['stdout' | 'stderr', string][] = []
child.stdout.on('data', (result) => {
let content = result.toString()
Auto source detection improvements (#14820) This PR introduces a new `source(…)` argument and improves on the existing `@source`. The goal of this PR is to make the automatic source detection configurable, let's dig in. By default, we will perform automatic source detection starting at the current working directory. Auto source detection will find plain text files (no binaries, images, ...) and will ignore git-ignored files. If you want to start from a different directory, you can use the new `source(…)` next to the `@import "tailwindcss/utilities" layer(utilities) source(…)`. E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss/utilities' layer(utilities) source('../../'); ``` Most people won't split their source files, and will just use the simple `@import "tailwindcss";`, because of this reason, you can use `source(…)` on the import as well: E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss' source('../../'); ``` Sometimes, you want to rely on auto source detection, but also want to look in another directory for source files. In this case, yuo can use the `@source` directive: ```css /* ./src/index.css */ @import 'tailwindcss'; /* Look for `blade.php` files in `../resources/views` */ @source '../resources/views/**/*.blade.php'; ``` However, you don't need to specify the extension, instead you can just point the directory and all the same automatic source detection rules will apply. ```css /* ./src/index.css */ @import 'tailwindcss'; @source '../resources/views'; ``` If, for whatever reason, you want to disable the default source detection feature entirely, and only want to rely on very specific glob patterns you define, then you can disable it via `source(none)`. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Only look at .blade.php files, nothing else */ @source "../resources/views/**/*.blade.php"; ``` Note: even with `source(none)`, if your `@source` points to a directory, then auto source detection will still be performed in that directory. If you don't want that, then you can simply add explicit files in the globs as seen in the previous example. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Run auto source detection in `../resources/views` */ @source "../resources/views"; ``` --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
if (debug || only) console.log(content)
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
combined.push(['stdout', content])
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
for (let line of content.split('\n')) {
stdoutMessages.push(stripVTControlCharacters(line))
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
notifyNext(stdoutActors, stdoutMessages)
})
child.stderr.on('data', (result) => {
let content = result.toString()
Auto source detection improvements (#14820) This PR introduces a new `source(…)` argument and improves on the existing `@source`. The goal of this PR is to make the automatic source detection configurable, let's dig in. By default, we will perform automatic source detection starting at the current working directory. Auto source detection will find plain text files (no binaries, images, ...) and will ignore git-ignored files. If you want to start from a different directory, you can use the new `source(…)` next to the `@import "tailwindcss/utilities" layer(utilities) source(…)`. E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss/utilities' layer(utilities) source('../../'); ``` Most people won't split their source files, and will just use the simple `@import "tailwindcss";`, because of this reason, you can use `source(…)` on the import as well: E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss' source('../../'); ``` Sometimes, you want to rely on auto source detection, but also want to look in another directory for source files. In this case, yuo can use the `@source` directive: ```css /* ./src/index.css */ @import 'tailwindcss'; /* Look for `blade.php` files in `../resources/views` */ @source '../resources/views/**/*.blade.php'; ``` However, you don't need to specify the extension, instead you can just point the directory and all the same automatic source detection rules will apply. ```css /* ./src/index.css */ @import 'tailwindcss'; @source '../resources/views'; ``` If, for whatever reason, you want to disable the default source detection feature entirely, and only want to rely on very specific glob patterns you define, then you can disable it via `source(none)`. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Only look at .blade.php files, nothing else */ @source "../resources/views/**/*.blade.php"; ``` Note: even with `source(none)`, if your `@source` points to a directory, then auto source detection will still be performed in that directory. If you don't want that, then you can simply add explicit files in the globs as seen in the previous example. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Run auto source detection in `../resources/views` */ @source "../resources/views"; ``` --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
if (debug || only) console.error(content)
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
combined.push(['stderr', content])
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
for (let line of content.split('\n')) {
stderrMessages.push(stripVTControlCharacters(line))
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
notifyNext(stderrActors, stderrMessages)
})
child.on('exit', onExit)
child.on('error', (error) => {
if (error.name !== 'AbortError') {
throw error
}
})
options.onTestFailed(() => {
2024-11-04 13:07:25 -05:00
// In only or debug mode, messages are logged to the console
// immediately.
if (only || debug) return
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
for (let [type, message] of combined) {
if (type === 'stdout') {
console.log(message)
} else {
console.error(message)
}
}
})
return {
dispose,
flush() {
stdoutActors.splice(0)
stderrActors.splice(0)
stdoutMessages.splice(0)
stderrMessages.splice(0)
},
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
onStdout(predicate: (message: string) => boolean) {
return new Promise<void>((resolve) => {
stdoutActors.push({ predicate, resolve })
notifyNext(stdoutActors, stdoutMessages)
})
},
onStderr(predicate: (message: string) => boolean) {
return new Promise<void>((resolve) => {
stderrActors.push({ predicate, resolve })
notifyNext(stderrActors, stderrMessages)
})
},
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
},
fs: {
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
async write(filename, content, encoding = 'utf8') {
let full = path.join(root, filename)
let dir = path.dirname(full)
await fs.mkdir(dir, { recursive: true })
if (typeof content !== 'string') {
return await fs.writeFile(full, content)
}
if (filename.endsWith('package.json')) {
content = await overwriteVersionsInPackageJson(content)
} else if (filename.endsWith('pnpm-workspace.yaml')) {
content = overwriteVersionsInPnpmWorkspace(content)
}
// Ensure that files written on Windows use \r\n line ending
if (IS_WINDOWS) {
content = content.replace(/\n/g, '\r\n')
}
await fs.writeFile(full, content, encoding)
},
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
async symlink(target, src) {
let targetAbsolute = path.join(root, target)
let targetParent = path.dirname(targetAbsolute)
await fs.mkdir(targetParent, { recursive: true })
let srcAbsolute = path.join(root, src)
let srcParent = path.dirname(srcAbsolute)
await fs.mkdir(srcParent, { recursive: true })
await fs.symlink(targetAbsolute, srcAbsolute)
},
async create(filenames) {
for (let filename of filenames) {
let full = path.join(root, filename)
let dir = path.dirname(full)
await fs.mkdir(dir, { recursive: true })
await fs.writeFile(full, '')
}
},
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
async delete(filename) {
Ensure `@tailwindcss/cli` recovers from a deleted transitive dependency (#20137) This PR fixes an issue where the `@tailwindcss/cli` can get into a non-recoverable state when any of the transitive dependencies break. Tailwind CSS has 2 kinds of dependencies: 1. All your templates 2. All dependencies that contribute to your configuration such as the `input.css`, any plugins, any `tailwind.config.js` files and so on. When a template changes, we just have to scan for new Tailwind CSS classes and emit a new CSS file. But when the `input.css` file, or any of its dependencies changes, then we want to perform a full rebuild. The idea is that your `@theme` might have changed, or new plugins have been added, or old plugins have been removed. If you have an `input.css` file: ```css @import "tailwindcss"; @config "./tailwind.config.js"; ``` That relies on a custom config: `tailwind.config.js`: ```js const theme = require('./my-custom-theme.js'); module.exports = { theme } ``` If that file relies on yet another file: `./my-custom-theme.js`, then changes there should also trigger a full rebuild. Since we're dealing with JavaScript here, we want to clear the require cache and rebuild the dependency tree such that another change to any of these files triggers a full fresh build. However, if any of those (transitive) dependencies are deleted, then we will end up in an invalid state. Creating a new compiler will result in a build error. The compiler won't be able to figure out the entire dependency tree, and we're stuck. Once the user fixes the potentially missing dependency, the watchers will not be watching any of those files because we created a fresh compiler. With this PR, we fix that by keeping track of old paths and using those while we are still in an invalid state. The moment everything is fixed, a fresh dependency tree is created and everything starts working again without you having to restart the `@tailwindcss/cli` command. Fixes: #20113 Closes: #20114 Closes: #20133 ## Test plan - Added an integration test that removes the transitive dependency. Re-adding that file later will recover the CLI state.
2026-06-01 19:15:40 +02:00
await fs.unlink(path.join(root, filename))
},
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
async read(filePath) {
Vite: Retain candidates between input CSS updates (#14228) This PR fixes an issue introduced with the changed candidate cache behavior in #14187. Prior to #14187, candidates were cached globally within an instance of Oxide. This meant that once a candidate was discovered, it would not reset until you either manually cleared the cache or restarted the Oxide process. With the changes in #14187 however, the cache was scoped to the instance of the `Scanner` class with the intention of making the caching behavior more easy to understand and to avoid a global cache. This, however, had an unforeseen side-effect in our Vite extension. Vite, in dev mode, discovers files _lazily_. So when a developer goes to `/index.html` the first time, we will scan the `/index.html` file for Tailwind candidates and then build a CSS file with those candidate. When they go to `/about.html` later, we will _append_ the candidates from the new file and so forth. The problem now arises when the dev server detects changes to the input CSS file. This requires us to do a re-scan of that CSS file which, after #14187, caused the candidate cache to be gone. This is usually fine since we would just scan files again for the changed candidate list but in the Vite case we would only get the input CSS file change _but no subsequent change events for all other files, including those currently rendered in the browser_). This caused updates to the CSS file to remove all candidates from the CSS file again. Ideally, we can separate between two concepts: The candidate cache and the CSS input file scan. An instance of the `Scanner` could re-parse the input CSS file without having to throw away previous candidates. This, however, would have another issue with the current Vite extension where we do not properly retain instances of the `Scanner` class anyways. To properly improve the cache behavior, we will have to fix the Vite `Scanner` retaining behavior first. Unfortunately this means that for the short term, we have to add some manual bookkeeping to the Vite client and retain the candidate cache between builds ourselves. --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
let content = await fs.readFile(path.resolve(root, filePath), 'utf8')
// Ensure that files read on Windows have \r\n line endings removed
if (IS_WINDOWS) {
content = content.replace(/\r\n/g, '\n')
}
return content
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
},
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
async glob(pattern) {
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
let files = await fastGlob(pattern, { cwd: root })
return Promise.all(
files.map(async (file) => {
let content = await fs.readFile(path.join(root, file), 'utf8')
Auto source detection improvements (#14820) This PR introduces a new `source(…)` argument and improves on the existing `@source`. The goal of this PR is to make the automatic source detection configurable, let's dig in. By default, we will perform automatic source detection starting at the current working directory. Auto source detection will find plain text files (no binaries, images, ...) and will ignore git-ignored files. If you want to start from a different directory, you can use the new `source(…)` next to the `@import "tailwindcss/utilities" layer(utilities) source(…)`. E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss/utilities' layer(utilities) source('../../'); ``` Most people won't split their source files, and will just use the simple `@import "tailwindcss";`, because of this reason, you can use `source(…)` on the import as well: E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss' source('../../'); ``` Sometimes, you want to rely on auto source detection, but also want to look in another directory for source files. In this case, yuo can use the `@source` directive: ```css /* ./src/index.css */ @import 'tailwindcss'; /* Look for `blade.php` files in `../resources/views` */ @source '../resources/views/**/*.blade.php'; ``` However, you don't need to specify the extension, instead you can just point the directory and all the same automatic source detection rules will apply. ```css /* ./src/index.css */ @import 'tailwindcss'; @source '../resources/views'; ``` If, for whatever reason, you want to disable the default source detection feature entirely, and only want to rely on very specific glob patterns you define, then you can disable it via `source(none)`. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Only look at .blade.php files, nothing else */ @source "../resources/views/**/*.blade.php"; ``` Note: even with `source(none)`, if your `@source` points to a directory, then auto source detection will still be performed in that directory. If you don't want that, then you can simply add explicit files in the globs as seen in the previous example. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Run auto source detection in `../resources/views` */ @source "../resources/views"; ``` --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
return [
file,
// Drop license comment
Watch CSS module files for changes (#17467) This PR is a follow-up PR for: https://github.com/tailwindlabs/tailwindcss/pull/17433 In the other PR we allow scanning CSS files for extracting usages of CSS variables. This is important for `.module.css` files that reference these variables but aren't in the same big AST of the main CSS file. This PR also makes sure to watch for changes in those registered CSS files and re-extract the variables when they change. This PR took a bit longer than expected because I was trying to make sure that writing to `./dist/out.css` works without infinite-looping (e.g.: we had issues with this in Tailwind CSS v3 with webpack). But I couldn't reproduce the issue at all. I did had some code that tried to detect if the CSS file contained license headers and skip in (because then it's very likely an output CSS file) but even without it the tests were fine. I setup integration tests with `@tailwindcss/cli` itself, and with tools that use webpack. Added a test for Next.js, and a dedicated webpack test as well. Even without tests, locally, I couldn't reproduce an infinite loop due to changes in an output CSS file... Eventually dropped the code that tries to detect output CSS files. One thing to keep in mind is that if you change any of your "main" CSS files, then we will trigger a full rebuild anyway, so this change is only required for unrelated CSS files (like CSS module files) that use CSS variables. ## Test plan 1. Added integration tests for the CLI and Next.js 2. Added new dedicated test for webpack
2025-03-31 18:44:06 +02:00
content.replace(/[\s\n]*\/\*![\s\S]*?\*\/[\s\n]*/g, ''),
Auto source detection improvements (#14820) This PR introduces a new `source(…)` argument and improves on the existing `@source`. The goal of this PR is to make the automatic source detection configurable, let's dig in. By default, we will perform automatic source detection starting at the current working directory. Auto source detection will find plain text files (no binaries, images, ...) and will ignore git-ignored files. If you want to start from a different directory, you can use the new `source(…)` next to the `@import "tailwindcss/utilities" layer(utilities) source(…)`. E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss/utilities' layer(utilities) source('../../'); ``` Most people won't split their source files, and will just use the simple `@import "tailwindcss";`, because of this reason, you can use `source(…)` on the import as well: E.g.: ```css /* ./src/styles/index.css */ @import 'tailwindcss' source('../../'); ``` Sometimes, you want to rely on auto source detection, but also want to look in another directory for source files. In this case, yuo can use the `@source` directive: ```css /* ./src/index.css */ @import 'tailwindcss'; /* Look for `blade.php` files in `../resources/views` */ @source '../resources/views/**/*.blade.php'; ``` However, you don't need to specify the extension, instead you can just point the directory and all the same automatic source detection rules will apply. ```css /* ./src/index.css */ @import 'tailwindcss'; @source '../resources/views'; ``` If, for whatever reason, you want to disable the default source detection feature entirely, and only want to rely on very specific glob patterns you define, then you can disable it via `source(none)`. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Only look at .blade.php files, nothing else */ @source "../resources/views/**/*.blade.php"; ``` Note: even with `source(none)`, if your `@source` points to a directory, then auto source detection will still be performed in that directory. If you don't want that, then you can simply add explicit files in the globs as seen in the previous example. ```css /* Completely disable the default auto source detection */ @import 'tailwindcss' source(none); /* Run auto source detection in `../resources/views` */ @source "../resources/views"; ``` --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
]
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
}),
)
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
},
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
async dumpFiles(pattern) {
let files = await context.fs.glob(pattern)
return `\n${files
.slice()
.sort((a: [string], z: [string]) => {
let aParts = a[0].split('/')
let zParts = z[0].split('/')
let aFile = aParts.at(-1)
Improve multi-root `@config` linking (#15001) This PR improves the discoverability of Tailwind config files when we are trying to link them to your CSS files. When you have multiple "root" CSS files in your project, and if they don't include an `@config` directive, then we tried to find the Tailwind config file in your current working directory. This means that if you run the upgrade command from the root of your project, and you have a nested folder with a separate Tailwind setup, then the nested CSS file would link to the root Tailwind config file. Visually, you can think of it like this: ``` . ├── admin │   ├── src │   │   └── styles │   │   └── index.css <-- This will be linked to (1) │   └── tailwind.config.js (2) ├── src │   └── styles │   └── index.css <-- This will be linked to (1) └── tailwind.config.js (1) ``` If you run the upgrade command from the root of your project, then the `/src/styles/index.css` will be linked to `/tailwind.config.js` which is what we expect. But `/admin/src/styles/index.css` will _also_ be linked to `/tailwind.config.js` With this PR we improve this behavior by looking at the CSS file, and crawling up the parent tree. This mens that the new behavior looks like this: ``` . ├── admin │   ├── src │   │   └── styles │   │   └── index.css <-- This will be linked to (2) │   └── tailwind.config.js (2) ├── src │   └── styles │   └── index.css <-- This will be linked to (1) └── tailwind.config.js (1) ``` Now `/src/styles/index.css` will be linked to `/tailwind.config.js`, and `/admin/src/styles/index.css` will be linked to `/admin/tailwind.config.js`. When we discover the Tailwind config file, we will also print a message to the user to let them know which CSS file is linked to which Tailwind config file. This should be a safe improvement because if your Tailwind config file had a different name, or if it lived in a sibling folder then Tailwind wouldn't find it either and you already required a `@config "…";` directive in your CSS file to point to the correct file. In the unlikely event that it turns out that 2 (or more) CSS files resolve to the same to the same Tailwind config file, then an upgrade might not be safe and some manual intervention might be needed. In this case, we will show a warning about this. <img width="1552" alt="image" src="https://github.com/user-attachments/assets/7a1ad11d-18c5-4b7d-9a02-14f0116ae955"> Test plan: --- - Added an integration test that properly links the nearest Tailwind config file by looking up the tree - Added an integration test that resolves 2 or more CSS files to the same config file, resulting in an error where manual intervention is needed - Ran it on the Tailwind UI codebase Running this on Tailwind UI's codebase it looks like this: <img width="1552" alt="image" src="https://github.com/user-attachments/assets/21785428-5e0d-47f7-80ec-dab497f58784"> --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-11-18 17:43:44 +01:00
let zFile = zParts.at(-1)
// Sort by depth, shallow first
if (aParts.length < zParts.length) return -1
if (aParts.length > zParts.length) return 1
Improve multi-root `@config` linking (#15001) This PR improves the discoverability of Tailwind config files when we are trying to link them to your CSS files. When you have multiple "root" CSS files in your project, and if they don't include an `@config` directive, then we tried to find the Tailwind config file in your current working directory. This means that if you run the upgrade command from the root of your project, and you have a nested folder with a separate Tailwind setup, then the nested CSS file would link to the root Tailwind config file. Visually, you can think of it like this: ``` . ├── admin │   ├── src │   │   └── styles │   │   └── index.css <-- This will be linked to (1) │   └── tailwind.config.js (2) ├── src │   └── styles │   └── index.css <-- This will be linked to (1) └── tailwind.config.js (1) ``` If you run the upgrade command from the root of your project, then the `/src/styles/index.css` will be linked to `/tailwind.config.js` which is what we expect. But `/admin/src/styles/index.css` will _also_ be linked to `/tailwind.config.js` With this PR we improve this behavior by looking at the CSS file, and crawling up the parent tree. This mens that the new behavior looks like this: ``` . ├── admin │   ├── src │   │   └── styles │   │   └── index.css <-- This will be linked to (2) │   └── tailwind.config.js (2) ├── src │   └── styles │   └── index.css <-- This will be linked to (1) └── tailwind.config.js (1) ``` Now `/src/styles/index.css` will be linked to `/tailwind.config.js`, and `/admin/src/styles/index.css` will be linked to `/admin/tailwind.config.js`. When we discover the Tailwind config file, we will also print a message to the user to let them know which CSS file is linked to which Tailwind config file. This should be a safe improvement because if your Tailwind config file had a different name, or if it lived in a sibling folder then Tailwind wouldn't find it either and you already required a `@config "…";` directive in your CSS file to point to the correct file. In the unlikely event that it turns out that 2 (or more) CSS files resolve to the same to the same Tailwind config file, then an upgrade might not be safe and some manual intervention might be needed. In this case, we will show a warning about this. <img width="1552" alt="image" src="https://github.com/user-attachments/assets/7a1ad11d-18c5-4b7d-9a02-14f0116ae955"> Test plan: --- - Added an integration test that properly links the nearest Tailwind config file by looking up the tree - Added an integration test that resolves 2 or more CSS files to the same config file, resulting in an error where manual intervention is needed - Ran it on the Tailwind UI codebase Running this on Tailwind UI's codebase it looks like this: <img width="1552" alt="image" src="https://github.com/user-attachments/assets/21785428-5e0d-47f7-80ec-dab497f58784"> --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-11-18 17:43:44 +01:00
// Sort by folder names, alphabetically
for (let i = 0; i < aParts.length - 1; i++) {
let diff = aParts[i].localeCompare(zParts[i])
if (diff !== 0) return diff
}
// Sort by filename, sort files named `index` before others
Improve multi-root `@config` linking (#15001) This PR improves the discoverability of Tailwind config files when we are trying to link them to your CSS files. When you have multiple "root" CSS files in your project, and if they don't include an `@config` directive, then we tried to find the Tailwind config file in your current working directory. This means that if you run the upgrade command from the root of your project, and you have a nested folder with a separate Tailwind setup, then the nested CSS file would link to the root Tailwind config file. Visually, you can think of it like this: ``` . ├── admin │   ├── src │   │   └── styles │   │   └── index.css <-- This will be linked to (1) │   └── tailwind.config.js (2) ├── src │   └── styles │   └── index.css <-- This will be linked to (1) └── tailwind.config.js (1) ``` If you run the upgrade command from the root of your project, then the `/src/styles/index.css` will be linked to `/tailwind.config.js` which is what we expect. But `/admin/src/styles/index.css` will _also_ be linked to `/tailwind.config.js` With this PR we improve this behavior by looking at the CSS file, and crawling up the parent tree. This mens that the new behavior looks like this: ``` . ├── admin │   ├── src │   │   └── styles │   │   └── index.css <-- This will be linked to (2) │   └── tailwind.config.js (2) ├── src │   └── styles │   └── index.css <-- This will be linked to (1) └── tailwind.config.js (1) ``` Now `/src/styles/index.css` will be linked to `/tailwind.config.js`, and `/admin/src/styles/index.css` will be linked to `/admin/tailwind.config.js`. When we discover the Tailwind config file, we will also print a message to the user to let them know which CSS file is linked to which Tailwind config file. This should be a safe improvement because if your Tailwind config file had a different name, or if it lived in a sibling folder then Tailwind wouldn't find it either and you already required a `@config "…";` directive in your CSS file to point to the correct file. In the unlikely event that it turns out that 2 (or more) CSS files resolve to the same to the same Tailwind config file, then an upgrade might not be safe and some manual intervention might be needed. In this case, we will show a warning about this. <img width="1552" alt="image" src="https://github.com/user-attachments/assets/7a1ad11d-18c5-4b7d-9a02-14f0116ae955"> Test plan: --- - Added an integration test that properly links the nearest Tailwind config file by looking up the tree - Added an integration test that resolves 2 or more CSS files to the same config file, resulting in an error where manual intervention is needed - Ran it on the Tailwind UI codebase Running this on Tailwind UI's codebase it looks like this: <img width="1552" alt="image" src="https://github.com/user-attachments/assets/21785428-5e0d-47f7-80ec-dab497f58784"> --------- Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-11-18 17:43:44 +01:00
if (aFile?.startsWith('index') && !zFile?.startsWith('index')) return -1
if (zFile?.startsWith('index') && !aFile?.startsWith('index')) return 1
// Sort by filename, alphabetically
return a[0].localeCompare(z[0])
})
.map(([file, content]) => `--- ${file} ---\n${content || '<EMPTY>'}`)
.join('\n\n')
.trim()}\n`
},
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
async expectFileToContain(filePath, contents) {
return retryAssertion(async () => {
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
let fileContent = await this.read(filePath)
for (let content of Array.isArray(contents) ? contents : [contents]) {
if (content instanceof RegExp) {
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
options.expect(fileContent).toMatch(content)
} else {
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
options.expect(fileContent).toContain(content)
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
})
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
},
Improve compatibility with `@tailwindcss/typography` and `@tailwindcss/forms` (#14221) This PR enables compatibility for the `@tailwindcss/typography` and `@tailwindcss/forms` plugins. This required the addition of new Plugin APIs and new package exports. ## New Plugin APIs and compatibility improvements We added support for `addComponents`, `matchComponents`, and `prefix`. The component APIs are an alias for the utilities APIs because the sorting in V4 is different and emitting components in a custom `@layer` is not necessary. Since `prefix` is not supported in V4, the `prefix()` API is currently an identity function. ```js addComponents({ '.btn': { padding: '.5rem 1rem', borderRadius: '.25rem', fontWeight: '600', }, '.btn-blue': { backgroundColor: '#3490dc', color: '#fff', '&:hover': { backgroundColor: '#2779bd', }, }, '.btn-red': { backgroundColor: '#e3342f', color: '#fff', '&:hover': { backgroundColor: '#cc1f1a', }, }, }) ``` The behavioral changes effect the `addUtilities` and `matchUtilities` functions, we now: - Allow arrays of CSS property objects to be emitted: ```js addUtilities({ '.text-trim': [ {'text-box-trim': 'both'}, {'text-box-edge': 'cap alphabetic'}, ], }) ``` - Allow arrays of utilities ```js addUtilities([ { '.text-trim':{ 'text-box-trim': 'both', 'text-box-edge': 'cap alphabetic', }, } ]) ``` - Allow more complicated selector names ```js addUtilities({ '.form-input, .form-select, .form-radio': { /* styles here */ }, '.form-input::placeholder': { /* styles here */ }, '.form-checkbox:indeterminate:checked': { /* styles here */ } }) ``` ## New `tailwindcss/color` and `tailwindcss/defaultTheme` export To be compatible to v3, we're adding two new exports to the tailwindcss package. These match the default theme values as defined in v3: ```ts import colors from 'tailwindcss/colors' console.log(colors.red[600]) ``` ```ts import theme from 'tailwindcss/defaultTheme' console.log(theme.spacing[4]) ``` --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com>
2024-08-22 08:06:21 -04:00
async expectFileNotToContain(filePath, contents) {
return retryAssertion(async () => {
let fileContent = await this.read(filePath)
for (let content of contents) {
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
options.expect(fileContent).not.toContain(content)
Improve compatibility with `@tailwindcss/typography` and `@tailwindcss/forms` (#14221) This PR enables compatibility for the `@tailwindcss/typography` and `@tailwindcss/forms` plugins. This required the addition of new Plugin APIs and new package exports. ## New Plugin APIs and compatibility improvements We added support for `addComponents`, `matchComponents`, and `prefix`. The component APIs are an alias for the utilities APIs because the sorting in V4 is different and emitting components in a custom `@layer` is not necessary. Since `prefix` is not supported in V4, the `prefix()` API is currently an identity function. ```js addComponents({ '.btn': { padding: '.5rem 1rem', borderRadius: '.25rem', fontWeight: '600', }, '.btn-blue': { backgroundColor: '#3490dc', color: '#fff', '&:hover': { backgroundColor: '#2779bd', }, }, '.btn-red': { backgroundColor: '#e3342f', color: '#fff', '&:hover': { backgroundColor: '#cc1f1a', }, }, }) ``` The behavioral changes effect the `addUtilities` and `matchUtilities` functions, we now: - Allow arrays of CSS property objects to be emitted: ```js addUtilities({ '.text-trim': [ {'text-box-trim': 'both'}, {'text-box-edge': 'cap alphabetic'}, ], }) ``` - Allow arrays of utilities ```js addUtilities([ { '.text-trim':{ 'text-box-trim': 'both', 'text-box-edge': 'cap alphabetic', }, } ]) ``` - Allow more complicated selector names ```js addUtilities({ '.form-input, .form-select, .form-radio': { /* styles here */ }, '.form-input::placeholder': { /* styles here */ }, '.form-checkbox:indeterminate:checked': { /* styles here */ } }) ``` ## New `tailwindcss/color` and `tailwindcss/defaultTheme` export To be compatible to v3, we're adding two new exports to the tailwindcss package. These match the default theme values as defined in v3: ```ts import colors from 'tailwindcss/colors' console.log(colors.red[600]) ``` ```ts import theme from 'tailwindcss/defaultTheme' console.log(theme.spacing[4]) ``` --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com>
2024-08-22 08:06:21 -04:00
}
})
},
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
},
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
} satisfies TestContext
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
Improve compatibility with `@tailwindcss/typography` and `@tailwindcss/forms` (#14221) This PR enables compatibility for the `@tailwindcss/typography` and `@tailwindcss/forms` plugins. This required the addition of new Plugin APIs and new package exports. ## New Plugin APIs and compatibility improvements We added support for `addComponents`, `matchComponents`, and `prefix`. The component APIs are an alias for the utilities APIs because the sorting in V4 is different and emitting components in a custom `@layer` is not necessary. Since `prefix` is not supported in V4, the `prefix()` API is currently an identity function. ```js addComponents({ '.btn': { padding: '.5rem 1rem', borderRadius: '.25rem', fontWeight: '600', }, '.btn-blue': { backgroundColor: '#3490dc', color: '#fff', '&:hover': { backgroundColor: '#2779bd', }, }, '.btn-red': { backgroundColor: '#e3342f', color: '#fff', '&:hover': { backgroundColor: '#cc1f1a', }, }, }) ``` The behavioral changes effect the `addUtilities` and `matchUtilities` functions, we now: - Allow arrays of CSS property objects to be emitted: ```js addUtilities({ '.text-trim': [ {'text-box-trim': 'both'}, {'text-box-edge': 'cap alphabetic'}, ], }) ``` - Allow arrays of utilities ```js addUtilities([ { '.text-trim':{ 'text-box-trim': 'both', 'text-box-edge': 'cap alphabetic', }, } ]) ``` - Allow more complicated selector names ```js addUtilities({ '.form-input, .form-select, .form-radio': { /* styles here */ }, '.form-input::placeholder': { /* styles here */ }, '.form-checkbox:indeterminate:checked': { /* styles here */ } }) ``` ## New `tailwindcss/color` and `tailwindcss/defaultTheme` export To be compatible to v3, we're adding two new exports to the tailwindcss package. These match the default theme values as defined in v3: ```ts import colors from 'tailwindcss/colors' console.log(colors.red[600]) ``` ```ts import theme from 'tailwindcss/defaultTheme' console.log(theme.spacing[4]) ``` --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com>
2024-08-22 08:06:21 -04:00
config.fs['.gitignore'] ??= txt`
node_modules/
`
// Disable Git's automatic line ending conversion in these throwaway test
// repositories. Tools like pnpm write files (e.g. `pnpm-lock.yaml`) with
// LF endings, which on Windows (where `core.autocrlf=true` is common)
// makes Git emit noisy "LF will be replaced by CRLF" warnings when it
// touches those files.
config.fs['.gitattributes'] ??= txt`
* -text
`
// Ensure there is always a root `pnpm-workspace.yaml` so that transitive
// dependency overrides (and the build allow list) are injected into it.
// As of pnpm v10, these can no longer live in the `pnpm.overrides` field
// of `package.json`.
config.fs['pnpm-workspace.yaml'] ??= ''
for (let [filename, content] of Object.entries(config.fs)) {
Ensure `@source` globs with symlinks are preserved (#20203) This PR fixes an issue when working with `@source` and `@source not` that involves symlinks. Internally our sources are mapped to a source entry where we have a `base` path and a `pattern`. You can think about this where we insert a `.gitignore` file in the `base` path for the given pattern. However, we optimize these entries to move as many "static" parts into the base path. For example: ```css @source "./some/folder/here/*.html"; ``` Is mapped to something like: ```ts { base: "/projects/my-project", pattern: "./some/folder/here/*.html" } ``` We then optimize it by turning it into: ```ts { base: "/projects/my-project/some/folder/here", pattern: "*.html" } ``` While doing this, we also use `dunce::canonicalize` to resolve the actual paths on disk. This means that a symlink is resolved to their real paths. This can cause issues because the "real" path is not what you wrote in the `@source` directives. So before, it could be that you have this: ```css @source "./some/symlinked-folder/here/*.html"; ``` Which was mapped to this on the Rust side: ```ts { base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" } ``` But was then optimized to: ```ts { base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" } ``` ...and we lost the `symlinked-folder` information. This causes issues as seen in #17985. With this PR, we keep the symlinked information in those globs since that's what you wrote in those `@source` directives. While setting up integration tests, I stumbled upon an issue because I wanted to test that ignoring a symlinked folder, but including a single particular file of that ignored folder resulted in that file being ignored as well. Let's look at an example: ```css @source '../lib'; @source not '../lib/ignored'; @source '../lib/ignored/except.html'; ``` Earlier I mentioned that we create `.gitignore` files based on these `@source` directives. In this case, when we're dealing with a folder, we use `**/*` as the contents. Looking at the example above, we should essentially have something like this: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored' **/* # @source '../lib/ignored/except.html' !except.html ``` Since it's a `.gitignore` file, we have to invert the globs. But the bug I noticed is that in reality the result of those gitignores didn't look like the above, it looked like: ```gitignore # lib/.gitignore # @source '../lib' !**/* # lib/ignored/.gitignore # @source '../lib/ignored/except.html' !except.html # @source '../lib/ignored' **/* ``` Notice how the `!except.html` and `**/*` are flipped. When dealing with `.gitignore` files, the order is important. This was caused because internally we kept a `BTreeMap` of `BTreeSet`s where the map was the base path and a set of patterns. The patterns were sorted because of the `BTreeSet`... which is not what we want. Fixes: #17985 Closes: #20091 ## Test plan 1. Added new tests in the scanner tests (on the Rust side) 2. Added integration tests with a symlink to another folder, outside of the current folder 2. Added integration tests with a symlink to another folder, inside of the current folder 2. Added integration tests to ensure that the order of `@source` files with a folder + file is sorted correctly. 3. Since we're dealing with symlinks in these tests, let's test all OSes [ci-all]
2026-06-07 20:44:22 +02:00
if (content.toString().startsWith('symlink:')) {
// The symlink path is relative to the target destination's path
let target = path.join(
filename,
content.toString().slice('symlink:'.length), // Relative path
)
await context.fs.symlink(target, filename)
} else {
await context.fs.write(filename, content)
}
}
let shouldInstallDependencies = config.installDependencies ?? true
try {
// Every test project gets its own root `pnpm-workspace.yaml` (see
// above). This makes the test directory its own workspace root, so even
// in debug mode — where the directory lives inside this repository's
// pnpm workspace — `pnpm install` won't attach to the parent workspace.
if (shouldInstallDependencies) {
await context.exec(`pnpm install`)
}
} catch (error: any) {
console.error(error)
console.error(error.stdout?.toString())
console.error(error.stderr?.toString())
throw error
}
let disposables: (() => Promise<void>)[] = []
async function dispose() {
let results = await Promise.allSettled(disposables.map((dispose) => dispose()))
if (!debug) {
await gracefullyRemove(root)
}
let errors = results.flatMap((result) =>
result.status === 'rejected' ? [result.reason] : [],
)
if (errors.length > 0) {
throw new AggregateError(errors, 'Failed to clean up spawned processes')
}
}
options.onTestFinished(dispose)
// Make it a git repository, and commit all files
if (only || debug) {
try {
await context.exec('git init', { cwd: root })
await context.exec('git add --all', { cwd: root })
Fix `@source` with folders that are ignored (#20214) This PR fixes an issue where a `@source` that's pointing to a folder that is git ignored, is also ignored by the `@source` even if it's explicitly added. Internally, we convert `@source` directives from `PublicSourceEntry`s to `SourceEntry`s where we have dedicated enum branches for `Auto`, `Pattern`, `Ignored` and `External`. The `Auto` one accepts a `base` path, and will be used for auto content detection. However, these paths will make use of all the default auto content detection rules, which includes git ignore rules. We also have `External` where we link to something that's "external" to the current repo. We can probably improve this name, but it's external in the sense that it won't show up on GitHub for example, aka ignored. We have some content dirs that we ignore by default, such as the `node_modules` folder. When you do use `@source` with `node_modules` in the path, then we will mark it as an `external` resource which does not look at the `gitignore` related rules and allowing it to be included this way. The idea with this is that, even though the folder is ignored by default, you can still include files from the folder by explicitly using the `@source` directive. The issue as seen in #19844 is using `vendor/` instead of `node_modules/` which is _not_ ignored by default. While we can add `vendor/` to this same ignored dirs list, it will result in a breaking change because this folder is often used by the Laravel community to store some resources in. This PR fixes this problem by not only looking at the content dirs we ignore by default, but also looking at the actual git ignore state of this folder. If it turns out that this is ignored, then we promote the `Auto` source to an `External` source. Fixes: #19844 Closes: #20057 ## Test plan 1. Added integration tests for this situation 2. Ran the fix on the reproduction from #19844. If we run the CLI with the `DEBUG=*` environment variable, the log file produces these results: ```diff diff --git a/./tailwindcss-29207.log b/./tailwindcss-30381.log index bc3017c..921af0e 100644 --- a/./tailwindcss-29207.log +++ b/./tailwindcss-30381.log @@ -6,8 +6,9 @@ INFO tailwindcss_oxide::scanner: Source: PublicSourceEntry { base: "/Users/robin INFO tailwindcss_oxide::scanner: Optimized sources: INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.phtml" } INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.xml" } -INFO tailwindcss_oxide::scanner: Source: Auto { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" } +INFO tailwindcss_oxide::scanner: Source: External { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" } INFO tailwindcss_oxide::scanner: Source: Ignored { base: "/Users/robin/.fnm/node-versions/v26.1.0/installation/bin", pattern: "/node" } INFO discover_sources: tailwindcss_oxide::scanner: enter -INFO discover_sources: tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml" +INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml" +INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme/module/templates/component.phtml" INFO discover_sources: tailwindcss_oxide::scanner: exit ``` We're checking some `.gitignore` related files, so let's check on each OS [ci-all]
2026-06-10 18:44:14 +02:00
await context.exec('git commit -m "before migration" --allow-empty', { cwd: root })
} catch (error: any) {
console.error(error)
console.error(error.stdout?.toString())
console.error(error.stderr?.toString())
throw error
}
}
return await testCallback(context)
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
},
)
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
test.only = (name: string, config: TestConfig, testCallback: TestCallback) => {
return test(name, config, testCallback, { only: true })
}
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
test.skip = (name: string, config: TestConfig, testCallback: TestCallback) => {
return test(name, config, testCallback, { skip: true })
}
test.concurrent = (name: string, config: TestConfig, testCallback: TestCallback) => {
return test(name, config, testCallback, { concurrent: true })
}
test.debug = (name: string, config: TestConfig, testCallback: TestCallback) => {
return test(name, config, testCallback, { debug: true })
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
// Maps package names to their tarball filenames. See scripts/pack-packages.ts
// for more details.
function pkgToFilename(name: string) {
return `${name.replace('@', '').replace('/', '-')}.tgz`
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
async function overwriteVersionsInPackageJson(content: string): Promise<string> {
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
let json = JSON.parse(content)
// Resolve all workspace:^ versions to local tarballs
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
for (let key of ['dependencies', 'devDependencies', 'peerDependencies', 'optionalDependencies']) {
let dependencies = json[key] || {}
for (let dependency in dependencies) {
if (dependencies[dependency] === 'workspace:^') {
dependencies[dependency] = resolveVersion(dependency)
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
}
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
return JSON.stringify(json, null, 2)
}
function overwriteVersionsInPnpmWorkspace(content: string): string {
let workspace = content.trim() === '' ? {} : Yaml.parse(content)
// Inherit information from the root workspace
workspace.allowBuilds = {
...ROOT_PNPM_WORKSPACE.allowBuilds,
...workspace.allowBuilds,
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
// Inject transitive dependency overwrite. This is necessary because
// @tailwindcss/vite internally depends on a specific version of
// @tailwindcss/oxide and we instead want to resolve it to the locally built
// version.
//
// As of pnpm v10, the `pnpm.overrides` field in `package.json` is no longer
// read. Overrides have to be defined in `pnpm-workspace.yaml` instead.
workspace.overrides ||= {}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
for (let pkg of PUBLIC_PACKAGES) {
Ensure clients pin the `tailwindcss` version (#15011) We noticed that in the current alpha 34 release, the `package.json` file of the `@tailwindcss/node` package only defines `tailwindcss` as a dev dependency. This makes it very easy for version mismatches to happen when a v3 version (or an earlier v4 alpha for that matter) was installed in the same project: ```json { "name": "@tailwindcss/node", "version": "4.0.0-alpha.34", "description": "A utility-first CSS framework for rapidly building custom user interfaces.", "license": "MIT", "repository": { "type": "git", "url": "https://github.com/tailwindlabs/tailwindcss.git", "directory": "packages/@tailwindcss-node" }, "bugs": "https://github.com/tailwindlabs/tailwindcss/issues", "homepage": "https://tailwindcss.com", "files": [ "dist/" ], "publishConfig": { "provenance": true, "access": "public" }, "exports": { ".": { "types": "./dist/index.d.ts", "import": "./dist/index.mjs", "require": "./dist/index.js" }, "./require-cache": { "types": "./dist/require-cache.d.ts", "default": "./dist/require-cache.js" }, "./esm-cache-loader": { "types": "./dist/esm-cache.loader.d.mts", "default": "./dist/esm-cache.loader.mjs" } }, "devDependencies": { "tailwindcss": "4.0.0-alpha.34" }, "dependencies": { "enhanced-resolve": "^5.17.1", "jiti": "^2.0.0-beta.3" }, "scripts": { "build": "tsup-node", "dev": "pnpm run build -- --watch" } } ``` Furthermore, we were trying to fix issues where our integration test setup could not install `tailwindcss@3` because of how we did pnpm overrides. This PR fixes this by: - Ensuring every client that calls into `tailwindcss` core marks it as a version-pinned dependency. You are still required to install `tailwindcss` in your project along side a client (e.g. `@tailwindcss/vite`) but we now only use your installed version for importing the respective `.css` files. For the core logic, we are now requiring each package to use `tailwindcss` at the same version. This should help resolve issues like https://github.com/tailwindlabs/tailwindcss/discussions/14652 - We tried to eliminate the dependency on `tailwindcss` from the `@tailwindcss/upgrade` package. Unfortunately this is not possible to do right now because we need to load the CSS files from v4 to create the right environment. In a future version we could bundle the required CSS files with `@tailwidncss/upgrade` but it doesn't seem necessary for now. - We then changed our integration test overrides to only override the `tailwindcss` package that are dependencies of the known list of packages that we have `tailwindcss` dependencies on: `@tailwindcss/node` and `@tailwindcss/upgrade`. This ensures that we can install v3 of `tailwindcss` in the integration tests and it will work. Something we want to do for some upgrade tests. # Test plan Integration work again. Furthermore we added a quick setup with the CLI using the local tarballs and ensured it works: ```bash pnpm init pnpm install ../../tailwindcss/dist/tailwindcss-cli.tgz pnpm install ../../tailwindcss/dist/tailwindcss.tgz echo '@import "tailwindcss";' > index.css echo '<div class="underline"></div>' > index.html pnpm tailwindcss -i index.css -o out.css cat out.css ```
2024-11-15 17:18:48 +01:00
if (pkg === 'tailwindcss') {
// We want to be explicit about the `tailwindcss` package so our tests can
// also import v3 without conflicting v4 tarballs.
workspace.overrides['@tailwindcss/node>tailwindcss'] = resolveVersion(pkg)
workspace.overrides['@tailwindcss/upgrade>tailwindcss'] = resolveVersion(pkg)
workspace.overrides['@tailwindcss/cli>tailwindcss'] = resolveVersion(pkg)
workspace.overrides['@tailwindcss/postcss>tailwindcss'] = resolveVersion(pkg)
2026-07-30 15:45:51 -04:00
workspace.overrides['@tailwindcss/turbopack>tailwindcss'] = resolveVersion(pkg)
workspace.overrides['@tailwindcss/vite>tailwindcss'] = resolveVersion(pkg)
workspace.overrides['@tailwindcss/webpack>tailwindcss'] = resolveVersion(pkg)
Ensure clients pin the `tailwindcss` version (#15011) We noticed that in the current alpha 34 release, the `package.json` file of the `@tailwindcss/node` package only defines `tailwindcss` as a dev dependency. This makes it very easy for version mismatches to happen when a v3 version (or an earlier v4 alpha for that matter) was installed in the same project: ```json { "name": "@tailwindcss/node", "version": "4.0.0-alpha.34", "description": "A utility-first CSS framework for rapidly building custom user interfaces.", "license": "MIT", "repository": { "type": "git", "url": "https://github.com/tailwindlabs/tailwindcss.git", "directory": "packages/@tailwindcss-node" }, "bugs": "https://github.com/tailwindlabs/tailwindcss/issues", "homepage": "https://tailwindcss.com", "files": [ "dist/" ], "publishConfig": { "provenance": true, "access": "public" }, "exports": { ".": { "types": "./dist/index.d.ts", "import": "./dist/index.mjs", "require": "./dist/index.js" }, "./require-cache": { "types": "./dist/require-cache.d.ts", "default": "./dist/require-cache.js" }, "./esm-cache-loader": { "types": "./dist/esm-cache.loader.d.mts", "default": "./dist/esm-cache.loader.mjs" } }, "devDependencies": { "tailwindcss": "4.0.0-alpha.34" }, "dependencies": { "enhanced-resolve": "^5.17.1", "jiti": "^2.0.0-beta.3" }, "scripts": { "build": "tsup-node", "dev": "pnpm run build -- --watch" } } ``` Furthermore, we were trying to fix issues where our integration test setup could not install `tailwindcss@3` because of how we did pnpm overrides. This PR fixes this by: - Ensuring every client that calls into `tailwindcss` core marks it as a version-pinned dependency. You are still required to install `tailwindcss` in your project along side a client (e.g. `@tailwindcss/vite`) but we now only use your installed version for importing the respective `.css` files. For the core logic, we are now requiring each package to use `tailwindcss` at the same version. This should help resolve issues like https://github.com/tailwindlabs/tailwindcss/discussions/14652 - We tried to eliminate the dependency on `tailwindcss` from the `@tailwindcss/upgrade` package. Unfortunately this is not possible to do right now because we need to load the CSS files from v4 to create the right environment. In a future version we could bundle the required CSS files with `@tailwidncss/upgrade` but it doesn't seem necessary for now. - We then changed our integration test overrides to only override the `tailwindcss` package that are dependencies of the known list of packages that we have `tailwindcss` dependencies on: `@tailwindcss/node` and `@tailwindcss/upgrade`. This ensures that we can install v3 of `tailwindcss` in the integration tests and it will work. Something we want to do for some upgrade tests. # Test plan Integration work again. Furthermore we added a quick setup with the CLI using the local tarballs and ensured it works: ```bash pnpm init pnpm install ../../tailwindcss/dist/tailwindcss-cli.tgz pnpm install ../../tailwindcss/dist/tailwindcss.tgz echo '@import "tailwindcss";' > index.css echo '<div class="underline"></div>' > index.html pnpm tailwindcss -i index.css -o out.css cat out.css ```
2024-11-15 17:18:48 +01:00
} else {
workspace.overrides[pkg] = resolveVersion(pkg)
Ensure clients pin the `tailwindcss` version (#15011) We noticed that in the current alpha 34 release, the `package.json` file of the `@tailwindcss/node` package only defines `tailwindcss` as a dev dependency. This makes it very easy for version mismatches to happen when a v3 version (or an earlier v4 alpha for that matter) was installed in the same project: ```json { "name": "@tailwindcss/node", "version": "4.0.0-alpha.34", "description": "A utility-first CSS framework for rapidly building custom user interfaces.", "license": "MIT", "repository": { "type": "git", "url": "https://github.com/tailwindlabs/tailwindcss.git", "directory": "packages/@tailwindcss-node" }, "bugs": "https://github.com/tailwindlabs/tailwindcss/issues", "homepage": "https://tailwindcss.com", "files": [ "dist/" ], "publishConfig": { "provenance": true, "access": "public" }, "exports": { ".": { "types": "./dist/index.d.ts", "import": "./dist/index.mjs", "require": "./dist/index.js" }, "./require-cache": { "types": "./dist/require-cache.d.ts", "default": "./dist/require-cache.js" }, "./esm-cache-loader": { "types": "./dist/esm-cache.loader.d.mts", "default": "./dist/esm-cache.loader.mjs" } }, "devDependencies": { "tailwindcss": "4.0.0-alpha.34" }, "dependencies": { "enhanced-resolve": "^5.17.1", "jiti": "^2.0.0-beta.3" }, "scripts": { "build": "tsup-node", "dev": "pnpm run build -- --watch" } } ``` Furthermore, we were trying to fix issues where our integration test setup could not install `tailwindcss@3` because of how we did pnpm overrides. This PR fixes this by: - Ensuring every client that calls into `tailwindcss` core marks it as a version-pinned dependency. You are still required to install `tailwindcss` in your project along side a client (e.g. `@tailwindcss/vite`) but we now only use your installed version for importing the respective `.css` files. For the core logic, we are now requiring each package to use `tailwindcss` at the same version. This should help resolve issues like https://github.com/tailwindlabs/tailwindcss/discussions/14652 - We tried to eliminate the dependency on `tailwindcss` from the `@tailwindcss/upgrade` package. Unfortunately this is not possible to do right now because we need to load the CSS files from v4 to create the right environment. In a future version we could bundle the required CSS files with `@tailwidncss/upgrade` but it doesn't seem necessary for now. - We then changed our integration test overrides to only override the `tailwindcss` package that are dependencies of the known list of packages that we have `tailwindcss` dependencies on: `@tailwindcss/node` and `@tailwindcss/upgrade`. This ensures that we can install v3 of `tailwindcss` in the integration tests and it will work. Something we want to do for some upgrade tests. # Test plan Integration work again. Furthermore we added a quick setup with the CLI using the local tarballs and ensured it works: ```bash pnpm init pnpm install ../../tailwindcss/dist/tailwindcss-cli.tgz pnpm install ../../tailwindcss/dist/tailwindcss.tgz echo '@import "tailwindcss";' > index.css echo '<div class="underline"></div>' > index.html pnpm tailwindcss -i index.css -o out.css cat out.css ```
2024-11-15 17:18:48 +01:00
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
}
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
return Yaml.stringify(workspace)
Add integration test setup and tests for the Vite integration (#14089) This PR adds a new root `/integrations` folder that will be the home of integration tests. The idea of these tests is to use Tailwind in various setups just like our users would (by only using the publishable npm builds). To avoid issues with concurrent tests making changes to the file system, to make it very easy to test through a range of versions, and to avoid changing configuration objects over and over in test runs, we decided to inline the scaffolding completely into the test file and have no examples checked into the repo. Here's an example of how this can look like for a simple Vite test: ```ts test('works with production builds', { fs: { 'package.json': json` { "type": "module", "dependencies": { "@tailwindcss/vite": "workspace:^", "tailwindcss": "workspace:^" }, "devDependencies": { "vite": "^5.3.5" } } `, 'vite.config.ts': ts` import tailwindcss from '@tailwindcss/vite' import { defineConfig } from 'vite' export default defineConfig({ build: { cssMinify: false }, plugins: [tailwindcss()], }) `, 'index.html': html` <head> <link rel="stylesheet" href="./src/index.css"> </head> <body> <div class="underline m-2">Hello, world!</div> </body> `, 'src/index.css': css` @import 'tailwindcss/theme' reference; @import 'tailwindcss/utilities'; `, }, }, async ({ fs, exec }) => { await exec('pnpm vite build') expect.assertions(2) for (let [path, content] of await fs.glob('dist/**/*.css')) { expect(path).toMatch(/\.css$/) expect(stripTailwindComment(content)).toMatchInlineSnapshot( ` ".m-2 { margin: var(--spacing-2, .5rem); } .underline { text-decoration-line: underline; }" `, ) } }, ) ``` By defining all dependencies this way, we never have to worry about which fixtures are checked in and can more easily describe changes to the setup. For ergonomics, we've also added the [`embed` prettier plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will mean that files inlined in the `fs` setup are properly indented. No extra work needed! If you're using VS Code, I can also recommend the [Language Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals) extension so that syntax highlighting also _just works_. A neat feature of inlining the scaffolding like this is to make it very simple to test through a variety of versions. For example, here's how we can set up a test against Vite 5 and Vite 4: ```js ;['^4.5.3', '^5.3.5'].forEach(viteVersion => { test(`works with production builds for Vite ${viteVersion}`, { fs: { 'package.json': json` { "type": "module", "devDependencies": { "vite": "${viteVersion}" } } `, async () => { // Do something }, ) }) ``` ## Philosophy Before we dive into the specifics, I want to clearly state the design considerations we have chosen for this new test suite: - All file mutations should be done in temp folders, nothing should ever mess with your working directory - Windows as a first-class citizen - Have a clean and simple API that describes the test setup only using public APIs - Focus on reliability (make sure cleanup scripts work and are tolerant to various error scenarios) - If a user reports an issue with a specific configuration, we want to be able to reproduce them with integration tests, no matter how obscure the setup (this means the test need to be in control of most of the variables) - Tests should be reasonably fast (obviously this depends on the integration. If we use a slow build tool, we can't magically speed it up, but our overhead should be minimal). ## How it works The current implementation provides a custom `test` helper function that, when used, sets up the environment according to the configuration. It'll create a new temporary directory and create all files, ensuring things like proper `\r\n` line endings on Windows. We do have to patch the `package.json` specifically, since we can not use public versions of the tailwindcss packages as we want to be able to test against a development build. To make this happen, every `pnpm build` run now creates tarballs of the npm modules (that contain only the files that would also in the published build). We then patch the `package.json` to rewrite `workspace:^` versions to link to those tarballs. We found this to work reliably on Windows and macOS as well as being fast enough to not cause any issues. Furthermore we also decided to use `pnpm` as the version manager for integration tests because of it's global module cache (so installing `vite` is fast as soon as you installed it once). The test function will receive a few utilities that it can use to more easily interact with the temp dir. One example is a `fs.glob` function that you can use to easily find files in eventual `dist/` directories or helpers around `spawn` and `exec` that make sure that processes are cleaned up correctly. Because we use tarballs from our build dependencies, working on changes requires a workflow where you run `pnpm build` before running `pnpm test:integrations`. However it also means we can run clients like our CLI client with no additional overhead—just install the dependency like any user would and set up your test cases this way. ## Test plan This PR also includes two Vite specific integration tests: One testing a static build (`pnpm vite build`) and one a dev mode build (`pnpm vite dev`) that also makes changes to the file system and asserts that the resources properly update. --------- Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
}
function resolveVersion(dependency: string) {
let tarball = path.join(REPO_ROOT, 'dist', pkgToFilename(dependency))
return `file:${tarball}`
}
export function stripTailwindComment(content: string) {
return content.replace(/\/\*! tailwindcss .*? \*\//g, '').trim()
}
export let svg = dedent
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
export let css = dedent
export let html = dedent
export let ts = dedent
export let js = dedent
export let jsx = dedent
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
export let json = dedent
export let yaml = dedent
export let txt = dedent
export function binary(str: string | TemplateStringsArray, ...values: unknown[]): Uint8Array {
let base64 = typeof str === 'string' ? str : String.raw(str, ...values)
return Uint8Array.from(atob(base64), (c) => c.charCodeAt(0))
}
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
export function candidate(strings: TemplateStringsArray, ...values: any[]) {
let output: string[] = []
for (let i = 0; i < strings.length; i++) {
output.push(strings[i])
if (i < values.length) {
output.push(values[i])
}
}
return `.${escape(output.join('').trim())}`
}
export async function retryAssertion<T>(
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
fn: () => Promise<T>,
{ timeout = ASSERTION_TIMEOUT, delay = 5 }: { timeout?: number; delay?: number } = {},
) {
let end = Date.now() + timeout
let error: any
while (Date.now() < end) {
try {
return await fn()
} catch (err) {
Fix `@source` with folders that are ignored (#20214) This PR fixes an issue where a `@source` that's pointing to a folder that is git ignored, is also ignored by the `@source` even if it's explicitly added. Internally, we convert `@source` directives from `PublicSourceEntry`s to `SourceEntry`s where we have dedicated enum branches for `Auto`, `Pattern`, `Ignored` and `External`. The `Auto` one accepts a `base` path, and will be used for auto content detection. However, these paths will make use of all the default auto content detection rules, which includes git ignore rules. We also have `External` where we link to something that's "external" to the current repo. We can probably improve this name, but it's external in the sense that it won't show up on GitHub for example, aka ignored. We have some content dirs that we ignore by default, such as the `node_modules` folder. When you do use `@source` with `node_modules` in the path, then we will mark it as an `external` resource which does not look at the `gitignore` related rules and allowing it to be included this way. The idea with this is that, even though the folder is ignored by default, you can still include files from the folder by explicitly using the `@source` directive. The issue as seen in #19844 is using `vendor/` instead of `node_modules/` which is _not_ ignored by default. While we can add `vendor/` to this same ignored dirs list, it will result in a breaking change because this folder is often used by the Laravel community to store some resources in. This PR fixes this problem by not only looking at the content dirs we ignore by default, but also looking at the actual git ignore state of this folder. If it turns out that this is ignored, then we promote the `Auto` source to an `External` source. Fixes: #19844 Closes: #20057 ## Test plan 1. Added integration tests for this situation 2. Ran the fix on the reproduction from #19844. If we run the CLI with the `DEBUG=*` environment variable, the log file produces these results: ```diff diff --git a/./tailwindcss-29207.log b/./tailwindcss-30381.log index bc3017c..921af0e 100644 --- a/./tailwindcss-29207.log +++ b/./tailwindcss-30381.log @@ -6,8 +6,9 @@ INFO tailwindcss_oxide::scanner: Source: PublicSourceEntry { base: "/Users/robin INFO tailwindcss_oxide::scanner: Optimized sources: INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.phtml" } INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.xml" } -INFO tailwindcss_oxide::scanner: Source: Auto { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" } +INFO tailwindcss_oxide::scanner: Source: External { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" } INFO tailwindcss_oxide::scanner: Source: Ignored { base: "/Users/robin/.fnm/node-versions/v26.1.0/installation/bin", pattern: "/node" } INFO discover_sources: tailwindcss_oxide::scanner: enter -INFO discover_sources: tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml" +INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml" +INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme/module/templates/component.phtml" INFO discover_sources: tailwindcss_oxide::scanner: exit ``` We're checking some `.gitignore` related files, so let's check on each OS [ci-all]
2026-06-10 18:44:14 +02:00
Error.captureStackTrace(err, retryAssertion)
Add `@source` support (#14078) This PR is an umbrella PR where we will add support for the new `@source` directive. This will allow you to add explicit content glob patterns if you want to look for Tailwind classes in other files that are not automatically detected yet. Right now this is an addition to the existing auto content detection that is automatically enabled in the `@tailwindcss/postcss` and `@tailwindcss/cli` packages. The `@tailwindcss/vite` package doesn't use the auto content detection, but uses the module graph instead. From an API perspective there is not a lot going on. There are only a few things that you have to know when using the `@source` directive, and you probably already know the rules: 1. You can use multiple `@source` directives if you want. 2. The `@source` accepts a glob pattern so that you can match multiple files at once 3. The pattern is relative to the current file you are in 4. The pattern includes all files it is matching, even git ignored files 1. The motivation for this is so that you can explicitly point to a `node_modules` folder if you want to look at `node_modules` for whatever reason. 6. Right now we don't support negative globs (starting with a `!`) yet, that will be available in the near future. Usage example: ```css /* ./src/input.css */ @import "tailwindcss"; @source "../laravel/resources/views/**/*.blade.php"; @source "../../packages/monorepo-package/**/*.js"; ``` It looks like the PR introduced a lot of changes, but this is a side effect of all the other plumbing work we had to do to make this work. For example: 1. We added dedicated integration tests that run on Linux and Windows in CI (just to make sure that all the `path` logic is correct) 2. We Have to make sure that the glob patterns are always correct even if you are using `@import` in your CSS and use `@source` in an imported file. This is because we receive the flattened CSS contents where all `@import`s are inlined. 3. We have to make sure that we also listen for changes in the files that match any of these patterns and trigger a rebuild. PRs: - [x] https://github.com/tailwindlabs/tailwindcss/pull/14063 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14085 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14079 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14067 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14076 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14080 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14127 - [x] https://github.com/tailwindlabs/tailwindcss/pull/14135 Once all the PRs are merged, then this umbrella PR can be merged. > [!IMPORTANT] > Make sure to merge this without rebasing such that each individual PR ends up on the main branch. --------- Co-authored-by: Philipp Spiess <hello@philippspiess.com> Co-authored-by: Jordan Pittman <jordan@cryptica.me> Co-authored-by: Adam Wathan <adam.wathan@gmail.com>
2024-08-07 16:38:44 +02:00
error = err
await new Promise((resolve) => setTimeout(resolve, delay))
}
}
throw error
}
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
export async function fetchStyles(base: string, path = '/'): Promise<string> {
while (base.endsWith('/')) {
base = base.slice(0, -1)
}
let index = await fetch(`${base}${path}`)
let html = await index.text()
let linkRegex = /<link rel="stylesheet" href="([a-zA-Z0-9/_.?=%-]+)"/gi
let styleRegex = /<style\b[^>]*>([\s\S]*?)<\/style>/gi
let stylesheets: string[] = []
let paths: string[] = []
for (let match of html.matchAll(linkRegex)) {
let path: string = match[1]
if (path.startsWith('./')) {
path = path.slice(1)
}
paths.push(path)
}
stylesheets.push(
...(await Promise.all(
paths.map(async (path) => {
Improve integration tests (stability + performance) (#15125) This PR improves the integration tests in two ways: 1. Make the integration tests more reliable and thus less flakey 2. Make the integration tests faster (by introducing concurrency) Tried a lot of different things to make sure that these tests are fast and stable. --- The biggest issue we noticed is that some tests are flakey, these are tests with long running dev-mode processes where watchers are being used and/or dev servers are created. To solve this, all the tests that spawn a process look at stdout/stderr and wait for a message from the process to know whether we can start making changes. For example, in case of an Astro project, you get a `watching for file changes` message. In case of Nuxt project you can wait for an `server warmed up in` and in case of Next.js there is a `Ready in` message. These depend on the tools being used, so this is hardcoded per test instead of a magically automatic solution. These messages allow us to wait until all the initial necessary work, internal watchers and/or dev servers are setup before we start making changes to the files and/or request CSS stylesheets before the server(s) are ready. --- Another improvement is how we setup the dev servers. Before, we used to try and get a free port on the system and use a `--port` flag or a `PORT` environment variable. Instead of doing this (which is slow), we rely on the process itself to show a URL with a port. Basically all tools will try to find a free port if the default port is in use. We can then use the stdout/stderr messages to get the URL and the port to use. To reduce the amount of potential conflicts in ports, we used to run every test and every file sequentially to basically guarantee that ports are free. With this new approach where we rely on the process, I noticed that we don't really run into this issue again (I reran the tests multiple times and they were always stable) <img width="316" alt="image" src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2" /> Note: these tests run Linux, Windows and macOS in this branch just for testing purposes. Once this is done, we will only run Linux tests on PRs and run all 3 of them on the `next` branch. We do make the tests concurrent by default now, which in theory means that there could be conflicts (which in practice means that the process has to do a few more tries to find a free port). To reduce these conflicts, we split up the integration tests such that Vite, PostCSS, CLI, … tests all run in a separate job in the GitHub actions workflow. <img width="312" alt="image" src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766" /> Comparing this branch against the `next` branch, this is what CI looks like right now: | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="594" alt="image" src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635" /> | <img width="672" alt="image" src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7" /> | There also was a point in time where I introduced sequential tests such that all spawned processes still run after each other, but so far I didn't run into issues if we keep them concurrent so I dropped that code. Some small changes I made to make things more reliable: 1. When relying on stdout/stderr messages, we split lines on `\n` and we strip all the ANSI escapes which allows us to not worry about special ANSI characters when finding the URL or a specific message to wait for. 2. Once a test is done, we `child.kill()` the spawned process. If that doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to force kill the process. This could technically lead to some memory or files not being cleaned up properly, but once CI is done, everything is thrown away anyway. 3. As you can see in the screenshots, I used some nicer names for the workflows. | `next` | `feat/improve-integration-tests` | | --- | --- | | <img width="276" alt="image" src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9" /> | <img width="179" alt="image" src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad" /> | They also look a bit nicer in the PR overview as well: <img width="929" alt="image" src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef" /> The very last commit just filters out Windows and macOS tests again for PRs (but they are executed on the `next` branch. --- ### Nest steps I think for now we are in a pretty good state, but there are some things we can do to further improve everything (mainly make things faster) but aren't necessary. I also ran into issue while trying it so there is more work to do. 1. More splits — instead of having a Vite folder and PostCSS folder, we can go a step further and have folders for Next.js, Astro, Nuxt, Remix, … 2. Caching — right now we have to run the build step for every OS on every "job". We can re-use the work here by introducing a setup job that the other jobs rely on. @thecrypticace and I tried it already, but were running into some Bun specific Standalone CLI issues when doing that. 3. Remote caching — we could re-enable remote caching such that the `build` step can be full turbo (e.g.: after a PR is merged in `next` and we run everything again)
2024-12-12 13:48:56 +01:00
let css = await fetch(`${base}${path}`, {
headers: {
Accept: 'text/css',
},
})
return await css.text()
}),
)),
)
for (let match of html.matchAll(styleRegex)) {
stylesheets.push(match[1])
}
return stylesheets.reduce((acc, css) => {
Add support for source maps (#17775) Closes #13694 Closes #13591 # Source Maps Support for Tailwind CSS This PR adds support for source maps to Tailwind CSS v4 allowing us to track where styles come from whether that be user CSS, imported stylesheets, or generated utilities. This will improve debuggability in browser dev tools and gives us a good foundation for producing better error messages. I'll go over the details on how end users can enable source maps, any limitations in our implementation, changes to the internal `compile(…)` API, and some details and reasoning around the implementation we chose. ## Usage ### CLI Source maps can be enabled in the CLI by using the command line argument `--map` which will generate an inline source map comment at the bottom of your CSS. A separate file may be generated by passing a file name to `--map`: ```bash # Generates an inline source map npx tailwindcss -i input.css -o output.css --map # Generates a separate source map file npx tailwindcss -i input.css -o output.css --map output.css.map ``` ### PostCSS Source maps are supported when using Tailwind as a PostCSS plugin *in development mode only*. They may or may not be enabled by default depending on your build tool. If they are not you may be able to configure them within your PostCSS config: ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "@tailwindcss/postcss": {}, }, } } ``` ### Vite Source maps are supported when using the Tailwind CSS Vite plugin in *development mode only* by enabling the `css.devSourcemap` setting: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { devSourcemap: true, }, }) ``` Now when a CSS file is requested by the browser it'll have an inline source map comment that the browser can use. ## Limitations - Production build source maps are currently disabled due to a bug in Lightning CSS. See https://github.com/parcel-bundler/lightningcss/pull/971 for more details. - In Vite, minified CSS build source maps are not supported at all. See https://github.com/vitejs/vite/issues/2830 for more details. - In PostCSS, minified CSS source maps are not supported. This is due to the complexity required around re-associating every AST node with a location in the generated, optimized CSS. This complexity would also have a non-trivial performance impact. ## Testing Here's how to test the source map functionality in different environments: ### Testing the CLI 1. Setup typical project that the CLI can use and with sources to scan. ```css @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 2. Build with source maps: ```bash bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map ``` 3. Open Chrome DevTools, inspect an element with utility classes, and you should see rules pointing to `input.css` or `node_modules/tailwindcss/index.css` ### Testing with Vite Testing in Vite will require building and installing necessary files under `dist/*.tgz`. 1. Create a Vite project and enable source maps in `vite.config.js`: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { // This line is required for them to work devSourcemap: true, }, }) ``` 2. Add a component that uses Tailwind classes and custom CSS: ```jsx // ./src/app.jsx export default function App() { return ( <div className="bg-blue-500 my-custom-class"> Hello World </div> ) } ``` ```css /* ./src/styles.css */ @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 3. Run `npm run dev`, open DevTools, and inspect elements to verify source mapping works for both utility classes and custom CSS. ### Testing with PostCSS CLI 1. Create a test file and update your PostCSS config: ```css /* input.css */ @import "tailwindcss"; @layer components { .card { @apply p-6 rounded-lg shadow-lg; } } ``` ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {} } } } ``` 2. Run PostCSS through Bun: ```bash bunx --bun postcss ./src/index.css -o out.css ``` 3. Inspect the output CSS - it should include an inline source map comment at the bottom. ### Testing with PostCSS + Next.js Testing in Next.js will require building and installing necessary files under `dist/*.tgz`. However, I've not been able to get CSS source maps to work in Next.js without this hack: ```js const nextConfig: NextConfig = { // next.js overwrites config.devtool so we prevent it from doing so // please don't actually do this… webpack: (config) => Object.defineProperty(config, "devtool", { get: () => "inline-source-map", set: () => {}, }), }; ``` This is definitely not supported and also doesn't work with turbopack. This can be used to test them temporarily but I suspect that they just don't work there. ### Manual source map analysis You can analyze source maps using Evan Wallace's [Source Map Visualization](https://evanw.github.io/source-map-visualization/) tool which will help to verify the accuracy and quality of source maps. This is what I used extensively while developing this implementation. It'll help verify that custom, user CSS maps back to itself in the input, that generated utilities all map back to `@tailwind utilities;`, that source locations from imported files are also handled correctly, etc… It also highlights the ranges of stuff so it's easy to see if there are off-by-one errors. It's easiest to use inline source maps with this tool because you can take the CSS file and drop it on the page and it'll analyze it while showing the file content. If you're using Vite you'll want to access the CSS file with `?direct` at the end so you don't get a JS module back. ## Implementation The source map implementation follows the ECMA-426 specification and includes several key components to aid in that goal: ### Source Location Tracking Each emittable AST node in the compilation pipeline tracks two types of source locations: - `src`: Original source location - [source file, start offset, end offset] - `dst`: Generated source location - [output file, start offset, end offset] This dual tracking allows us to maintain mappings between the original source and generated output for things like user CSS, generated utilities, uses of `@apply`, and tracking theme variables. It is important to note that source locations for nodes _never overlap_ within a file which helps simplify source map generation. As such each type of node tracks a specific piece of itself rather than its entire "block": | Node | What a `SourceLocation` represents | | ----------- | ---------------------------------------------------------------- | | Style Rule | The selector | | At Rule | Rule name and params, includes the `@` | | Declaration | Property name and value, excludes the semicolon | | Comment | The entire comment, includes the start `/*` and end `*/` markers | ### Windows line endings when parsing CSS Because our AST tracks nodes through offsets we must ensure that any mutations to the file do *not* change the lenth of the string. We were previously replacing `\r\n` with `\n` (see [filter code points](https://drafts.csswg.org/css-syntax/#css-filter-code-points) from the spec) — which changes the length of the string and all offsets may end up incorrect. The CSS parser was updated to handle the CRLF token directly by skipping over the `\r` and letting remaining code handle `\n` as it did previously. Some additional tweaks were required when "peeking" the input but those changes were fairly small. ### Tracking of imports Source maps need paths to the actual imported stylesheets but the resolve step for stylesheets happens inside the call to `loadStylesheet` which make the file path unavailable to us. Because of this the `loadStylesheet` API was augmented such that it has to return a `path` property that we can then use to identify imported sources. I've also made the same change to the `loadModule` API for consistency but nothing currently uses this property. The `path` property likely makes `base` redundant but elminating that (if we even want to) is a future task. ### Optimizing the AST Our optimization pass may intoduce some nodes, for example, fallbacks we create for `@property`. These nodes are linked back to `@tailwind utilities` as ultimately that is what is responsible for creating them. ### Line Offset Tables A key component to our source map generation is the line offset table, which was inspired by some ESBuild internals. It stores a sorted list of offsets for the start of each line allowing us to translate offsets to line/column `Position`s in `O(log N)` time and from `Position`s to offsets in `O(1)` time. Creation of the table takes `O(N)` time. This means that we can store code point offsets for source locations and not have to worry about computing or tracking line/column numbers during parsing and serialization. Only when a source map is generated do these offsets need to be computed. This ensures the performance penalty when not using source maps is minimal. ### Source Map Generation The source map returned by `buildSourceMap()` is designed to follow the [ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not completely finalized we consider the result of `buildSourceMap()` to be internal API that may change as the spec chamges. The produces source map is a "decoded" map such that all sources and mappings are in an object graph. A library like `source-map-js` must be used to convert this to an encoded source map of the right version where mappings are encoded with base 64 VLQs. Any specific integration (Vite, PostCSS, etc…) can then use `toSourceMap()` from `@tailwindcss/node` to convert from the internal source map to an spec-compliant encoded source map that can be understood by other tools. ### Handling minification in Lightning Since we use Lightning CSS for optimization, and it takes in an input map, we generate an encoded source map that we then pass to lightning. The output source map *from lighting itself* is then passed back in during the second optimization pass. The final map is then passed from lightning to the CLI (but not Vite or PostCSS — see the limitations section for details). In some cases we have to "fix up" the output CSS. When this happens we use `magic-string` to do the replacement in a way that is trackable and `@amppproject/remapping` to map that change back onto the original source map. Once the need for these fix ups disappear these dependencies can go away. Notes: - The accuracy of source maps run though lightning is reduced as it only tracks on a per-rule level. This is sufficient enough for browser dev tools so should be fine. - Source maps during optimization do not function properly at this time because of a bug in Lightning CSS regarding license comments. Once this bug is fixed they will start working as expected. ### How source locations flow through the system 1. During initial CSS parsing, source locations are preserved. 2. During parsing these source locations are also mapped to the destinations which supports an optimization for when no utilities are generated. 3. Throughout the compilation process, transformations maintain source location data 4. Generated utilities are explicitly pointed to `@tailwind utilities` unless generated by `@apply`. 5. When optimization is enabled, source maps are remapped through lightningcss 6. Final source maps are written in the requested format (inline or separate file)
2025-05-08 16:29:49 -04:00
if (acc.length > 0) acc += '\n'
acc += css
return acc
}, '')
}
export async function getRandomPort() {
return new Promise<number>((resolve, reject) => {
let server = createServer()
server.unref()
server.on('error', reject)
server.listen(0, '127.0.0.1', () => {
let address = server.address()
server.close(() => {
if (address && typeof address === 'object') {
resolve(address.port)
} else {
reject(new Error('Unable to allocate random port'))
}
})
})
})
}
async function killProcessTree(child: ChildProcess) {
if (child.exitCode !== null || child.signalCode !== null || child.pid === undefined) {
return
}
if (IS_WINDOWS) {
await execFileAsync('taskkill', ['/pid', String(child.pid), '/T', '/F'], {
timeout: ASSERTION_TIMEOUT,
windowsHide: true,
}).catch(() => {})
return
}
try {
process.kill(-child.pid, 'SIGTERM')
} catch (error: any) {
if (error?.code !== 'ESRCH') {
child.kill()
}
}
}
function forceKillProcessTree(child: ChildProcess) {
if (child.exitCode !== null || child.signalCode !== null || child.pid === undefined) {
return
}
if (IS_WINDOWS) {
execFile('taskkill', ['/pid', String(child.pid), '/T', '/F'], { windowsHide: true }, () => {})
return
}
try {
process.kill(-child.pid, 'SIGKILL')
} catch (error: any) {
if (error?.code !== 'ESRCH') {
child.kill('SIGKILL')
}
}
}
async function gracefullyRemove(dir: string) {
// Skip removing the directory in CI because it can stall on Windows
if (!process.env.CI) {
await fs.rm(dir, { recursive: true, force: true }).catch((error) => {
console.log(`Failed to remove ${dir}`, error)
})
}
}
Add support for source maps (#17775) Closes #13694 Closes #13591 # Source Maps Support for Tailwind CSS This PR adds support for source maps to Tailwind CSS v4 allowing us to track where styles come from whether that be user CSS, imported stylesheets, or generated utilities. This will improve debuggability in browser dev tools and gives us a good foundation for producing better error messages. I'll go over the details on how end users can enable source maps, any limitations in our implementation, changes to the internal `compile(…)` API, and some details and reasoning around the implementation we chose. ## Usage ### CLI Source maps can be enabled in the CLI by using the command line argument `--map` which will generate an inline source map comment at the bottom of your CSS. A separate file may be generated by passing a file name to `--map`: ```bash # Generates an inline source map npx tailwindcss -i input.css -o output.css --map # Generates a separate source map file npx tailwindcss -i input.css -o output.css --map output.css.map ``` ### PostCSS Source maps are supported when using Tailwind as a PostCSS plugin *in development mode only*. They may or may not be enabled by default depending on your build tool. If they are not you may be able to configure them within your PostCSS config: ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "@tailwindcss/postcss": {}, }, } } ``` ### Vite Source maps are supported when using the Tailwind CSS Vite plugin in *development mode only* by enabling the `css.devSourcemap` setting: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { devSourcemap: true, }, }) ``` Now when a CSS file is requested by the browser it'll have an inline source map comment that the browser can use. ## Limitations - Production build source maps are currently disabled due to a bug in Lightning CSS. See https://github.com/parcel-bundler/lightningcss/pull/971 for more details. - In Vite, minified CSS build source maps are not supported at all. See https://github.com/vitejs/vite/issues/2830 for more details. - In PostCSS, minified CSS source maps are not supported. This is due to the complexity required around re-associating every AST node with a location in the generated, optimized CSS. This complexity would also have a non-trivial performance impact. ## Testing Here's how to test the source map functionality in different environments: ### Testing the CLI 1. Setup typical project that the CLI can use and with sources to scan. ```css @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 2. Build with source maps: ```bash bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map ``` 3. Open Chrome DevTools, inspect an element with utility classes, and you should see rules pointing to `input.css` or `node_modules/tailwindcss/index.css` ### Testing with Vite Testing in Vite will require building and installing necessary files under `dist/*.tgz`. 1. Create a Vite project and enable source maps in `vite.config.js`: ```js import tailwindcss from "@tailwindcss/vite"; import { defineConfig } from "vite"; export default defineConfig({ plugins: [tailwindcss()], css: { // This line is required for them to work devSourcemap: true, }, }) ``` 2. Add a component that uses Tailwind classes and custom CSS: ```jsx // ./src/app.jsx export default function App() { return ( <div className="bg-blue-500 my-custom-class"> Hello World </div> ) } ``` ```css /* ./src/styles.css */ @import "tailwindcss"; @utilty my-custom-utility { color: red; } /* to test `@apply` */ .card { @apply bg-white text-center shadow-md; } ``` 3. Run `npm run dev`, open DevTools, and inspect elements to verify source mapping works for both utility classes and custom CSS. ### Testing with PostCSS CLI 1. Create a test file and update your PostCSS config: ```css /* input.css */ @import "tailwindcss"; @layer components { .card { @apply p-6 rounded-lg shadow-lg; } } ``` ```jsonc // package.json { // … "postcss": { "map": { "inline": true }, "plugins": { "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {} } } } ``` 2. Run PostCSS through Bun: ```bash bunx --bun postcss ./src/index.css -o out.css ``` 3. Inspect the output CSS - it should include an inline source map comment at the bottom. ### Testing with PostCSS + Next.js Testing in Next.js will require building and installing necessary files under `dist/*.tgz`. However, I've not been able to get CSS source maps to work in Next.js without this hack: ```js const nextConfig: NextConfig = { // next.js overwrites config.devtool so we prevent it from doing so // please don't actually do this… webpack: (config) => Object.defineProperty(config, "devtool", { get: () => "inline-source-map", set: () => {}, }), }; ``` This is definitely not supported and also doesn't work with turbopack. This can be used to test them temporarily but I suspect that they just don't work there. ### Manual source map analysis You can analyze source maps using Evan Wallace's [Source Map Visualization](https://evanw.github.io/source-map-visualization/) tool which will help to verify the accuracy and quality of source maps. This is what I used extensively while developing this implementation. It'll help verify that custom, user CSS maps back to itself in the input, that generated utilities all map back to `@tailwind utilities;`, that source locations from imported files are also handled correctly, etc… It also highlights the ranges of stuff so it's easy to see if there are off-by-one errors. It's easiest to use inline source maps with this tool because you can take the CSS file and drop it on the page and it'll analyze it while showing the file content. If you're using Vite you'll want to access the CSS file with `?direct` at the end so you don't get a JS module back. ## Implementation The source map implementation follows the ECMA-426 specification and includes several key components to aid in that goal: ### Source Location Tracking Each emittable AST node in the compilation pipeline tracks two types of source locations: - `src`: Original source location - [source file, start offset, end offset] - `dst`: Generated source location - [output file, start offset, end offset] This dual tracking allows us to maintain mappings between the original source and generated output for things like user CSS, generated utilities, uses of `@apply`, and tracking theme variables. It is important to note that source locations for nodes _never overlap_ within a file which helps simplify source map generation. As such each type of node tracks a specific piece of itself rather than its entire "block": | Node | What a `SourceLocation` represents | | ----------- | ---------------------------------------------------------------- | | Style Rule | The selector | | At Rule | Rule name and params, includes the `@` | | Declaration | Property name and value, excludes the semicolon | | Comment | The entire comment, includes the start `/*` and end `*/` markers | ### Windows line endings when parsing CSS Because our AST tracks nodes through offsets we must ensure that any mutations to the file do *not* change the lenth of the string. We were previously replacing `\r\n` with `\n` (see [filter code points](https://drafts.csswg.org/css-syntax/#css-filter-code-points) from the spec) — which changes the length of the string and all offsets may end up incorrect. The CSS parser was updated to handle the CRLF token directly by skipping over the `\r` and letting remaining code handle `\n` as it did previously. Some additional tweaks were required when "peeking" the input but those changes were fairly small. ### Tracking of imports Source maps need paths to the actual imported stylesheets but the resolve step for stylesheets happens inside the call to `loadStylesheet` which make the file path unavailable to us. Because of this the `loadStylesheet` API was augmented such that it has to return a `path` property that we can then use to identify imported sources. I've also made the same change to the `loadModule` API for consistency but nothing currently uses this property. The `path` property likely makes `base` redundant but elminating that (if we even want to) is a future task. ### Optimizing the AST Our optimization pass may intoduce some nodes, for example, fallbacks we create for `@property`. These nodes are linked back to `@tailwind utilities` as ultimately that is what is responsible for creating them. ### Line Offset Tables A key component to our source map generation is the line offset table, which was inspired by some ESBuild internals. It stores a sorted list of offsets for the start of each line allowing us to translate offsets to line/column `Position`s in `O(log N)` time and from `Position`s to offsets in `O(1)` time. Creation of the table takes `O(N)` time. This means that we can store code point offsets for source locations and not have to worry about computing or tracking line/column numbers during parsing and serialization. Only when a source map is generated do these offsets need to be computed. This ensures the performance penalty when not using source maps is minimal. ### Source Map Generation The source map returned by `buildSourceMap()` is designed to follow the [ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not completely finalized we consider the result of `buildSourceMap()` to be internal API that may change as the spec chamges. The produces source map is a "decoded" map such that all sources and mappings are in an object graph. A library like `source-map-js` must be used to convert this to an encoded source map of the right version where mappings are encoded with base 64 VLQs. Any specific integration (Vite, PostCSS, etc…) can then use `toSourceMap()` from `@tailwindcss/node` to convert from the internal source map to an spec-compliant encoded source map that can be understood by other tools. ### Handling minification in Lightning Since we use Lightning CSS for optimization, and it takes in an input map, we generate an encoded source map that we then pass to lightning. The output source map *from lighting itself* is then passed back in during the second optimization pass. The final map is then passed from lightning to the CLI (but not Vite or PostCSS — see the limitations section for details). In some cases we have to "fix up" the output CSS. When this happens we use `magic-string` to do the replacement in a way that is trackable and `@amppproject/remapping` to map that change back onto the original source map. Once the need for these fix ups disappear these dependencies can go away. Notes: - The accuracy of source maps run though lightning is reduced as it only tracks on a per-rule level. This is sufficient enough for browser dev tools so should be fine. - Source maps during optimization do not function properly at this time because of a bug in Lightning CSS regarding license comments. Once this bug is fixed they will start working as expected. ### How source locations flow through the system 1. During initial CSS parsing, source locations are preserved. 2. During parsing these source locations are also mapped to the destinations which supports an optimization for when no utilities are generated. 3. Throughout the compilation process, transformations maintain source location data 4. Generated utilities are explicitly pointed to `@tailwind utilities` unless generated by `@apply`. 5. When optimization is enabled, source maps are remapped through lightningcss 6. Final source maps are written in the requested format (inline or separate file)
2025-05-08 16:29:49 -04:00
const SOURCE_MAP_COMMENT = /^\/\*# sourceMappingURL=data:application\/json;base64,(.*) \*\/$/
export interface SourceMap {
at(
line: number,
column: number,
): {
source: string | null
original: string
generated: string
}
}
interface SourceMapOptions {
/**
* A raw source map
*
* This may be a string or an object. Strings will be decoded.
*/
map: string | object
/**
* The content of the generated file the source map is for
*/
content: string
/**
* The encoding of the source map
*
* Can be used to decode a base64 map (e.g. an inline source map URI)
*/
encoding?: BufferEncoding
}
function parseSourceMap(opts: string | SourceMapOptions): SourceMap {
if (typeof opts === 'string') {
let lines = opts.trimEnd().split('\n')
let comment = lines.at(-1) ?? ''
let map = String(comment).match(SOURCE_MAP_COMMENT)?.[1] ?? null
if (!map) throw new Error('No source map comment found')
return parseSourceMap({
map,
content: lines.slice(0, -1).join('\n'),
encoding: 'base64',
})
}
let rawMap: RawSourceMap
let content = opts.content
if (typeof opts.map === 'object') {
rawMap = opts.map as RawSourceMap
} else {
rawMap = JSON.parse(Buffer.from(opts.map, opts.encoding ?? 'utf-8').toString())
}
let map = new SourceMapConsumer(rawMap)
let generatedTable = createLineTable(content)
return {
at(line: number, column: number) {
let pos = map.originalPositionFor({ line, column })
let source = pos.source ? map.sourceContentFor(pos.source) : null
let originalTable = createLineTable(source ?? '')
let originalOffset = originalTable.findOffset(pos)
let generatedOffset = generatedTable.findOffset({ line, column })
return {
source: pos.source,
original: source
? source.slice(originalOffset, originalOffset + 10).trim() + '...'
: '(none)',
generated: content.slice(generatedOffset, generatedOffset + 10).trim() + '...',
}
},
}
}