Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
import dedent from 'dedent'
|
|
|
|
|
import fastGlob from 'fast-glob'
|
2026-04-30 23:23:36 +02:00
|
|
|
import { exec, execFile, spawn, type ChildProcess } from 'node:child_process'
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
import fs from 'node:fs/promises'
|
2026-04-30 23:23:36 +02:00
|
|
|
import { createServer } from 'node:net'
|
2024-09-21 03:59:18 +08:00
|
|
|
import { platform, tmpdir } from 'node:os'
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
import path from 'node:path'
|
2026-04-30 23:23:36 +02:00
|
|
|
import { promisify, stripVTControlCharacters } from 'node:util'
|
Add support for source maps (#17775)
Closes #13694
Closes #13591
# Source Maps Support for Tailwind CSS
This PR adds support for source maps to Tailwind CSS v4 allowing us to
track where styles come from whether that be user CSS, imported
stylesheets, or generated utilities. This will improve debuggability in
browser dev tools and gives us a good foundation for producing better
error messages. I'll go over the details on how end users can enable
source maps, any limitations in our implementation, changes to the
internal `compile(…)` API, and some details and reasoning around the
implementation we chose.
## Usage
### CLI
Source maps can be enabled in the CLI by using the command line argument
`--map` which will generate an inline source map comment at the bottom
of your CSS. A separate file may be generated by passing a file name to
`--map`:
```bash
# Generates an inline source map
npx tailwindcss -i input.css -o output.css --map
# Generates a separate source map file
npx tailwindcss -i input.css -o output.css --map output.css.map
```
### PostCSS
Source maps are supported when using Tailwind as a PostCSS plugin *in
development mode only*. They may or may not be enabled by default
depending on your build tool. If they are not you may be able to
configure them within your PostCSS config:
```jsonc
// package.json
{
// …
"postcss": {
"map": { "inline": true },
"plugins": {
"@tailwindcss/postcss": {},
},
}
}
```
### Vite
Source maps are supported when using the Tailwind CSS Vite plugin in
*development mode only* by enabling the `css.devSourcemap` setting:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
devSourcemap: true,
},
})
```
Now when a CSS file is requested by the browser it'll have an inline
source map comment that the browser can use.
## Limitations
- Production build source maps are currently disabled due to a bug in
Lightning CSS. See
https://github.com/parcel-bundler/lightningcss/pull/971 for more
details.
- In Vite, minified CSS build source maps are not supported at all. See
https://github.com/vitejs/vite/issues/2830 for more details.
- In PostCSS, minified CSS source maps are not supported. This is due to
the complexity required around re-associating every AST node with a
location in the generated, optimized CSS. This complexity would also
have a non-trivial performance impact.
## Testing
Here's how to test the source map functionality in different
environments:
### Testing the CLI
1. Setup typical project that the CLI can use and with sources to scan.
```css
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
2. Build with source maps:
```bash
bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map
```
3. Open Chrome DevTools, inspect an element with utility classes, and
you should see rules pointing to `input.css` or
`node_modules/tailwindcss/index.css`
### Testing with Vite
Testing in Vite will require building and installing necessary files
under `dist/*.tgz`.
1. Create a Vite project and enable source maps in `vite.config.js`:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
// This line is required for them to work
devSourcemap: true,
},
})
```
2. Add a component that uses Tailwind classes and custom CSS:
```jsx
// ./src/app.jsx
export default function App() {
return (
<div className="bg-blue-500 my-custom-class">
Hello World
</div>
)
}
```
```css
/* ./src/styles.css */
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
3. Run `npm run dev`, open DevTools, and inspect elements to verify
source mapping works for both utility classes and custom CSS.
### Testing with PostCSS CLI
1. Create a test file and update your PostCSS config:
```css
/* input.css */
@import "tailwindcss";
@layer components {
.card {
@apply p-6 rounded-lg shadow-lg;
}
}
```
```jsonc
// package.json
{
// …
"postcss": {
"map": {
"inline": true
},
"plugins": {
"/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {}
}
}
}
```
2. Run PostCSS through Bun:
```bash
bunx --bun postcss ./src/index.css -o out.css
```
3. Inspect the output CSS - it should include an inline source map
comment at the bottom.
### Testing with PostCSS + Next.js
Testing in Next.js will require building and installing necessary files
under `dist/*.tgz`. However, I've not been able to get CSS source maps
to work in Next.js without this hack:
```js
const nextConfig: NextConfig = {
// next.js overwrites config.devtool so we prevent it from doing so
// please don't actually do this…
webpack: (config) =>
Object.defineProperty(config, "devtool", {
get: () => "inline-source-map",
set: () => {},
}),
};
```
This is definitely not supported and also doesn't work with turbopack.
This can be used to test them temporarily but I suspect that they just
don't work there.
### Manual source map analysis
You can analyze source maps using Evan Wallace's [Source Map
Visualization](https://evanw.github.io/source-map-visualization/) tool
which will help to verify the accuracy and quality of source maps. This
is what I used extensively while developing this implementation.
It'll help verify that custom, user CSS maps back to itself in the
input, that generated utilities all map back to `@tailwind utilities;`,
that source locations from imported files are also handled correctly,
etc… It also highlights the ranges of stuff so it's easy to see if there
are off-by-one errors.
It's easiest to use inline source maps with this tool because you can
take the CSS file and drop it on the page and it'll analyze it while
showing the file content.
If you're using Vite you'll want to access the CSS file with `?direct`
at the end so you don't get a JS module back.
## Implementation
The source map implementation follows the ECMA-426 specification and
includes several key components to aid in that goal:
### Source Location Tracking
Each emittable AST node in the compilation pipeline tracks two types of
source locations:
- `src`: Original source location - [source file, start offset, end
offset]
- `dst`: Generated source location - [output file, start offset, end
offset]
This dual tracking allows us to maintain mappings between the original
source and generated output for things like user CSS, generated
utilities, uses of `@apply`, and tracking theme variables.
It is important to note that source locations for nodes _never overlap_
within a file which helps simplify source map generation. As such each
type of node tracks a specific piece of itself rather than its entire
"block":
| Node | What a `SourceLocation` represents |
| ----------- |
---------------------------------------------------------------- |
| Style Rule | The selector |
| At Rule | Rule name and params, includes the `@` |
| Declaration | Property name and value, excludes the semicolon |
| Comment | The entire comment, includes the start `/*` and end `*/`
markers |
### Windows line endings when parsing CSS
Because our AST tracks nodes through offsets we must ensure that any
mutations to the file do *not* change the lenth of the string. We were
previously replacing `\r\n` with `\n` (see [filter code
points](https://drafts.csswg.org/css-syntax/#css-filter-code-points)
from the spec) — which changes the length of the string and all offsets
may end up incorrect. The CSS parser was updated to handle the CRLF
token directly by skipping over the `\r` and letting remaining code
handle `\n` as it did previously. Some additional tweaks were required
when "peeking" the input but those changes were fairly small.
### Tracking of imports
Source maps need paths to the actual imported stylesheets but the
resolve step for stylesheets happens inside the call to `loadStylesheet`
which make the file path unavailable to us. Because of this the
`loadStylesheet` API was augmented such that it has to return a `path`
property that we can then use to identify imported sources. I've also
made the same change to the `loadModule` API for consistency but nothing
currently uses this property.
The `path` property likely makes `base` redundant but elminating that
(if we even want to) is a future task.
### Optimizing the AST
Our optimization pass may intoduce some nodes, for example, fallbacks we
create for `@property`. These nodes are linked back to `@tailwind
utilities` as ultimately that is what is responsible for creating them.
### Line Offset Tables
A key component to our source map generation is the line offset table,
which was inspired by some ESBuild internals. It stores a sorted list of
offsets for the start of each line allowing us to translate offsets to
line/column `Position`s in `O(log N)` time and from `Position`s to
offsets in `O(1)` time. Creation of the table takes `O(N)` time.
This means that we can store code point offsets for source locations and
not have to worry about computing or tracking line/column numbers during
parsing and serialization. Only when a source map is generated do these
offsets need to be computed. This ensures the performance penalty when
not using source maps is minimal.
### Source Map Generation
The source map returned by `buildSourceMap()` is designed to follow the
[ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not
completely finalized we consider the result of `buildSourceMap()` to be
internal API that may change as the spec chamges.
The produces source map is a "decoded" map such that all sources and
mappings are in an object graph. A library like `source-map-js` must be
used to convert this to an encoded source map of the right version where
mappings are encoded with base 64 VLQs.
Any specific integration (Vite, PostCSS, etc…) can then use
`toSourceMap()` from `@tailwindcss/node` to convert from the internal
source map to an spec-compliant encoded source map that can be
understood by other tools.
### Handling minification in Lightning
Since we use Lightning CSS for optimization, and it takes in an input
map, we generate an encoded source map that we then pass to lightning.
The output source map *from lighting itself* is then passed back in
during the second optimization pass. The final map is then passed from
lightning to the CLI (but not Vite or PostCSS — see the limitations
section for details).
In some cases we have to "fix up" the output CSS. When this happens we
use `magic-string` to do the replacement in a way that is trackable and
`@amppproject/remapping` to map that change back onto the original
source map. Once the need for these fix ups disappear these dependencies
can go away.
Notes:
- The accuracy of source maps run though lightning is reduced as it only
tracks on a per-rule level. This is sufficient enough for browser dev
tools so should be fine.
- Source maps during optimization do not function properly at this time
because of a bug in Lightning CSS regarding license comments. Once this
bug is fixed they will start working as expected.
### How source locations flow through the system
1. During initial CSS parsing, source locations are preserved.
2. During parsing these source locations are also mapped to the
destinations which supports an optimization for when no utilities are
generated.
3. Throughout the compilation process, transformations maintain source
location data
4. Generated utilities are explicitly pointed to `@tailwind utilities`
unless generated by `@apply`.
5. When optimization is enabled, source maps are remapped through
lightningcss
6. Final source maps are written in the requested format (inline or
separate file)
2025-05-08 16:29:49 -04:00
|
|
|
import { RawSourceMap, SourceMapConsumer } from 'source-map-js'
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
import { test as defaultTest, type ExpectStatic } from 'vitest'
|
2026-06-25 19:14:10 +02:00
|
|
|
import * as Yaml from 'yaml'
|
Add support for source maps (#17775)
Closes #13694
Closes #13591
# Source Maps Support for Tailwind CSS
This PR adds support for source maps to Tailwind CSS v4 allowing us to
track where styles come from whether that be user CSS, imported
stylesheets, or generated utilities. This will improve debuggability in
browser dev tools and gives us a good foundation for producing better
error messages. I'll go over the details on how end users can enable
source maps, any limitations in our implementation, changes to the
internal `compile(…)` API, and some details and reasoning around the
implementation we chose.
## Usage
### CLI
Source maps can be enabled in the CLI by using the command line argument
`--map` which will generate an inline source map comment at the bottom
of your CSS. A separate file may be generated by passing a file name to
`--map`:
```bash
# Generates an inline source map
npx tailwindcss -i input.css -o output.css --map
# Generates a separate source map file
npx tailwindcss -i input.css -o output.css --map output.css.map
```
### PostCSS
Source maps are supported when using Tailwind as a PostCSS plugin *in
development mode only*. They may or may not be enabled by default
depending on your build tool. If they are not you may be able to
configure them within your PostCSS config:
```jsonc
// package.json
{
// …
"postcss": {
"map": { "inline": true },
"plugins": {
"@tailwindcss/postcss": {},
},
}
}
```
### Vite
Source maps are supported when using the Tailwind CSS Vite plugin in
*development mode only* by enabling the `css.devSourcemap` setting:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
devSourcemap: true,
},
})
```
Now when a CSS file is requested by the browser it'll have an inline
source map comment that the browser can use.
## Limitations
- Production build source maps are currently disabled due to a bug in
Lightning CSS. See
https://github.com/parcel-bundler/lightningcss/pull/971 for more
details.
- In Vite, minified CSS build source maps are not supported at all. See
https://github.com/vitejs/vite/issues/2830 for more details.
- In PostCSS, minified CSS source maps are not supported. This is due to
the complexity required around re-associating every AST node with a
location in the generated, optimized CSS. This complexity would also
have a non-trivial performance impact.
## Testing
Here's how to test the source map functionality in different
environments:
### Testing the CLI
1. Setup typical project that the CLI can use and with sources to scan.
```css
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
2. Build with source maps:
```bash
bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map
```
3. Open Chrome DevTools, inspect an element with utility classes, and
you should see rules pointing to `input.css` or
`node_modules/tailwindcss/index.css`
### Testing with Vite
Testing in Vite will require building and installing necessary files
under `dist/*.tgz`.
1. Create a Vite project and enable source maps in `vite.config.js`:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
// This line is required for them to work
devSourcemap: true,
},
})
```
2. Add a component that uses Tailwind classes and custom CSS:
```jsx
// ./src/app.jsx
export default function App() {
return (
<div className="bg-blue-500 my-custom-class">
Hello World
</div>
)
}
```
```css
/* ./src/styles.css */
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
3. Run `npm run dev`, open DevTools, and inspect elements to verify
source mapping works for both utility classes and custom CSS.
### Testing with PostCSS CLI
1. Create a test file and update your PostCSS config:
```css
/* input.css */
@import "tailwindcss";
@layer components {
.card {
@apply p-6 rounded-lg shadow-lg;
}
}
```
```jsonc
// package.json
{
// …
"postcss": {
"map": {
"inline": true
},
"plugins": {
"/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {}
}
}
}
```
2. Run PostCSS through Bun:
```bash
bunx --bun postcss ./src/index.css -o out.css
```
3. Inspect the output CSS - it should include an inline source map
comment at the bottom.
### Testing with PostCSS + Next.js
Testing in Next.js will require building and installing necessary files
under `dist/*.tgz`. However, I've not been able to get CSS source maps
to work in Next.js without this hack:
```js
const nextConfig: NextConfig = {
// next.js overwrites config.devtool so we prevent it from doing so
// please don't actually do this…
webpack: (config) =>
Object.defineProperty(config, "devtool", {
get: () => "inline-source-map",
set: () => {},
}),
};
```
This is definitely not supported and also doesn't work with turbopack.
This can be used to test them temporarily but I suspect that they just
don't work there.
### Manual source map analysis
You can analyze source maps using Evan Wallace's [Source Map
Visualization](https://evanw.github.io/source-map-visualization/) tool
which will help to verify the accuracy and quality of source maps. This
is what I used extensively while developing this implementation.
It'll help verify that custom, user CSS maps back to itself in the
input, that generated utilities all map back to `@tailwind utilities;`,
that source locations from imported files are also handled correctly,
etc… It also highlights the ranges of stuff so it's easy to see if there
are off-by-one errors.
It's easiest to use inline source maps with this tool because you can
take the CSS file and drop it on the page and it'll analyze it while
showing the file content.
If you're using Vite you'll want to access the CSS file with `?direct`
at the end so you don't get a JS module back.
## Implementation
The source map implementation follows the ECMA-426 specification and
includes several key components to aid in that goal:
### Source Location Tracking
Each emittable AST node in the compilation pipeline tracks two types of
source locations:
- `src`: Original source location - [source file, start offset, end
offset]
- `dst`: Generated source location - [output file, start offset, end
offset]
This dual tracking allows us to maintain mappings between the original
source and generated output for things like user CSS, generated
utilities, uses of `@apply`, and tracking theme variables.
It is important to note that source locations for nodes _never overlap_
within a file which helps simplify source map generation. As such each
type of node tracks a specific piece of itself rather than its entire
"block":
| Node | What a `SourceLocation` represents |
| ----------- |
---------------------------------------------------------------- |
| Style Rule | The selector |
| At Rule | Rule name and params, includes the `@` |
| Declaration | Property name and value, excludes the semicolon |
| Comment | The entire comment, includes the start `/*` and end `*/`
markers |
### Windows line endings when parsing CSS
Because our AST tracks nodes through offsets we must ensure that any
mutations to the file do *not* change the lenth of the string. We were
previously replacing `\r\n` with `\n` (see [filter code
points](https://drafts.csswg.org/css-syntax/#css-filter-code-points)
from the spec) — which changes the length of the string and all offsets
may end up incorrect. The CSS parser was updated to handle the CRLF
token directly by skipping over the `\r` and letting remaining code
handle `\n` as it did previously. Some additional tweaks were required
when "peeking" the input but those changes were fairly small.
### Tracking of imports
Source maps need paths to the actual imported stylesheets but the
resolve step for stylesheets happens inside the call to `loadStylesheet`
which make the file path unavailable to us. Because of this the
`loadStylesheet` API was augmented such that it has to return a `path`
property that we can then use to identify imported sources. I've also
made the same change to the `loadModule` API for consistency but nothing
currently uses this property.
The `path` property likely makes `base` redundant but elminating that
(if we even want to) is a future task.
### Optimizing the AST
Our optimization pass may intoduce some nodes, for example, fallbacks we
create for `@property`. These nodes are linked back to `@tailwind
utilities` as ultimately that is what is responsible for creating them.
### Line Offset Tables
A key component to our source map generation is the line offset table,
which was inspired by some ESBuild internals. It stores a sorted list of
offsets for the start of each line allowing us to translate offsets to
line/column `Position`s in `O(log N)` time and from `Position`s to
offsets in `O(1)` time. Creation of the table takes `O(N)` time.
This means that we can store code point offsets for source locations and
not have to worry about computing or tracking line/column numbers during
parsing and serialization. Only when a source map is generated do these
offsets need to be computed. This ensures the performance penalty when
not using source maps is minimal.
### Source Map Generation
The source map returned by `buildSourceMap()` is designed to follow the
[ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not
completely finalized we consider the result of `buildSourceMap()` to be
internal API that may change as the spec chamges.
The produces source map is a "decoded" map such that all sources and
mappings are in an object graph. A library like `source-map-js` must be
used to convert this to an encoded source map of the right version where
mappings are encoded with base 64 VLQs.
Any specific integration (Vite, PostCSS, etc…) can then use
`toSourceMap()` from `@tailwindcss/node` to convert from the internal
source map to an spec-compliant encoded source map that can be
understood by other tools.
### Handling minification in Lightning
Since we use Lightning CSS for optimization, and it takes in an input
map, we generate an encoded source map that we then pass to lightning.
The output source map *from lighting itself* is then passed back in
during the second optimization pass. The final map is then passed from
lightning to the CLI (but not Vite or PostCSS — see the limitations
section for details).
In some cases we have to "fix up" the output CSS. When this happens we
use `magic-string` to do the replacement in a way that is trackable and
`@amppproject/remapping` to map that change back onto the original
source map. Once the need for these fix ups disappear these dependencies
can go away.
Notes:
- The accuracy of source maps run though lightning is reduced as it only
tracks on a per-rule level. This is sufficient enough for browser dev
tools so should be fine.
- Source maps during optimization do not function properly at this time
because of a bug in Lightning CSS regarding license comments. Once this
bug is fixed they will start working as expected.
### How source locations flow through the system
1. During initial CSS parsing, source locations are preserved.
2. During parsing these source locations are also mapped to the
destinations which supports an optimization for when no utilities are
generated.
3. Throughout the compilation process, transformations maintain source
location data
4. Generated utilities are explicitly pointed to `@tailwind utilities`
unless generated by `@apply`.
5. When optimization is enabled, source maps are remapped through
lightningcss
6. Final source maps are written in the requested format (inline or
separate file)
2025-05-08 16:29:49 -04:00
|
|
|
import { createLineTable } from '../packages/tailwindcss/src/source-maps/line-table'
|
2025-02-17 20:49:17 +08:00
|
|
|
import { escape } from '../packages/tailwindcss/src/utils/escape'
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
|
|
|
|
const REPO_ROOT = path.join(__dirname, '..')
|
2026-06-25 19:14:10 +02:00
|
|
|
const ROOT_PNPM_WORKSPACE = Yaml.parse(
|
|
|
|
|
await fs.readFile(path.join(REPO_ROOT, 'pnpm-workspace.yaml'), 'utf8'),
|
|
|
|
|
)
|
2024-08-07 16:38:44 +02:00
|
|
|
const PUBLIC_PACKAGES = (await fs.readdir(path.join(REPO_ROOT, 'dist'))).map((name) =>
|
|
|
|
|
name.replace('tailwindcss-', '@tailwindcss/').replace('.tgz', ''),
|
|
|
|
|
)
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
|
|
|
|
interface SpawnedProcess {
|
Make upgrade tooling more stable (#19846)
This PR is an attempt to make the upgrade tooling more stable.
### TL;DR
1. When migrating from Tailwind CSS v3 → Tailwind CSS v4, only migrate
files listed in the `config.content` instead of relying on v4's auto
content detection feature
2. Skip writing files that have not been changed
3. Write changed files in a safe way: first write to a temporary file,
then rename the file atomically
4. Never migrate files that are git ignored, even if they are listed in
the `config.content` file
5. Always ignore `.env` and `.env.*` files when scanning for files. Most
people will have this in their `.gitignore` file, but if not, then this
is a fallback mechanism.
---
Looking at the #18972 issue, it looks like some people are running into
weird situations where some of the contents is just gone.
I have never been able to reproduce this on my own devices and in my own
projects unfortunately. But there is definitely _something_ happening
that's not right that people are running into.
Therefore, this PR is an attempt to fix what I think _might_ be wrong,
but I'm not 100% sure if these fixes are enough, or if something else is
still happening here.
This builds on top of the #19779 PR which has some small fixes, but is
incomplete to make this work.
### What's happening
Looking at some of the comments, it looks like a few things are
happening such as:
1. The upgrade tool is emptying out my files — it looks like these are
only happening if you ctrl+c while the process is taking a while. It
could be that a lot of files are being checked and therefore the tooling
looks like its stuck.
6. The upgrade tool is looking at files it shouldn't look at — in
Tailwind CSS v4 we have this concept of the auto-content detection. This
means that we will look at any plain text file that is not git ignored.
### Fixes
#### Emptying out files
The files being emptied looks like it's because how `fs.writeFile`
behaves by default. It opens the file handle with the `w` flag, which
will first truncate the file before writing the new contents. This is
not a single atomic operation, so a killed process in the middle will
cause invalid state.
When we migrate your template files, everything is happening in promises
to migrate things at the same time. When a lot of files are being
scanned, truncating might have happened already before we write the new
content. Since we migrate a bunch of files in parallel, a ctrl+c could
cause data loss in multiple files.
To mitigate this, I switched to an alternative way of writing files.
1. First, we do some quick checks where if the contents didn't change we
just bail out immediately. Files that don't include Tailwind CSS classes
won't change, and therefore we don't need to override these files with
the same contents.
2. When the migrated contents is empty, we bail out as well. I'm 100%
sure that this is not the spot where the "emptying out" happens, I still
believe it happens in the `writeFile` itself, but added it just in case.
7. Next, I introduced a safe write, where we first write to a temporary
file in the same folder. We could write it to `/tmp`, but then we can't
guarantee that we are on the same file system.
If we ctrl+c at this stage, then the worst case scenario is that you
have additional temporary files in your project, but your original files
are still there.
Once that file was written, we will use the atomic `fs.rename`. This
should be atomic as long as we are on the same file system, so either
the rename didn't happen yet, or it completed.
I added an integration test for this, but I had to change the
`writeFile` implementation slightly. In the test, we will truncate the
file first, after that we will write the new contents. This is so that
we have enough time to kill the current process and allows us to verify
that we didn't clear out the file. Again, this is a hacky way of testing
this, just because I can't reproduce this issue myself, let alone
reproduce it reliable in a CI environment.
Note: we are also using `realpath` to make sure that we are updating the
real file. Otherwise, if we were dealing with a symlinked file, we would
override the symlink with a "hard" copy instead.
#### Touching files that should not be touched
During the migration, we rely on the Tailwind CSS v4 auto detection
logic which means that it will scan any plain text file that is not git
ignored. Therefore changes to php files could happen because in theory
they could contain Tailwind CSS classes.
To solve this, when migrating from Tailwind CSS v3 to Tailwind CSS v4,
we will _only_ take the sources into account that were listed in the
`config.content` array. Since this was a requirement in Tailwind CSS v3,
it should be safe to rely on this array.
Additionally, this will make sure that we are dealing with way fewer
files to migrate as well.
On top of that, files that match the patterns in the content array that
are git ignored will also be skipped. This is to prevent that we mutate
files in `node_modules` for example.
In one of the comments I read that `.env` files were emptied out. In
most cases people will have these files gitignored but I explicitly
added `.env` and `.env.*` as files to never ever touch by default when
scanning.
Last but not least, this also updates the output a little bit of the
upgrade tool in case we skip content files (because of git ignore) and
if we changed a file.
<img width="1122" height="1376" alt="image"
src="https://github.com/user-attachments/assets/318fdbbf-e319-4c7e-9648-ee9283842624"
/>
- "Git ignored folder, skipping: `./node_modules`": this is because the
content array looks like this while the `node_modules` are being
ignored:
<img width="1090" height="398" alt="image"
src="https://github.com/user-attachments/assets/7d694720-5671-47ec-bb2e-f24c5f2c4248"
/>
- "Migrated
`./resources/views/vendor/filament-panels/components/logo.blade.php`":
this is because **I** made a change to showcase this feature.
Fixes: #18972
Closes: #19779
### Test plan
1. Existing tests still pass
1. Added a dedicated integration test to ensure that we only take
`config.content` into account when migrating from Tailwind CSS v3 to
Tailwind CSS v4 projects.
1. Added a dedicated integration test to make sure that files listed in
`config.content` that are also git ignored, will still be skipped.
1. Added a dedicated integration test to ensure that when `writeFile` is
cancelled mid-write that our old files are still present.
1. Added a dedicated integration test to ensure that we ignore `.env`
and `.env.*` files even if you didn't git ignore them.
[ci-all] To verify on Windows
---------
Co-authored-by: Sami <sychocouldy@gmail.com>
2026-03-24 17:24:12 +01:00
|
|
|
dispose: () => Promise<void>
|
2025-04-03 17:13:15 +02:00
|
|
|
flush: () => void
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
onStdout: (predicate: (message: string) => boolean) => Promise<void>
|
|
|
|
|
onStderr: (predicate: (message: string) => boolean) => Promise<void>
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
interface ChildProcessOptions {
|
|
|
|
|
cwd?: string
|
2024-11-07 21:31:06 +01:00
|
|
|
env?: Record<string, string>
|
2024-08-07 16:38:44 +02:00
|
|
|
}
|
|
|
|
|
|
2024-09-11 17:02:24 +02:00
|
|
|
interface ExecOptions {
|
|
|
|
|
ignoreStdErr?: boolean
|
2025-01-17 16:23:34 +01:00
|
|
|
stdin?: string
|
2024-09-11 17:02:24 +02:00
|
|
|
}
|
|
|
|
|
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
interface TestConfig {
|
|
|
|
|
fs: {
|
2024-11-07 09:51:58 -05:00
|
|
|
[filePath: string]: string | Uint8Array
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
2025-01-28 11:13:16 -05:00
|
|
|
|
2025-11-21 00:16:20 +01:00
|
|
|
timeout?: number
|
2025-01-28 11:13:16 -05:00
|
|
|
installDependencies?: boolean
|
2026-06-25 19:14:10 +02:00
|
|
|
retry?: number
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
interface TestContext {
|
2024-08-07 16:38:44 +02:00
|
|
|
root: string
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
expect: ExpectStatic
|
2024-09-11 17:02:24 +02:00
|
|
|
exec(command: string, options?: ChildProcessOptions, execOptions?: ExecOptions): Promise<string>
|
2024-08-07 16:38:44 +02:00
|
|
|
spawn(command: string, options?: ChildProcessOptions): Promise<SpawnedProcess>
|
Add support for source maps (#17775)
Closes #13694
Closes #13591
# Source Maps Support for Tailwind CSS
This PR adds support for source maps to Tailwind CSS v4 allowing us to
track where styles come from whether that be user CSS, imported
stylesheets, or generated utilities. This will improve debuggability in
browser dev tools and gives us a good foundation for producing better
error messages. I'll go over the details on how end users can enable
source maps, any limitations in our implementation, changes to the
internal `compile(…)` API, and some details and reasoning around the
implementation we chose.
## Usage
### CLI
Source maps can be enabled in the CLI by using the command line argument
`--map` which will generate an inline source map comment at the bottom
of your CSS. A separate file may be generated by passing a file name to
`--map`:
```bash
# Generates an inline source map
npx tailwindcss -i input.css -o output.css --map
# Generates a separate source map file
npx tailwindcss -i input.css -o output.css --map output.css.map
```
### PostCSS
Source maps are supported when using Tailwind as a PostCSS plugin *in
development mode only*. They may or may not be enabled by default
depending on your build tool. If they are not you may be able to
configure them within your PostCSS config:
```jsonc
// package.json
{
// …
"postcss": {
"map": { "inline": true },
"plugins": {
"@tailwindcss/postcss": {},
},
}
}
```
### Vite
Source maps are supported when using the Tailwind CSS Vite plugin in
*development mode only* by enabling the `css.devSourcemap` setting:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
devSourcemap: true,
},
})
```
Now when a CSS file is requested by the browser it'll have an inline
source map comment that the browser can use.
## Limitations
- Production build source maps are currently disabled due to a bug in
Lightning CSS. See
https://github.com/parcel-bundler/lightningcss/pull/971 for more
details.
- In Vite, minified CSS build source maps are not supported at all. See
https://github.com/vitejs/vite/issues/2830 for more details.
- In PostCSS, minified CSS source maps are not supported. This is due to
the complexity required around re-associating every AST node with a
location in the generated, optimized CSS. This complexity would also
have a non-trivial performance impact.
## Testing
Here's how to test the source map functionality in different
environments:
### Testing the CLI
1. Setup typical project that the CLI can use and with sources to scan.
```css
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
2. Build with source maps:
```bash
bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map
```
3. Open Chrome DevTools, inspect an element with utility classes, and
you should see rules pointing to `input.css` or
`node_modules/tailwindcss/index.css`
### Testing with Vite
Testing in Vite will require building and installing necessary files
under `dist/*.tgz`.
1. Create a Vite project and enable source maps in `vite.config.js`:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
// This line is required for them to work
devSourcemap: true,
},
})
```
2. Add a component that uses Tailwind classes and custom CSS:
```jsx
// ./src/app.jsx
export default function App() {
return (
<div className="bg-blue-500 my-custom-class">
Hello World
</div>
)
}
```
```css
/* ./src/styles.css */
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
3. Run `npm run dev`, open DevTools, and inspect elements to verify
source mapping works for both utility classes and custom CSS.
### Testing with PostCSS CLI
1. Create a test file and update your PostCSS config:
```css
/* input.css */
@import "tailwindcss";
@layer components {
.card {
@apply p-6 rounded-lg shadow-lg;
}
}
```
```jsonc
// package.json
{
// …
"postcss": {
"map": {
"inline": true
},
"plugins": {
"/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {}
}
}
}
```
2. Run PostCSS through Bun:
```bash
bunx --bun postcss ./src/index.css -o out.css
```
3. Inspect the output CSS - it should include an inline source map
comment at the bottom.
### Testing with PostCSS + Next.js
Testing in Next.js will require building and installing necessary files
under `dist/*.tgz`. However, I've not been able to get CSS source maps
to work in Next.js without this hack:
```js
const nextConfig: NextConfig = {
// next.js overwrites config.devtool so we prevent it from doing so
// please don't actually do this…
webpack: (config) =>
Object.defineProperty(config, "devtool", {
get: () => "inline-source-map",
set: () => {},
}),
};
```
This is definitely not supported and also doesn't work with turbopack.
This can be used to test them temporarily but I suspect that they just
don't work there.
### Manual source map analysis
You can analyze source maps using Evan Wallace's [Source Map
Visualization](https://evanw.github.io/source-map-visualization/) tool
which will help to verify the accuracy and quality of source maps. This
is what I used extensively while developing this implementation.
It'll help verify that custom, user CSS maps back to itself in the
input, that generated utilities all map back to `@tailwind utilities;`,
that source locations from imported files are also handled correctly,
etc… It also highlights the ranges of stuff so it's easy to see if there
are off-by-one errors.
It's easiest to use inline source maps with this tool because you can
take the CSS file and drop it on the page and it'll analyze it while
showing the file content.
If you're using Vite you'll want to access the CSS file with `?direct`
at the end so you don't get a JS module back.
## Implementation
The source map implementation follows the ECMA-426 specification and
includes several key components to aid in that goal:
### Source Location Tracking
Each emittable AST node in the compilation pipeline tracks two types of
source locations:
- `src`: Original source location - [source file, start offset, end
offset]
- `dst`: Generated source location - [output file, start offset, end
offset]
This dual tracking allows us to maintain mappings between the original
source and generated output for things like user CSS, generated
utilities, uses of `@apply`, and tracking theme variables.
It is important to note that source locations for nodes _never overlap_
within a file which helps simplify source map generation. As such each
type of node tracks a specific piece of itself rather than its entire
"block":
| Node | What a `SourceLocation` represents |
| ----------- |
---------------------------------------------------------------- |
| Style Rule | The selector |
| At Rule | Rule name and params, includes the `@` |
| Declaration | Property name and value, excludes the semicolon |
| Comment | The entire comment, includes the start `/*` and end `*/`
markers |
### Windows line endings when parsing CSS
Because our AST tracks nodes through offsets we must ensure that any
mutations to the file do *not* change the lenth of the string. We were
previously replacing `\r\n` with `\n` (see [filter code
points](https://drafts.csswg.org/css-syntax/#css-filter-code-points)
from the spec) — which changes the length of the string and all offsets
may end up incorrect. The CSS parser was updated to handle the CRLF
token directly by skipping over the `\r` and letting remaining code
handle `\n` as it did previously. Some additional tweaks were required
when "peeking" the input but those changes were fairly small.
### Tracking of imports
Source maps need paths to the actual imported stylesheets but the
resolve step for stylesheets happens inside the call to `loadStylesheet`
which make the file path unavailable to us. Because of this the
`loadStylesheet` API was augmented such that it has to return a `path`
property that we can then use to identify imported sources. I've also
made the same change to the `loadModule` API for consistency but nothing
currently uses this property.
The `path` property likely makes `base` redundant but elminating that
(if we even want to) is a future task.
### Optimizing the AST
Our optimization pass may intoduce some nodes, for example, fallbacks we
create for `@property`. These nodes are linked back to `@tailwind
utilities` as ultimately that is what is responsible for creating them.
### Line Offset Tables
A key component to our source map generation is the line offset table,
which was inspired by some ESBuild internals. It stores a sorted list of
offsets for the start of each line allowing us to translate offsets to
line/column `Position`s in `O(log N)` time and from `Position`s to
offsets in `O(1)` time. Creation of the table takes `O(N)` time.
This means that we can store code point offsets for source locations and
not have to worry about computing or tracking line/column numbers during
parsing and serialization. Only when a source map is generated do these
offsets need to be computed. This ensures the performance penalty when
not using source maps is minimal.
### Source Map Generation
The source map returned by `buildSourceMap()` is designed to follow the
[ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not
completely finalized we consider the result of `buildSourceMap()` to be
internal API that may change as the spec chamges.
The produces source map is a "decoded" map such that all sources and
mappings are in an object graph. A library like `source-map-js` must be
used to convert this to an encoded source map of the right version where
mappings are encoded with base 64 VLQs.
Any specific integration (Vite, PostCSS, etc…) can then use
`toSourceMap()` from `@tailwindcss/node` to convert from the internal
source map to an spec-compliant encoded source map that can be
understood by other tools.
### Handling minification in Lightning
Since we use Lightning CSS for optimization, and it takes in an input
map, we generate an encoded source map that we then pass to lightning.
The output source map *from lighting itself* is then passed back in
during the second optimization pass. The final map is then passed from
lightning to the CLI (but not Vite or PostCSS — see the limitations
section for details).
In some cases we have to "fix up" the output CSS. When this happens we
use `magic-string` to do the replacement in a way that is trackable and
`@amppproject/remapping` to map that change back onto the original
source map. Once the need for these fix ups disappear these dependencies
can go away.
Notes:
- The accuracy of source maps run though lightning is reduced as it only
tracks on a per-rule level. This is sufficient enough for browser dev
tools so should be fine.
- Source maps during optimization do not function properly at this time
because of a bug in Lightning CSS regarding license comments. Once this
bug is fixed they will start working as expected.
### How source locations flow through the system
1. During initial CSS parsing, source locations are preserved.
2. During parsing these source locations are also mapped to the
destinations which supports an optimization for when no utilities are
generated.
3. Throughout the compilation process, transformations maintain source
location data
4. Generated utilities are explicitly pointed to `@tailwind utilities`
unless generated by `@apply`.
5. When optimization is enabled, source maps are remapped through
lightningcss
6. Final source maps are written in the requested format (inline or
separate file)
2025-05-08 16:29:49 -04:00
|
|
|
parseSourceMap(opts: string | SourceMapOptions): SourceMap
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
fs: {
|
2025-02-25 16:07:16 +01:00
|
|
|
write(filePath: string, content: string, encoding?: BufferEncoding): Promise<void>
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
symlink(dst: string, src: string): Promise<void>
|
2024-10-30 10:56:55 -04:00
|
|
|
create(filePaths: string[]): Promise<void>
|
2024-08-07 16:38:44 +02:00
|
|
|
read(filePath: string): Promise<string>
|
Ensure `@tailwindcss/cli` recovers from a deleted transitive dependency (#20137)
This PR fixes an issue where the `@tailwindcss/cli` can get into a
non-recoverable state when any of the transitive dependencies break.
Tailwind CSS has 2 kinds of dependencies:
1. All your templates
2. All dependencies that contribute to your configuration such as the
`input.css`, any plugins, any `tailwind.config.js` files and so on.
When a template changes, we just have to scan for new Tailwind CSS
classes and emit a new CSS file. But when the `input.css` file, or any
of its dependencies changes, then we want to perform a full rebuild.
The idea is that your `@theme` might have changed, or new plugins have
been added, or old plugins have been removed.
If you have an `input.css` file:
```css
@import "tailwindcss";
@config "./tailwind.config.js";
```
That relies on a custom config: `tailwind.config.js`:
```js
const theme = require('./my-custom-theme.js');
module.exports = {
theme
}
```
If that file relies on yet another file: `./my-custom-theme.js`, then
changes there should also trigger a full rebuild.
Since we're dealing with JavaScript here, we want to clear the require
cache and rebuild the dependency tree such that another change to any of
these files triggers a full fresh build.
However, if any of those (transitive) dependencies are deleted, then we
will end up in an invalid state. Creating a new compiler will result in
a build error. The compiler won't be able to figure out the entire
dependency tree, and we're stuck.
Once the user fixes the potentially missing dependency, the watchers
will not be watching any of those files because we created a fresh
compiler.
With this PR, we fix that by keeping track of old paths and using those
while we are still in an invalid state. The moment everything is fixed,
a fresh dependency tree is created and everything starts working again
without you having to restart the `@tailwindcss/cli` command.
Fixes: #20113
Closes: #20114
Closes: #20133
## Test plan
- Added an integration test that removes the transitive dependency.
Re-adding that file later will recover the CLI state.
2026-06-01 19:15:40 +02:00
|
|
|
delete(filePath: string): Promise<void>
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
glob(pattern: string): Promise<[string, string][]>
|
2024-10-10 09:44:04 -04:00
|
|
|
dumpFiles(pattern: string): Promise<string>
|
2024-09-05 14:24:50 +02:00
|
|
|
expectFileToContain(
|
|
|
|
|
filePath: string,
|
2024-11-13 16:42:49 +01:00
|
|
|
contents: string | RegExp | (string | RegExp)[],
|
2024-09-05 14:24:50 +02:00
|
|
|
): Promise<void>
|
Fix `@source` with folders that are ignored (#20214)
This PR fixes an issue where a `@source` that's pointing to a folder
that is git ignored, is also ignored by the `@source` even if it's
explicitly added.
Internally, we convert `@source` directives from `PublicSourceEntry`s to
`SourceEntry`s where we have dedicated enum branches for `Auto`,
`Pattern`, `Ignored` and `External`.
The `Auto` one accepts a `base` path, and will be used for auto content
detection. However, these paths will make use of all the default auto
content detection rules, which includes git ignore rules.
We also have `External` where we link to something that's "external" to
the current repo. We can probably improve this name, but it's external
in the sense that it won't show up on GitHub for example, aka ignored.
We have some content dirs that we ignore by default, such as the
`node_modules` folder. When you do use `@source` with `node_modules` in
the path, then we will mark it as an `external` resource which does not
look at the `gitignore` related rules and allowing it to be included
this way.
The idea with this is that, even though the folder is ignored by
default, you can still include files from the folder by explicitly using
the `@source` directive.
The issue as seen in #19844 is using `vendor/` instead of
`node_modules/` which is _not_ ignored by default. While we can add
`vendor/` to this same ignored dirs list, it will result in a breaking
change because this folder is often used by the Laravel community to
store some resources in.
This PR fixes this problem by not only looking at the content dirs we
ignore by default, but also looking at the actual git ignore state of
this folder. If it turns out that this is ignored, then we promote the
`Auto` source to an `External` source.
Fixes: #19844
Closes: #20057
## Test plan
1. Added integration tests for this situation
2. Ran the fix on the reproduction from #19844. If we run the CLI with
the `DEBUG=*` environment variable, the log file produces these results:
```diff
diff --git a/./tailwindcss-29207.log b/./tailwindcss-30381.log
index bc3017c..921af0e 100644
--- a/./tailwindcss-29207.log
+++ b/./tailwindcss-30381.log
@@ -6,8 +6,9 @@ INFO tailwindcss_oxide::scanner: Source: PublicSourceEntry { base: "/Users/robin
INFO tailwindcss_oxide::scanner: Optimized sources:
INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.phtml" }
INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.xml" }
-INFO tailwindcss_oxide::scanner: Source: Auto { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" }
+INFO tailwindcss_oxide::scanner: Source: External { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" }
INFO tailwindcss_oxide::scanner: Source: Ignored { base: "/Users/robin/.fnm/node-versions/v26.1.0/installation/bin", pattern: "/node" }
INFO discover_sources: tailwindcss_oxide::scanner: enter
-INFO discover_sources: tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml"
+INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml"
+INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme/module/templates/component.phtml"
INFO discover_sources: tailwindcss_oxide::scanner: exit
```
We're checking some `.gitignore` related files, so let's check on each
OS [ci-all]
2026-06-10 18:44:14 +02:00
|
|
|
expectFileNotToContain(filePath: string, contents: string[]): Promise<void>
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
type TestCallback = (context: TestContext) => Promise<void> | void
|
2024-08-08 12:02:19 +02:00
|
|
|
interface TestFlags {
|
|
|
|
|
only?: boolean
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
skip?: boolean
|
2024-08-08 12:02:19 +02:00
|
|
|
debug?: boolean
|
2026-04-30 23:23:36 +02:00
|
|
|
concurrent?: boolean
|
2024-08-08 12:02:19 +02:00
|
|
|
}
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
|
|
|
|
type SpawnActor = { predicate: (message: string) => boolean; resolve: () => void }
|
|
|
|
|
|
2025-04-28 10:53:39 +02:00
|
|
|
export const IS_WINDOWS = platform() === 'win32'
|
2024-08-16 15:45:52 +02:00
|
|
|
|
2026-04-30 23:23:36 +02:00
|
|
|
const execFileAsync = promisify(execFile)
|
|
|
|
|
|
2024-08-16 15:45:52 +02:00
|
|
|
const TEST_TIMEOUT = IS_WINDOWS ? 120000 : 60000
|
|
|
|
|
const ASSERTION_TIMEOUT = IS_WINDOWS ? 10000 : 5000
|
|
|
|
|
|
|
|
|
|
// On Windows CI, tmpdir returns a path containing a weird RUNNER~1 folder that
|
|
|
|
|
// apparently causes the vite builds to not work.
|
2024-09-21 03:59:18 +08:00
|
|
|
const TMP_ROOT =
|
|
|
|
|
process.env.CI && IS_WINDOWS ? path.dirname(process.env.GITHUB_WORKSPACE!) : tmpdir()
|
2024-08-07 16:38:44 +02:00
|
|
|
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
export function test(
|
|
|
|
|
name: string,
|
|
|
|
|
config: TestConfig,
|
|
|
|
|
testCallback: TestCallback,
|
2026-04-30 23:23:36 +02:00
|
|
|
{ only = false, skip = false, debug = false, concurrent = false }: TestFlags = {},
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
) {
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
return defaultTest(
|
2024-08-07 16:38:44 +02:00
|
|
|
name,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
{
|
2025-11-21 00:16:20 +01:00
|
|
|
timeout: config.timeout ?? TEST_TIMEOUT,
|
2026-06-25 19:14:10 +02:00
|
|
|
retry: config.retry ?? (process.env.CI ? 2 : 0),
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
only: only || (!process.env.CI && debug),
|
|
|
|
|
skip,
|
2026-04-30 23:23:36 +02:00
|
|
|
concurrent,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
},
|
2024-08-07 16:38:44 +02:00
|
|
|
async (options) => {
|
2024-08-16 15:45:52 +02:00
|
|
|
let rootDir = debug ? path.join(REPO_ROOT, '.debug') : TMP_ROOT
|
2024-08-08 12:02:19 +02:00
|
|
|
await fs.mkdir(rootDir, { recursive: true })
|
|
|
|
|
|
|
|
|
|
let root = await fs.mkdtemp(path.join(rootDir, 'tailwind-integrations'))
|
|
|
|
|
|
|
|
|
|
if (debug) {
|
|
|
|
|
console.log('Running test in debug mode. File system will be written to:')
|
|
|
|
|
console.log(root)
|
|
|
|
|
console.log()
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
let context = {
|
|
|
|
|
root,
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
expect: options.expect,
|
Add support for source maps (#17775)
Closes #13694
Closes #13591
# Source Maps Support for Tailwind CSS
This PR adds support for source maps to Tailwind CSS v4 allowing us to
track where styles come from whether that be user CSS, imported
stylesheets, or generated utilities. This will improve debuggability in
browser dev tools and gives us a good foundation for producing better
error messages. I'll go over the details on how end users can enable
source maps, any limitations in our implementation, changes to the
internal `compile(…)` API, and some details and reasoning around the
implementation we chose.
## Usage
### CLI
Source maps can be enabled in the CLI by using the command line argument
`--map` which will generate an inline source map comment at the bottom
of your CSS. A separate file may be generated by passing a file name to
`--map`:
```bash
# Generates an inline source map
npx tailwindcss -i input.css -o output.css --map
# Generates a separate source map file
npx tailwindcss -i input.css -o output.css --map output.css.map
```
### PostCSS
Source maps are supported when using Tailwind as a PostCSS plugin *in
development mode only*. They may or may not be enabled by default
depending on your build tool. If they are not you may be able to
configure them within your PostCSS config:
```jsonc
// package.json
{
// …
"postcss": {
"map": { "inline": true },
"plugins": {
"@tailwindcss/postcss": {},
},
}
}
```
### Vite
Source maps are supported when using the Tailwind CSS Vite plugin in
*development mode only* by enabling the `css.devSourcemap` setting:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
devSourcemap: true,
},
})
```
Now when a CSS file is requested by the browser it'll have an inline
source map comment that the browser can use.
## Limitations
- Production build source maps are currently disabled due to a bug in
Lightning CSS. See
https://github.com/parcel-bundler/lightningcss/pull/971 for more
details.
- In Vite, minified CSS build source maps are not supported at all. See
https://github.com/vitejs/vite/issues/2830 for more details.
- In PostCSS, minified CSS source maps are not supported. This is due to
the complexity required around re-associating every AST node with a
location in the generated, optimized CSS. This complexity would also
have a non-trivial performance impact.
## Testing
Here's how to test the source map functionality in different
environments:
### Testing the CLI
1. Setup typical project that the CLI can use and with sources to scan.
```css
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
2. Build with source maps:
```bash
bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map
```
3. Open Chrome DevTools, inspect an element with utility classes, and
you should see rules pointing to `input.css` or
`node_modules/tailwindcss/index.css`
### Testing with Vite
Testing in Vite will require building and installing necessary files
under `dist/*.tgz`.
1. Create a Vite project and enable source maps in `vite.config.js`:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
// This line is required for them to work
devSourcemap: true,
},
})
```
2. Add a component that uses Tailwind classes and custom CSS:
```jsx
// ./src/app.jsx
export default function App() {
return (
<div className="bg-blue-500 my-custom-class">
Hello World
</div>
)
}
```
```css
/* ./src/styles.css */
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
3. Run `npm run dev`, open DevTools, and inspect elements to verify
source mapping works for both utility classes and custom CSS.
### Testing with PostCSS CLI
1. Create a test file and update your PostCSS config:
```css
/* input.css */
@import "tailwindcss";
@layer components {
.card {
@apply p-6 rounded-lg shadow-lg;
}
}
```
```jsonc
// package.json
{
// …
"postcss": {
"map": {
"inline": true
},
"plugins": {
"/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {}
}
}
}
```
2. Run PostCSS through Bun:
```bash
bunx --bun postcss ./src/index.css -o out.css
```
3. Inspect the output CSS - it should include an inline source map
comment at the bottom.
### Testing with PostCSS + Next.js
Testing in Next.js will require building and installing necessary files
under `dist/*.tgz`. However, I've not been able to get CSS source maps
to work in Next.js without this hack:
```js
const nextConfig: NextConfig = {
// next.js overwrites config.devtool so we prevent it from doing so
// please don't actually do this…
webpack: (config) =>
Object.defineProperty(config, "devtool", {
get: () => "inline-source-map",
set: () => {},
}),
};
```
This is definitely not supported and also doesn't work with turbopack.
This can be used to test them temporarily but I suspect that they just
don't work there.
### Manual source map analysis
You can analyze source maps using Evan Wallace's [Source Map
Visualization](https://evanw.github.io/source-map-visualization/) tool
which will help to verify the accuracy and quality of source maps. This
is what I used extensively while developing this implementation.
It'll help verify that custom, user CSS maps back to itself in the
input, that generated utilities all map back to `@tailwind utilities;`,
that source locations from imported files are also handled correctly,
etc… It also highlights the ranges of stuff so it's easy to see if there
are off-by-one errors.
It's easiest to use inline source maps with this tool because you can
take the CSS file and drop it on the page and it'll analyze it while
showing the file content.
If you're using Vite you'll want to access the CSS file with `?direct`
at the end so you don't get a JS module back.
## Implementation
The source map implementation follows the ECMA-426 specification and
includes several key components to aid in that goal:
### Source Location Tracking
Each emittable AST node in the compilation pipeline tracks two types of
source locations:
- `src`: Original source location - [source file, start offset, end
offset]
- `dst`: Generated source location - [output file, start offset, end
offset]
This dual tracking allows us to maintain mappings between the original
source and generated output for things like user CSS, generated
utilities, uses of `@apply`, and tracking theme variables.
It is important to note that source locations for nodes _never overlap_
within a file which helps simplify source map generation. As such each
type of node tracks a specific piece of itself rather than its entire
"block":
| Node | What a `SourceLocation` represents |
| ----------- |
---------------------------------------------------------------- |
| Style Rule | The selector |
| At Rule | Rule name and params, includes the `@` |
| Declaration | Property name and value, excludes the semicolon |
| Comment | The entire comment, includes the start `/*` and end `*/`
markers |
### Windows line endings when parsing CSS
Because our AST tracks nodes through offsets we must ensure that any
mutations to the file do *not* change the lenth of the string. We were
previously replacing `\r\n` with `\n` (see [filter code
points](https://drafts.csswg.org/css-syntax/#css-filter-code-points)
from the spec) — which changes the length of the string and all offsets
may end up incorrect. The CSS parser was updated to handle the CRLF
token directly by skipping over the `\r` and letting remaining code
handle `\n` as it did previously. Some additional tweaks were required
when "peeking" the input but those changes were fairly small.
### Tracking of imports
Source maps need paths to the actual imported stylesheets but the
resolve step for stylesheets happens inside the call to `loadStylesheet`
which make the file path unavailable to us. Because of this the
`loadStylesheet` API was augmented such that it has to return a `path`
property that we can then use to identify imported sources. I've also
made the same change to the `loadModule` API for consistency but nothing
currently uses this property.
The `path` property likely makes `base` redundant but elminating that
(if we even want to) is a future task.
### Optimizing the AST
Our optimization pass may intoduce some nodes, for example, fallbacks we
create for `@property`. These nodes are linked back to `@tailwind
utilities` as ultimately that is what is responsible for creating them.
### Line Offset Tables
A key component to our source map generation is the line offset table,
which was inspired by some ESBuild internals. It stores a sorted list of
offsets for the start of each line allowing us to translate offsets to
line/column `Position`s in `O(log N)` time and from `Position`s to
offsets in `O(1)` time. Creation of the table takes `O(N)` time.
This means that we can store code point offsets for source locations and
not have to worry about computing or tracking line/column numbers during
parsing and serialization. Only when a source map is generated do these
offsets need to be computed. This ensures the performance penalty when
not using source maps is minimal.
### Source Map Generation
The source map returned by `buildSourceMap()` is designed to follow the
[ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not
completely finalized we consider the result of `buildSourceMap()` to be
internal API that may change as the spec chamges.
The produces source map is a "decoded" map such that all sources and
mappings are in an object graph. A library like `source-map-js` must be
used to convert this to an encoded source map of the right version where
mappings are encoded with base 64 VLQs.
Any specific integration (Vite, PostCSS, etc…) can then use
`toSourceMap()` from `@tailwindcss/node` to convert from the internal
source map to an spec-compliant encoded source map that can be
understood by other tools.
### Handling minification in Lightning
Since we use Lightning CSS for optimization, and it takes in an input
map, we generate an encoded source map that we then pass to lightning.
The output source map *from lighting itself* is then passed back in
during the second optimization pass. The final map is then passed from
lightning to the CLI (but not Vite or PostCSS — see the limitations
section for details).
In some cases we have to "fix up" the output CSS. When this happens we
use `magic-string` to do the replacement in a way that is trackable and
`@amppproject/remapping` to map that change back onto the original
source map. Once the need for these fix ups disappear these dependencies
can go away.
Notes:
- The accuracy of source maps run though lightning is reduced as it only
tracks on a per-rule level. This is sufficient enough for browser dev
tools so should be fine.
- Source maps during optimization do not function properly at this time
because of a bug in Lightning CSS regarding license comments. Once this
bug is fixed they will start working as expected.
### How source locations flow through the system
1. During initial CSS parsing, source locations are preserved.
2. During parsing these source locations are also mapped to the
destinations which supports an optimization for when no utilities are
generated.
3. Throughout the compilation process, transformations maintain source
location data
4. Generated utilities are explicitly pointed to `@tailwind utilities`
unless generated by `@apply`.
5. When optimization is enabled, source maps are remapped through
lightningcss
6. Final source maps are written in the requested format (inline or
separate file)
2025-05-08 16:29:49 -04:00
|
|
|
parseSourceMap,
|
2024-09-11 17:02:24 +02:00
|
|
|
async exec(
|
|
|
|
|
command: string,
|
|
|
|
|
childProcessOptions: ChildProcessOptions = {},
|
|
|
|
|
execOptions: ExecOptions = {},
|
|
|
|
|
) {
|
2024-08-08 12:02:19 +02:00
|
|
|
let cwd = childProcessOptions.cwd ?? root
|
2026-06-25 19:14:10 +02:00
|
|
|
let originalCommand = command
|
|
|
|
|
|
|
|
|
|
// Avoid `pnpm exec upgrade` on Windows because the upgrader runs
|
|
|
|
|
// `pnpm add`, which rewrites Windows `.cmd` shims while the current
|
|
|
|
|
// shim is still executing.
|
|
|
|
|
//
|
|
|
|
|
// Pretty sure this is a pnpm v11 bug on Windows.
|
|
|
|
|
if (IS_WINDOWS && command.includes('pnpm exec upgrade')) {
|
|
|
|
|
for (let base = cwd; ; base = path.dirname(base)) {
|
|
|
|
|
let upgradeBin = path.join(
|
|
|
|
|
base,
|
|
|
|
|
'node_modules',
|
|
|
|
|
'@tailwindcss',
|
|
|
|
|
'upgrade',
|
|
|
|
|
'dist',
|
|
|
|
|
'index.mjs',
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
try {
|
|
|
|
|
await fs.access(upgradeBin)
|
|
|
|
|
command = command.replace('pnpm exec upgrade', `node "${upgradeBin}"`)
|
|
|
|
|
break
|
|
|
|
|
} catch (error: any) {
|
|
|
|
|
if (error?.code !== 'ENOENT') throw error
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let parent = path.dirname(base)
|
|
|
|
|
if (parent === base) {
|
|
|
|
|
throw new Error(`Unable to resolve @tailwindcss/upgrade from ${cwd}`)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-08 12:02:19 +02:00
|
|
|
if (debug && cwd !== root) {
|
|
|
|
|
let relative = path.relative(root, cwd)
|
|
|
|
|
if (relative[0] !== '.') relative = `./${relative}`
|
|
|
|
|
console.log(`> cd ${relative}`)
|
|
|
|
|
}
|
|
|
|
|
if (debug) console.log(`> ${command}`)
|
2024-09-04 10:09:24 +02:00
|
|
|
return new Promise((resolve, reject) => {
|
2025-01-17 16:23:34 +01:00
|
|
|
let child = exec(
|
2024-09-04 10:09:24 +02:00
|
|
|
command,
|
|
|
|
|
{
|
|
|
|
|
cwd,
|
|
|
|
|
...childProcessOptions,
|
2025-05-09 14:55:02 +02:00
|
|
|
env: {
|
|
|
|
|
...process.env,
|
|
|
|
|
...childProcessOptions.env,
|
|
|
|
|
},
|
2024-09-04 10:09:24 +02:00
|
|
|
},
|
|
|
|
|
(error, stdout, stderr) => {
|
|
|
|
|
if (error) {
|
2026-06-25 19:14:10 +02:00
|
|
|
if (command !== originalCommand) {
|
|
|
|
|
error.message = error.message.replace(command, originalCommand)
|
|
|
|
|
let mappedError = error as any
|
|
|
|
|
mappedError.cmd = originalCommand
|
|
|
|
|
}
|
2024-09-11 17:02:24 +02:00
|
|
|
if (execOptions.ignoreStdErr !== true) console.error(stderr)
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
if (only || debug) {
|
|
|
|
|
console.error(stdout)
|
|
|
|
|
}
|
2024-09-04 10:09:24 +02:00
|
|
|
reject(error)
|
|
|
|
|
} else {
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
if (only || debug) {
|
|
|
|
|
console.log(stdout.toString() + '\n\n' + stderr.toString())
|
|
|
|
|
}
|
2024-10-10 09:44:04 -04:00
|
|
|
resolve(stdout.toString() + '\n\n' + stderr.toString())
|
2024-09-04 10:09:24 +02:00
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
)
|
2025-01-17 16:23:34 +01:00
|
|
|
if (execOptions.stdin) {
|
|
|
|
|
child.stdin?.write(execOptions.stdin)
|
|
|
|
|
child.stdin?.end()
|
|
|
|
|
}
|
2024-09-04 10:09:24 +02:00
|
|
|
})
|
2024-08-07 16:38:44 +02:00
|
|
|
},
|
|
|
|
|
async spawn(command: string, childProcessOptions: ChildProcessOptions = {}) {
|
|
|
|
|
let resolveDisposal: (() => void) | undefined
|
|
|
|
|
let rejectDisposal: ((error: Error) => void) | undefined
|
|
|
|
|
let disposePromise = new Promise<void>((resolve, reject) => {
|
|
|
|
|
resolveDisposal = resolve
|
|
|
|
|
rejectDisposal = reject
|
|
|
|
|
})
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
2024-08-08 12:02:19 +02:00
|
|
|
let cwd = childProcessOptions.cwd ?? root
|
|
|
|
|
if (debug && cwd !== root) {
|
|
|
|
|
let relative = path.relative(root, cwd)
|
|
|
|
|
if (relative[0] !== '.') relative = `./${relative}`
|
|
|
|
|
console.log(`> cd ${relative}`)
|
|
|
|
|
}
|
|
|
|
|
if (debug) console.log(`>& ${command}`)
|
2024-08-07 16:38:44 +02:00
|
|
|
let child = spawn(command, {
|
2024-08-08 12:02:19 +02:00
|
|
|
cwd,
|
2026-04-30 23:23:36 +02:00
|
|
|
detached: !IS_WINDOWS,
|
2024-08-07 16:38:44 +02:00
|
|
|
shell: true,
|
2024-11-07 21:31:06 +01:00
|
|
|
...childProcessOptions,
|
2024-08-07 16:38:44 +02:00
|
|
|
env: {
|
|
|
|
|
...process.env,
|
2024-11-07 21:31:06 +01:00
|
|
|
...childProcessOptions.env,
|
2024-08-07 16:38:44 +02:00
|
|
|
},
|
|
|
|
|
})
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
2026-04-30 23:23:36 +02:00
|
|
|
let disposed = false
|
|
|
|
|
|
|
|
|
|
async function dispose() {
|
|
|
|
|
if (disposed) return disposePromise
|
|
|
|
|
disposed = true
|
|
|
|
|
|
|
|
|
|
await killProcessTree(child)
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
2026-04-30 23:23:36 +02:00
|
|
|
let timer = setTimeout(() => {
|
|
|
|
|
forceKillProcessTree(child)
|
|
|
|
|
rejectDisposal?.(new Error(`spawned process (${command}) did not exit in time`))
|
|
|
|
|
}, ASSERTION_TIMEOUT)
|
|
|
|
|
disposePromise.then(
|
|
|
|
|
() => clearTimeout(timer),
|
|
|
|
|
() => clearTimeout(timer),
|
2024-08-07 16:38:44 +02:00
|
|
|
)
|
|
|
|
|
return disposePromise
|
|
|
|
|
}
|
|
|
|
|
disposables.push(dispose)
|
Make upgrade tooling more stable (#19846)
This PR is an attempt to make the upgrade tooling more stable.
### TL;DR
1. When migrating from Tailwind CSS v3 → Tailwind CSS v4, only migrate
files listed in the `config.content` instead of relying on v4's auto
content detection feature
2. Skip writing files that have not been changed
3. Write changed files in a safe way: first write to a temporary file,
then rename the file atomically
4. Never migrate files that are git ignored, even if they are listed in
the `config.content` file
5. Always ignore `.env` and `.env.*` files when scanning for files. Most
people will have this in their `.gitignore` file, but if not, then this
is a fallback mechanism.
---
Looking at the #18972 issue, it looks like some people are running into
weird situations where some of the contents is just gone.
I have never been able to reproduce this on my own devices and in my own
projects unfortunately. But there is definitely _something_ happening
that's not right that people are running into.
Therefore, this PR is an attempt to fix what I think _might_ be wrong,
but I'm not 100% sure if these fixes are enough, or if something else is
still happening here.
This builds on top of the #19779 PR which has some small fixes, but is
incomplete to make this work.
### What's happening
Looking at some of the comments, it looks like a few things are
happening such as:
1. The upgrade tool is emptying out my files — it looks like these are
only happening if you ctrl+c while the process is taking a while. It
could be that a lot of files are being checked and therefore the tooling
looks like its stuck.
6. The upgrade tool is looking at files it shouldn't look at — in
Tailwind CSS v4 we have this concept of the auto-content detection. This
means that we will look at any plain text file that is not git ignored.
### Fixes
#### Emptying out files
The files being emptied looks like it's because how `fs.writeFile`
behaves by default. It opens the file handle with the `w` flag, which
will first truncate the file before writing the new contents. This is
not a single atomic operation, so a killed process in the middle will
cause invalid state.
When we migrate your template files, everything is happening in promises
to migrate things at the same time. When a lot of files are being
scanned, truncating might have happened already before we write the new
content. Since we migrate a bunch of files in parallel, a ctrl+c could
cause data loss in multiple files.
To mitigate this, I switched to an alternative way of writing files.
1. First, we do some quick checks where if the contents didn't change we
just bail out immediately. Files that don't include Tailwind CSS classes
won't change, and therefore we don't need to override these files with
the same contents.
2. When the migrated contents is empty, we bail out as well. I'm 100%
sure that this is not the spot where the "emptying out" happens, I still
believe it happens in the `writeFile` itself, but added it just in case.
7. Next, I introduced a safe write, where we first write to a temporary
file in the same folder. We could write it to `/tmp`, but then we can't
guarantee that we are on the same file system.
If we ctrl+c at this stage, then the worst case scenario is that you
have additional temporary files in your project, but your original files
are still there.
Once that file was written, we will use the atomic `fs.rename`. This
should be atomic as long as we are on the same file system, so either
the rename didn't happen yet, or it completed.
I added an integration test for this, but I had to change the
`writeFile` implementation slightly. In the test, we will truncate the
file first, after that we will write the new contents. This is so that
we have enough time to kill the current process and allows us to verify
that we didn't clear out the file. Again, this is a hacky way of testing
this, just because I can't reproduce this issue myself, let alone
reproduce it reliable in a CI environment.
Note: we are also using `realpath` to make sure that we are updating the
real file. Otherwise, if we were dealing with a symlinked file, we would
override the symlink with a "hard" copy instead.
#### Touching files that should not be touched
During the migration, we rely on the Tailwind CSS v4 auto detection
logic which means that it will scan any plain text file that is not git
ignored. Therefore changes to php files could happen because in theory
they could contain Tailwind CSS classes.
To solve this, when migrating from Tailwind CSS v3 to Tailwind CSS v4,
we will _only_ take the sources into account that were listed in the
`config.content` array. Since this was a requirement in Tailwind CSS v3,
it should be safe to rely on this array.
Additionally, this will make sure that we are dealing with way fewer
files to migrate as well.
On top of that, files that match the patterns in the content array that
are git ignored will also be skipped. This is to prevent that we mutate
files in `node_modules` for example.
In one of the comments I read that `.env` files were emptied out. In
most cases people will have these files gitignored but I explicitly
added `.env` and `.env.*` as files to never ever touch by default when
scanning.
Last but not least, this also updates the output a little bit of the
upgrade tool in case we skip content files (because of git ignore) and
if we changed a file.
<img width="1122" height="1376" alt="image"
src="https://github.com/user-attachments/assets/318fdbbf-e319-4c7e-9648-ee9283842624"
/>
- "Git ignored folder, skipping: `./node_modules`": this is because the
content array looks like this while the `node_modules` are being
ignored:
<img width="1090" height="398" alt="image"
src="https://github.com/user-attachments/assets/7d694720-5671-47ec-bb2e-f24c5f2c4248"
/>
- "Migrated
`./resources/views/vendor/filament-panels/components/logo.blade.php`":
this is because **I** made a change to showcase this feature.
Fixes: #18972
Closes: #19779
### Test plan
1. Existing tests still pass
1. Added a dedicated integration test to ensure that we only take
`config.content` into account when migrating from Tailwind CSS v3 to
Tailwind CSS v4 projects.
1. Added a dedicated integration test to make sure that files listed in
`config.content` that are also git ignored, will still be skipped.
1. Added a dedicated integration test to ensure that when `writeFile` is
cancelled mid-write that our old files are still present.
1. Added a dedicated integration test to ensure that we ignore `.env`
and `.env.*` files even if you didn't git ignore them.
[ci-all] To verify on Windows
---------
Co-authored-by: Sami <sychocouldy@gmail.com>
2026-03-24 17:24:12 +01:00
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
function onExit() {
|
|
|
|
|
resolveDisposal?.()
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let stdoutMessages: string[] = []
|
|
|
|
|
let stderrMessages: string[] = []
|
|
|
|
|
|
|
|
|
|
let stdoutActors: SpawnActor[] = []
|
|
|
|
|
let stderrActors: SpawnActor[] = []
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
function notifyNext(actors: SpawnActor[], messages: string[]) {
|
|
|
|
|
if (actors.length <= 0) return
|
|
|
|
|
let [next] = actors
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
for (let [idx, message] of messages.entries()) {
|
|
|
|
|
if (next.predicate(message)) {
|
|
|
|
|
messages.splice(0, idx + 1)
|
|
|
|
|
let actorIdx = actors.indexOf(next)
|
|
|
|
|
actors.splice(actorIdx, 1)
|
|
|
|
|
next.resolve()
|
|
|
|
|
break
|
|
|
|
|
}
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
let combined: ['stdout' | 'stderr', string][] = []
|
|
|
|
|
|
|
|
|
|
child.stdout.on('data', (result) => {
|
|
|
|
|
let content = result.toString()
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
if (debug || only) console.log(content)
|
2024-08-07 16:38:44 +02:00
|
|
|
combined.push(['stdout', content])
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
for (let line of content.split('\n')) {
|
|
|
|
|
stdoutMessages.push(stripVTControlCharacters(line))
|
|
|
|
|
}
|
2024-08-07 16:38:44 +02:00
|
|
|
notifyNext(stdoutActors, stdoutMessages)
|
|
|
|
|
})
|
|
|
|
|
child.stderr.on('data', (result) => {
|
|
|
|
|
let content = result.toString()
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
if (debug || only) console.error(content)
|
2024-08-07 16:38:44 +02:00
|
|
|
combined.push(['stderr', content])
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
for (let line of content.split('\n')) {
|
|
|
|
|
stderrMessages.push(stripVTControlCharacters(line))
|
|
|
|
|
}
|
2024-08-07 16:38:44 +02:00
|
|
|
notifyNext(stderrActors, stderrMessages)
|
|
|
|
|
})
|
|
|
|
|
child.on('exit', onExit)
|
|
|
|
|
child.on('error', (error) => {
|
|
|
|
|
if (error.name !== 'AbortError') {
|
|
|
|
|
throw error
|
|
|
|
|
}
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
options.onTestFailed(() => {
|
2024-11-04 13:07:25 -05:00
|
|
|
// In only or debug mode, messages are logged to the console
|
|
|
|
|
// immediately.
|
|
|
|
|
if (only || debug) return
|
2024-08-08 12:02:19 +02:00
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
for (let [type, message] of combined) {
|
|
|
|
|
if (type === 'stdout') {
|
|
|
|
|
console.log(message)
|
|
|
|
|
} else {
|
|
|
|
|
console.error(message)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
return {
|
|
|
|
|
dispose,
|
2025-04-03 17:13:15 +02:00
|
|
|
flush() {
|
|
|
|
|
stdoutActors.splice(0)
|
|
|
|
|
stderrActors.splice(0)
|
|
|
|
|
|
|
|
|
|
stdoutMessages.splice(0)
|
|
|
|
|
stderrMessages.splice(0)
|
|
|
|
|
},
|
2024-08-07 16:38:44 +02:00
|
|
|
onStdout(predicate: (message: string) => boolean) {
|
|
|
|
|
return new Promise<void>((resolve) => {
|
|
|
|
|
stdoutActors.push({ predicate, resolve })
|
|
|
|
|
notifyNext(stdoutActors, stdoutMessages)
|
|
|
|
|
})
|
|
|
|
|
},
|
|
|
|
|
onStderr(predicate: (message: string) => boolean) {
|
|
|
|
|
return new Promise<void>((resolve) => {
|
|
|
|
|
stderrActors.push({ predicate, resolve })
|
|
|
|
|
notifyNext(stderrActors, stderrMessages)
|
|
|
|
|
})
|
|
|
|
|
},
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
2024-08-07 16:38:44 +02:00
|
|
|
},
|
|
|
|
|
fs: {
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
async write(filename, content, encoding = 'utf8') {
|
2024-08-08 12:02:19 +02:00
|
|
|
let full = path.join(root, filename)
|
2024-11-07 09:51:58 -05:00
|
|
|
let dir = path.dirname(full)
|
|
|
|
|
await fs.mkdir(dir, { recursive: true })
|
|
|
|
|
|
|
|
|
|
if (typeof content !== 'string') {
|
|
|
|
|
return await fs.writeFile(full, content)
|
|
|
|
|
}
|
2024-08-08 12:02:19 +02:00
|
|
|
|
|
|
|
|
if (filename.endsWith('package.json')) {
|
|
|
|
|
content = await overwriteVersionsInPackageJson(content)
|
2026-06-25 19:14:10 +02:00
|
|
|
} else if (filename.endsWith('pnpm-workspace.yaml')) {
|
|
|
|
|
content = overwriteVersionsInPnpmWorkspace(content)
|
2024-08-08 12:02:19 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Ensure that files written on Windows use \r\n line ending
|
2024-08-16 15:45:52 +02:00
|
|
|
if (IS_WINDOWS) {
|
2024-08-08 12:02:19 +02:00
|
|
|
content = content.replace(/\n/g, '\r\n')
|
|
|
|
|
}
|
|
|
|
|
|
2025-02-25 16:07:16 +01:00
|
|
|
await fs.writeFile(full, content, encoding)
|
2024-08-08 12:02:19 +02:00
|
|
|
},
|
2024-10-30 10:56:55 -04:00
|
|
|
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
async symlink(target, src) {
|
|
|
|
|
let targetAbsolute = path.join(root, target)
|
|
|
|
|
let targetParent = path.dirname(targetAbsolute)
|
|
|
|
|
await fs.mkdir(targetParent, { recursive: true })
|
|
|
|
|
|
|
|
|
|
let srcAbsolute = path.join(root, src)
|
|
|
|
|
let srcParent = path.dirname(srcAbsolute)
|
|
|
|
|
await fs.mkdir(srcParent, { recursive: true })
|
|
|
|
|
|
|
|
|
|
await fs.symlink(targetAbsolute, srcAbsolute)
|
|
|
|
|
},
|
|
|
|
|
|
|
|
|
|
async create(filenames) {
|
2024-10-30 10:56:55 -04:00
|
|
|
for (let filename of filenames) {
|
|
|
|
|
let full = path.join(root, filename)
|
|
|
|
|
|
|
|
|
|
let dir = path.dirname(full)
|
|
|
|
|
await fs.mkdir(dir, { recursive: true })
|
|
|
|
|
await fs.writeFile(full, '')
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
async delete(filename) {
|
Ensure `@tailwindcss/cli` recovers from a deleted transitive dependency (#20137)
This PR fixes an issue where the `@tailwindcss/cli` can get into a
non-recoverable state when any of the transitive dependencies break.
Tailwind CSS has 2 kinds of dependencies:
1. All your templates
2. All dependencies that contribute to your configuration such as the
`input.css`, any plugins, any `tailwind.config.js` files and so on.
When a template changes, we just have to scan for new Tailwind CSS
classes and emit a new CSS file. But when the `input.css` file, or any
of its dependencies changes, then we want to perform a full rebuild.
The idea is that your `@theme` might have changed, or new plugins have
been added, or old plugins have been removed.
If you have an `input.css` file:
```css
@import "tailwindcss";
@config "./tailwind.config.js";
```
That relies on a custom config: `tailwind.config.js`:
```js
const theme = require('./my-custom-theme.js');
module.exports = {
theme
}
```
If that file relies on yet another file: `./my-custom-theme.js`, then
changes there should also trigger a full rebuild.
Since we're dealing with JavaScript here, we want to clear the require
cache and rebuild the dependency tree such that another change to any of
these files triggers a full fresh build.
However, if any of those (transitive) dependencies are deleted, then we
will end up in an invalid state. Creating a new compiler will result in
a build error. The compiler won't be able to figure out the entire
dependency tree, and we're stuck.
Once the user fixes the potentially missing dependency, the watchers
will not be watching any of those files because we created a fresh
compiler.
With this PR, we fix that by keeping track of old paths and using those
while we are still in an invalid state. The moment everything is fixed,
a fresh dependency tree is created and everything starts working again
without you having to restart the `@tailwindcss/cli` command.
Fixes: #20113
Closes: #20114
Closes: #20133
## Test plan
- Added an integration test that removes the transitive dependency.
Re-adding that file later will recover the CLI state.
2026-06-01 19:15:40 +02:00
|
|
|
await fs.unlink(path.join(root, filename))
|
|
|
|
|
},
|
|
|
|
|
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
async read(filePath) {
|
Vite: Retain candidates between input CSS updates (#14228)
This PR fixes an issue introduced with the changed candidate cache
behavior in #14187.
Prior to #14187, candidates were cached globally within an instance of
Oxide. This meant that once a candidate was discovered, it would not
reset until you either manually cleared the cache or restarted the Oxide
process. With the changes in #14187 however, the cache was scoped to the
instance of the `Scanner` class with the intention of making the caching
behavior more easy to understand and to avoid a global cache.
This, however, had an unforeseen side-effect in our Vite extension.
Vite, in dev mode, discovers files _lazily_. So when a developer goes to
`/index.html` the first time, we will scan the `/index.html` file for
Tailwind candidates and then build a CSS file with those candidate. When
they go to `/about.html` later, we will _append_ the candidates from the
new file and so forth.
The problem now arises when the dev server detects changes to the input
CSS file. This requires us to do a re-scan of that CSS file which, after
#14187, caused the candidate cache to be gone. This is usually fine
since we would just scan files again for the changed candidate list but
in the Vite case we would only get the input CSS file change _but no
subsequent change events for all other files, including those currently
rendered in the browser_). This caused updates to the CSS file to remove
all candidates from the CSS file again.
Ideally, we can separate between two concepts: The candidate cache and
the CSS input file scan. An instance of the `Scanner` could re-parse the
input CSS file without having to throw away previous candidates. This,
however, would have another issue with the current Vite extension where
we do not properly retain instances of the `Scanner` class anyways. To
properly improve the cache behavior, we will have to fix the Vite
`Scanner` retaining behavior first. Unfortunately this means that for
the short term, we have to add some manual bookkeeping to the Vite
client and retain the candidate cache between builds ourselves.
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-08-21 12:54:42 +02:00
|
|
|
let content = await fs.readFile(path.resolve(root, filePath), 'utf8')
|
|
|
|
|
|
|
|
|
|
// Ensure that files read on Windows have \r\n line endings removed
|
|
|
|
|
if (IS_WINDOWS) {
|
|
|
|
|
content = content.replace(/\r\n/g, '\n')
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return content
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
},
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
async glob(pattern) {
|
2024-08-07 16:38:44 +02:00
|
|
|
let files = await fastGlob(pattern, { cwd: root })
|
|
|
|
|
return Promise.all(
|
|
|
|
|
files.map(async (file) => {
|
|
|
|
|
let content = await fs.readFile(path.join(root, file), 'utf8')
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
return [
|
|
|
|
|
file,
|
|
|
|
|
// Drop license comment
|
2025-03-31 18:44:06 +02:00
|
|
|
content.replace(/[\s\n]*\/\*![\s\S]*?\*\/[\s\n]*/g, ''),
|
Auto source detection improvements (#14820)
This PR introduces a new `source(…)` argument and improves on the
existing `@source`. The goal of this PR is to make the automatic source
detection configurable, let's dig in.
By default, we will perform automatic source detection starting at the
current working directory. Auto source detection will find plain text
files (no binaries, images, ...) and will ignore git-ignored files.
If you want to start from a different directory, you can use the new
`source(…)` next to the `@import "tailwindcss/utilities"
layer(utilities) source(…)`.
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss/utilities' layer(utilities) source('../../');
```
Most people won't split their source files, and will just use the simple
`@import "tailwindcss";`, because of this reason, you can use
`source(…)` on the import as well:
E.g.:
```css
/* ./src/styles/index.css */
@import 'tailwindcss' source('../../');
```
Sometimes, you want to rely on auto source detection, but also want to
look in another directory for source files. In this case, yuo can use
the `@source` directive:
```css
/* ./src/index.css */
@import 'tailwindcss';
/* Look for `blade.php` files in `../resources/views` */
@source '../resources/views/**/*.blade.php';
```
However, you don't need to specify the extension, instead you can just
point the directory and all the same automatic source detection rules
will apply.
```css
/* ./src/index.css */
@import 'tailwindcss';
@source '../resources/views';
```
If, for whatever reason, you want to disable the default source
detection feature entirely, and only want to rely on very specific glob
patterns you define, then you can disable it via `source(none)`.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Only look at .blade.php files, nothing else */
@source "../resources/views/**/*.blade.php";
```
Note: even with `source(none)`, if your `@source` points to a directory,
then auto source detection will still be performed in that directory. If
you don't want that, then you can simply add explicit files in the globs
as seen in the previous example.
```css
/* Completely disable the default auto source detection */
@import 'tailwindcss' source(none);
/* Run auto source detection in `../resources/views` */
@source "../resources/views";
```
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
Co-authored-by: Adam Wathan <4323180+adamwathan@users.noreply.github.com>
2024-10-29 21:33:34 +01:00
|
|
|
]
|
2024-08-07 16:38:44 +02:00
|
|
|
}),
|
|
|
|
|
)
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
},
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
async dumpFiles(pattern) {
|
2024-10-10 09:44:04 -04:00
|
|
|
let files = await context.fs.glob(pattern)
|
|
|
|
|
return `\n${files
|
|
|
|
|
.slice()
|
|
|
|
|
.sort((a: [string], z: [string]) => {
|
|
|
|
|
let aParts = a[0].split('/')
|
|
|
|
|
let zParts = z[0].split('/')
|
|
|
|
|
|
|
|
|
|
let aFile = aParts.at(-1)
|
Improve multi-root `@config` linking (#15001)
This PR improves the discoverability of Tailwind config files when we
are trying to link them to your CSS files.
When you have multiple "root" CSS files in your project, and if they
don't include an `@config` directive, then we tried to find the Tailwind
config file in your current working directory.
This means that if you run the upgrade command from the root of your
project, and you have a nested folder with a separate Tailwind setup,
then the nested CSS file would link to the root Tailwind config file.
Visually, you can think of it like this:
```
.
├── admin
│ ├── src
│ │ └── styles
│ │ └── index.css <-- This will be linked to (1)
│ └── tailwind.config.js (2)
├── src
│ └── styles
│ └── index.css <-- This will be linked to (1)
└── tailwind.config.js (1)
```
If you run the upgrade command from the root of your project, then the
`/src/styles/index.css` will be linked to `/tailwind.config.js` which is
what we expect.
But `/admin/src/styles/index.css` will _also_ be linked to
`/tailwind.config.js`
With this PR we improve this behavior by looking at the CSS file, and
crawling up the parent tree. This mens that the new behavior looks like
this:
```
.
├── admin
│ ├── src
│ │ └── styles
│ │ └── index.css <-- This will be linked to (2)
│ └── tailwind.config.js (2)
├── src
│ └── styles
│ └── index.css <-- This will be linked to (1)
└── tailwind.config.js (1)
```
Now `/src/styles/index.css` will be linked to `/tailwind.config.js`, and
`/admin/src/styles/index.css` will be linked to
`/admin/tailwind.config.js`.
When we discover the Tailwind config file, we will also print a message
to the user to let them know which CSS file is linked to which Tailwind
config file.
This should be a safe improvement because if your Tailwind config file
had a different name, or if it lived in a sibling folder then Tailwind
wouldn't find it either and you already required a `@config "…";`
directive in your CSS file to point to the correct file.
In the unlikely event that it turns out that 2 (or more) CSS files
resolve to the same to the same Tailwind config file, then an upgrade
might not be safe and some manual intervention might be needed. In this
case, we will show a warning about this.
<img width="1552" alt="image"
src="https://github.com/user-attachments/assets/7a1ad11d-18c5-4b7d-9a02-14f0116ae955">
Test plan:
---
- Added an integration test that properly links the nearest Tailwind
config file by looking up the tree
- Added an integration test that resolves 2 or more CSS files to the
same config file, resulting in an error where manual intervention is
needed
- Ran it on the Tailwind UI codebase
Running this on Tailwind UI's codebase it looks like this:
<img width="1552" alt="image"
src="https://github.com/user-attachments/assets/21785428-5e0d-47f7-80ec-dab497f58784">
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-11-18 17:43:44 +01:00
|
|
|
let zFile = zParts.at(-1)
|
2024-10-10 09:44:04 -04:00
|
|
|
|
|
|
|
|
// Sort by depth, shallow first
|
|
|
|
|
if (aParts.length < zParts.length) return -1
|
|
|
|
|
if (aParts.length > zParts.length) return 1
|
|
|
|
|
|
Improve multi-root `@config` linking (#15001)
This PR improves the discoverability of Tailwind config files when we
are trying to link them to your CSS files.
When you have multiple "root" CSS files in your project, and if they
don't include an `@config` directive, then we tried to find the Tailwind
config file in your current working directory.
This means that if you run the upgrade command from the root of your
project, and you have a nested folder with a separate Tailwind setup,
then the nested CSS file would link to the root Tailwind config file.
Visually, you can think of it like this:
```
.
├── admin
│ ├── src
│ │ └── styles
│ │ └── index.css <-- This will be linked to (1)
│ └── tailwind.config.js (2)
├── src
│ └── styles
│ └── index.css <-- This will be linked to (1)
└── tailwind.config.js (1)
```
If you run the upgrade command from the root of your project, then the
`/src/styles/index.css` will be linked to `/tailwind.config.js` which is
what we expect.
But `/admin/src/styles/index.css` will _also_ be linked to
`/tailwind.config.js`
With this PR we improve this behavior by looking at the CSS file, and
crawling up the parent tree. This mens that the new behavior looks like
this:
```
.
├── admin
│ ├── src
│ │ └── styles
│ │ └── index.css <-- This will be linked to (2)
│ └── tailwind.config.js (2)
├── src
│ └── styles
│ └── index.css <-- This will be linked to (1)
└── tailwind.config.js (1)
```
Now `/src/styles/index.css` will be linked to `/tailwind.config.js`, and
`/admin/src/styles/index.css` will be linked to
`/admin/tailwind.config.js`.
When we discover the Tailwind config file, we will also print a message
to the user to let them know which CSS file is linked to which Tailwind
config file.
This should be a safe improvement because if your Tailwind config file
had a different name, or if it lived in a sibling folder then Tailwind
wouldn't find it either and you already required a `@config "…";`
directive in your CSS file to point to the correct file.
In the unlikely event that it turns out that 2 (or more) CSS files
resolve to the same to the same Tailwind config file, then an upgrade
might not be safe and some manual intervention might be needed. In this
case, we will show a warning about this.
<img width="1552" alt="image"
src="https://github.com/user-attachments/assets/7a1ad11d-18c5-4b7d-9a02-14f0116ae955">
Test plan:
---
- Added an integration test that properly links the nearest Tailwind
config file by looking up the tree
- Added an integration test that resolves 2 or more CSS files to the
same config file, resulting in an error where manual intervention is
needed
- Ran it on the Tailwind UI codebase
Running this on Tailwind UI's codebase it looks like this:
<img width="1552" alt="image"
src="https://github.com/user-attachments/assets/21785428-5e0d-47f7-80ec-dab497f58784">
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-11-18 17:43:44 +01:00
|
|
|
// Sort by folder names, alphabetically
|
|
|
|
|
for (let i = 0; i < aParts.length - 1; i++) {
|
|
|
|
|
let diff = aParts[i].localeCompare(zParts[i])
|
|
|
|
|
if (diff !== 0) return diff
|
|
|
|
|
}
|
|
|
|
|
|
2024-10-10 09:44:04 -04:00
|
|
|
// Sort by filename, sort files named `index` before others
|
Improve multi-root `@config` linking (#15001)
This PR improves the discoverability of Tailwind config files when we
are trying to link them to your CSS files.
When you have multiple "root" CSS files in your project, and if they
don't include an `@config` directive, then we tried to find the Tailwind
config file in your current working directory.
This means that if you run the upgrade command from the root of your
project, and you have a nested folder with a separate Tailwind setup,
then the nested CSS file would link to the root Tailwind config file.
Visually, you can think of it like this:
```
.
├── admin
│ ├── src
│ │ └── styles
│ │ └── index.css <-- This will be linked to (1)
│ └── tailwind.config.js (2)
├── src
│ └── styles
│ └── index.css <-- This will be linked to (1)
└── tailwind.config.js (1)
```
If you run the upgrade command from the root of your project, then the
`/src/styles/index.css` will be linked to `/tailwind.config.js` which is
what we expect.
But `/admin/src/styles/index.css` will _also_ be linked to
`/tailwind.config.js`
With this PR we improve this behavior by looking at the CSS file, and
crawling up the parent tree. This mens that the new behavior looks like
this:
```
.
├── admin
│ ├── src
│ │ └── styles
│ │ └── index.css <-- This will be linked to (2)
│ └── tailwind.config.js (2)
├── src
│ └── styles
│ └── index.css <-- This will be linked to (1)
└── tailwind.config.js (1)
```
Now `/src/styles/index.css` will be linked to `/tailwind.config.js`, and
`/admin/src/styles/index.css` will be linked to
`/admin/tailwind.config.js`.
When we discover the Tailwind config file, we will also print a message
to the user to let them know which CSS file is linked to which Tailwind
config file.
This should be a safe improvement because if your Tailwind config file
had a different name, or if it lived in a sibling folder then Tailwind
wouldn't find it either and you already required a `@config "…";`
directive in your CSS file to point to the correct file.
In the unlikely event that it turns out that 2 (or more) CSS files
resolve to the same to the same Tailwind config file, then an upgrade
might not be safe and some manual intervention might be needed. In this
case, we will show a warning about this.
<img width="1552" alt="image"
src="https://github.com/user-attachments/assets/7a1ad11d-18c5-4b7d-9a02-14f0116ae955">
Test plan:
---
- Added an integration test that properly links the nearest Tailwind
config file by looking up the tree
- Added an integration test that resolves 2 or more CSS files to the
same config file, resulting in an error where manual intervention is
needed
- Ran it on the Tailwind UI codebase
Running this on Tailwind UI's codebase it looks like this:
<img width="1552" alt="image"
src="https://github.com/user-attachments/assets/21785428-5e0d-47f7-80ec-dab497f58784">
---------
Co-authored-by: Jordan Pittman <jordan@cryptica.me>
2024-11-18 17:43:44 +01:00
|
|
|
if (aFile?.startsWith('index') && !zFile?.startsWith('index')) return -1
|
|
|
|
|
if (zFile?.startsWith('index') && !aFile?.startsWith('index')) return 1
|
2024-10-10 09:44:04 -04:00
|
|
|
|
|
|
|
|
// Sort by filename, alphabetically
|
|
|
|
|
return a[0].localeCompare(z[0])
|
|
|
|
|
})
|
|
|
|
|
.map(([file, content]) => `--- ${file} ---\n${content || '<EMPTY>'}`)
|
2024-10-10 16:02:42 +02:00
|
|
|
.join('\n\n')
|
|
|
|
|
.trim()}\n`
|
2024-10-10 09:44:04 -04:00
|
|
|
},
|
2024-08-07 16:38:44 +02:00
|
|
|
async expectFileToContain(filePath, contents) {
|
2024-08-16 15:45:52 +02:00
|
|
|
return retryAssertion(async () => {
|
2024-08-07 16:38:44 +02:00
|
|
|
let fileContent = await this.read(filePath)
|
2024-09-05 14:24:50 +02:00
|
|
|
for (let content of Array.isArray(contents) ? contents : [contents]) {
|
|
|
|
|
if (content instanceof RegExp) {
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
options.expect(fileContent).toMatch(content)
|
2024-09-05 14:24:50 +02:00
|
|
|
} else {
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
options.expect(fileContent).toContain(content)
|
2024-09-05 14:24:50 +02:00
|
|
|
}
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
})
|
2024-08-07 16:38:44 +02:00
|
|
|
},
|
Improve compatibility with `@tailwindcss/typography` and `@tailwindcss/forms` (#14221)
This PR enables compatibility for the `@tailwindcss/typography` and
`@tailwindcss/forms` plugins. This required the addition of new Plugin
APIs and new package exports.
## New Plugin APIs and compatibility improvements
We added support for `addComponents`, `matchComponents`, and `prefix`.
The component APIs are an alias for the utilities APIs because the
sorting in V4 is different and emitting components in a custom `@layer`
is not necessary. Since `prefix` is not supported in V4, the `prefix()`
API is currently an identity function.
```js
addComponents({
'.btn': {
padding: '.5rem 1rem',
borderRadius: '.25rem',
fontWeight: '600',
},
'.btn-blue': {
backgroundColor: '#3490dc',
color: '#fff',
'&:hover': {
backgroundColor: '#2779bd',
},
},
'.btn-red': {
backgroundColor: '#e3342f',
color: '#fff',
'&:hover': {
backgroundColor: '#cc1f1a',
},
},
})
```
The behavioral changes effect the `addUtilities` and `matchUtilities`
functions, we now:
- Allow arrays of CSS property objects to be emitted:
```js
addUtilities({
'.text-trim': [
{'text-box-trim': 'both'},
{'text-box-edge': 'cap alphabetic'},
],
})
```
- Allow arrays of utilities
```js
addUtilities([
{
'.text-trim':{
'text-box-trim': 'both',
'text-box-edge': 'cap alphabetic',
},
}
])
```
- Allow more complicated selector names
```js
addUtilities({
'.form-input, .form-select, .form-radio': {
/* styles here */
},
'.form-input::placeholder': {
/* styles here */
},
'.form-checkbox:indeterminate:checked': {
/* styles here */
}
})
```
## New `tailwindcss/color` and `tailwindcss/defaultTheme` export
To be compatible to v3, we're adding two new exports to the tailwindcss
package. These match the default theme values as defined in v3:
```ts
import colors from 'tailwindcss/colors'
console.log(colors.red[600])
```
```ts
import theme from 'tailwindcss/defaultTheme'
console.log(theme.spacing[4])
```
---------
Co-authored-by: Philipp Spiess <hello@philippspiess.com>
2024-08-22 08:06:21 -04:00
|
|
|
async expectFileNotToContain(filePath, contents) {
|
|
|
|
|
return retryAssertion(async () => {
|
|
|
|
|
let fileContent = await this.read(filePath)
|
|
|
|
|
for (let content of contents) {
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
options.expect(fileContent).not.toContain(content)
|
Improve compatibility with `@tailwindcss/typography` and `@tailwindcss/forms` (#14221)
This PR enables compatibility for the `@tailwindcss/typography` and
`@tailwindcss/forms` plugins. This required the addition of new Plugin
APIs and new package exports.
## New Plugin APIs and compatibility improvements
We added support for `addComponents`, `matchComponents`, and `prefix`.
The component APIs are an alias for the utilities APIs because the
sorting in V4 is different and emitting components in a custom `@layer`
is not necessary. Since `prefix` is not supported in V4, the `prefix()`
API is currently an identity function.
```js
addComponents({
'.btn': {
padding: '.5rem 1rem',
borderRadius: '.25rem',
fontWeight: '600',
},
'.btn-blue': {
backgroundColor: '#3490dc',
color: '#fff',
'&:hover': {
backgroundColor: '#2779bd',
},
},
'.btn-red': {
backgroundColor: '#e3342f',
color: '#fff',
'&:hover': {
backgroundColor: '#cc1f1a',
},
},
})
```
The behavioral changes effect the `addUtilities` and `matchUtilities`
functions, we now:
- Allow arrays of CSS property objects to be emitted:
```js
addUtilities({
'.text-trim': [
{'text-box-trim': 'both'},
{'text-box-edge': 'cap alphabetic'},
],
})
```
- Allow arrays of utilities
```js
addUtilities([
{
'.text-trim':{
'text-box-trim': 'both',
'text-box-edge': 'cap alphabetic',
},
}
])
```
- Allow more complicated selector names
```js
addUtilities({
'.form-input, .form-select, .form-radio': {
/* styles here */
},
'.form-input::placeholder': {
/* styles here */
},
'.form-checkbox:indeterminate:checked': {
/* styles here */
}
})
```
## New `tailwindcss/color` and `tailwindcss/defaultTheme` export
To be compatible to v3, we're adding two new exports to the tailwindcss
package. These match the default theme values as defined in v3:
```ts
import colors from 'tailwindcss/colors'
console.log(colors.red[600])
```
```ts
import theme from 'tailwindcss/defaultTheme'
console.log(theme.spacing[4])
```
---------
Co-authored-by: Philipp Spiess <hello@philippspiess.com>
2024-08-22 08:06:21 -04:00
|
|
|
}
|
|
|
|
|
})
|
|
|
|
|
},
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
},
|
2024-08-07 16:38:44 +02:00
|
|
|
} satisfies TestContext
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
Improve compatibility with `@tailwindcss/typography` and `@tailwindcss/forms` (#14221)
This PR enables compatibility for the `@tailwindcss/typography` and
`@tailwindcss/forms` plugins. This required the addition of new Plugin
APIs and new package exports.
## New Plugin APIs and compatibility improvements
We added support for `addComponents`, `matchComponents`, and `prefix`.
The component APIs are an alias for the utilities APIs because the
sorting in V4 is different and emitting components in a custom `@layer`
is not necessary. Since `prefix` is not supported in V4, the `prefix()`
API is currently an identity function.
```js
addComponents({
'.btn': {
padding: '.5rem 1rem',
borderRadius: '.25rem',
fontWeight: '600',
},
'.btn-blue': {
backgroundColor: '#3490dc',
color: '#fff',
'&:hover': {
backgroundColor: '#2779bd',
},
},
'.btn-red': {
backgroundColor: '#e3342f',
color: '#fff',
'&:hover': {
backgroundColor: '#cc1f1a',
},
},
})
```
The behavioral changes effect the `addUtilities` and `matchUtilities`
functions, we now:
- Allow arrays of CSS property objects to be emitted:
```js
addUtilities({
'.text-trim': [
{'text-box-trim': 'both'},
{'text-box-edge': 'cap alphabetic'},
],
})
```
- Allow arrays of utilities
```js
addUtilities([
{
'.text-trim':{
'text-box-trim': 'both',
'text-box-edge': 'cap alphabetic',
},
}
])
```
- Allow more complicated selector names
```js
addUtilities({
'.form-input, .form-select, .form-radio': {
/* styles here */
},
'.form-input::placeholder': {
/* styles here */
},
'.form-checkbox:indeterminate:checked': {
/* styles here */
}
})
```
## New `tailwindcss/color` and `tailwindcss/defaultTheme` export
To be compatible to v3, we're adding two new exports to the tailwindcss
package. These match the default theme values as defined in v3:
```ts
import colors from 'tailwindcss/colors'
console.log(colors.red[600])
```
```ts
import theme from 'tailwindcss/defaultTheme'
console.log(theme.spacing[4])
```
---------
Co-authored-by: Philipp Spiess <hello@philippspiess.com>
2024-08-22 08:06:21 -04:00
|
|
|
config.fs['.gitignore'] ??= txt`
|
|
|
|
|
node_modules/
|
|
|
|
|
`
|
|
|
|
|
|
2026-06-25 19:14:10 +02:00
|
|
|
// Disable Git's automatic line ending conversion in these throwaway test
|
|
|
|
|
// repositories. Tools like pnpm write files (e.g. `pnpm-lock.yaml`) with
|
|
|
|
|
// LF endings, which on Windows (where `core.autocrlf=true` is common)
|
|
|
|
|
// makes Git emit noisy "LF will be replaced by CRLF" warnings when it
|
|
|
|
|
// touches those files.
|
|
|
|
|
config.fs['.gitattributes'] ??= txt`
|
|
|
|
|
* -text
|
|
|
|
|
`
|
|
|
|
|
|
|
|
|
|
// Ensure there is always a root `pnpm-workspace.yaml` so that transitive
|
|
|
|
|
// dependency overrides (and the build allow list) are injected into it.
|
|
|
|
|
// As of pnpm v10, these can no longer live in the `pnpm.overrides` field
|
|
|
|
|
// of `package.json`.
|
|
|
|
|
config.fs['pnpm-workspace.yaml'] ??= ''
|
|
|
|
|
|
2024-08-08 12:02:19 +02:00
|
|
|
for (let [filename, content] of Object.entries(config.fs)) {
|
Ensure `@source` globs with symlinks are preserved (#20203)
This PR fixes an issue when working with `@source` and `@source not`
that involves symlinks.
Internally our sources are mapped to a source entry where we have a
`base` path and a `pattern`. You can think about this where we insert a
`.gitignore` file in the `base` path for the given pattern.
However, we optimize these entries to move as many "static" parts into
the base path. For example:
```css
@source "./some/folder/here/*.html";
```
Is mapped to something like:
```ts
{ base: "/projects/my-project", pattern: "./some/folder/here/*.html" }
```
We then optimize it by turning it into:
```ts
{ base: "/projects/my-project/some/folder/here", pattern: "*.html" }
```
While doing this, we also use `dunce::canonicalize` to resolve the
actual paths on disk. This means that a symlink is resolved to their
real paths. This can cause issues because the "real" path is not what
you wrote in the `@source` directives.
So before, it could be that you have this:
```css
@source "./some/symlinked-folder/here/*.html";
```
Which was mapped to this on the Rust side:
```ts
{ base: "/projects/my-project", pattern: "./some/symlinked-folder/here/*.html" }
```
But was then optimized to:
```ts
{ base: "/projects/my-project/some/actual-folder/here", pattern: "*.html" }
```
...and we lost the `symlinked-folder` information. This causes issues as
seen in #17985.
With this PR, we keep the symlinked information in those globs since
that's what you wrote in those `@source` directives.
While setting up integration tests, I stumbled upon an issue because I
wanted to test that ignoring a symlinked folder, but including a single
particular file of that ignored folder resulted in that file being
ignored as well. Let's look at an example:
```css
@source '../lib';
@source not '../lib/ignored';
@source '../lib/ignored/except.html';
```
Earlier I mentioned that we create `.gitignore` files based on these
`@source` directives. In this case, when we're dealing with a folder, we
use `**/*` as the contents.
Looking at the example above, we should essentially have something like
this:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored'
**/*
# @source '../lib/ignored/except.html'
!except.html
```
Since it's a `.gitignore` file, we have to invert the globs. But the bug
I noticed is that in reality the result of those gitignores didn't look
like the above, it looked like:
```gitignore
# lib/.gitignore
# @source '../lib'
!**/*
# lib/ignored/.gitignore
# @source '../lib/ignored/except.html'
!except.html
# @source '../lib/ignored'
**/*
```
Notice how the `!except.html` and `**/*` are flipped. When dealing with
`.gitignore` files, the order is important.
This was caused because internally we kept a `BTreeMap` of `BTreeSet`s
where the map was the base path and a set of patterns. The patterns were
sorted because of the `BTreeSet`... which is not what we want.
Fixes: #17985
Closes: #20091
## Test plan
1. Added new tests in the scanner tests (on the Rust side)
2. Added integration tests with a symlink to another folder, outside of
the current folder
2. Added integration tests with a symlink to another folder, inside of
the current folder
2. Added integration tests to ensure that the order of `@source` files
with a folder + file is sorted correctly.
3. Since we're dealing with symlinks in these tests, let's test all OSes
[ci-all]
2026-06-07 20:44:22 +02:00
|
|
|
if (content.toString().startsWith('symlink:')) {
|
|
|
|
|
// The symlink path is relative to the target destination's path
|
|
|
|
|
let target = path.join(
|
|
|
|
|
filename,
|
|
|
|
|
content.toString().slice('symlink:'.length), // Relative path
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
await context.fs.symlink(target, filename)
|
|
|
|
|
} else {
|
|
|
|
|
await context.fs.write(filename, content)
|
|
|
|
|
}
|
2024-08-08 12:02:19 +02:00
|
|
|
}
|
|
|
|
|
|
2025-01-28 11:13:16 -05:00
|
|
|
let shouldInstallDependencies = config.installDependencies ?? true
|
|
|
|
|
|
2024-08-08 12:02:19 +02:00
|
|
|
try {
|
2026-06-25 19:14:10 +02:00
|
|
|
// Every test project gets its own root `pnpm-workspace.yaml` (see
|
|
|
|
|
// above). This makes the test directory its own workspace root, so even
|
|
|
|
|
// in debug mode — where the directory lives inside this repository's
|
|
|
|
|
// pnpm workspace — `pnpm install` won't attach to the parent workspace.
|
2025-01-28 11:13:16 -05:00
|
|
|
if (shouldInstallDependencies) {
|
2026-06-25 19:14:10 +02:00
|
|
|
await context.exec(`pnpm install`)
|
2025-01-28 11:13:16 -05:00
|
|
|
}
|
2024-08-08 12:02:19 +02:00
|
|
|
} catch (error: any) {
|
|
|
|
|
console.error(error)
|
2024-08-16 15:45:52 +02:00
|
|
|
console.error(error.stdout?.toString())
|
|
|
|
|
console.error(error.stderr?.toString())
|
2024-08-08 12:02:19 +02:00
|
|
|
throw error
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let disposables: (() => Promise<void>)[] = []
|
|
|
|
|
|
|
|
|
|
async function dispose() {
|
2026-04-30 23:23:36 +02:00
|
|
|
let results = await Promise.allSettled(disposables.map((dispose) => dispose()))
|
2024-08-08 17:49:06 +02:00
|
|
|
|
2024-08-16 15:45:52 +02:00
|
|
|
if (!debug) {
|
|
|
|
|
await gracefullyRemove(root)
|
2024-08-08 12:02:19 +02:00
|
|
|
}
|
2026-04-30 23:23:36 +02:00
|
|
|
|
|
|
|
|
let errors = results.flatMap((result) =>
|
|
|
|
|
result.status === 'rejected' ? [result.reason] : [],
|
|
|
|
|
)
|
|
|
|
|
if (errors.length > 0) {
|
|
|
|
|
throw new AggregateError(errors, 'Failed to clean up spawned processes')
|
|
|
|
|
}
|
2024-08-08 12:02:19 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
options.onTestFinished(dispose)
|
|
|
|
|
|
2024-11-14 11:31:05 +01:00
|
|
|
// Make it a git repository, and commit all files
|
|
|
|
|
if (only || debug) {
|
|
|
|
|
try {
|
|
|
|
|
await context.exec('git init', { cwd: root })
|
|
|
|
|
await context.exec('git add --all', { cwd: root })
|
Fix `@source` with folders that are ignored (#20214)
This PR fixes an issue where a `@source` that's pointing to a folder
that is git ignored, is also ignored by the `@source` even if it's
explicitly added.
Internally, we convert `@source` directives from `PublicSourceEntry`s to
`SourceEntry`s where we have dedicated enum branches for `Auto`,
`Pattern`, `Ignored` and `External`.
The `Auto` one accepts a `base` path, and will be used for auto content
detection. However, these paths will make use of all the default auto
content detection rules, which includes git ignore rules.
We also have `External` where we link to something that's "external" to
the current repo. We can probably improve this name, but it's external
in the sense that it won't show up on GitHub for example, aka ignored.
We have some content dirs that we ignore by default, such as the
`node_modules` folder. When you do use `@source` with `node_modules` in
the path, then we will mark it as an `external` resource which does not
look at the `gitignore` related rules and allowing it to be included
this way.
The idea with this is that, even though the folder is ignored by
default, you can still include files from the folder by explicitly using
the `@source` directive.
The issue as seen in #19844 is using `vendor/` instead of
`node_modules/` which is _not_ ignored by default. While we can add
`vendor/` to this same ignored dirs list, it will result in a breaking
change because this folder is often used by the Laravel community to
store some resources in.
This PR fixes this problem by not only looking at the content dirs we
ignore by default, but also looking at the actual git ignore state of
this folder. If it turns out that this is ignored, then we promote the
`Auto` source to an `External` source.
Fixes: #19844
Closes: #20057
## Test plan
1. Added integration tests for this situation
2. Ran the fix on the reproduction from #19844. If we run the CLI with
the `DEBUG=*` environment variable, the log file produces these results:
```diff
diff --git a/./tailwindcss-29207.log b/./tailwindcss-30381.log
index bc3017c..921af0e 100644
--- a/./tailwindcss-29207.log
+++ b/./tailwindcss-30381.log
@@ -6,8 +6,9 @@ INFO tailwindcss_oxide::scanner: Source: PublicSourceEntry { base: "/Users/robin
INFO tailwindcss_oxide::scanner: Optimized sources:
INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.phtml" }
INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.xml" }
-INFO tailwindcss_oxide::scanner: Source: Auto { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" }
+INFO tailwindcss_oxide::scanner: Source: External { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" }
INFO tailwindcss_oxide::scanner: Source: Ignored { base: "/Users/robin/.fnm/node-versions/v26.1.0/installation/bin", pattern: "/node" }
INFO discover_sources: tailwindcss_oxide::scanner: enter
-INFO discover_sources: tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml"
+INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml"
+INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme/module/templates/component.phtml"
INFO discover_sources: tailwindcss_oxide::scanner: exit
```
We're checking some `.gitignore` related files, so let's check on each
OS [ci-all]
2026-06-10 18:44:14 +02:00
|
|
|
await context.exec('git commit -m "before migration" --allow-empty', { cwd: root })
|
2024-11-14 11:31:05 +01:00
|
|
|
} catch (error: any) {
|
|
|
|
|
console.error(error)
|
|
|
|
|
console.error(error.stdout?.toString())
|
|
|
|
|
console.error(error.stderr?.toString())
|
|
|
|
|
throw error
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-08 17:49:06 +02:00
|
|
|
return await testCallback(context)
|
2024-08-07 16:38:44 +02:00
|
|
|
},
|
|
|
|
|
)
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
test.only = (name: string, config: TestConfig, testCallback: TestCallback) => {
|
|
|
|
|
return test(name, config, testCallback, { only: true })
|
|
|
|
|
}
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
test.skip = (name: string, config: TestConfig, testCallback: TestCallback) => {
|
|
|
|
|
return test(name, config, testCallback, { skip: true })
|
|
|
|
|
}
|
2026-04-30 23:23:36 +02:00
|
|
|
test.concurrent = (name: string, config: TestConfig, testCallback: TestCallback) => {
|
|
|
|
|
return test(name, config, testCallback, { concurrent: true })
|
|
|
|
|
}
|
2024-08-08 12:02:19 +02:00
|
|
|
test.debug = (name: string, config: TestConfig, testCallback: TestCallback) => {
|
2024-08-16 15:45:52 +02:00
|
|
|
return test(name, config, testCallback, { debug: true })
|
2024-08-08 12:02:19 +02:00
|
|
|
}
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
|
|
|
|
// Maps package names to their tarball filenames. See scripts/pack-packages.ts
|
|
|
|
|
// for more details.
|
|
|
|
|
function pkgToFilename(name: string) {
|
|
|
|
|
return `${name.replace('@', '').replace('/', '-')}.tgz`
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
async function overwriteVersionsInPackageJson(content: string): Promise<string> {
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
let json = JSON.parse(content)
|
|
|
|
|
|
|
|
|
|
// Resolve all workspace:^ versions to local tarballs
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
for (let key of ['dependencies', 'devDependencies', 'peerDependencies', 'optionalDependencies']) {
|
|
|
|
|
let dependencies = json[key] || {}
|
|
|
|
|
for (let dependency in dependencies) {
|
|
|
|
|
if (dependencies[dependency] === 'workspace:^') {
|
|
|
|
|
dependencies[dependency] = resolveVersion(dependency)
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
}
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
2026-06-25 19:14:10 +02:00
|
|
|
return JSON.stringify(json, null, 2)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function overwriteVersionsInPnpmWorkspace(content: string): string {
|
|
|
|
|
let workspace = content.trim() === '' ? {} : Yaml.parse(content)
|
|
|
|
|
|
|
|
|
|
// Inherit information from the root workspace
|
|
|
|
|
workspace.allowBuilds = {
|
|
|
|
|
...ROOT_PNPM_WORKSPACE.allowBuilds,
|
|
|
|
|
...workspace.allowBuilds,
|
|
|
|
|
}
|
|
|
|
|
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
// Inject transitive dependency overwrite. This is necessary because
|
|
|
|
|
// @tailwindcss/vite internally depends on a specific version of
|
|
|
|
|
// @tailwindcss/oxide and we instead want to resolve it to the locally built
|
|
|
|
|
// version.
|
2026-06-25 19:14:10 +02:00
|
|
|
//
|
|
|
|
|
// As of pnpm v10, the `pnpm.overrides` field in `package.json` is no longer
|
|
|
|
|
// read. Overrides have to be defined in `pnpm-workspace.yaml` instead.
|
|
|
|
|
workspace.overrides ||= {}
|
2024-08-07 16:38:44 +02:00
|
|
|
for (let pkg of PUBLIC_PACKAGES) {
|
Ensure clients pin the `tailwindcss` version (#15011)
We noticed that in the current alpha 34 release, the `package.json` file
of the `@tailwindcss/node` package only defines `tailwindcss` as a dev
dependency. This makes it very easy for version mismatches to happen
when a v3 version (or an earlier v4 alpha for that matter) was installed
in the same project:
```json
{
"name": "@tailwindcss/node",
"version": "4.0.0-alpha.34",
"description": "A utility-first CSS framework for rapidly building custom user interfaces.",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/tailwindlabs/tailwindcss.git",
"directory": "packages/@tailwindcss-node"
},
"bugs": "https://github.com/tailwindlabs/tailwindcss/issues",
"homepage": "https://tailwindcss.com",
"files": [
"dist/"
],
"publishConfig": {
"provenance": true,
"access": "public"
},
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.mjs",
"require": "./dist/index.js"
},
"./require-cache": {
"types": "./dist/require-cache.d.ts",
"default": "./dist/require-cache.js"
},
"./esm-cache-loader": {
"types": "./dist/esm-cache.loader.d.mts",
"default": "./dist/esm-cache.loader.mjs"
}
},
"devDependencies": {
"tailwindcss": "4.0.0-alpha.34"
},
"dependencies": {
"enhanced-resolve": "^5.17.1",
"jiti": "^2.0.0-beta.3"
},
"scripts": {
"build": "tsup-node",
"dev": "pnpm run build -- --watch"
}
}
```
Furthermore, we were trying to fix issues where our integration test
setup could not install `tailwindcss@3` because of how we did pnpm
overrides.
This PR fixes this by:
- Ensuring every client that calls into `tailwindcss` core marks it as a
version-pinned dependency. You are still required to install
`tailwindcss` in your project along side a client (e.g.
`@tailwindcss/vite`) but we now only use your installed version for
importing the respective `.css` files. For the core logic, we are now
requiring each package to use `tailwindcss` at the same version. This
should help resolve issues like
https://github.com/tailwindlabs/tailwindcss/discussions/14652
- We tried to eliminate the dependency on `tailwindcss` from the
`@tailwindcss/upgrade` package. Unfortunately this is not possible to do
right now because we need to load the CSS files from v4 to create the
right environment. In a future version we could bundle the required CSS
files with `@tailwidncss/upgrade` but it doesn't seem necessary for now.
- We then changed our integration test overrides to only override the
`tailwindcss` package that are dependencies of the known list of
packages that we have `tailwindcss` dependencies on: `@tailwindcss/node`
and `@tailwindcss/upgrade`. This ensures that we can install v3 of
`tailwindcss` in the integration tests and it will work. Something we
want to do for some upgrade tests.
# Test plan
Integration work again. Furthermore we added a quick setup with the CLI
using the local tarballs and ensured it works:
```bash
pnpm init
pnpm install ../../tailwindcss/dist/tailwindcss-cli.tgz
pnpm install ../../tailwindcss/dist/tailwindcss.tgz
echo '@import "tailwindcss";' > index.css
echo '<div class="underline"></div>' > index.html
pnpm tailwindcss -i index.css -o out.css
cat out.css
```
2024-11-15 17:18:48 +01:00
|
|
|
if (pkg === 'tailwindcss') {
|
|
|
|
|
// We want to be explicit about the `tailwindcss` package so our tests can
|
|
|
|
|
// also import v3 without conflicting v4 tarballs.
|
2026-06-25 19:14:10 +02:00
|
|
|
workspace.overrides['@tailwindcss/node>tailwindcss'] = resolveVersion(pkg)
|
|
|
|
|
workspace.overrides['@tailwindcss/upgrade>tailwindcss'] = resolveVersion(pkg)
|
|
|
|
|
workspace.overrides['@tailwindcss/cli>tailwindcss'] = resolveVersion(pkg)
|
|
|
|
|
workspace.overrides['@tailwindcss/postcss>tailwindcss'] = resolveVersion(pkg)
|
|
|
|
|
workspace.overrides['@tailwindcss/vite>tailwindcss'] = resolveVersion(pkg)
|
|
|
|
|
workspace.overrides['@tailwindcss/webpack>tailwindcss'] = resolveVersion(pkg)
|
Ensure clients pin the `tailwindcss` version (#15011)
We noticed that in the current alpha 34 release, the `package.json` file
of the `@tailwindcss/node` package only defines `tailwindcss` as a dev
dependency. This makes it very easy for version mismatches to happen
when a v3 version (or an earlier v4 alpha for that matter) was installed
in the same project:
```json
{
"name": "@tailwindcss/node",
"version": "4.0.0-alpha.34",
"description": "A utility-first CSS framework for rapidly building custom user interfaces.",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/tailwindlabs/tailwindcss.git",
"directory": "packages/@tailwindcss-node"
},
"bugs": "https://github.com/tailwindlabs/tailwindcss/issues",
"homepage": "https://tailwindcss.com",
"files": [
"dist/"
],
"publishConfig": {
"provenance": true,
"access": "public"
},
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.mjs",
"require": "./dist/index.js"
},
"./require-cache": {
"types": "./dist/require-cache.d.ts",
"default": "./dist/require-cache.js"
},
"./esm-cache-loader": {
"types": "./dist/esm-cache.loader.d.mts",
"default": "./dist/esm-cache.loader.mjs"
}
},
"devDependencies": {
"tailwindcss": "4.0.0-alpha.34"
},
"dependencies": {
"enhanced-resolve": "^5.17.1",
"jiti": "^2.0.0-beta.3"
},
"scripts": {
"build": "tsup-node",
"dev": "pnpm run build -- --watch"
}
}
```
Furthermore, we were trying to fix issues where our integration test
setup could not install `tailwindcss@3` because of how we did pnpm
overrides.
This PR fixes this by:
- Ensuring every client that calls into `tailwindcss` core marks it as a
version-pinned dependency. You are still required to install
`tailwindcss` in your project along side a client (e.g.
`@tailwindcss/vite`) but we now only use your installed version for
importing the respective `.css` files. For the core logic, we are now
requiring each package to use `tailwindcss` at the same version. This
should help resolve issues like
https://github.com/tailwindlabs/tailwindcss/discussions/14652
- We tried to eliminate the dependency on `tailwindcss` from the
`@tailwindcss/upgrade` package. Unfortunately this is not possible to do
right now because we need to load the CSS files from v4 to create the
right environment. In a future version we could bundle the required CSS
files with `@tailwidncss/upgrade` but it doesn't seem necessary for now.
- We then changed our integration test overrides to only override the
`tailwindcss` package that are dependencies of the known list of
packages that we have `tailwindcss` dependencies on: `@tailwindcss/node`
and `@tailwindcss/upgrade`. This ensures that we can install v3 of
`tailwindcss` in the integration tests and it will work. Something we
want to do for some upgrade tests.
# Test plan
Integration work again. Furthermore we added a quick setup with the CLI
using the local tarballs and ensured it works:
```bash
pnpm init
pnpm install ../../tailwindcss/dist/tailwindcss-cli.tgz
pnpm install ../../tailwindcss/dist/tailwindcss.tgz
echo '@import "tailwindcss";' > index.css
echo '<div class="underline"></div>' > index.html
pnpm tailwindcss -i index.css -o out.css
cat out.css
```
2024-11-15 17:18:48 +01:00
|
|
|
} else {
|
2026-06-25 19:14:10 +02:00
|
|
|
workspace.overrides[pkg] = resolveVersion(pkg)
|
Ensure clients pin the `tailwindcss` version (#15011)
We noticed that in the current alpha 34 release, the `package.json` file
of the `@tailwindcss/node` package only defines `tailwindcss` as a dev
dependency. This makes it very easy for version mismatches to happen
when a v3 version (or an earlier v4 alpha for that matter) was installed
in the same project:
```json
{
"name": "@tailwindcss/node",
"version": "4.0.0-alpha.34",
"description": "A utility-first CSS framework for rapidly building custom user interfaces.",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/tailwindlabs/tailwindcss.git",
"directory": "packages/@tailwindcss-node"
},
"bugs": "https://github.com/tailwindlabs/tailwindcss/issues",
"homepage": "https://tailwindcss.com",
"files": [
"dist/"
],
"publishConfig": {
"provenance": true,
"access": "public"
},
"exports": {
".": {
"types": "./dist/index.d.ts",
"import": "./dist/index.mjs",
"require": "./dist/index.js"
},
"./require-cache": {
"types": "./dist/require-cache.d.ts",
"default": "./dist/require-cache.js"
},
"./esm-cache-loader": {
"types": "./dist/esm-cache.loader.d.mts",
"default": "./dist/esm-cache.loader.mjs"
}
},
"devDependencies": {
"tailwindcss": "4.0.0-alpha.34"
},
"dependencies": {
"enhanced-resolve": "^5.17.1",
"jiti": "^2.0.0-beta.3"
},
"scripts": {
"build": "tsup-node",
"dev": "pnpm run build -- --watch"
}
}
```
Furthermore, we were trying to fix issues where our integration test
setup could not install `tailwindcss@3` because of how we did pnpm
overrides.
This PR fixes this by:
- Ensuring every client that calls into `tailwindcss` core marks it as a
version-pinned dependency. You are still required to install
`tailwindcss` in your project along side a client (e.g.
`@tailwindcss/vite`) but we now only use your installed version for
importing the respective `.css` files. For the core logic, we are now
requiring each package to use `tailwindcss` at the same version. This
should help resolve issues like
https://github.com/tailwindlabs/tailwindcss/discussions/14652
- We tried to eliminate the dependency on `tailwindcss` from the
`@tailwindcss/upgrade` package. Unfortunately this is not possible to do
right now because we need to load the CSS files from v4 to create the
right environment. In a future version we could bundle the required CSS
files with `@tailwidncss/upgrade` but it doesn't seem necessary for now.
- We then changed our integration test overrides to only override the
`tailwindcss` package that are dependencies of the known list of
packages that we have `tailwindcss` dependencies on: `@tailwindcss/node`
and `@tailwindcss/upgrade`. This ensures that we can install v3 of
`tailwindcss` in the integration tests and it will work. Something we
want to do for some upgrade tests.
# Test plan
Integration work again. Furthermore we added a quick setup with the CLI
using the local tarballs and ensured it works:
```bash
pnpm init
pnpm install ../../tailwindcss/dist/tailwindcss-cli.tgz
pnpm install ../../tailwindcss/dist/tailwindcss.tgz
echo '@import "tailwindcss";' > index.css
echo '<div class="underline"></div>' > index.html
pnpm tailwindcss -i index.css -o out.css
cat out.css
```
2024-11-15 17:18:48 +01:00
|
|
|
}
|
2024-08-07 16:38:44 +02:00
|
|
|
}
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
|
2026-06-25 19:14:10 +02:00
|
|
|
return Yaml.stringify(workspace)
|
Add integration test setup and tests for the Vite integration (#14089)
This PR adds a new root `/integrations` folder that will be the home of
integration tests. The idea of these tests is to use Tailwind in various
setups just like our users would (by only using the publishable npm
builds).
To avoid issues with concurrent tests making changes to the file system,
to make it very easy to test through a range of versions, and to avoid
changing configuration objects over and over in test runs, we decided to
inline the scaffolding completely into the test file and have no
examples checked into the repo.
Here's an example of how this can look like for a simple Vite test:
```ts
test('works with production builds', {
fs: {
'package.json': json`
{
"type": "module",
"dependencies": {
"@tailwindcss/vite": "workspace:^",
"tailwindcss": "workspace:^"
},
"devDependencies": {
"vite": "^5.3.5"
}
}
`,
'vite.config.ts': ts`
import tailwindcss from '@tailwindcss/vite'
import { defineConfig } from 'vite'
export default defineConfig({
build: { cssMinify: false },
plugins: [tailwindcss()],
})
`,
'index.html': html`
<head>
<link rel="stylesheet" href="./src/index.css">
</head>
<body>
<div class="underline m-2">Hello, world!</div>
</body>
`,
'src/index.css': css`
@import 'tailwindcss/theme' reference;
@import 'tailwindcss/utilities';
`,
},
},
async ({ fs, exec }) => {
await exec('pnpm vite build')
expect.assertions(2)
for (let [path, content] of await fs.glob('dist/**/*.css')) {
expect(path).toMatch(/\.css$/)
expect(stripTailwindComment(content)).toMatchInlineSnapshot(
`
".m-2 {
margin: var(--spacing-2, .5rem);
}
.underline {
text-decoration-line: underline;
}"
`,
)
}
},
)
```
By defining all dependencies this way, we never have to worry about
which fixtures are checked in and can more easily describe changes to
the setup.
For ergonomics, we've also added the [`embed` prettier
plugin](https://github.com/Sec-ant/prettier-plugin-embed). This will
mean that files inlined in the `fs` setup are properly indented. No
extra work needed!
If you're using VS Code, I can also recommend the [Language
Literals](https://marketplace.visualstudio.com/items?itemName=sissel.language-literals)
extension so that syntax highlighting also _just works_.
A neat feature of inlining the scaffolding like this is to make it very
simple to test through a variety of versions. For example, here's how we
can set up a test against Vite 5 and Vite 4:
```js
;['^4.5.3', '^5.3.5'].forEach(viteVersion => {
test(`works with production builds for Vite ${viteVersion}`, {
fs: {
'package.json': json`
{
"type": "module",
"devDependencies": {
"vite": "${viteVersion}"
}
}
`,
async () => {
// Do something
},
)
})
```
## Philosophy
Before we dive into the specifics, I want to clearly state the design
considerations we have chosen for this new test suite:
- All file mutations should be done in temp folders, nothing should ever
mess with your working directory
- Windows as a first-class citizen
- Have a clean and simple API that describes the test setup only using
public APIs
- Focus on reliability (make sure cleanup scripts work and are tolerant
to various error scenarios)
- If a user reports an issue with a specific configuration, we want to
be able to reproduce them with integration tests, no matter how obscure
the setup (this means the test need to be in control of most of the
variables)
- Tests should be reasonably fast (obviously this depends on the
integration. If we use a slow build tool, we can't magically speed it
up, but our overhead should be minimal).
## How it works
The current implementation provides a custom `test` helper function
that, when used, sets up the environment according to the configuration.
It'll create a new temporary directory and create all files, ensuring
things like proper `\r\n` line endings on Windows.
We do have to patch the `package.json` specifically, since we can not
use public versions of the tailwindcss packages as we want to be able to
test against a development build. To make this happen, every `pnpm
build` run now creates tarballs of the npm modules (that contain only
the files that would also in the published build). We then patch the
`package.json` to rewrite `workspace:^` versions to link to those
tarballs. We found this to work reliably on Windows and macOS as well as
being fast enough to not cause any issues. Furthermore we also decided
to use `pnpm` as the version manager for integration tests because of
it's global module cache (so installing `vite` is fast as soon as you
installed it once).
The test function will receive a few utilities that it can use to more
easily interact with the temp dir. One example is a `fs.glob` function
that you can use to easily find files in eventual `dist/` directories or
helpers around `spawn` and `exec` that make sure that processes are
cleaned up correctly.
Because we use tarballs from our build dependencies, working on changes
requires a workflow where you run `pnpm build` before running `pnpm
test:integrations`. However it also means we can run clients like our
CLI client with no additional overhead—just install the dependency like
any user would and set up your test cases this way.
## Test plan
This PR also includes two Vite specific integration tests: One testing a
static build (`pnpm vite build`) and one a dev mode build (`pnpm vite
dev`) that also makes changes to the file system and asserts that the
resources properly update.
---------
Co-authored-by: Robin Malfait <malfait.robin@gmail.com>
2024-08-02 11:50:49 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function resolveVersion(dependency: string) {
|
|
|
|
|
let tarball = path.join(REPO_ROOT, 'dist', pkgToFilename(dependency))
|
|
|
|
|
return `file:${tarball}`
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
export function stripTailwindComment(content: string) {
|
|
|
|
|
return content.replace(/\/\*! tailwindcss .*? \*\//g, '').trim()
|
|
|
|
|
}
|
|
|
|
|
|
2024-11-07 09:51:58 -05:00
|
|
|
export let svg = dedent
|
2024-08-07 16:38:44 +02:00
|
|
|
export let css = dedent
|
|
|
|
|
export let html = dedent
|
|
|
|
|
export let ts = dedent
|
|
|
|
|
export let js = dedent
|
2025-04-03 17:13:15 +02:00
|
|
|
export let jsx = dedent
|
2024-08-07 16:38:44 +02:00
|
|
|
export let json = dedent
|
|
|
|
|
export let yaml = dedent
|
|
|
|
|
export let txt = dedent
|
|
|
|
|
|
2024-11-07 09:51:58 -05:00
|
|
|
export function binary(str: string | TemplateStringsArray, ...values: unknown[]): Uint8Array {
|
|
|
|
|
let base64 = typeof str === 'string' ? str : String.raw(str, ...values)
|
|
|
|
|
|
|
|
|
|
return Uint8Array.from(atob(base64), (c) => c.charCodeAt(0))
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-07 16:38:44 +02:00
|
|
|
export function candidate(strings: TemplateStringsArray, ...values: any[]) {
|
|
|
|
|
let output: string[] = []
|
|
|
|
|
for (let i = 0; i < strings.length; i++) {
|
|
|
|
|
output.push(strings[i])
|
|
|
|
|
if (i < values.length) {
|
|
|
|
|
output.push(values[i])
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return `.${escape(output.join('').trim())}`
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-16 15:45:52 +02:00
|
|
|
export async function retryAssertion<T>(
|
2024-08-07 16:38:44 +02:00
|
|
|
fn: () => Promise<T>,
|
|
|
|
|
{ timeout = ASSERTION_TIMEOUT, delay = 5 }: { timeout?: number; delay?: number } = {},
|
|
|
|
|
) {
|
|
|
|
|
let end = Date.now() + timeout
|
|
|
|
|
let error: any
|
|
|
|
|
while (Date.now() < end) {
|
|
|
|
|
try {
|
|
|
|
|
return await fn()
|
|
|
|
|
} catch (err) {
|
Fix `@source` with folders that are ignored (#20214)
This PR fixes an issue where a `@source` that's pointing to a folder
that is git ignored, is also ignored by the `@source` even if it's
explicitly added.
Internally, we convert `@source` directives from `PublicSourceEntry`s to
`SourceEntry`s where we have dedicated enum branches for `Auto`,
`Pattern`, `Ignored` and `External`.
The `Auto` one accepts a `base` path, and will be used for auto content
detection. However, these paths will make use of all the default auto
content detection rules, which includes git ignore rules.
We also have `External` where we link to something that's "external" to
the current repo. We can probably improve this name, but it's external
in the sense that it won't show up on GitHub for example, aka ignored.
We have some content dirs that we ignore by default, such as the
`node_modules` folder. When you do use `@source` with `node_modules` in
the path, then we will mark it as an `external` resource which does not
look at the `gitignore` related rules and allowing it to be included
this way.
The idea with this is that, even though the folder is ignored by
default, you can still include files from the folder by explicitly using
the `@source` directive.
The issue as seen in #19844 is using `vendor/` instead of
`node_modules/` which is _not_ ignored by default. While we can add
`vendor/` to this same ignored dirs list, it will result in a breaking
change because this folder is often used by the Laravel community to
store some resources in.
This PR fixes this problem by not only looking at the content dirs we
ignore by default, but also looking at the actual git ignore state of
this folder. If it turns out that this is ignored, then we promote the
`Auto` source to an `External` source.
Fixes: #19844
Closes: #20057
## Test plan
1. Added integration tests for this situation
2. Ran the fix on the reproduction from #19844. If we run the CLI with
the `DEBUG=*` environment variable, the log file produces these results:
```diff
diff --git a/./tailwindcss-29207.log b/./tailwindcss-30381.log
index bc3017c..921af0e 100644
--- a/./tailwindcss-29207.log
+++ b/./tailwindcss-30381.log
@@ -6,8 +6,9 @@ INFO tailwindcss_oxide::scanner: Source: PublicSourceEntry { base: "/Users/robin
INFO tailwindcss_oxide::scanner: Optimized sources:
INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.phtml" }
INFO tailwindcss_oxide::scanner: Source: Pattern { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme", pattern: "/**/*.xml" }
-INFO tailwindcss_oxide::scanner: Source: Auto { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" }
+INFO tailwindcss_oxide::scanner: Source: External { base: "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme" }
INFO tailwindcss_oxide::scanner: Source: Ignored { base: "/Users/robin/.fnm/node-versions/v26.1.0/installation/bin", pattern: "/node" }
INFO discover_sources: tailwindcss_oxide::scanner: enter
-INFO discover_sources: tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml"
+INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/app/design/frontend/theme/index.phtml"
+INFO tailwindcss_oxide::scanner: Reading "/Users/robin/github.com/GrimLink/tailwind-gitignore-bug/vendor/acme/theme/module/templates/component.phtml"
INFO discover_sources: tailwindcss_oxide::scanner: exit
```
We're checking some `.gitignore` related files, so let's check on each
OS [ci-all]
2026-06-10 18:44:14 +02:00
|
|
|
Error.captureStackTrace(err, retryAssertion)
|
2024-08-07 16:38:44 +02:00
|
|
|
error = err
|
|
|
|
|
await new Promise((resolve) => setTimeout(resolve, delay))
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
throw error
|
|
|
|
|
}
|
2024-08-16 15:45:52 +02:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
export async function fetchStyles(base: string, path = '/'): Promise<string> {
|
|
|
|
|
while (base.endsWith('/')) {
|
|
|
|
|
base = base.slice(0, -1)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let index = await fetch(`${base}${path}`)
|
2024-08-16 15:45:52 +02:00
|
|
|
let html = await index.text()
|
|
|
|
|
|
2026-03-15 22:04:58 +01:00
|
|
|
let linkRegex = /<link rel="stylesheet" href="([a-zA-Z0-9/_.?=%-]+)"/gi
|
2024-09-04 19:14:05 +02:00
|
|
|
let styleRegex = /<style\b[^>]*>([\s\S]*?)<\/style>/gi
|
|
|
|
|
|
|
|
|
|
let stylesheets: string[] = []
|
2024-08-16 15:45:52 +02:00
|
|
|
|
|
|
|
|
let paths: string[] = []
|
2024-09-04 19:14:05 +02:00
|
|
|
for (let match of html.matchAll(linkRegex)) {
|
2024-08-16 15:45:52 +02:00
|
|
|
let path: string = match[1]
|
|
|
|
|
if (path.startsWith('./')) {
|
|
|
|
|
path = path.slice(1)
|
|
|
|
|
}
|
|
|
|
|
paths.push(path)
|
|
|
|
|
}
|
2024-09-04 19:14:05 +02:00
|
|
|
stylesheets.push(
|
|
|
|
|
...(await Promise.all(
|
|
|
|
|
paths.map(async (path) => {
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let css = await fetch(`${base}${path}`, {
|
2024-09-04 19:14:05 +02:00
|
|
|
headers: {
|
|
|
|
|
Accept: 'text/css',
|
|
|
|
|
},
|
|
|
|
|
})
|
|
|
|
|
return await css.text()
|
|
|
|
|
}),
|
|
|
|
|
)),
|
2024-08-16 15:45:52 +02:00
|
|
|
)
|
|
|
|
|
|
2024-09-04 19:14:05 +02:00
|
|
|
for (let match of html.matchAll(styleRegex)) {
|
|
|
|
|
stylesheets.push(match[1])
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-16 15:45:52 +02:00
|
|
|
return stylesheets.reduce((acc, css) => {
|
Add support for source maps (#17775)
Closes #13694
Closes #13591
# Source Maps Support for Tailwind CSS
This PR adds support for source maps to Tailwind CSS v4 allowing us to
track where styles come from whether that be user CSS, imported
stylesheets, or generated utilities. This will improve debuggability in
browser dev tools and gives us a good foundation for producing better
error messages. I'll go over the details on how end users can enable
source maps, any limitations in our implementation, changes to the
internal `compile(…)` API, and some details and reasoning around the
implementation we chose.
## Usage
### CLI
Source maps can be enabled in the CLI by using the command line argument
`--map` which will generate an inline source map comment at the bottom
of your CSS. A separate file may be generated by passing a file name to
`--map`:
```bash
# Generates an inline source map
npx tailwindcss -i input.css -o output.css --map
# Generates a separate source map file
npx tailwindcss -i input.css -o output.css --map output.css.map
```
### PostCSS
Source maps are supported when using Tailwind as a PostCSS plugin *in
development mode only*. They may or may not be enabled by default
depending on your build tool. If they are not you may be able to
configure them within your PostCSS config:
```jsonc
// package.json
{
// …
"postcss": {
"map": { "inline": true },
"plugins": {
"@tailwindcss/postcss": {},
},
}
}
```
### Vite
Source maps are supported when using the Tailwind CSS Vite plugin in
*development mode only* by enabling the `css.devSourcemap` setting:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
devSourcemap: true,
},
})
```
Now when a CSS file is requested by the browser it'll have an inline
source map comment that the browser can use.
## Limitations
- Production build source maps are currently disabled due to a bug in
Lightning CSS. See
https://github.com/parcel-bundler/lightningcss/pull/971 for more
details.
- In Vite, minified CSS build source maps are not supported at all. See
https://github.com/vitejs/vite/issues/2830 for more details.
- In PostCSS, minified CSS source maps are not supported. This is due to
the complexity required around re-associating every AST node with a
location in the generated, optimized CSS. This complexity would also
have a non-trivial performance impact.
## Testing
Here's how to test the source map functionality in different
environments:
### Testing the CLI
1. Setup typical project that the CLI can use and with sources to scan.
```css
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
2. Build with source maps:
```bash
bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map
```
3. Open Chrome DevTools, inspect an element with utility classes, and
you should see rules pointing to `input.css` or
`node_modules/tailwindcss/index.css`
### Testing with Vite
Testing in Vite will require building and installing necessary files
under `dist/*.tgz`.
1. Create a Vite project and enable source maps in `vite.config.js`:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
// This line is required for them to work
devSourcemap: true,
},
})
```
2. Add a component that uses Tailwind classes and custom CSS:
```jsx
// ./src/app.jsx
export default function App() {
return (
<div className="bg-blue-500 my-custom-class">
Hello World
</div>
)
}
```
```css
/* ./src/styles.css */
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
3. Run `npm run dev`, open DevTools, and inspect elements to verify
source mapping works for both utility classes and custom CSS.
### Testing with PostCSS CLI
1. Create a test file and update your PostCSS config:
```css
/* input.css */
@import "tailwindcss";
@layer components {
.card {
@apply p-6 rounded-lg shadow-lg;
}
}
```
```jsonc
// package.json
{
// …
"postcss": {
"map": {
"inline": true
},
"plugins": {
"/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {}
}
}
}
```
2. Run PostCSS through Bun:
```bash
bunx --bun postcss ./src/index.css -o out.css
```
3. Inspect the output CSS - it should include an inline source map
comment at the bottom.
### Testing with PostCSS + Next.js
Testing in Next.js will require building and installing necessary files
under `dist/*.tgz`. However, I've not been able to get CSS source maps
to work in Next.js without this hack:
```js
const nextConfig: NextConfig = {
// next.js overwrites config.devtool so we prevent it from doing so
// please don't actually do this…
webpack: (config) =>
Object.defineProperty(config, "devtool", {
get: () => "inline-source-map",
set: () => {},
}),
};
```
This is definitely not supported and also doesn't work with turbopack.
This can be used to test them temporarily but I suspect that they just
don't work there.
### Manual source map analysis
You can analyze source maps using Evan Wallace's [Source Map
Visualization](https://evanw.github.io/source-map-visualization/) tool
which will help to verify the accuracy and quality of source maps. This
is what I used extensively while developing this implementation.
It'll help verify that custom, user CSS maps back to itself in the
input, that generated utilities all map back to `@tailwind utilities;`,
that source locations from imported files are also handled correctly,
etc… It also highlights the ranges of stuff so it's easy to see if there
are off-by-one errors.
It's easiest to use inline source maps with this tool because you can
take the CSS file and drop it on the page and it'll analyze it while
showing the file content.
If you're using Vite you'll want to access the CSS file with `?direct`
at the end so you don't get a JS module back.
## Implementation
The source map implementation follows the ECMA-426 specification and
includes several key components to aid in that goal:
### Source Location Tracking
Each emittable AST node in the compilation pipeline tracks two types of
source locations:
- `src`: Original source location - [source file, start offset, end
offset]
- `dst`: Generated source location - [output file, start offset, end
offset]
This dual tracking allows us to maintain mappings between the original
source and generated output for things like user CSS, generated
utilities, uses of `@apply`, and tracking theme variables.
It is important to note that source locations for nodes _never overlap_
within a file which helps simplify source map generation. As such each
type of node tracks a specific piece of itself rather than its entire
"block":
| Node | What a `SourceLocation` represents |
| ----------- |
---------------------------------------------------------------- |
| Style Rule | The selector |
| At Rule | Rule name and params, includes the `@` |
| Declaration | Property name and value, excludes the semicolon |
| Comment | The entire comment, includes the start `/*` and end `*/`
markers |
### Windows line endings when parsing CSS
Because our AST tracks nodes through offsets we must ensure that any
mutations to the file do *not* change the lenth of the string. We were
previously replacing `\r\n` with `\n` (see [filter code
points](https://drafts.csswg.org/css-syntax/#css-filter-code-points)
from the spec) — which changes the length of the string and all offsets
may end up incorrect. The CSS parser was updated to handle the CRLF
token directly by skipping over the `\r` and letting remaining code
handle `\n` as it did previously. Some additional tweaks were required
when "peeking" the input but those changes were fairly small.
### Tracking of imports
Source maps need paths to the actual imported stylesheets but the
resolve step for stylesheets happens inside the call to `loadStylesheet`
which make the file path unavailable to us. Because of this the
`loadStylesheet` API was augmented such that it has to return a `path`
property that we can then use to identify imported sources. I've also
made the same change to the `loadModule` API for consistency but nothing
currently uses this property.
The `path` property likely makes `base` redundant but elminating that
(if we even want to) is a future task.
### Optimizing the AST
Our optimization pass may intoduce some nodes, for example, fallbacks we
create for `@property`. These nodes are linked back to `@tailwind
utilities` as ultimately that is what is responsible for creating them.
### Line Offset Tables
A key component to our source map generation is the line offset table,
which was inspired by some ESBuild internals. It stores a sorted list of
offsets for the start of each line allowing us to translate offsets to
line/column `Position`s in `O(log N)` time and from `Position`s to
offsets in `O(1)` time. Creation of the table takes `O(N)` time.
This means that we can store code point offsets for source locations and
not have to worry about computing or tracking line/column numbers during
parsing and serialization. Only when a source map is generated do these
offsets need to be computed. This ensures the performance penalty when
not using source maps is minimal.
### Source Map Generation
The source map returned by `buildSourceMap()` is designed to follow the
[ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not
completely finalized we consider the result of `buildSourceMap()` to be
internal API that may change as the spec chamges.
The produces source map is a "decoded" map such that all sources and
mappings are in an object graph. A library like `source-map-js` must be
used to convert this to an encoded source map of the right version where
mappings are encoded with base 64 VLQs.
Any specific integration (Vite, PostCSS, etc…) can then use
`toSourceMap()` from `@tailwindcss/node` to convert from the internal
source map to an spec-compliant encoded source map that can be
understood by other tools.
### Handling minification in Lightning
Since we use Lightning CSS for optimization, and it takes in an input
map, we generate an encoded source map that we then pass to lightning.
The output source map *from lighting itself* is then passed back in
during the second optimization pass. The final map is then passed from
lightning to the CLI (but not Vite or PostCSS — see the limitations
section for details).
In some cases we have to "fix up" the output CSS. When this happens we
use `magic-string` to do the replacement in a way that is trackable and
`@amppproject/remapping` to map that change back onto the original
source map. Once the need for these fix ups disappear these dependencies
can go away.
Notes:
- The accuracy of source maps run though lightning is reduced as it only
tracks on a per-rule level. This is sufficient enough for browser dev
tools so should be fine.
- Source maps during optimization do not function properly at this time
because of a bug in Lightning CSS regarding license comments. Once this
bug is fixed they will start working as expected.
### How source locations flow through the system
1. During initial CSS parsing, source locations are preserved.
2. During parsing these source locations are also mapped to the
destinations which supports an optimization for when no utilities are
generated.
3. Throughout the compilation process, transformations maintain source
location data
4. Generated utilities are explicitly pointed to `@tailwind utilities`
unless generated by `@apply`.
5. When optimization is enabled, source maps are remapped through
lightningcss
6. Final source maps are written in the requested format (inline or
separate file)
2025-05-08 16:29:49 -04:00
|
|
|
if (acc.length > 0) acc += '\n'
|
|
|
|
|
acc += css
|
|
|
|
|
return acc
|
2024-09-04 19:14:05 +02:00
|
|
|
}, '')
|
2024-08-16 15:45:52 +02:00
|
|
|
}
|
|
|
|
|
|
2026-04-30 23:23:36 +02:00
|
|
|
export async function getRandomPort() {
|
|
|
|
|
return new Promise<number>((resolve, reject) => {
|
|
|
|
|
let server = createServer()
|
|
|
|
|
server.unref()
|
|
|
|
|
server.on('error', reject)
|
|
|
|
|
server.listen(0, '127.0.0.1', () => {
|
|
|
|
|
let address = server.address()
|
|
|
|
|
server.close(() => {
|
|
|
|
|
if (address && typeof address === 'object') {
|
|
|
|
|
resolve(address.port)
|
|
|
|
|
} else {
|
|
|
|
|
reject(new Error('Unable to allocate random port'))
|
|
|
|
|
}
|
|
|
|
|
})
|
|
|
|
|
})
|
|
|
|
|
})
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
async function killProcessTree(child: ChildProcess) {
|
|
|
|
|
if (child.exitCode !== null || child.signalCode !== null || child.pid === undefined) {
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (IS_WINDOWS) {
|
|
|
|
|
await execFileAsync('taskkill', ['/pid', String(child.pid), '/T', '/F'], {
|
|
|
|
|
timeout: ASSERTION_TIMEOUT,
|
|
|
|
|
windowsHide: true,
|
|
|
|
|
}).catch(() => {})
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
try {
|
|
|
|
|
process.kill(-child.pid, 'SIGTERM')
|
|
|
|
|
} catch (error: any) {
|
|
|
|
|
if (error?.code !== 'ESRCH') {
|
|
|
|
|
child.kill()
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function forceKillProcessTree(child: ChildProcess) {
|
|
|
|
|
if (child.exitCode !== null || child.signalCode !== null || child.pid === undefined) {
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (IS_WINDOWS) {
|
|
|
|
|
execFile('taskkill', ['/pid', String(child.pid), '/T', '/F'], { windowsHide: true }, () => {})
|
|
|
|
|
return
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
try {
|
|
|
|
|
process.kill(-child.pid, 'SIGKILL')
|
|
|
|
|
} catch (error: any) {
|
|
|
|
|
if (error?.code !== 'ESRCH') {
|
|
|
|
|
child.kill('SIGKILL')
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2024-08-16 15:45:52 +02:00
|
|
|
async function gracefullyRemove(dir: string) {
|
|
|
|
|
// Skip removing the directory in CI because it can stall on Windows
|
|
|
|
|
if (!process.env.CI) {
|
2025-04-03 17:07:38 +02:00
|
|
|
await fs.rm(dir, { recursive: true, force: true }).catch((error) => {
|
|
|
|
|
console.log(`Failed to remove ${dir}`, error)
|
|
|
|
|
})
|
2024-08-16 15:45:52 +02:00
|
|
|
}
|
|
|
|
|
}
|
Add support for source maps (#17775)
Closes #13694
Closes #13591
# Source Maps Support for Tailwind CSS
This PR adds support for source maps to Tailwind CSS v4 allowing us to
track where styles come from whether that be user CSS, imported
stylesheets, or generated utilities. This will improve debuggability in
browser dev tools and gives us a good foundation for producing better
error messages. I'll go over the details on how end users can enable
source maps, any limitations in our implementation, changes to the
internal `compile(…)` API, and some details and reasoning around the
implementation we chose.
## Usage
### CLI
Source maps can be enabled in the CLI by using the command line argument
`--map` which will generate an inline source map comment at the bottom
of your CSS. A separate file may be generated by passing a file name to
`--map`:
```bash
# Generates an inline source map
npx tailwindcss -i input.css -o output.css --map
# Generates a separate source map file
npx tailwindcss -i input.css -o output.css --map output.css.map
```
### PostCSS
Source maps are supported when using Tailwind as a PostCSS plugin *in
development mode only*. They may or may not be enabled by default
depending on your build tool. If they are not you may be able to
configure them within your PostCSS config:
```jsonc
// package.json
{
// …
"postcss": {
"map": { "inline": true },
"plugins": {
"@tailwindcss/postcss": {},
},
}
}
```
### Vite
Source maps are supported when using the Tailwind CSS Vite plugin in
*development mode only* by enabling the `css.devSourcemap` setting:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
devSourcemap: true,
},
})
```
Now when a CSS file is requested by the browser it'll have an inline
source map comment that the browser can use.
## Limitations
- Production build source maps are currently disabled due to a bug in
Lightning CSS. See
https://github.com/parcel-bundler/lightningcss/pull/971 for more
details.
- In Vite, minified CSS build source maps are not supported at all. See
https://github.com/vitejs/vite/issues/2830 for more details.
- In PostCSS, minified CSS source maps are not supported. This is due to
the complexity required around re-associating every AST node with a
location in the generated, optimized CSS. This complexity would also
have a non-trivial performance impact.
## Testing
Here's how to test the source map functionality in different
environments:
### Testing the CLI
1. Setup typical project that the CLI can use and with sources to scan.
```css
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
2. Build with source maps:
```bash
bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map
```
3. Open Chrome DevTools, inspect an element with utility classes, and
you should see rules pointing to `input.css` or
`node_modules/tailwindcss/index.css`
### Testing with Vite
Testing in Vite will require building and installing necessary files
under `dist/*.tgz`.
1. Create a Vite project and enable source maps in `vite.config.js`:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";
export default defineConfig({
plugins: [tailwindcss()],
css: {
// This line is required for them to work
devSourcemap: true,
},
})
```
2. Add a component that uses Tailwind classes and custom CSS:
```jsx
// ./src/app.jsx
export default function App() {
return (
<div className="bg-blue-500 my-custom-class">
Hello World
</div>
)
}
```
```css
/* ./src/styles.css */
@import "tailwindcss";
@utilty my-custom-utility {
color: red;
}
/* to test `@apply` */
.card {
@apply bg-white text-center shadow-md;
}
```
3. Run `npm run dev`, open DevTools, and inspect elements to verify
source mapping works for both utility classes and custom CSS.
### Testing with PostCSS CLI
1. Create a test file and update your PostCSS config:
```css
/* input.css */
@import "tailwindcss";
@layer components {
.card {
@apply p-6 rounded-lg shadow-lg;
}
}
```
```jsonc
// package.json
{
// …
"postcss": {
"map": {
"inline": true
},
"plugins": {
"/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {}
}
}
}
```
2. Run PostCSS through Bun:
```bash
bunx --bun postcss ./src/index.css -o out.css
```
3. Inspect the output CSS - it should include an inline source map
comment at the bottom.
### Testing with PostCSS + Next.js
Testing in Next.js will require building and installing necessary files
under `dist/*.tgz`. However, I've not been able to get CSS source maps
to work in Next.js without this hack:
```js
const nextConfig: NextConfig = {
// next.js overwrites config.devtool so we prevent it from doing so
// please don't actually do this…
webpack: (config) =>
Object.defineProperty(config, "devtool", {
get: () => "inline-source-map",
set: () => {},
}),
};
```
This is definitely not supported and also doesn't work with turbopack.
This can be used to test them temporarily but I suspect that they just
don't work there.
### Manual source map analysis
You can analyze source maps using Evan Wallace's [Source Map
Visualization](https://evanw.github.io/source-map-visualization/) tool
which will help to verify the accuracy and quality of source maps. This
is what I used extensively while developing this implementation.
It'll help verify that custom, user CSS maps back to itself in the
input, that generated utilities all map back to `@tailwind utilities;`,
that source locations from imported files are also handled correctly,
etc… It also highlights the ranges of stuff so it's easy to see if there
are off-by-one errors.
It's easiest to use inline source maps with this tool because you can
take the CSS file and drop it on the page and it'll analyze it while
showing the file content.
If you're using Vite you'll want to access the CSS file with `?direct`
at the end so you don't get a JS module back.
## Implementation
The source map implementation follows the ECMA-426 specification and
includes several key components to aid in that goal:
### Source Location Tracking
Each emittable AST node in the compilation pipeline tracks two types of
source locations:
- `src`: Original source location - [source file, start offset, end
offset]
- `dst`: Generated source location - [output file, start offset, end
offset]
This dual tracking allows us to maintain mappings between the original
source and generated output for things like user CSS, generated
utilities, uses of `@apply`, and tracking theme variables.
It is important to note that source locations for nodes _never overlap_
within a file which helps simplify source map generation. As such each
type of node tracks a specific piece of itself rather than its entire
"block":
| Node | What a `SourceLocation` represents |
| ----------- |
---------------------------------------------------------------- |
| Style Rule | The selector |
| At Rule | Rule name and params, includes the `@` |
| Declaration | Property name and value, excludes the semicolon |
| Comment | The entire comment, includes the start `/*` and end `*/`
markers |
### Windows line endings when parsing CSS
Because our AST tracks nodes through offsets we must ensure that any
mutations to the file do *not* change the lenth of the string. We were
previously replacing `\r\n` with `\n` (see [filter code
points](https://drafts.csswg.org/css-syntax/#css-filter-code-points)
from the spec) — which changes the length of the string and all offsets
may end up incorrect. The CSS parser was updated to handle the CRLF
token directly by skipping over the `\r` and letting remaining code
handle `\n` as it did previously. Some additional tweaks were required
when "peeking" the input but those changes were fairly small.
### Tracking of imports
Source maps need paths to the actual imported stylesheets but the
resolve step for stylesheets happens inside the call to `loadStylesheet`
which make the file path unavailable to us. Because of this the
`loadStylesheet` API was augmented such that it has to return a `path`
property that we can then use to identify imported sources. I've also
made the same change to the `loadModule` API for consistency but nothing
currently uses this property.
The `path` property likely makes `base` redundant but elminating that
(if we even want to) is a future task.
### Optimizing the AST
Our optimization pass may intoduce some nodes, for example, fallbacks we
create for `@property`. These nodes are linked back to `@tailwind
utilities` as ultimately that is what is responsible for creating them.
### Line Offset Tables
A key component to our source map generation is the line offset table,
which was inspired by some ESBuild internals. It stores a sorted list of
offsets for the start of each line allowing us to translate offsets to
line/column `Position`s in `O(log N)` time and from `Position`s to
offsets in `O(1)` time. Creation of the table takes `O(N)` time.
This means that we can store code point offsets for source locations and
not have to worry about computing or tracking line/column numbers during
parsing and serialization. Only when a source map is generated do these
offsets need to be computed. This ensures the performance penalty when
not using source maps is minimal.
### Source Map Generation
The source map returned by `buildSourceMap()` is designed to follow the
[ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not
completely finalized we consider the result of `buildSourceMap()` to be
internal API that may change as the spec chamges.
The produces source map is a "decoded" map such that all sources and
mappings are in an object graph. A library like `source-map-js` must be
used to convert this to an encoded source map of the right version where
mappings are encoded with base 64 VLQs.
Any specific integration (Vite, PostCSS, etc…) can then use
`toSourceMap()` from `@tailwindcss/node` to convert from the internal
source map to an spec-compliant encoded source map that can be
understood by other tools.
### Handling minification in Lightning
Since we use Lightning CSS for optimization, and it takes in an input
map, we generate an encoded source map that we then pass to lightning.
The output source map *from lighting itself* is then passed back in
during the second optimization pass. The final map is then passed from
lightning to the CLI (but not Vite or PostCSS — see the limitations
section for details).
In some cases we have to "fix up" the output CSS. When this happens we
use `magic-string` to do the replacement in a way that is trackable and
`@amppproject/remapping` to map that change back onto the original
source map. Once the need for these fix ups disappear these dependencies
can go away.
Notes:
- The accuracy of source maps run though lightning is reduced as it only
tracks on a per-rule level. This is sufficient enough for browser dev
tools so should be fine.
- Source maps during optimization do not function properly at this time
because of a bug in Lightning CSS regarding license comments. Once this
bug is fixed they will start working as expected.
### How source locations flow through the system
1. During initial CSS parsing, source locations are preserved.
2. During parsing these source locations are also mapped to the
destinations which supports an optimization for when no utilities are
generated.
3. Throughout the compilation process, transformations maintain source
location data
4. Generated utilities are explicitly pointed to `@tailwind utilities`
unless generated by `@apply`.
5. When optimization is enabled, source maps are remapped through
lightningcss
6. Final source maps are written in the requested format (inline or
separate file)
2025-05-08 16:29:49 -04:00
|
|
|
|
|
|
|
|
const SOURCE_MAP_COMMENT = /^\/\*# sourceMappingURL=data:application\/json;base64,(.*) \*\/$/
|
|
|
|
|
|
|
|
|
|
export interface SourceMap {
|
|
|
|
|
at(
|
|
|
|
|
line: number,
|
|
|
|
|
column: number,
|
|
|
|
|
): {
|
|
|
|
|
source: string | null
|
|
|
|
|
original: string
|
|
|
|
|
generated: string
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
interface SourceMapOptions {
|
|
|
|
|
/**
|
|
|
|
|
* A raw source map
|
|
|
|
|
*
|
|
|
|
|
* This may be a string or an object. Strings will be decoded.
|
|
|
|
|
*/
|
|
|
|
|
map: string | object
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The content of the generated file the source map is for
|
|
|
|
|
*/
|
|
|
|
|
content: string
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The encoding of the source map
|
|
|
|
|
*
|
|
|
|
|
* Can be used to decode a base64 map (e.g. an inline source map URI)
|
|
|
|
|
*/
|
|
|
|
|
encoding?: BufferEncoding
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
function parseSourceMap(opts: string | SourceMapOptions): SourceMap {
|
|
|
|
|
if (typeof opts === 'string') {
|
|
|
|
|
let lines = opts.trimEnd().split('\n')
|
|
|
|
|
let comment = lines.at(-1) ?? ''
|
|
|
|
|
let map = String(comment).match(SOURCE_MAP_COMMENT)?.[1] ?? null
|
|
|
|
|
if (!map) throw new Error('No source map comment found')
|
|
|
|
|
|
|
|
|
|
return parseSourceMap({
|
|
|
|
|
map,
|
|
|
|
|
content: lines.slice(0, -1).join('\n'),
|
|
|
|
|
encoding: 'base64',
|
|
|
|
|
})
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let rawMap: RawSourceMap
|
|
|
|
|
let content = opts.content
|
|
|
|
|
|
|
|
|
|
if (typeof opts.map === 'object') {
|
|
|
|
|
rawMap = opts.map as RawSourceMap
|
|
|
|
|
} else {
|
|
|
|
|
rawMap = JSON.parse(Buffer.from(opts.map, opts.encoding ?? 'utf-8').toString())
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
let map = new SourceMapConsumer(rawMap)
|
|
|
|
|
let generatedTable = createLineTable(content)
|
|
|
|
|
|
|
|
|
|
return {
|
|
|
|
|
at(line: number, column: number) {
|
|
|
|
|
let pos = map.originalPositionFor({ line, column })
|
|
|
|
|
let source = pos.source ? map.sourceContentFor(pos.source) : null
|
|
|
|
|
let originalTable = createLineTable(source ?? '')
|
|
|
|
|
let originalOffset = originalTable.findOffset(pos)
|
|
|
|
|
let generatedOffset = generatedTable.findOffset({ line, column })
|
|
|
|
|
|
|
|
|
|
return {
|
|
|
|
|
source: pos.source,
|
|
|
|
|
original: source
|
|
|
|
|
? source.slice(originalOffset, originalOffset + 10).trim() + '...'
|
|
|
|
|
: '(none)',
|
|
|
|
|
generated: content.slice(generatedOffset, generatedOffset + 10).trim() + '...',
|
|
|
|
|
}
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
}
|