Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
import { describe } from 'vitest'
|
2025-04-03 17:13:15 +02:00
|
|
|
import { candidate, css, fetchStyles, js, json, jsx, retryAssertion, test, txt } from '../utils'
|
2024-08-16 15:45:52 +02:00
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
'production build',
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`
|
|
|
|
|
{
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"react": "^18",
|
|
|
|
|
"react-dom": "^18",
|
|
|
|
|
"next": "^14"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
|
|
|
|
"@tailwindcss/postcss": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'postcss.config.mjs': js`
|
2025-02-25 11:36:43 +01:00
|
|
|
export default {
|
2024-08-16 15:45:52 +02:00
|
|
|
plugins: {
|
|
|
|
|
'@tailwindcss/postcss': {},
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
`,
|
2025-02-25 11:36:43 +01:00
|
|
|
'next.config.mjs': js`export default {}`,
|
2024-08-16 15:45:52 +02:00
|
|
|
'app/layout.js': js`
|
|
|
|
|
import './globals.css'
|
|
|
|
|
|
|
|
|
|
export default function RootLayout({ children }) {
|
|
|
|
|
return (
|
|
|
|
|
<html>
|
|
|
|
|
<body>{children}</body>
|
|
|
|
|
</html>
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/page.js': js`
|
2025-02-25 11:36:43 +01:00
|
|
|
import styles from './page.module.css'
|
2024-08-16 15:45:52 +02:00
|
|
|
export default function Page() {
|
2025-02-25 11:36:43 +01:00
|
|
|
return (
|
|
|
|
|
<h1 className={styles.heading + ' text-3xl font-bold underline'}>Hello, Next.js!</h1>
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/page.module.css': css`
|
|
|
|
|
@reference './globals.css';
|
|
|
|
|
.heading {
|
Improve compatibility with Safari 15 (#17435)
This PR improves the compatibility with Tailwind CSS v4 with unsupported
browsers with the goal to greatly improve compatibility with Safari 15.
To make this work, this PR makes the following changes to all code
- Change `oklab(…)` default theme values to use a percentage in the
first place (so instead of `--color-red-500: oklch(0.637 0.237 25.331);`
we now define it as `--color-red-500: oklch(63.7% 0.237 25.331);` since
this syntax has much broader support on Safari).
- Polyfill `@property` with a `@supports` query targeting older versions
of Safari and Firefox *
- Create fallbacks for the `color-mix(…)` function that use _inlined
color values from your theme_ so that they can be computed a compile
time by `lightningcss`. These fallbacks will convert to srgb to increase
compatibility.
- Create fallbacks for the _relative color_ feature used in the new
shadow utilities and using `color-mix(…)` in case _relative color_ is
applied on `currentcolor` (due to limited browser support)
- Create fallbacks for gradient interpolation methods (e.g. to support
`bg-linear-to-r/oklab`)
- Polyfill `@media` queries range syntax.
## A simplified example
Given this example CSS input:
```css
@import 'tailwindcss';
@source inline('from-cyan-500/50 bg-linear-45');
```
Here's the updated output CSS including the newly added polyfills and
updated `oklab` values:
```css
.bg-linear-45 {
--tw-gradient-position: 45deg;
background-image: linear-gradient(var(--tw-gradient-stops));
}
@supports (background-image: linear-gradient(in lab, red, red)) {
.bg-linear-45 {
--tw-gradient-position: 45deg in oklab;
}
}
.from-cyan-500\\/50 {
--tw-gradient-from: oklab(71.5% -.11682 -.08247 / .5);
--tw-gradient-stops: var(--tw-gradient-via-stops, var(--tw-gradient-position), var(--tw-gradient-from) var(--tw-gradient-from-position), var(--tw-gradient-to) var(--tw-gradient-to-position));
}
@supports (color: color-mix(in lab, red, red)) {
.from-cyan-500\\/50 {
--tw-gradient-from: color-mix(in oklab, var(--color-cyan-500) 50%, transparent);
}
}
:root, :host {
--color-cyan-500: oklch(71.5% .143 215.221);
}
@supports (((-webkit-hyphens: none)) and (not (margin-trim: 1lh))) or ((-moz-orient: inline) and (not (color: rgb(from red r g b)))) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
--tw-gradient-from: #0000;
--tw-gradient-via: #0000;
--tw-gradient-to: #0000;
--tw-gradient-stops: initial;
--tw-gradient-via-stops: initial;
--tw-gradient-from-position: 0%;
--tw-gradient-via-position: 50%;
--tw-gradient-to-position: 100%;
}
}
}
@property --tw-gradient-position {
syntax: "*";
inherits: false
}
@property --tw-gradient-from {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-via {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-to {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-via-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-from-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 0%;
}
@property --tw-gradient-via-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 50%;
}
@property --tw-gradient-to-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 100%;
}
```
## \* A note on `@property` polyfills and CSS modules
On Next.js, CSS module files are required to be _pure_, meaning that all
selectors must either be scoped to a class or an ID. Fortunatnyl for us,
this does not apply to `@property` rules which we've been using before
to initialize CSS variables.
However, since we're now bringing back the `@property` polyfills, that
would cause unexpected rules to be exported from the CSS file as this:
```css
@reference "tailwindcss";
.skew {
@apply skew-7;
}
```
Would turn to the following file:
```css
.skew {
/* … */
}
@supports (/*…*/) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
}
}
}
@property /* … */
```
Notice that this adds a `*` selector which is not considered pure.
Unfortunately there is no way for us to silence this warning or work
around it, as the dependency causing this errors
([`postcss-modules-local-by-default`](https://github.com/css-modules/postcss-modules-local-by-default))
is bundled into Next.js. To work around crashes, these polyfills will
not apply to CSS modules processed by the PostCSS extension for now.
## Testing on tailwindcss.com
To see the changes in effect, take a look at this screencast that
compares tailwindcss.com on iOS 15.5 with a version that has the patches
of this PR applied:
https://github.com/user-attachments/assets/1279d6f5-3c63-4f30-839c-198a789f4292
## Test plan
- Tested on tailwindcss.com via a preview build:
https://tailwindcss-com-git-legacy-browsers-tailwindlabs.vercel.app/
- Updated tests
- Ensure we also test on Chrome 111, Safari 16.4, Firefox 128 to
make sure we have no regressions. Also tested on Safari 16.4, 15.5, 18.0
2025-04-01 13:33:22 +02:00
|
|
|
@apply text-red-500 animate-ping skew-7;
|
2024-08-16 15:45:52 +02:00
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/globals.css': css`
|
2025-02-25 11:36:43 +01:00
|
|
|
@reference 'tailwindcss/theme';
|
2024-08-16 15:45:52 +02:00
|
|
|
@import 'tailwindcss/utilities';
|
|
|
|
|
`,
|
|
|
|
|
},
|
|
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
async ({ fs, exec, expect }) => {
|
2024-08-16 15:45:52 +02:00
|
|
|
await exec('pnpm next build')
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('.next/static/css/**/*.css')
|
2025-02-25 11:36:43 +01:00
|
|
|
expect(files).toHaveLength(2)
|
2024-08-16 15:45:52 +02:00
|
|
|
|
2025-02-25 11:36:43 +01:00
|
|
|
let globalCss: string | null = null
|
|
|
|
|
let moduleCss: string | null = null
|
|
|
|
|
for (let [filename, content] of files) {
|
|
|
|
|
if (content.includes('@keyframes page_ping')) moduleCss = filename
|
|
|
|
|
else globalCss = filename
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
await fs.expectFileToContain(globalCss!, [
|
2024-08-16 15:45:52 +02:00
|
|
|
candidate`underline`,
|
|
|
|
|
candidate`font-bold`,
|
|
|
|
|
candidate`text-3xl`,
|
|
|
|
|
])
|
2025-02-25 11:36:43 +01:00
|
|
|
|
|
|
|
|
await fs.expectFileToContain(moduleCss!, [
|
Improve compatibility with Safari 15 (#17435)
This PR improves the compatibility with Tailwind CSS v4 with unsupported
browsers with the goal to greatly improve compatibility with Safari 15.
To make this work, this PR makes the following changes to all code
- Change `oklab(…)` default theme values to use a percentage in the
first place (so instead of `--color-red-500: oklch(0.637 0.237 25.331);`
we now define it as `--color-red-500: oklch(63.7% 0.237 25.331);` since
this syntax has much broader support on Safari).
- Polyfill `@property` with a `@supports` query targeting older versions
of Safari and Firefox *
- Create fallbacks for the `color-mix(…)` function that use _inlined
color values from your theme_ so that they can be computed a compile
time by `lightningcss`. These fallbacks will convert to srgb to increase
compatibility.
- Create fallbacks for the _relative color_ feature used in the new
shadow utilities and using `color-mix(…)` in case _relative color_ is
applied on `currentcolor` (due to limited browser support)
- Create fallbacks for gradient interpolation methods (e.g. to support
`bg-linear-to-r/oklab`)
- Polyfill `@media` queries range syntax.
## A simplified example
Given this example CSS input:
```css
@import 'tailwindcss';
@source inline('from-cyan-500/50 bg-linear-45');
```
Here's the updated output CSS including the newly added polyfills and
updated `oklab` values:
```css
.bg-linear-45 {
--tw-gradient-position: 45deg;
background-image: linear-gradient(var(--tw-gradient-stops));
}
@supports (background-image: linear-gradient(in lab, red, red)) {
.bg-linear-45 {
--tw-gradient-position: 45deg in oklab;
}
}
.from-cyan-500\\/50 {
--tw-gradient-from: oklab(71.5% -.11682 -.08247 / .5);
--tw-gradient-stops: var(--tw-gradient-via-stops, var(--tw-gradient-position), var(--tw-gradient-from) var(--tw-gradient-from-position), var(--tw-gradient-to) var(--tw-gradient-to-position));
}
@supports (color: color-mix(in lab, red, red)) {
.from-cyan-500\\/50 {
--tw-gradient-from: color-mix(in oklab, var(--color-cyan-500) 50%, transparent);
}
}
:root, :host {
--color-cyan-500: oklch(71.5% .143 215.221);
}
@supports (((-webkit-hyphens: none)) and (not (margin-trim: 1lh))) or ((-moz-orient: inline) and (not (color: rgb(from red r g b)))) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
--tw-gradient-from: #0000;
--tw-gradient-via: #0000;
--tw-gradient-to: #0000;
--tw-gradient-stops: initial;
--tw-gradient-via-stops: initial;
--tw-gradient-from-position: 0%;
--tw-gradient-via-position: 50%;
--tw-gradient-to-position: 100%;
}
}
}
@property --tw-gradient-position {
syntax: "*";
inherits: false
}
@property --tw-gradient-from {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-via {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-to {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-via-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-from-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 0%;
}
@property --tw-gradient-via-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 50%;
}
@property --tw-gradient-to-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 100%;
}
```
## \* A note on `@property` polyfills and CSS modules
On Next.js, CSS module files are required to be _pure_, meaning that all
selectors must either be scoped to a class or an ID. Fortunatnyl for us,
this does not apply to `@property` rules which we've been using before
to initialize CSS variables.
However, since we're now bringing back the `@property` polyfills, that
would cause unexpected rules to be exported from the CSS file as this:
```css
@reference "tailwindcss";
.skew {
@apply skew-7;
}
```
Would turn to the following file:
```css
.skew {
/* … */
}
@supports (/*…*/) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
}
}
}
@property /* … */
```
Notice that this adds a `*` selector which is not considered pure.
Unfortunately there is no way for us to silence this warning or work
around it, as the dependency causing this errors
([`postcss-modules-local-by-default`](https://github.com/css-modules/postcss-modules-local-by-default))
is bundled into Next.js. To work around crashes, these polyfills will
not apply to CSS modules processed by the PostCSS extension for now.
## Testing on tailwindcss.com
To see the changes in effect, take a look at this screencast that
compares tailwindcss.com on iOS 15.5 with a version that has the patches
of this PR applied:
https://github.com/user-attachments/assets/1279d6f5-3c63-4f30-839c-198a789f4292
## Test plan
- Tested on tailwindcss.com via a preview build:
https://tailwindcss-com-git-legacy-browsers-tailwindlabs.vercel.app/
- Updated tests
- Ensure we also test on Chrome 111, Safari 16.4, Firefox 128 to
make sure we have no regressions. Also tested on Safari 16.4, 15.5, 18.0
2025-04-01 13:33:22 +02:00
|
|
|
'color:var(--color-red-500,oklch(63.7% .237 25.331)',
|
2025-02-25 11:36:43 +01:00
|
|
|
'animation:var(--animate-ping,ping 1s cubic-bezier(0,0,.2,1) infinite)',
|
|
|
|
|
/@keyframes page_ping.*{75%,to{transform:scale\(2\);opacity:0}/,
|
Improve compatibility with Safari 15 (#17435)
This PR improves the compatibility with Tailwind CSS v4 with unsupported
browsers with the goal to greatly improve compatibility with Safari 15.
To make this work, this PR makes the following changes to all code
- Change `oklab(…)` default theme values to use a percentage in the
first place (so instead of `--color-red-500: oklch(0.637 0.237 25.331);`
we now define it as `--color-red-500: oklch(63.7% 0.237 25.331);` since
this syntax has much broader support on Safari).
- Polyfill `@property` with a `@supports` query targeting older versions
of Safari and Firefox *
- Create fallbacks for the `color-mix(…)` function that use _inlined
color values from your theme_ so that they can be computed a compile
time by `lightningcss`. These fallbacks will convert to srgb to increase
compatibility.
- Create fallbacks for the _relative color_ feature used in the new
shadow utilities and using `color-mix(…)` in case _relative color_ is
applied on `currentcolor` (due to limited browser support)
- Create fallbacks for gradient interpolation methods (e.g. to support
`bg-linear-to-r/oklab`)
- Polyfill `@media` queries range syntax.
## A simplified example
Given this example CSS input:
```css
@import 'tailwindcss';
@source inline('from-cyan-500/50 bg-linear-45');
```
Here's the updated output CSS including the newly added polyfills and
updated `oklab` values:
```css
.bg-linear-45 {
--tw-gradient-position: 45deg;
background-image: linear-gradient(var(--tw-gradient-stops));
}
@supports (background-image: linear-gradient(in lab, red, red)) {
.bg-linear-45 {
--tw-gradient-position: 45deg in oklab;
}
}
.from-cyan-500\\/50 {
--tw-gradient-from: oklab(71.5% -.11682 -.08247 / .5);
--tw-gradient-stops: var(--tw-gradient-via-stops, var(--tw-gradient-position), var(--tw-gradient-from) var(--tw-gradient-from-position), var(--tw-gradient-to) var(--tw-gradient-to-position));
}
@supports (color: color-mix(in lab, red, red)) {
.from-cyan-500\\/50 {
--tw-gradient-from: color-mix(in oklab, var(--color-cyan-500) 50%, transparent);
}
}
:root, :host {
--color-cyan-500: oklch(71.5% .143 215.221);
}
@supports (((-webkit-hyphens: none)) and (not (margin-trim: 1lh))) or ((-moz-orient: inline) and (not (color: rgb(from red r g b)))) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
--tw-gradient-from: #0000;
--tw-gradient-via: #0000;
--tw-gradient-to: #0000;
--tw-gradient-stops: initial;
--tw-gradient-via-stops: initial;
--tw-gradient-from-position: 0%;
--tw-gradient-via-position: 50%;
--tw-gradient-to-position: 100%;
}
}
}
@property --tw-gradient-position {
syntax: "*";
inherits: false
}
@property --tw-gradient-from {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-via {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-to {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-via-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-from-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 0%;
}
@property --tw-gradient-via-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 50%;
}
@property --tw-gradient-to-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 100%;
}
```
## \* A note on `@property` polyfills and CSS modules
On Next.js, CSS module files are required to be _pure_, meaning that all
selectors must either be scoped to a class or an ID. Fortunatnyl for us,
this does not apply to `@property` rules which we've been using before
to initialize CSS variables.
However, since we're now bringing back the `@property` polyfills, that
would cause unexpected rules to be exported from the CSS file as this:
```css
@reference "tailwindcss";
.skew {
@apply skew-7;
}
```
Would turn to the following file:
```css
.skew {
/* … */
}
@supports (/*…*/) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
}
}
}
@property /* … */
```
Notice that this adds a `*` selector which is not considered pure.
Unfortunately there is no way for us to silence this warning or work
around it, as the dependency causing this errors
([`postcss-modules-local-by-default`](https://github.com/css-modules/postcss-modules-local-by-default))
is bundled into Next.js. To work around crashes, these polyfills will
not apply to CSS modules processed by the PostCSS extension for now.
## Testing on tailwindcss.com
To see the changes in effect, take a look at this screencast that
compares tailwindcss.com on iOS 15.5 with a version that has the patches
of this PR applied:
https://github.com/user-attachments/assets/1279d6f5-3c63-4f30-839c-198a789f4292
## Test plan
- Tested on tailwindcss.com via a preview build:
https://tailwindcss-com-git-legacy-browsers-tailwindlabs.vercel.app/
- Updated tests
- Ensure we also test on Chrome 111, Safari 16.4, Firefox 128 to
make sure we have no regressions. Also tested on Safari 16.4, 15.5, 18.0
2025-04-01 13:33:22 +02:00
|
|
|
'--tw-skew-x:skewX(7deg);',
|
2025-02-25 11:36:43 +01:00
|
|
|
])
|
2024-08-16 15:45:52 +02:00
|
|
|
},
|
|
|
|
|
)
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
|
|
|
|
|
describe.each(['turbo', 'webpack'])('%s', (bundler) => {
|
2024-08-16 15:45:52 +02:00
|
|
|
test(
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
'dev mode',
|
2024-08-16 15:45:52 +02:00
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`
|
|
|
|
|
{
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"react": "^18",
|
|
|
|
|
"react-dom": "^18",
|
|
|
|
|
"next": "^14"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
|
|
|
|
"@tailwindcss/postcss": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'postcss.config.mjs': js`
|
2025-02-25 11:36:43 +01:00
|
|
|
export default {
|
2024-08-16 15:45:52 +02:00
|
|
|
plugins: {
|
|
|
|
|
'@tailwindcss/postcss': {},
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
`,
|
2025-02-25 11:36:43 +01:00
|
|
|
'next.config.mjs': js`export default {}`,
|
2024-08-16 15:45:52 +02:00
|
|
|
'app/layout.js': js`
|
|
|
|
|
import './globals.css'
|
|
|
|
|
|
|
|
|
|
export default function RootLayout({ children }) {
|
|
|
|
|
return (
|
|
|
|
|
<html>
|
|
|
|
|
<body>{children}</body>
|
|
|
|
|
</html>
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/page.js': js`
|
2025-02-25 11:36:43 +01:00
|
|
|
import styles from './page.module.css'
|
2024-08-16 15:45:52 +02:00
|
|
|
export default function Page() {
|
2025-02-25 11:36:43 +01:00
|
|
|
return <h1 className={styles.heading + ' underline'}>Hello, Next.js!</h1>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/page.module.css': css`
|
|
|
|
|
@reference './globals.css';
|
|
|
|
|
.heading {
|
Improve compatibility with Safari 15 (#17435)
This PR improves the compatibility with Tailwind CSS v4 with unsupported
browsers with the goal to greatly improve compatibility with Safari 15.
To make this work, this PR makes the following changes to all code
- Change `oklab(…)` default theme values to use a percentage in the
first place (so instead of `--color-red-500: oklch(0.637 0.237 25.331);`
we now define it as `--color-red-500: oklch(63.7% 0.237 25.331);` since
this syntax has much broader support on Safari).
- Polyfill `@property` with a `@supports` query targeting older versions
of Safari and Firefox *
- Create fallbacks for the `color-mix(…)` function that use _inlined
color values from your theme_ so that they can be computed a compile
time by `lightningcss`. These fallbacks will convert to srgb to increase
compatibility.
- Create fallbacks for the _relative color_ feature used in the new
shadow utilities and using `color-mix(…)` in case _relative color_ is
applied on `currentcolor` (due to limited browser support)
- Create fallbacks for gradient interpolation methods (e.g. to support
`bg-linear-to-r/oklab`)
- Polyfill `@media` queries range syntax.
## A simplified example
Given this example CSS input:
```css
@import 'tailwindcss';
@source inline('from-cyan-500/50 bg-linear-45');
```
Here's the updated output CSS including the newly added polyfills and
updated `oklab` values:
```css
.bg-linear-45 {
--tw-gradient-position: 45deg;
background-image: linear-gradient(var(--tw-gradient-stops));
}
@supports (background-image: linear-gradient(in lab, red, red)) {
.bg-linear-45 {
--tw-gradient-position: 45deg in oklab;
}
}
.from-cyan-500\\/50 {
--tw-gradient-from: oklab(71.5% -.11682 -.08247 / .5);
--tw-gradient-stops: var(--tw-gradient-via-stops, var(--tw-gradient-position), var(--tw-gradient-from) var(--tw-gradient-from-position), var(--tw-gradient-to) var(--tw-gradient-to-position));
}
@supports (color: color-mix(in lab, red, red)) {
.from-cyan-500\\/50 {
--tw-gradient-from: color-mix(in oklab, var(--color-cyan-500) 50%, transparent);
}
}
:root, :host {
--color-cyan-500: oklch(71.5% .143 215.221);
}
@supports (((-webkit-hyphens: none)) and (not (margin-trim: 1lh))) or ((-moz-orient: inline) and (not (color: rgb(from red r g b)))) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
--tw-gradient-from: #0000;
--tw-gradient-via: #0000;
--tw-gradient-to: #0000;
--tw-gradient-stops: initial;
--tw-gradient-via-stops: initial;
--tw-gradient-from-position: 0%;
--tw-gradient-via-position: 50%;
--tw-gradient-to-position: 100%;
}
}
}
@property --tw-gradient-position {
syntax: "*";
inherits: false
}
@property --tw-gradient-from {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-via {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-to {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-via-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-from-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 0%;
}
@property --tw-gradient-via-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 50%;
}
@property --tw-gradient-to-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 100%;
}
```
## \* A note on `@property` polyfills and CSS modules
On Next.js, CSS module files are required to be _pure_, meaning that all
selectors must either be scoped to a class or an ID. Fortunatnyl for us,
this does not apply to `@property` rules which we've been using before
to initialize CSS variables.
However, since we're now bringing back the `@property` polyfills, that
would cause unexpected rules to be exported from the CSS file as this:
```css
@reference "tailwindcss";
.skew {
@apply skew-7;
}
```
Would turn to the following file:
```css
.skew {
/* … */
}
@supports (/*…*/) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
}
}
}
@property /* … */
```
Notice that this adds a `*` selector which is not considered pure.
Unfortunately there is no way for us to silence this warning or work
around it, as the dependency causing this errors
([`postcss-modules-local-by-default`](https://github.com/css-modules/postcss-modules-local-by-default))
is bundled into Next.js. To work around crashes, these polyfills will
not apply to CSS modules processed by the PostCSS extension for now.
## Testing on tailwindcss.com
To see the changes in effect, take a look at this screencast that
compares tailwindcss.com on iOS 15.5 with a version that has the patches
of this PR applied:
https://github.com/user-attachments/assets/1279d6f5-3c63-4f30-839c-198a789f4292
## Test plan
- Tested on tailwindcss.com via a preview build:
https://tailwindcss-com-git-legacy-browsers-tailwindlabs.vercel.app/
- Updated tests
- Ensure we also test on Chrome 111, Safari 16.4, Firefox 128 to
make sure we have no regressions. Also tested on Safari 16.4, 15.5, 18.0
2025-04-01 13:33:22 +02:00
|
|
|
@apply text-red-500 animate-ping skew-7 content-['module'];
|
2024-08-16 15:45:52 +02:00
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/globals.css': css`
|
2025-02-25 11:36:43 +01:00
|
|
|
@reference 'tailwindcss/theme';
|
2024-08-16 15:45:52 +02:00
|
|
|
@import 'tailwindcss/utilities';
|
|
|
|
|
`,
|
|
|
|
|
},
|
|
|
|
|
},
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
async ({ fs, spawn, expect }) => {
|
|
|
|
|
let process = await spawn(`pnpm next dev ${bundler === 'turbo' ? '--turbo' : ''}`)
|
|
|
|
|
|
|
|
|
|
let url = ''
|
|
|
|
|
await process.onStdout((m) => {
|
|
|
|
|
let match = /Local:\s*(http.*)/.exec(m)
|
|
|
|
|
if (match) url = match[1]
|
|
|
|
|
return Boolean(url)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await process.onStdout((m) => m.includes('Ready in'))
|
2024-08-16 15:45:52 +02:00
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
let css = await fetchStyles(url)
|
2024-08-16 15:45:52 +02:00
|
|
|
expect(css).toContain(candidate`underline`)
|
2025-02-25 11:36:43 +01:00
|
|
|
expect(css).toContain('content: var(--tw-content)')
|
|
|
|
|
expect(css).toContain('@keyframes')
|
2024-08-16 15:45:52 +02:00
|
|
|
})
|
|
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
await fs.write(
|
|
|
|
|
'app/page.js',
|
|
|
|
|
js`
|
2025-02-25 11:36:43 +01:00
|
|
|
import styles from './page.module.css'
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
export default function Page() {
|
2025-02-25 11:36:43 +01:00
|
|
|
return <h1 className={styles.heading + ' underline bg-red-500'}>Hello, Next.js!</h1>
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
)
|
|
|
|
|
await process.onStdout((m) => m.includes('Compiled in'))
|
2024-09-06 10:48:49 +02:00
|
|
|
|
Improve integration tests (stability + performance) (#15125)
This PR improves the integration tests in two ways:
1. Make the integration tests more reliable and thus less flakey
2. Make the integration tests faster (by introducing concurrency)
Tried a lot of different things to make sure that these tests are fast
and stable.
---
The biggest issue we noticed is that some tests are flakey, these are
tests with long running dev-mode processes where watchers are being used
and/or dev servers are created.
To solve this, all the tests that spawn a process look at stdout/stderr
and wait for a message from the process to know whether we can start
making changes.
For example, in case of an Astro project, you get a `watching for file
changes` message. In case of Nuxt project you can wait for an `server
warmed up in` and in case of Next.js there is a `Ready in` message.
These depend on the tools being used, so this is hardcoded per test
instead of a magically automatic solution.
These messages allow us to wait until all the initial necessary work,
internal watchers and/or dev servers are setup before we start making
changes to the files and/or request CSS stylesheets before the server(s)
are ready.
---
Another improvement is how we setup the dev servers. Before, we used to
try and get a free port on the system and use a `--port` flag or a
`PORT` environment variable. Instead of doing this (which is slow), we
rely on the process itself to show a URL with a port. Basically all
tools will try to find a free port if the default port is in use. We can
then use the stdout/stderr messages to get the URL and the port to use.
To reduce the amount of potential conflicts in ports, we used to run
every test and every file sequentially to basically guarantee that ports
are free. With this new approach where we rely on the process, I noticed
that we don't really run into this issue again (I reran the tests
multiple times and they were always stable)
<img width="316" alt="image"
src="https://github.com/user-attachments/assets/b75ddab4-f919-4995-85d0-f212b603e5c2"
/>
Note: these tests run Linux, Windows and macOS in this branch just for
testing purposes. Once this is done, we will only run Linux tests on PRs
and run all 3 of them on the `next` branch.
We do make the tests concurrent by default now, which in theory means
that there could be conflicts (which in practice means that the process
has to do a few more tries to find a free port). To reduce these
conflicts, we split up the integration tests such that Vite, PostCSS,
CLI, … tests all run in a separate job in the GitHub actions workflow.
<img width="312" alt="image"
src="https://github.com/user-attachments/assets/fe9a58a1-98eb-4d9b-8845-a7c8a7af5766"
/>
Comparing this branch against the `next` branch, this is what CI looks
like right now:
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="594" alt="image"
src="https://github.com/user-attachments/assets/540d21eb-ab03-42e8-9f6f-b3a071fc7635"
/> | <img width="672" alt="image"
src="https://github.com/user-attachments/assets/8ef2e891-08a1-464b-9954-4153174ebce7"
/> |
There also was a point in time where I introduced sequential tests such
that all spawned processes still run after each other, but so far I
didn't run into issues if we keep them concurrent so I dropped that
code.
Some small changes I made to make things more reliable:
1. When relying on stdout/stderr messages, we split lines on `\n` and we
strip all the ANSI escapes which allows us to not worry about special
ANSI characters when finding the URL or a specific message to wait for.
2. Once a test is done, we `child.kill()` the spawned process. If that
doesn't work, for whatever reason, we run a `child.kill('SIGKILL')` to
force kill the process. This could technically lead to some memory or
files not being cleaned up properly, but once CI is done, everything is
thrown away anyway.
3. As you can see in the screenshots, I used some nicer names for the
workflows.
| `next` | `feat/improve-integration-tests` |
| --- | --- |
| <img width="276" alt="image"
src="https://github.com/user-attachments/assets/e574bb53-e21b-4619-9cdb-515431b255b9"
/> | <img width="179" alt="image"
src="https://github.com/user-attachments/assets/8bc75119-fb91-4500-a1d0-bd09f74c93ad"
/> |
They also look a bit nicer in the PR overview as well:
<img width="929" alt="image"
src="https://github.com/user-attachments/assets/04fc71fc-74b0-4e7c-9047-2aada664efef"
/>
The very last commit just filters out Windows and macOS tests again for
PRs (but they are executed on the `next` branch.
---
### Nest steps
I think for now we are in a pretty good state, but there are some things
we can do to further improve everything (mainly make things faster) but
aren't necessary. I also ran into issue while trying it so there is more
work to do.
1. More splits — instead of having a Vite folder and PostCSS folder, we
can go a step further and have folders for Next.js, Astro, Nuxt, Remix,
…
2. Caching — right now we have to run the build step for every OS on
every "job". We can re-use the work here by introducing a setup job that
the other jobs rely on. @thecrypticace and I tried it already, but were
running into some Bun specific Standalone CLI issues when doing that.
3. Remote caching — we could re-enable remote caching such that the
`build` step can be full turbo (e.g.: after a PR is merged in `next` and
we run everything again)
2024-12-12 13:48:56 +01:00
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
let css = await fetchStyles(url)
|
2024-08-16 15:45:52 +02:00
|
|
|
expect(css).toContain(candidate`underline`)
|
2025-02-25 11:36:43 +01:00
|
|
|
expect(css).toContain(candidate`bg-red-500`)
|
Improve compatibility with Safari 15 (#17435)
This PR improves the compatibility with Tailwind CSS v4 with unsupported
browsers with the goal to greatly improve compatibility with Safari 15.
To make this work, this PR makes the following changes to all code
- Change `oklab(…)` default theme values to use a percentage in the
first place (so instead of `--color-red-500: oklch(0.637 0.237 25.331);`
we now define it as `--color-red-500: oklch(63.7% 0.237 25.331);` since
this syntax has much broader support on Safari).
- Polyfill `@property` with a `@supports` query targeting older versions
of Safari and Firefox *
- Create fallbacks for the `color-mix(…)` function that use _inlined
color values from your theme_ so that they can be computed a compile
time by `lightningcss`. These fallbacks will convert to srgb to increase
compatibility.
- Create fallbacks for the _relative color_ feature used in the new
shadow utilities and using `color-mix(…)` in case _relative color_ is
applied on `currentcolor` (due to limited browser support)
- Create fallbacks for gradient interpolation methods (e.g. to support
`bg-linear-to-r/oklab`)
- Polyfill `@media` queries range syntax.
## A simplified example
Given this example CSS input:
```css
@import 'tailwindcss';
@source inline('from-cyan-500/50 bg-linear-45');
```
Here's the updated output CSS including the newly added polyfills and
updated `oklab` values:
```css
.bg-linear-45 {
--tw-gradient-position: 45deg;
background-image: linear-gradient(var(--tw-gradient-stops));
}
@supports (background-image: linear-gradient(in lab, red, red)) {
.bg-linear-45 {
--tw-gradient-position: 45deg in oklab;
}
}
.from-cyan-500\\/50 {
--tw-gradient-from: oklab(71.5% -.11682 -.08247 / .5);
--tw-gradient-stops: var(--tw-gradient-via-stops, var(--tw-gradient-position), var(--tw-gradient-from) var(--tw-gradient-from-position), var(--tw-gradient-to) var(--tw-gradient-to-position));
}
@supports (color: color-mix(in lab, red, red)) {
.from-cyan-500\\/50 {
--tw-gradient-from: color-mix(in oklab, var(--color-cyan-500) 50%, transparent);
}
}
:root, :host {
--color-cyan-500: oklch(71.5% .143 215.221);
}
@supports (((-webkit-hyphens: none)) and (not (margin-trim: 1lh))) or ((-moz-orient: inline) and (not (color: rgb(from red r g b)))) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
--tw-gradient-from: #0000;
--tw-gradient-via: #0000;
--tw-gradient-to: #0000;
--tw-gradient-stops: initial;
--tw-gradient-via-stops: initial;
--tw-gradient-from-position: 0%;
--tw-gradient-via-position: 50%;
--tw-gradient-to-position: 100%;
}
}
}
@property --tw-gradient-position {
syntax: "*";
inherits: false
}
@property --tw-gradient-from {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-via {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-to {
syntax: "<color>";
inherits: false;
initial-value: #0000;
}
@property --tw-gradient-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-via-stops {
syntax: "*";
inherits: false
}
@property --tw-gradient-from-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 0%;
}
@property --tw-gradient-via-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 50%;
}
@property --tw-gradient-to-position {
syntax: "<length-percentage>";
inherits: false;
initial-value: 100%;
}
```
## \* A note on `@property` polyfills and CSS modules
On Next.js, CSS module files are required to be _pure_, meaning that all
selectors must either be scoped to a class or an ID. Fortunatnyl for us,
this does not apply to `@property` rules which we've been using before
to initialize CSS variables.
However, since we're now bringing back the `@property` polyfills, that
would cause unexpected rules to be exported from the CSS file as this:
```css
@reference "tailwindcss";
.skew {
@apply skew-7;
}
```
Would turn to the following file:
```css
.skew {
/* … */
}
@supports (/*…*/) {
@layer base {
*, :before, :after, ::backdrop {
--tw-gradient-position: initial;
}
}
}
@property /* … */
```
Notice that this adds a `*` selector which is not considered pure.
Unfortunately there is no way for us to silence this warning or work
around it, as the dependency causing this errors
([`postcss-modules-local-by-default`](https://github.com/css-modules/postcss-modules-local-by-default))
is bundled into Next.js. To work around crashes, these polyfills will
not apply to CSS modules processed by the PostCSS extension for now.
## Testing on tailwindcss.com
To see the changes in effect, take a look at this screencast that
compares tailwindcss.com on iOS 15.5 with a version that has the patches
of this PR applied:
https://github.com/user-attachments/assets/1279d6f5-3c63-4f30-839c-198a789f4292
## Test plan
- Tested on tailwindcss.com via a preview build:
https://tailwindcss-com-git-legacy-browsers-tailwindlabs.vercel.app/
- Updated tests
- Ensure we also test on Chrome 111, Safari 16.4, Firefox 128 to
make sure we have no regressions. Also tested on Safari 16.4, 15.5, 18.0
2025-04-01 13:33:22 +02:00
|
|
|
expect(css).toContain('--tw-skew-x: skewX(7deg);')
|
2025-02-25 11:36:43 +01:00
|
|
|
expect(css).toContain('content: var(--tw-content)')
|
|
|
|
|
expect(css).toContain('@keyframes')
|
2024-08-16 15:45:52 +02:00
|
|
|
})
|
|
|
|
|
},
|
|
|
|
|
)
|
|
|
|
|
})
|
2025-02-18 11:51:58 +01:00
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
'should scan dynamic route segments',
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`
|
|
|
|
|
{
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"react": "^18",
|
|
|
|
|
"react-dom": "^18",
|
|
|
|
|
"next": "^14"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
|
|
|
|
"@tailwindcss/postcss": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'postcss.config.mjs': js`
|
2025-02-25 11:36:43 +01:00
|
|
|
export default {
|
2025-02-18 11:51:58 +01:00
|
|
|
plugins: {
|
|
|
|
|
'@tailwindcss/postcss': {},
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
`,
|
2025-02-25 11:36:43 +01:00
|
|
|
'next.config.mjs': js`export default {}`,
|
2025-02-18 11:51:58 +01:00
|
|
|
'app/a/[slug]/page.js': js`
|
|
|
|
|
export default function Page() {
|
|
|
|
|
return <h1 className="content-['[slug]']">Hello, Next.js!</h1>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/b/[...slug]/page.js': js`
|
|
|
|
|
export default function Page() {
|
|
|
|
|
return <h1 className="content-['[...slug]']">Hello, Next.js!</h1>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/c/[[...slug]]/page.js': js`
|
|
|
|
|
export default function Page() {
|
|
|
|
|
return <h1 className="content-['[[...slug]]']">Hello, Next.js!</h1>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/d/(theme)/page.js': js`
|
|
|
|
|
export default function Page() {
|
|
|
|
|
return <h1 className="content-['(theme)']">Hello, Next.js!</h1>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/layout.js': js`
|
|
|
|
|
import './globals.css'
|
|
|
|
|
|
|
|
|
|
export default function RootLayout({ children }) {
|
|
|
|
|
return (
|
|
|
|
|
<html>
|
|
|
|
|
<body>{children}</body>
|
|
|
|
|
</html>
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/globals.css': css`
|
|
|
|
|
@import 'tailwindcss/utilities' source(none);
|
|
|
|
|
@source './**/*.{js,ts,jsx,tsx,mdx}';
|
|
|
|
|
`,
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
async ({ fs, exec, expect }) => {
|
|
|
|
|
await exec('pnpm next build')
|
|
|
|
|
|
|
|
|
|
let files = await fs.glob('.next/static/css/**/*.css')
|
|
|
|
|
expect(files).toHaveLength(1)
|
|
|
|
|
let [filename] = files[0]
|
|
|
|
|
|
|
|
|
|
await fs.expectFileToContain(filename, [
|
|
|
|
|
candidate`content-['[slug]']`,
|
|
|
|
|
candidate`content-['[...slug]']`,
|
|
|
|
|
candidate`content-['[[...slug]]']`,
|
|
|
|
|
candidate`content-['(theme)']`,
|
|
|
|
|
])
|
|
|
|
|
},
|
|
|
|
|
)
|
2025-03-31 18:44:06 +02:00
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
'changes to CSS files should pick up new CSS variables (if any)',
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`
|
|
|
|
|
{
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"react": "^18",
|
|
|
|
|
"react-dom": "^18",
|
|
|
|
|
"next": "^14"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
|
|
|
|
"@tailwindcss/postcss": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'postcss.config.mjs': js`
|
|
|
|
|
export default {
|
|
|
|
|
plugins: {
|
|
|
|
|
'@tailwindcss/postcss': {},
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'next.config.mjs': js`export default {}`,
|
|
|
|
|
'app/layout.js': js`
|
|
|
|
|
import './globals.css'
|
|
|
|
|
|
|
|
|
|
export default function RootLayout({ children }) {
|
|
|
|
|
return (
|
|
|
|
|
<html>
|
|
|
|
|
<body>{children}</body>
|
|
|
|
|
</html>
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/page.js': js`
|
|
|
|
|
export default function Page() {
|
|
|
|
|
return <div className="flex"></div>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'unrelated.module.css': css`
|
|
|
|
|
.module {
|
|
|
|
|
color: var(--color-blue-500);
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/globals.css': css`
|
|
|
|
|
@import 'tailwindcss/theme';
|
|
|
|
|
@import 'tailwindcss/utilities';
|
|
|
|
|
`,
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
async ({ spawn, exec, fs, expect }) => {
|
|
|
|
|
// Generate the initial build so output CSS files exist on disk
|
|
|
|
|
await exec('pnpm next build')
|
|
|
|
|
|
|
|
|
|
// NOTE: We are writing to an output CSS file which is not being ignored by
|
|
|
|
|
// `.gitignore` nor marked with `@source not`. This should not result in an
|
|
|
|
|
// infinite loop.
|
|
|
|
|
let process = await spawn(`pnpm next dev`)
|
|
|
|
|
|
|
|
|
|
let url = ''
|
|
|
|
|
await process.onStdout((m) => {
|
|
|
|
|
let match = /Local:\s*(http.*)/.exec(m)
|
|
|
|
|
if (match) url = match[1]
|
|
|
|
|
return Boolean(url)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await process.onStdout((m) => m.includes('Ready in'))
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
let css = await fetchStyles(url)
|
|
|
|
|
expect(css).toContain(candidate`flex`)
|
|
|
|
|
expect(css).toContain('--color-blue-500:')
|
|
|
|
|
expect(css).not.toContain('--color-red-500:')
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await fs.write(
|
|
|
|
|
'unrelated.module.css',
|
|
|
|
|
css`
|
|
|
|
|
.module {
|
|
|
|
|
color: var(--color-blue-500);
|
|
|
|
|
background-color: var(--color-red-500);
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
)
|
|
|
|
|
await process.onStdout((m) => m.includes('Compiled in'))
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
let css = await fetchStyles(url)
|
|
|
|
|
expect(css).toContain(candidate`flex`)
|
|
|
|
|
expect(css).toContain('--color-blue-500:')
|
|
|
|
|
expect(css).toContain('--color-red-500:')
|
|
|
|
|
})
|
|
|
|
|
},
|
|
|
|
|
)
|
2025-04-03 17:13:15 +02:00
|
|
|
|
|
|
|
|
test(
|
|
|
|
|
'changes to `public/` should not trigger an infinite loop',
|
|
|
|
|
{
|
|
|
|
|
fs: {
|
|
|
|
|
'package.json': json`
|
|
|
|
|
{
|
|
|
|
|
"dependencies": {
|
|
|
|
|
"react": "^18",
|
|
|
|
|
"react-dom": "^18",
|
|
|
|
|
"next": "^15",
|
|
|
|
|
"@ducanh2912/next-pwa": "^10.2.9"
|
|
|
|
|
},
|
|
|
|
|
"devDependencies": {
|
|
|
|
|
"@tailwindcss/postcss": "workspace:^",
|
|
|
|
|
"tailwindcss": "workspace:^"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'.gitignore': txt`
|
|
|
|
|
.next/
|
|
|
|
|
public/workbox-*.js
|
|
|
|
|
public/sw.js
|
|
|
|
|
`,
|
|
|
|
|
'postcss.config.mjs': js`
|
|
|
|
|
export default {
|
|
|
|
|
plugins: {
|
|
|
|
|
'@tailwindcss/postcss': {},
|
|
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'next.config.mjs': js`
|
|
|
|
|
import withPWA from '@ducanh2912/next-pwa'
|
|
|
|
|
|
|
|
|
|
const pwaConfig = {
|
|
|
|
|
dest: 'public',
|
|
|
|
|
register: true,
|
|
|
|
|
skipWaiting: true,
|
|
|
|
|
reloadOnOnline: false,
|
|
|
|
|
cleanupOutdatedCaches: true,
|
|
|
|
|
clientsClaim: true,
|
|
|
|
|
maximumFileSizeToCacheInBytes: 20 * 1024 * 1024,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
const nextConfig = {}
|
|
|
|
|
|
|
|
|
|
const configWithPWA = withPWA(pwaConfig)(nextConfig)
|
|
|
|
|
|
|
|
|
|
export default configWithPWA
|
|
|
|
|
`,
|
|
|
|
|
'app/layout.js': js`
|
|
|
|
|
import './globals.css'
|
|
|
|
|
|
|
|
|
|
export default function RootLayout({ children }) {
|
|
|
|
|
return (
|
|
|
|
|
<html>
|
|
|
|
|
<body>{children}</body>
|
|
|
|
|
</html>
|
|
|
|
|
)
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/page.js': js`
|
|
|
|
|
export default function Page() {
|
|
|
|
|
return <div className="flex"></div>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
'app/globals.css': css`
|
|
|
|
|
@import 'tailwindcss/theme';
|
|
|
|
|
@import 'tailwindcss/utilities';
|
|
|
|
|
`,
|
|
|
|
|
},
|
|
|
|
|
},
|
|
|
|
|
async ({ spawn, fs, expect }) => {
|
|
|
|
|
let process = await spawn('pnpm next dev')
|
|
|
|
|
|
|
|
|
|
let url = ''
|
|
|
|
|
await process.onStdout((m) => {
|
|
|
|
|
let match = /Local:\s*(http.*)/.exec(m)
|
|
|
|
|
if (match) url = match[1]
|
|
|
|
|
return Boolean(url)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await process.onStdout((m) => m.includes('Ready in'))
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
let css = await fetchStyles(url)
|
|
|
|
|
expect(css).toContain(candidate`flex`)
|
|
|
|
|
expect(css).not.toContain(candidate`underline`)
|
|
|
|
|
})
|
|
|
|
|
|
|
|
|
|
await fs.write(
|
|
|
|
|
'app/page.js',
|
|
|
|
|
jsx`
|
|
|
|
|
export default function Page() {
|
|
|
|
|
return <div className="flex underline"></div>
|
|
|
|
|
}
|
|
|
|
|
`,
|
|
|
|
|
)
|
|
|
|
|
await process.onStdout((m) => m.includes('Compiled in'))
|
|
|
|
|
|
|
|
|
|
await retryAssertion(async () => {
|
|
|
|
|
let css = await fetchStyles(url)
|
|
|
|
|
expect(css).toContain(candidate`flex`)
|
|
|
|
|
expect(css).toContain(candidate`underline`)
|
|
|
|
|
})
|
|
|
|
|
// Flush all existing messages in the queue
|
|
|
|
|
process.flush()
|
|
|
|
|
|
|
|
|
|
// Fetch the styles one more time, to ensure we see the latest version of
|
|
|
|
|
// the CSS
|
|
|
|
|
await fetchStyles(url)
|
|
|
|
|
|
2025-12-22 21:09:19 +08:00
|
|
|
// At this point, no changes should trigger a compile step. If we see any
|
2025-04-03 17:13:15 +02:00
|
|
|
// changes, there is an infinite loop because we (the user) didn't write any
|
|
|
|
|
// files to disk.
|
|
|
|
|
//
|
|
|
|
|
// Ensure there are no more changes in stdout (signaling no infinite loop)
|
|
|
|
|
let result = await Promise.race([
|
|
|
|
|
// If this succeeds, it means that it saw another change which indicates
|
|
|
|
|
// an infinite loop.
|
|
|
|
|
process.onStdout((m) => m.includes('Compiled in')).then(() => 'infinite loop detected'),
|
|
|
|
|
|
|
|
|
|
// There should be no changes in stdout
|
|
|
|
|
new Promise((resolve) => setTimeout(() => resolve('no infinite loop detected'), 2_000)),
|
|
|
|
|
])
|
|
|
|
|
expect(result).toBe('no infinite loop detected')
|
|
|
|
|
},
|
|
|
|
|
)
|